43 Commits

Author SHA1 Message Date
Godzil
e9cf0c353b 1.1.17 2017-02-12 23:37:19 +00:00
Godzil
6bc39083b9 - Support more episode naming schemes
- Display when we are going to fetch from a single URL (@http://...)
- Display a warning when a series looks to have no episodes
- Make a backup of the .crpersistent before changing it
2017-02-12 23:10:51 +00:00
Godzil
67d06246d4 1.1.17-0 2017-02-11 19:24:59 +00:00
Godzil
2ab1daf2b3 Another lint pass on episode.ts
Correct a stupid bug where it try to download an episode two times (and led to a failure) if metadata can't be fetch as expected. Doh!
2017-02-11 19:23:48 +00:00
Godzil
065d3b4c66 1.1.16 2017-02-10 23:52:23 +00:00
Godzil
cfe73f5ca8 More lint cleaning, add a way to download a single episode by URL 2017-02-10 23:51:22 +00:00
Godzil
2fea379484 Fancy output also works under windows, so it's now enabled for all platform! 2017-02-10 23:51:04 +00:00
Godzil
bee3f33e20 Update npm packages, cleanup the code, cleanup all tslint complain 2017-02-10 17:43:52 +00:00
Manoël Trapier
5d9c25491d Typo on typings 2017-02-08 16:47:52 +00:00
Godzil
58f4dc61ff Fix login issue 2017-02-07 20:22:25 +00:00
Godzil
b96efacbd2 - Revert login using the token method
- Use the cloudscraper layer on top of request to pass through the cloudfare browser check
- switch from tsd to typings
2017-02-07 20:22:01 +00:00
Godzil
a346ab8854 1.1.14 2017-02-01 08:53:23 +00:00
Godzil
499530141e Disable debug message about ffmpeg 2017-02-01 08:53:14 +00:00
Godzil
d1457bb893 1.1.13 2017-01-28 13:38:20 +00:00
Godzil
8dfd1b447c Add a log objet to do some fancy output on the command line (not fully enabled under windows as it need some tests) 2017-01-28 13:38:14 +00:00
Godzil
ce63ae9a16 Upgrade to 1.1.12 to fix login issue 2017-01-23 21:13:13 +00:00
Godzil
70d80ccd17 Update dependency to more recent version, and correct a few warnings reported by ts 2017-01-23 21:06:34 +00:00
Manoël Trapier
7833fbe292 Merge pull request #13 from majewskim/master
Fix a login issue
2017-01-23 20:46:10 +00:00
Mateusz Majewski
fa6aa74442 Merge branch 'master' into master 2017-01-18 12:49:49 +02:00
Mateusz Majewski
fe2ed9fb76 Fixing login issue by bypassing the login form and making a request directly. 2017-01-18 11:19:41 +02:00
Mateusz Majewski
cc655b9e00 Fixing login issue by bypassing the login form and making a request directly. 2017-01-18 11:08:45 +02:00
Manoël Trapier
e1d2a55a01 Update README.md 2016-10-21 17:21:36 +01:00
Manoël Trapier
a31de0ef9d Remove .js from the name 2016-10-21 17:21:05 +01:00
Godzil
2853334d7f 1.1.11
- Update login mechanism to march CR september 2016 changes
2016-09-16 22:20:43 +01:00
Godzil
69dd28d31b Update login to match latest CR changes 2016-09-16 22:20:39 +01:00
Godzil
56afce02ea 1.1.10
- Change name format to follow Plex forvourite one.
- Remplace ":" in file name to prevent issue on Windows
2016-09-10 20:22:10 +01:00
Godzil
bc4697061e Remplace ':' in filename to make Windows happy 2016-09-10 20:17:19 +01:00
Godzil
55ffe85f77 Make the name to be more Plex friendly 2016-09-10 11:53:45 +01:00
Godzil
ec8c2c7716 1.1.9
New features:
 - Should correctly handle 10.5 or 1A episode numbers
 - Change characters in series name that are not allowed in filename (or could cause issues, like slash ( / ) or single quote ( ' )
 - Prevent overwrite of existing files (usefull when different season report the same number)
 - If an error during episode metadata scraping, it will set to Season 0, episode 0 (look at the logs!)
 - If an episode is not available yet, it will be skipped
2016-09-07 22:09:14 +01:00
Godzil
714a528f8b Prevent overwriting an existing output file 2016-09-07 21:55:14 +01:00
Godzil
8314d91bd7 Add functionality to ignore (instead of stopping) if an episode is not available yet 2016-09-07 21:51:36 +01:00
Godzil
5bd31f9e0b Add better episode numbering scheme 2016-09-07 21:34:29 +01:00
Godzil
95a93930f3 Merge branch 'master' of github.com:Godzil/crunchyroll.js 2016-08-22 13:10:02 +02:00
Godzil
4a9e1d0410 Update LICENSE 2016-08-22 12:03:05 +01:00
Godzil
1eacd0a5ca add license to npm 2016-08-22 13:03:02 +02:00
Godzil
3c32726745 1.1.8 2016-08-22 12:32:25 +02:00
Godzil
42ae0ae1fb Forget to rename main executable 2016-08-22 12:30:43 +02:00
Godzil
e4b3871919 1.1.7 2016-08-22 12:22:45 +02:00
Godzil
58e4a557e2 Update readme 2016-08-22 12:22:36 +02:00
Godzil
8371d68113 Correct errors 2016-08-22 12:21:17 +02:00
Godzil
b7d496fc9d Update README.md 2016-08-22 10:51:23 +01:00
Godzil
14260d04b3 Update README.md 2016-08-22 10:50:13 +01:00
Godzil
3d46b65d67 Update package.json 2016-08-22 10:47:18 +01:00
23 changed files with 992 additions and 442 deletions

View File

@@ -1,4 +1,5 @@
Copyright (c) 2015 Roel van Uden Copyright (c) 2015 Roel van Uden
Copyright (c) 2016 Manoel <Godzil> Trapier
Permission is hereby granted, free of charge, to any person obtaining a copy Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to of this software and associated documentation files (the "Software"), to

1
README Symbolic link
View File

@@ -0,0 +1 @@
README.md

View File

@@ -1,6 +1,6 @@
# CrunchyRoll.js # Crunchy: a fork of Deathspike/CrunchyRoll.js
*CrunchyRoll.js* is capable of downloading *anime* episodes from the popular *CrunchyRoll* streaming service. An episode is stored in the original video format (often H.264 in a MP4 container) and the configured subtitle format (ASS or SRT).The two output files are then merged into a single MKV file. *Crunchy* is capable of downloading *anime* episodes from the popular *CrunchyRoll* streaming service. An episode is stored in the original video format (often H.264 in a MP4 container) and the configured subtitle format (ASS or SRT).The two output files are then merged into a single MKV file.
## Motivation ## Motivation
@@ -10,6 +10,8 @@
This application is not endorsed or affliated with *CrunchyRoll*. The usage of this application enables episodes to be downloaded for offline convenience which may be forbidden by law in your country. Usage of this application may also cause a violation of the agreed *Terms of Service* between you and the stream provider. A tool is not responsible for your actions; please make an informed decision prior to using this application. This application is not endorsed or affliated with *CrunchyRoll*. The usage of this application enables episodes to be downloaded for offline convenience which may be forbidden by law in your country. Usage of this application may also cause a violation of the agreed *Terms of Service* between you and the stream provider. A tool is not responsible for your actions; please make an informed decision prior to using this application.
**PLEASE _ONLY_ USE THIS TOOL IF YOU HAVE A _PREMIUM ACCOUNT_**
## Configuration ## Configuration
It is recommended to enable authentication (`-p` and `-u`) so your account permissions and settings are available for use. It is not possible to download non-free material without an account and premium subscription. Furthermore, the default account settings are used when downloading. If you want the highest quality videos, configure these preferences at https://www.crunchyroll.com/acct/?action=video. It is recommended to enable authentication (`-p` and `-u`) so your account permissions and settings are available for use. It is not possible to download non-free material without an account and premium subscription. Furthermore, the default account settings are used when downloading. If you want the highest quality videos, configure these preferences at https://www.crunchyroll.com/acct/?action=video.
@@ -28,28 +30,28 @@ Use the applicable instructions to install. Is your operating system not listed?
1. Run in *Terminal*: `sudo apt-get install nodejs npm mkvtoolnix rtmpdump ffmpeg` 1. Run in *Terminal*: `sudo apt-get install nodejs npm mkvtoolnix rtmpdump ffmpeg`
2. Run in *Terminal*: `sudo ln -s /usr/bin/nodejs /usr/bin/node` 2. Run in *Terminal*: `sudo ln -s /usr/bin/nodejs /usr/bin/node`
3. Run in *Terminal*: `sudo npm install -g crunchyroll` 3. Run in *Terminal*: `sudo npm install -g crunchy`
### Mac OS X ### Mac OS X
1. Install *Homebrew* following the instructions at http://brew.sh/ 1. Install *Homebrew* following the instructions at http://brew.sh/
2. Run in *Terminal*: `brew install node mkvtoolnix rtmpdump ffmpeg` 2. Run in *Terminal*: `brew install node mkvtoolnix rtmpdump ffmpeg`
3. Run in *Terminal*: `npm install -g crunchyroll` 3. Run in *Terminal*: `npm install -g crunchy`
### Windows ### Windows
1. Install *NodeJS* following the instructions at http://nodejs.org/ 1. Install *NodeJS* following the instructions at http://nodejs.org/
3. Run in *Command Prompt*: `npm install -g crunchyroll` 3. Run in *Command Prompt*: `npm install -g crunchy`
## Instructions ## Instructions
Use the applicable instructions for the interface of your choice (currently limited to command-line). Use the applicable instructions for the interface of your choice (currently limited to command-line).
### Command-line Interface (`crunchyroll`) ### Command-line Interface (`crunchy`)
The [command-line interface](http://en.wikipedia.org/wiki/Command-line_interface) does not have a graphical component and is ideal for automation purposes and headless machines. The interface can run using a sequence of series addresses (the site address containing the episode listing), or with a batch-mode source file. The `crunchyroll --help` command will produce the following output: The [command-line interface](http://en.wikipedia.org/wiki/Command-line_interface) does not have a graphical component and is ideal for automation purposes and headless machines. The interface can run using a sequence of series addresses (the site address containing the episode listing), or with a batch-mode source file. The `crunchy --help` command will produce the following output:
Usage: crunchyroll [options] Usage: crunchy [options]
Options: Options:
@@ -74,15 +76,15 @@ When no sequence of series addresses is provided, the batch-mode source file wil
Download in batch-mode: Download in batch-mode:
crunchyroll crunchy
Download *Fairy Tail* to the current work directory: Download *Fairy Tail* to the current work directory:
crunchyroll http://www.crunchyroll.com/fairy-tail crunchy http://www.crunchyroll.com/fairy-tail
Download *Fairy Tail* to `C:\Anime`: Download *Fairy Tail* to `C:\Anime`:
crunchyroll --output C:\Anime http://www.crunchyroll.com/fairy-tail crunchy --output C:\Anime http://www.crunchyroll.com/fairy-tail
#### Switches #### Switches

View File

@@ -1,36 +1,46 @@
{ {
"author": "Roel van Uden", "author": "Godzil",
"description": "CrunchyRoll.js is capable of downloading anime episodes from the popular CrunchyRoll streaming service.", "description": "Crunchy is a fork of Crunchyroll.js, capable of downloading anime episodes from the popular CrunchyRoll streaming service.",
"license": "MIT",
"keywords": [ "keywords": [
"anime", "anime",
"download", "download",
"crunchyroll" "crunchyroll"
], ],
"name": "crunchyroll", "name": "crunchy",
"repository": { "repository": {
"type": "git", "type": "git",
"url": "git://github.com/Deathspike/crunchyroll.js.git" "url": "git://github.com/Godzil/crunchyroll.js.git"
}, },
"version": "1.1.5", "version": "1.1.17",
"bin": { "bin": {
"crunchyroll": "./bin/crunchyroll" "crunchy": "./bin/crunchy"
}, },
"dependencies": { "dependencies": {
"big-integer": "1.4.4", "big-integer": "^1.4.4",
"cheerio": "0.18.0", "cheerio": "^0.22.0",
"commander": "2.6.0", "cloudscraper": "^1.4.1",
"mkdirp": "0.5.0", "commander": "^2.6.0",
"request": "2.53.0", "fs-extra": "^2.0.0",
"xml2js": "0.4.5" "mkdirp": "^0.5.0",
"request": "^2.74.0",
"xml2js": "^0.4.5"
}, },
"devDependencies": { "devDependencies": {
"tsd": "0.5.7", "tsconfig-lint": "^0.12.0",
"tslint": "2.3.0-beta", "tslint": "^4.4.2",
"typescript": "1.5.0-beta" "typescript": "^2.2.0",
"typings": "^2.1.0"
}, },
"scripts": { "scripts": {
"prepublish": "npm run tsd && tsc", "prepublish": "npm run types && tsc",
"test": "node ts --only-test", "compile": "tsc",
"tsd": "tsd reinstall -o -s" "test": "tslint -c ./tslint.json --project ./tsconfig.json ./src/**/*.ts",
"types": "typings install",
"reinstall": "tsd reinstall; npm run types",
"start": "node ./bin/crunchy"
},
"bugs": {
"url": "https://github.com/Godzil/Crunchy/issues"
} }
} }

View File

@@ -7,16 +7,33 @@ import series from './series';
/** /**
* Streams the batch of series to disk. * Streams the batch of series to disk.
*/ */
export default function(args: string[], done: (err?: Error) => void) { export default function(args: string[], done: (err?: Error) => void)
var config = parse(args); {
var batchPath = path.join(config.output || process.cwd(), 'CrunchyRoll.txt'); const config = parse(args);
tasks(config, batchPath, (err, tasks) => { const batchPath = path.join(config.output || process.cwd(), 'CrunchyRoll.txt');
if (err) return done(err);
var i = 0; tasks(config, batchPath, (err, tasks) =>
(function next() { {
if (i >= tasks.length) return done(); if (err)
series(tasks[i].config, tasks[i].address, err => { {
if (err) return done(err); return done(err);
}
let i = 0;
(function next()
{
if (i >= tasks.length)
{
return done();
}
series(tasks[i].config, tasks[i].address, (errin) =>
{
if (errin)
{
return done(errin);
}
i += 1; i += 1;
next(); next();
}); });
@@ -27,43 +44,85 @@ export default function(args: string[], done: (err?: Error) => void) {
/** /**
* Splits the value into arguments. * Splits the value into arguments.
*/ */
function split(value: string): string[] { function split(value: string): string[]
var inQuote = false; {
var i: number; let inQuote = false;
var pieces: string[] = []; let i: number;
var previous = 0; const pieces: string[] = [];
for (i = 0; i < value.length; i += 1) { let previous = 0;
if (value.charAt(i) === '"') inQuote = !inQuote;
if (!inQuote && value.charAt(i) === ' ') { for (i = 0; i < value.length; i += 1)
{
if (value.charAt(i) === '"')
{
inQuote = !inQuote;
}
if (!inQuote && value.charAt(i) === ' ')
{
pieces.push(value.substring(previous, i).match(/^"?(.+?)"?$/)[1]); pieces.push(value.substring(previous, i).match(/^"?(.+?)"?$/)[1]);
previous = i + 1; previous = i + 1;
} }
} }
var lastPiece = value.substring(previous, i).match(/^"?(.+?)"?$/);
if (lastPiece) pieces.push(lastPiece[1]); const lastPiece = value.substring(previous, i).match(/^"?(.+?)"?$/);
if (lastPiece)
{
pieces.push(lastPiece[1]);
}
return pieces; return pieces;
} }
/** /**
* Parses the configuration or reads the batch-mode file for tasks. * Parses the configuration or reads the batch-mode file for tasks.
*/ */
function tasks(config: IConfigLine, batchPath: string, done: (err: Error, tasks?: IConfigTask[]) => void) { function tasks(config: IConfigLine, batchPath: string, done: (err: Error, tasks?: IConfigTask[]) => void)
if (config.args.length) { {
return done(null, config.args.map(address => { if (config.args.length)
return {address: address, config: config}; {
const configIn = config;
return done(null, config.args.map((addressIn) =>
{
return {address: addressIn, config: configIn};
})); }));
} }
fs.exists(batchPath, exists => {
if (!exists) return done(null, []); fs.exists(batchPath, (exists) =>
fs.readFile(batchPath, 'utf8', (err, data) => { {
if (err) return done(err); if (!exists)
var map: IConfigTask[] = []; {
data.split(/\r?\n/).forEach(line => { return done(null, []);
if (/^(\/\/|#)/.test(line)) return; }
var lineConfig = parse(process.argv.concat(split(line)));
lineConfig.args.forEach(address => { fs.readFile(batchPath, 'utf8', (err, data) =>
if (!address) return; {
map.push({address: address, config: lineConfig}); if (err)
{
return done(err);
}
const map: IConfigTask[] = [];
data.split(/\r?\n/).forEach((line) =>
{
if (/^(\/\/|#)/.test(line))
{
return;
}
const lineConfig = parse(process.argv.concat(split(line)));
lineConfig.args.forEach((addressIn) =>
{
if (!addressIn)
{
return;
}
map.push({address: addressIn, config: lineConfig});
}); });
}); });
done(null, map); done(null, map);
@@ -74,7 +133,8 @@ function tasks(config: IConfigLine, batchPath: string, done: (err: Error, tasks?
/** /**
* Parses the arguments and returns a configuration. * Parses the arguments and returns a configuration.
*/ */
function parse(args: string[]): IConfigLine { function parse(args: string[]): IConfigLine
{
return new commander.Command().version(require('../package').version) return new commander.Command().version(require('../package').version)
// Authentication // Authentication
.option('-p, --pass <s>', 'The password.') .option('-p, --pass <s>', 'The password.')

View File

@@ -1,6 +1,10 @@
'use strict'; 'use strict';
import batch from './batch'; import batch from './batch';
batch(process.argv, (err: any) => { batch(process.argv, (err: any) =>
if (err) console.error(err.stack || err); {
if (err)
{
console.error(err.stack || err);
}
}); });

View File

@@ -7,15 +7,27 @@ import path = require('path');
import subtitle from './subtitle/index'; import subtitle from './subtitle/index';
import video from './video/index'; import video from './video/index';
import xml2js = require('xml2js'); import xml2js = require('xml2js');
import log = require('./log');
/** /**
* Streams the episode to disk. * Streams the episode to disk.
*/ */
export default function(config: IConfig, address: string, done: (err: Error) => void) { export default function(config: IConfig, address: string, done: (err: Error, ign: boolean) => void)
scrapePage(config, address, (err, page) => { {
if (err) return done(err); scrapePage(config, address, (err, page) =>
scrapePlayer(config, address, page.id, (err, player) => { {
if (err) return done(err); if (err)
{
return done(err, false);
}
scrapePlayer(config, address, page.id, (errS, player) =>
{
if (errS)
{
return done(errS, false);
}
download(config, page, player, done); download(config, page, player, done);
}); });
}); });
@@ -24,37 +36,107 @@ export default function(config: IConfig, address: string, done: (err: Error) =>
/** /**
* Completes a download and writes the message with an elapsed time. * Completes a download and writes the message with an elapsed time.
*/ */
function complete(message: string, begin: number, done: (err: Error) => void) { function complete(epName: string, message: string, begin: number, done: (err: Error, ign: boolean) => void)
var timeInMs = Date.now() - begin; {
var seconds = prefix(Math.floor(timeInMs / 1000) % 60, 2); const timeInMs = Date.now() - begin;
var minutes = prefix(Math.floor(timeInMs / 1000 / 60) % 60, 2); const seconds = prefix(Math.floor(timeInMs / 1000) % 60, 2);
var hours = prefix(Math.floor(timeInMs / 1000 / 60 / 60), 2); const minutes = prefix(Math.floor(timeInMs / 1000 / 60) % 60, 2);
console.log(message + ' (' + hours + ':' + minutes + ':' + seconds + ')'); const hours = prefix(Math.floor(timeInMs / 1000 / 60 / 60), 2);
done(null);
log.dispEpisode(epName, message + ' (' + hours + ':' + minutes + ':' + seconds + ')', true);
done(null, false);
}
/**
* Check if a file exist..
*/
function fileExist(path: string)
{
try
{
fs.statSync(path);
return true;
} catch (e)
{
return false;
}
} }
/** /**
* Downloads the subtitle and video. * Downloads the subtitle and video.
*/ */
function download(config: IConfig, page: IEpisodePage, player: IEpisodePlayer, done: (err: Error) => void) { function download(config: IConfig, page: IEpisodePage, player: IEpisodePlayer, done: (err: Error, ign: boolean) => void)
var series = config.series || page.series; {
var fileName = name(config, page, series); let series = config.series || page.series;
var filePath = path.join(config.output || process.cwd(), series, fileName);
mkdirp(path.dirname(filePath), (err: Error) => { series = series.replace('/', '_').replace('\'', '_').replace(':', '_');
if (err) return done(err); let fileName = name(config, page, series, '').replace('/', '_').replace('\'', '_').replace(':', '_');
downloadSubtitle(config, player, filePath, err => { let filePath = path.join(config.output || process.cwd(), series, fileName);
if (err) return done(err);
var now = Date.now(); if (fileExist(filePath + '.mkv'))
console.log('Fetching ' + fileName); {
downloadVideo(config, page, player, filePath, err => { let count = 0;
if (err) return done(err); log.warn('File \'' + fileName + '\' already exist...');
if (config.merge) return complete('Finished ' + fileName, now, done);
var isSubtited = Boolean(player.subtitle); do
video.merge(config, isSubtited, player.video.file, filePath, player.mode, err => { {
if (err) return done(err); count = count + 1;
complete('Finished ' + fileName, now, done); fileName = name(config, page, series, '-' + count).replace('/', '_').replace('\'', '_').replace(':', '_');
filePath = path.join(config.output || process.cwd(), series, fileName);
} while (fileExist(filePath + '.mkv'));
log.warn('Renaming to \'' + fileName + '\'...');
}
mkdirp(path.dirname(filePath), (errM: Error) =>
{
if (errM)
{
return done(errM, false);
}
downloadSubtitle(config, player, filePath, (errDS) =>
{
if (errDS)
{
return done(errDS, false);
}
const now = Date.now();
if (player.video.file !== undefined)
{
log.dispEpisode(fileName, 'Fetching...', false);
downloadVideo(config, page, player, filePath, (errDV) =>
{
if (errDV)
{
return done(errDV, false);
}
if (config.merge)
{
return complete(fileName, 'Finished!', now, done);
}
const isSubtited = Boolean(player.subtitle);
video.merge(config, isSubtited, player.video.file, filePath, player.video.mode, (errVM) =>
{
if (errVM)
{
return done(errVM, false);
}
complete(fileName, 'Finished!', now, done);
});
}); });
}); }
else
{
log.dispEpisode(fileName, 'Ignoring: not released yet', true);
done(null, true);
}
}); });
}); });
} }
@@ -62,15 +144,32 @@ function download(config: IConfig, page: IEpisodePage, player: IEpisodePlayer, d
/** /**
* Saves the subtitles to disk. * Saves the subtitles to disk.
*/ */
function downloadSubtitle(config: IConfig, player: IEpisodePlayer, filePath: string, done: (err?: Error) => void) { function downloadSubtitle(config: IConfig, player: IEpisodePlayer, filePath: string, done: (err?: Error) => void)
var enc = player.subtitle; {
if (!enc) return done(); const enc = player.subtitle;
subtitle.decode(enc.id, enc.iv, enc.data, (err, data) => {
if (err) return done(err); if (!enc)
var formats = subtitle.formats; {
var format = formats[config.format] ? config.format : 'ass'; return done();
formats[format](data, (err: Error, decodedSubtitle: string) => { }
if (err) return done(err);
subtitle.decode(enc.id, enc.iv, enc.data, (errSD, data) =>
{
if (errSD)
{
return done(errSD);
}
const formats = subtitle.formats;
const format = formats[config.format] ? config.format : 'ass';
formats[format](data, (errF: Error, decodedSubtitle: string) =>
{
if (errF)
{
return done(errF);
}
fs.writeFile(filePath + '.' + format, '\ufeff' + decodedSubtitle, done); fs.writeFile(filePath + '.' + format, '\ufeff' + decodedSubtitle, done);
}); });
}); });
@@ -79,98 +178,149 @@ function downloadSubtitle(config: IConfig, player: IEpisodePlayer, filePath: str
/** /**
* Streams the video to disk. * Streams the video to disk.
*/ */
function downloadVideo(config: IConfig, function downloadVideo(ignored/*config*/: IConfig, page: IEpisodePage, player: IEpisodePlayer,
page: IEpisodePage, filePath: string, done: (err: Error) => void)
player: IEpisodePlayer, {
filePath: string, video.stream(player.video.host, player.video.file, page.swf, filePath,
done: (err: Error) => void) { path.extname(player.video.file), player.video.mode, done);
video.stream(
player.video.host,
player.video.file,
page.swf,
filePath, path.extname(player.video.file),
player.video.mode,
done);
} }
/** /**
* Names the file based on the config, page, series and tag. * Names the file based on the config, page, series and tag.
*/ */
function name(config: IConfig, page: IEpisodePage, series: string) { function name(config: IConfig, page: IEpisodePage, series: string, extra: string)
var episode = (page.episode < 10 ? '0' : '') + page.episode; {
var volume = (page.volume < 10 ? '0' : '') + page.volume; const episodeNum = parseInt(page.episode, 10);
var tag = config.tag || 'CrunchyRoll'; const volumeNum = parseInt(page.volume, 10);
return series + ' ' + volume + 'x' + episode + ' [' + tag + ']'; const episode = (episodeNum < 10 ? '0' : '') + page.episode;
const volume = (volumeNum < 10 ? '0' : '') + page.volume;
const tag = config.tag || 'CrunchyRoll';
return series + ' - s' + volume + 'e' + episode + ' - [' + tag + ']' + extra;
} }
/** /**
* Prefixes a value. * Prefixes a value.
*/ */
function prefix(value: number|string, length: number) { function prefix(value: number|string, length: number)
var valueString = typeof value !== 'string' ? String(value) : value; {
while (valueString.length < length) valueString = '0' + valueString; let valueString = (typeof value !== 'string') ? String(value) : value;
while (valueString.length < length)
{
valueString = '0' + valueString;
}
return valueString; return valueString;
} }
/** /**
* Requests the page data and scrapes the id, episode, series and swf. * Requests the page data and scrapes the id, episode, series and swf.
*/ */
function scrapePage(config: IConfig, address: string, done: (err: Error, page?: IEpisodePage) => void) { function scrapePage(config: IConfig, address: string, done: (err: Error, page?: IEpisodePage) => void)
var id = parseInt((address.match(/[0-9]+$/) || ['0'])[0], 10); {
if (!id) return done(new Error('Invalid address.')); const epId = parseInt((address.match(/[0-9]+$/) || ['0'])[0], 10);
request.get(config, address, (err, result) => {
if (err) return done(err); if (!epId)
var $ = cheerio.load(result); {
var swf = /^([^?]+)/.exec($('link[rel=video_src]').attr('href')); return done(new Error('Invalid address.'));
var regexp = /-\s+(?:Watch\s+)?(.+?)(?:\s+Season\s+([0-9]+))?(?:\s+-)?\s+Episode\s+([0-9]+)/; }
var data = regexp.exec($('title').text());
if (!swf || !data) return done(new Error('Invalid page.')); request.get(config, address, (err, result) =>
done(null, { {
id: id, if (err)
episode: parseInt(data[3], 10), {
series: data[1], return done(err);
swf: swf[1], }
volume: parseInt(data[2], 10) || 1
}); const $ = cheerio.load(result);
const swf = /^([^?]+)/.exec($('link[rel=video_src]').attr('href'));
const regexp = /\s*([^\n\r\t\f]+)\n?\s*[^0-9]*([0-9][\-0-9.]*)?,?\n?\s\s*[^0-9]*((PV )?[S0-9][P0-9.]*[a-fA-F]?)/;
const look = $('#showmedia_about_media').text();
const seasonTitle = $('span[itemprop="title"]').text();
const data = regexp.exec(look);
if (!swf || !data)
{
log.warn('Somethig unexpected in the page at ' + address + ' (data are: ' + look + ')');
log.warn('Setting Season to 0 and episode to 0...');
done(null, {
episode: '0',
id: epId,
series: seasonTitle,
swf: swf[1],
volume: '0',
});
}
else
{
done(null, {
episode: data[3],
id: epId,
series: data[1],
swf: swf[1],
volume: data[2] || '1',
});
}
}); });
} }
/** /**
* Requests the player data and scrapes the subtitle and video data. * Requests the player data and scrapes the subtitle and video data.
*/ */
function scrapePlayer(config: IConfig, address: string, id: number, done: (err: Error, player?: IEpisodePlayer) => void) { function scrapePlayer(config: IConfig, address: string, id: number, done: (err: Error, player?: IEpisodePlayer) => void)
var url = address.match(/^(https?:\/\/[^\/]+)/); {
if (!url) return done(new Error('Invalid address.')); const url = address.match(/^(https?:\/\/[^\/]+)/);
if (!url)
{
return done(new Error('Invalid address.'));
}
request.post(config, { request.post(config, {
form: {current_page: address}, form: {current_page: address},
url: url[1] + '/xml/?req=RpcApiVideoPlayer_GetStandardConfig&media_id=' + id url: url[1] + '/xml/?req=RpcApiVideoPlayer_GetStandardConfig&media_id=' + id,
}, (err, result) => { }, (err, result) =>
if (err) return done(err); {
if (err)
{
return done(err);
}
xml2js.parseString(result, { xml2js.parseString(result, {
explicitArray: false, explicitArray: false,
explicitRoot: false explicitRoot: false,
}, (err: Error, player: IEpisodePlayerConfig) => { }, (errPS: Error, player: IEpisodePlayerConfig) =>
if (err) return done(err); {
try { if (errPS)
var isSubtitled = Boolean(player['default:preload'].subtitle); {
var streamMode="RTMP"; return done(errPS);
if (player['default:preload'].stream_info.host == "") }
try
{
const isSubtitled = Boolean(player['default:preload'].subtitle);
let streamMode = 'RTMP';
if (player['default:preload'].stream_info.host === '')
{ {
streamMode="HLS"; streamMode = 'HLS';
} }
done(null, { done(null, {
subtitle: isSubtitled ? { subtitle: isSubtitled ? {
data: player['default:preload'].subtitle.data,
id: parseInt(player['default:preload'].subtitle.$.id, 10), id: parseInt(player['default:preload'].subtitle.$.id, 10),
iv: player['default:preload'].subtitle.iv, iv: player['default:preload'].subtitle.iv,
data: player['default:preload'].subtitle.data
} : null, } : null,
video: { video: {
mode: streamMode;
file: player['default:preload'].stream_info.file, file: player['default:preload'].stream_info.file,
host: player['default:preload'].stream_info.host host: player['default:preload'].stream_info.host,
} mode: streamMode,
},
}); });
} catch (parseError) { } catch (parseError)
{
done(parseError); done(parseError);
} }
}); });

View File

@@ -1,7 +1,7 @@
interface IEpisodePage { interface IEpisodePage {
id: number; id: number;
episode: number; episode: string;
series: string; series: string;
volume: number; volume: string;
swf: string; swf: string;
} }

View File

@@ -5,6 +5,7 @@ interface IEpisodePlayer {
data: string; data: string;
}; };
video: { video: {
mode: string;
file: string; file: string;
host: string; host: string;
}; };

View File

@@ -1,5 +1,5 @@
interface ISeriesEpisode { interface ISeriesEpisode {
address: string; address: string;
episode: number; episode: string;
volume: number; volume: number;
} }

37
src/log.ts Normal file
View File

@@ -0,0 +1,37 @@
'use strict';
import os = require('os');
export function error(str: string)
{
/* Do fancy output */
console.error(' \x1B[1;31m* ERROR\x1B[0m: ' + str);
}
export function info(str: string)
{
/* Do fancy output */
console.log(' \x1B[1;32m* INFO \x1B[0m: ' + str);
}
export function debug(str: string)
{
/* Do fancy output */
console.log(' \x1B[1;35m* DEBUG\x1B[0m: ' + str);
}
export function warn(str: string)
{
/* Do fancy output */
console.log(' \x1B[1;33m* WARN \x1B[0m: ' + str);
}
export function dispEpisode(name: string, status: string, addNL: boolean)
{
/* Do fancy output */
process.stdout.write(' \x1B[1;33m> \x1B[37m' + name + '\x1B[0m : \x1B[33m' + status + '\x1B[0m\x1B[0G');
if (addNL)
{
console.log('');
}
}

View File

@@ -1,15 +1,37 @@
'use strict'; 'use strict';
import request = require('request'); import request = require('request');
var isAuthenticated = false; import cheerio = require('cheerio');
import log = require('./log');
const cloudscraper = require('cloudscraper');
let isAuthenticated = false;
let isPremium = false;
const defaultHeaders: request.Headers =
{
'User-Agent': 'Mozilla/5.0 (Windows NT 6.2; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0',
'Connection': 'keep-alive'
};
/** /**
* Performs a GET request for the resource. * Performs a GET request for the resource.
*/ */
export function get(config: IConfig, options: request.Options, done: (err: Error, result?: string) => void) { export function get(config: IConfig, options: request.Options, done: (err: Error, result?: string) => void)
authenticate(config, err => { {
if (err) return done(err); authenticate(config, err =>
request.get(modify(options), (err: Error, response: any, body: any) => { {
if (err) return done(err); if (err)
{
return done(err);
}
cloudscraper.request(modify(options, 'GET'), (err: Error, response: any, body: any) =>
{
if (err)
{
return done(err);
}
done(null, typeof body === 'string' ? body : String(body)); done(null, typeof body === 'string' ? body : String(body));
}); });
}); });
@@ -18,11 +40,22 @@ export function get(config: IConfig, options: request.Options, done: (err: Error
/** /**
* Performs a POST request for the resource. * Performs a POST request for the resource.
*/ */
export function post(config: IConfig, options: request.Options, done: (err: Error, result?: string) => void) { export function post(config: IConfig, options: request.Options, done: (err: Error, result?: string) => void)
authenticate(config, err => { {
if (err) return done(err); authenticate(config, err =>
request.post(modify(options), (err: Error, response: any, body: any) => { {
if (err) return done(err); if (err)
{
return done(err);
}
cloudscraper.request(modify(options, 'POST'), (err: Error, response: any, body: any) =>
{
if (err)
{
return done(err);
}
done(null, typeof body === 'string' ? body : String(body)); done(null, typeof body === 'string' ? body : String(body));
}); });
}); });
@@ -31,32 +64,127 @@ export function post(config: IConfig, options: request.Options, done: (err: Erro
/** /**
* Authenticates using the configured pass and user. * Authenticates using the configured pass and user.
*/ */
function authenticate(config: IConfig, done: (err: Error) => void) { function authenticate(config: IConfig, done: (err: Error) => void)
if (isAuthenticated || !config.pass || !config.user) return done(null); {
var options = { if (isAuthenticated || !config.pass || !config.user)
form: { {
formname: 'RpcApiUser_Login', return done(null);
fail_url: 'https://www.crunchyroll.com/login', }
name: config.user,
password: config.pass /* Bypass the login page and send a login request directly */
}, let options =
{
headers: defaultHeaders,
jar: true, jar: true,
url: 'https://www.crunchyroll.com/?a=formhandler' gzip: false,
method: 'GET',
url: 'https://www.crunchyroll.com/login'
}; };
request.post(options, (err: Error) => {
cloudscraper.request(options, (err: Error, rep: string, body: string) =>
{
if (err) return done(err); if (err) return done(err);
isAuthenticated = true;
done(null); const $ = cheerio.load(body);
/* Get the token from the login page */
const token = $('input[name="login_form[_token]"]').attr('value');
if (token === '')
{
return done(new Error('Can`t find token!'));
}
let options =
{
headers: defaultHeaders,
form:
{
'login_form[redirect_url]': '/',
'login_form[name]': config.user,
'login_form[password]': config.pass,
'login_form[_token]': token
},
jar: true,
gzip: false,
method: 'POST',
url: 'https://www.crunchyroll.com/login'
};
cloudscraper.request(options, (err: Error, rep: string, body: string) =>
{
if (err)
{
return done(err);
}
/* The page return with a meta based redirection, as we wan't to check that everything is fine, reload
* the main page. A bit convoluted, but more sure.
*/
let options =
{
headers: defaultHeaders,
jar: true,
url: 'http://www.crunchyroll.com/',
method: 'GET'
};
cloudscraper.request(options, (err: Error, rep: string, body: string) =>
{
if (err)
{
return done(err);
}
let $ = cheerio.load(body);
/* Check if auth worked */
const regexps = /ga\('set', 'dimension[5-8]', '([^']*)'\);/g;
const dims = regexps.exec($('script').text());
for (let i = 1; i < 5; i++)
{
if ((dims[i] !== undefined) && (dims[i] !== '') && (dims[i] !== 'not-registered'))
{
isAuthenticated = true;
}
if ((dims[i] === 'premium') || (dims[i] === 'premiumplus'))
{
isPremium = true;
}
}
if (isAuthenticated === false)
{
const error = $('ul.message, li.error').text();
return done(new Error('Authentication failed: ' + error));
}
if (isPremium === false)
{
log.warn('Do not use this app without a premium account.');
}
else
{
log.info('You have a premium account! Good!');
}
done(null);
});
});
}); });
} }
/** /**
* Modifies the options to use the authenticated cookie jar. * Modifies the options to use the authenticated cookie jar.
*/ */
function modify(options: string|request.Options): request.Options { function modify(options: string|request.Options, reqMethod: string): request.Options
if (typeof options !== 'string') { {
if (typeof options !== 'string')
{
options.jar = true; options.jar = true;
options.headers = defaultHeaders;
options.method = reqMethod;
return options; return options;
} }
return {jar: true, url: options.toString()}; return { jar: true, headers: defaultHeaders, url: options.toString(), method: reqMethod };
} }

View File

@@ -2,31 +2,64 @@
import cheerio = require('cheerio'); import cheerio = require('cheerio');
import episode from './episode'; import episode from './episode';
import fs = require('fs'); import fs = require('fs');
const fse = require('fs-extra');
import request = require('./request'); import request = require('./request');
import path = require('path'); import path = require('path');
import url = require('url'); import url = require('url');
var persistent = '.crpersistent'; import log = require('./log');
const persistent = '.crpersistent';
/** /**
* Streams the series to disk. * Streams the series to disk.
*/ */
export default function(config: IConfig, address: string, done: (err: Error) => void) { export default function(config: IConfig, address: string, done: (err: Error) => void)
var persistentPath = path.join(config.output || process.cwd(), persistent); {
fs.readFile(persistentPath, 'utf8', (err, contents) => { const persistentPath = path.join(config.output || process.cwd(), persistent);
var cache = config.cache ? {} : JSON.parse(contents || '{}');
page(config, address, (err, page) => { /* Make a backup of the persistent file in case of */
if (err) return done(err); fse.copySync(persistentPath, persistentPath + '.backup');
var i = 0;
(function next() { fs.readFile(persistentPath, 'utf8', (err: Error, contents: string) =>
{
const cache = config.cache ? {} : JSON.parse(contents || '{}');
page(config, address, (errP, page) =>
{
if (errP)
{
return done(errP);
}
let i = 0;
(function next()
{
if (i >= page.episodes.length) return done(null); if (i >= page.episodes.length) return done(null);
download(cache, config, address, page.episodes[i], err => { download(cache, config, address, page.episodes[i], (errD, ignored) =>
if (err) return done(err); {
var newCache = JSON.stringify(cache, null, ' '); if (errD)
fs.writeFile(persistentPath, newCache, err => { {
if (err) return done(err); return done(errD);
}
if ((ignored === false) || (ignored === undefined))
{
const newCache = JSON.stringify(cache, null, ' ');
fs.writeFile(persistentPath, newCache, (errW: Error) =>
{
if (errW)
{
return done(errW);
}
i += 1;
next();
});
}
else
{
i += 1; i += 1;
next(); next();
}); }
}); });
})(); })();
}); });
@@ -36,60 +69,117 @@ export default function(config: IConfig, address: string, done: (err: Error) =>
/** /**
* Downloads the episode. * Downloads the episode.
*/ */
function download(cache: {[address: string]: number}, function download(cache: {[address: string]: number}, config: IConfig,
config: IConfig, baseAddress: string, item: ISeriesEpisode,
baseAddress: string, done: (err: Error, ign: boolean) => void)
item: ISeriesEpisode, {
done: (err: Error) => void) { if (!filter(config, item))
if (!filter(config, item)) return done(null); {
var address = url.resolve(baseAddress, item.address); return done(null, false);
if (cache[address]) return done(null); }
episode(config, address, err => {
if (err) return done(err); const address = url.resolve(baseAddress, item.address);
if (cache[address])
{
return done(null, false);
}
episode(config, address, (err, ignored) =>
{
if (err)
{
return done(err, false);
}
cache[address] = Date.now(); cache[address] = Date.now();
done(null); done(null, ignored);
}); });
} }
/** /**
* Filters the item based on the configuration. * Filters the item based on the configuration.
*/ */
function filter(config: IConfig, item: ISeriesEpisode) { function filter(config: IConfig, item: ISeriesEpisode)
{
// Filter on chapter. // Filter on chapter.
var episodeFilter = config.episode; const episodeFilter = config.episode;
if (episodeFilter > 0 && item.episode <= episodeFilter) return false;
if (episodeFilter < 0 && item.episode >= -episodeFilter) return false;
// Filter on volume. // Filter on volume.
var volumeFilter = config.volume; const volumeFilter = config.volume;
if (volumeFilter > 0 && item.volume <= volumeFilter) return false;
if (volumeFilter < 0 && item.volume >= -volumeFilter) return false; const currentEpisode = parseInt(item.episode, 10);
const currentVolume = item.volume;
if ( ( (episodeFilter > 0) && (currentEpisode <= episodeFilter) ) ||
( (episodeFilter < 0) && (currentEpisode >= -episodeFilter) ) ||
( (volumeFilter > 0) && (currentVolume <= volumeFilter ) ) ||
( (volumeFilter < 0) && (currentVolume >= -volumeFilter ) ) )
{
return false;
}
return true; return true;
} }
/** /**
* Requests the page and scrapes the episodes and series. * Requests the page and scrapes the episodes and series.
*/ */
function page(config: IConfig, address: string, done: (err: Error, result?: ISeries) => void) { function page(config: IConfig, address: string, done: (err: Error, result?: ISeries) => void)
request.get(config, address, (err, result) => { {
if (err) return done(err); if (address[0] === '@')
var $ = cheerio.load(result); {
var title = $('span[itemprop=name]').text(); log.info('Trying to fetch from ' + address.substr(1));
if (!title) return done(new Error('Invalid page.')); const episodes: ISeriesEpisode[] = [];
var episodes: ISeriesEpisode[] = []; episodes.push({
$('.episode').each((i, el) => { address: address.substr(1),
if ($(el).children('img[src*=coming_soon]').length) return; episode: '',
var volume = /([0-9]+)\s*$/.exec($(el).closest('ul').prev('a').text()); volume: 0,
var regexp = /Episode\s+([0-9]+)\s*$/i;
var episode = regexp.exec($(el).children('.series-title').text());
var address = $(el).attr('href');
if (!address || !episode) return;
episodes.push({
address: address,
episode: parseInt(episode[0], 10),
volume: volume ? parseInt(volume[0], 10) : 1
});
}); });
done(null, {episodes: episodes.reverse(), series: title}); done(null, {episodes: episodes.reverse(), series: ""});
}); }
else
{
let episodeCount = 0;
request.get(config, address, (err, result) => {
if (err) {
return done(err);
}
const $ = cheerio.load(result);
const title = $('span[itemprop=name]').text();
if (!title) {
return done(new Error('Invalid page.(' + address + ')'));
}
log.info('Checking availability for ' + title);
const episodes: ISeriesEpisode[] = [];
$('.episode').each((i, el) => {
if ($(el).children('img[src*=coming_soon]').length) {
return;
}
const volume = /([0-9]+)\s*$/.exec($(el).closest('ul').prev('a').text());
const regexp = /Episode\s+((PV )?[S0-9][\-P0-9.]*[a-fA-F]?)\s*$/i;
const episode = regexp.exec($(el).children('.series-title').text());
const url = $(el).attr('href');
if ((!url) || (!episode)) {
return;
}
episodeCount += 1;
episodes.push({
address: url,
episode: episode[1],
volume: volume ? parseInt(volume[0], 10) : 1,
});
});
if (episodeCount === 0)
{
log.warn("No episodes found for " + title + ". Could it be a movie?");
}
done(null, {episodes: episodes.reverse(), series: title});
});
}
} }

View File

@@ -7,10 +7,14 @@ import zlib = require('zlib');
/** /**
* Decodes the data. * Decodes the data.
*/ */
export default function(id: number, iv: Buffer|string, data: Buffer|string, done: (err?: Error, result?: Buffer) => void) { export default function(id: number, iv: Buffer|string, data: Buffer|string,
try { done: (err?: Error, result?: Buffer) => void)
{
try
{
decompress(decrypt(id, iv, data), done); decompress(decrypt(id, iv, data), done);
} catch (e) { } catch (e)
{
done(e); done(e);
} }
} }
@@ -18,21 +22,27 @@ import zlib = require('zlib');
/** /**
* Decrypts the data. * Decrypts the data.
*/ */
function decrypt(id: number, iv: Buffer|string, data: Buffer|string) { function decrypt(id: number, iv: Buffer|string, data: Buffer|string)
var ivBuffer = typeof iv === 'string' ? new Buffer(iv, 'base64') : iv; {
var dataBuffer = typeof data === 'string' ? new Buffer(data, 'base64') : data; const ivBuffer = typeof iv === 'string' ? new Buffer(iv, 'base64') : iv;
var decipher = crypto.createDecipheriv('aes-256-cbc', key(id), ivBuffer); const dataBuffer = typeof data === 'string' ? new Buffer(data, 'base64') : data;
const decipher = crypto.createDecipheriv('aes-256-cbc', key(id), ivBuffer);
decipher.setAutoPadding(false); decipher.setAutoPadding(false);
return Buffer.concat([decipher.update(dataBuffer), decipher.final()]); return Buffer.concat([decipher.update(dataBuffer), decipher.final()]);
} }
/** /**
* Decompresses the data. * Decompresses the data.
*/ */
function decompress(data: Buffer, done: (err: Error, result?: Buffer) => void) { function decompress(data: Buffer, done: (err: Error, result?: Buffer) => void)
try { {
try
{
zlib.inflate(data, done); zlib.inflate(data, done);
} catch (e) { } catch (e)
{
done(null, data); done(null, data);
} }
} }
@@ -40,36 +50,45 @@ function decompress(data: Buffer, done: (err: Error, result?: Buffer) => void) {
/** /**
* Generates a key. * Generates a key.
*/ */
function key(subtitleId: number): Buffer { function key(subtitleId: number): Buffer
var hash = secret(20, 97, 1, 2) + magic(subtitleId); {
var result = new Buffer(32); const hash = secret(20, 97, 1, 2) + magic(subtitleId);
const result = new Buffer(32);
result.fill(0); result.fill(0);
crypto.createHash('sha1').update(hash).digest().copy(result); crypto.createHash('sha1').update(hash).digest().copy(result);
return result; return result;
} }
/** /**
* Generates a magic number. * Generates a magic number.
*/ */
function magic(subtitleId: number): number { function magic(subtitleId: number): number
var base = Math.floor(Math.sqrt(6.9) * Math.pow(2, 25)); {
var hash = bigInt(base).xor(subtitleId).toJSNumber(); const base = Math.floor(Math.sqrt(6.9) * Math.pow(2, 25));
var multipliedHash = bigInt(hash).multiply(32).toJSNumber(); const hash = bigInt(base).xor(subtitleId).toJSNumber();
const multipliedHash = bigInt(hash).multiply(32).toJSNumber();
return bigInt(hash).xor(hash >> 3).xor(multipliedHash).toJSNumber(); return bigInt(hash).xor(hash >> 3).xor(multipliedHash).toJSNumber();
} }
/** /**
* Generates a secret string based on a Fibonacci sequence. * Generates a secret string based on a Fibonacci sequence.
*/ */
function secret(size: number, modulo: number, firstSeed: number, secondSeed: number): string { function secret(size: number, modulo: number, firstSeed: number, secondSeed: number): string
var currentValue = firstSeed + secondSeed; {
var previousValue = secondSeed; let currentValue = firstSeed + secondSeed;
var result = ''; let previousValue = secondSeed;
for (var i = 0; i < size; i += 1) { let result = '';
var oldValue = currentValue;
for (let i = 0; i < size; i += 1)
{
const oldValue = currentValue;
result += String.fromCharCode(currentValue % modulo + 33); result += String.fromCharCode(currentValue % modulo + 33);
currentValue += previousValue; currentValue += previousValue;
previousValue = oldValue; previousValue = oldValue;
} }
return result; return result;
} }

View File

@@ -4,17 +4,25 @@ import xml2js = require('xml2js');
/** /**
* Converts an input buffer to a SubStation Alpha subtitle. * Converts an input buffer to a SubStation Alpha subtitle.
*/ */
export default function(input: string|Buffer, done: (err: Error, subtitle?: string) => void) { export default function(input: string|Buffer, done: (err: Error, subtitle?: string) => void)
{
xml2js.parseString(input.toString(), { xml2js.parseString(input.toString(), {
explicitArray: false, explicitArray: false,
explicitRoot: false explicitRoot: false
}, (err: Error, xml: ISubtitle) => { }, (err: Error, xml: ISubtitle) =>
if (err) return done(err); {
try { if (err)
{
return done(err);
}
try
{
done(null, script(xml) + '\n' + done(null, script(xml) + '\n' +
style(xml.styles) + '\n' + style(xml.styles) + '\n' +
event(xml.events)); event(xml.events));
} catch (err) { } catch (err)
{
done(err); done(err);
} }
}); });
@@ -23,69 +31,73 @@ export default function(input: string|Buffer, done: (err: Error, subtitle?: stri
/** /**
* Converts the event block. * Converts the event block.
*/ */
function event(block: ISubtitleEvent): string { function event(block: ISubtitleEvent): string
{
var format = 'Layer,Start,End,Style,Name,MarginL,MarginR,MarginV,Effect,Text'; var format = 'Layer,Start,End,Style,Name,MarginL,MarginR,MarginV,Effect,Text';
return '[Events]\n' + return '[Events]\n' +
'Format: ' + format + '\n' + 'Format: ' + format + '\n' + [].concat(block.event).map(style => ('Dialogue: 0,' +
[].concat(block.event).map(style => ('Dialogue: 0,' + style.$.start + ',' +
style.$.start + ',' + style.$.end + ',' +
style.$.end + ',' + style.$.style + ',' +
style.$.style + ',' + style.$.name + ',' +
style.$.name + ',' + style.$.margin_l + ',' +
style.$.margin_l + ',' + style.$.margin_r + ',' +
style.$.margin_r + ',' + style.$.margin_v + ',' +
style.$.margin_v + ',' + style.$.effect + ',' +
style.$.effect + ',' + style.$.text)).join('\n') + '\n';
style.$.text)).join('\n') + '\n';
} }
/** /**
* Converts the script block. * Converts the script block.
*/ */
function script(block: ISubtitle): string { function script(block: ISubtitle): string
{
return '[Script Info]\n' + return '[Script Info]\n' +
'Title: ' + block.$.title + '\n' + 'Title: ' + block.$.title + '\n' +
'ScriptType: v4.00+\n' + 'ScriptType: v4.00+\n' +
'WrapStyle: ' + block.$.wrap_style + '\n' + 'WrapStyle: ' + block.$.wrap_style + '\n' +
'PlayResX: ' + block.$.play_res_x + '\n' + 'PlayResX: ' + block.$.play_res_x + '\n' +
'PlayResY: ' + block.$.play_res_y + '\n' + 'PlayResY: ' + block.$.play_res_y + '\n' +
'Subtitle ID: ' + block.$.id + '\n' + 'Subtitle ID: ' + block.$.id + '\n' +
'Language: ' + block.$.lang_string + '\n' + 'Language: ' + block.$.lang_string + '\n' +
'Created: ' + block.$.created + '\n'; 'Created: ' + block.$.created + '\n';
} }
/** /**
* Converts the style block. * Converts the style block.
*/ */
function style(block: ISubtitleStyle): string { function style(block: ISubtitleStyle): string
{
var format = 'Name,Fontname,Fontsize,PrimaryColour,SecondaryColour,' + var format = 'Name,Fontname,Fontsize,PrimaryColour,SecondaryColour,' +
'OutlineColour,BackColour,Bold,Italic,Underline,StrikeOut,ScaleX,' + 'OutlineColour,BackColour,Bold,Italic,Underline,StrikeOut,ScaleX,' +
'ScaleY,Spacing,Angle,BorderStyle,Outline,Shadow,Alignment,' + 'ScaleY,Spacing,Angle,BorderStyle,Outline,Shadow,Alignment,' +
'MarginL,MarginR,MarginV,Encoding'; 'MarginL,MarginR,MarginV,Encoding';
return '[V4+ Styles]\n' + return '[V4+ Styles]\n' +
'Format: ' + format + '\n' + 'Format: ' + format + '\n' + [].concat(block.style).map(style => 'Style: ' +
[].concat(block.style).map(style => 'Style: ' + style.$.name + ',' +
style.$.name + ',' + style.$.font_name + ',' +
style.$.font_name + ',' + style.$.font_size + ',' +
style.$.font_size + ',' + style.$.primary_colour + ',' +
style.$.primary_colour + ',' + style.$.secondary_colour + ',' +
style.$.secondary_colour + ',' + style.$.outline_colour + ',' +
style.$.outline_colour + ',' + style.$.back_colour + ',' +
style.$.back_colour + ',' + style.$.bold + ',' +
style.$.bold + ',' + style.$.italic + ',' +
style.$.italic + ',' + style.$.underline + ',' +
style.$.underline + ',' + style.$.strikeout + ',' +
style.$.strikeout + ',' + style.$.scale_x + ',' +
style.$.scale_x + ',' + style.$.scale_y + ',' +
style.$.scale_y + ',' + style.$.spacing + ',' +
style.$.spacing + ',' + style.$.angle + ',' +
style.$.angle + ',' + style.$.border_style + ',' +
style.$.border_style + ',' + style.$.outline + ',' +
style.$.outline + ',' + style.$.shadow + ',' +
style.$.shadow + ',' + style.$.alignment + ',' +
style.$.alignment + ',' + style.$.margin_l + ',' +
style.$.margin_l + ',' + style.$.margin_r + ',' +
style.$.margin_r + ',' + style.$.margin_v + ',' +
style.$.margin_v + ',' + style.$.encoding).join('\n') + '\n';
style.$.encoding).join('\n') + '\n';
} }

View File

@@ -4,18 +4,30 @@ import xml2js = require('xml2js');
/** /**
* Converts an input buffer to a SubRip subtitle. * Converts an input buffer to a SubRip subtitle.
*/ */
export default function(input: Buffer|string, done: (err: Error, subtitle?: string) => void) { export default function(input: Buffer|string, done: (err: Error, subtitle?: string) => void)
var options = {explicitArray: false, explicitRoot: false}; {
xml2js.parseString(input.toString(), options, (err: Error, xml: ISubtitle) => { const options = {explicitArray: false, explicitRoot: false};
try {
if (err) return done(err); xml2js.parseString(input.toString(), options, (err: Error, xml: ISubtitle) =>
done(null, xml.events.event.map((event, index) => { {
var attributes = event.$; try
{
if (err)
{
return done(err);
}
done(null, xml.events.event.map((event, index) =>
{
const attributes = event.$;
return (index + 1) + '\n' + return (index + 1) + '\n' +
time(attributes.start) + ' --> ' + time(attributes.end) + '\n' + time(attributes.start) + ' --> ' + time(attributes.end) + '\n' +
text(attributes.text) + '\n'; text(attributes.text) + '\n';
}).join('\n')); }).join('\n'));
} catch (err) {
} catch (err)
{
done(err); done(err);
} }
}); });
@@ -24,41 +36,59 @@ import xml2js = require('xml2js');
/** /**
* Prefixes a value. * Prefixes a value.
*/ */
function prefix(value: string, length: number): string { function prefix(value: string, length: number): string
while (value.length < length) value = '0' + value; {
while (value.length < length)
{
value = '0' + value;
}
return value; return value;
} }
/** /**
* Suffixes a value. * Suffixes a value.
*/ */
function suffix(value: string, length: number): string { function suffix(value: string, length: number): string
while (value.length < length) value = value + '0'; {
while (value.length < length)
{
value = value + '0';
}
return value; return value;
} }
/** /**
* Formats a text value. * Formats a text value.
*/ */
function text(value: string): string { function text(value: string): string
{
return value return value
.replace(/{\\i1}/g, '<i>').replace(/{\\i0}/g, '</i>') .replace(/{\\i1}/g, '<i>').replace(/{\\i0}/g, '</i>')
.replace(/{\\b1}/g, '<b>').replace(/{\\b0}/g, '</b>') .replace(/{\\b1}/g, '<b>').replace(/{\\b0}/g, '</b>')
.replace(/{\\u1}/g, '<u>').replace(/{\\u0}/g, '</u>') .replace(/{\\u1}/g, '<u>').replace(/{\\u0}/g, '</u>')
.replace(/{[^}]+}/g, '') .replace(/{[^}]+}/g, '')
.replace(/(\s+)?\\n(\s+)?/ig, '\n') .replace(/(\s+)?\\n(\s+)?/ig, '\n')
.trim(); .trim();
} }
/** /**
* Formats a time stamp. * Formats a time stamp.
*/ */
function time(value: string): string { function time(value: string): string
var all = value.match(/^([0-9]+):([0-9]+):([0-9]+)\.([0-9]+)$/); {
if (!all) throw new Error('Invalid time.'); const all = value.match(/^([0-9]+):([0-9]+):([0-9]+)\.([0-9]+)$/);
var hours = prefix(all[1], 2);
var minutes = prefix(all[2], 2); if (!all)
var seconds = prefix(all[3], 2); {
var milliseconds = suffix(all[4], 3); throw new Error('Invalid time.');
}
const hours = prefix(all[1], 2);
const minutes = prefix(all[2], 2);
const seconds = prefix(all[3], 2);
const milliseconds = suffix(all[4], 3);
return hours + ':' + minutes + ':' + seconds + ',' + milliseconds; return hours + ':' + minutes + ':' + seconds + ',' + milliseconds;
} }

View File

@@ -8,36 +8,55 @@ import subtitle from '../subtitle/index';
/** /**
* Merges the subtitle and video files into a Matroska Multimedia Container. * Merges the subtitle and video files into a Matroska Multimedia Container.
*/ */
export default function(config: IConfig, isSubtitled: boolean, rtmpInputPath: string, filePath: string, streamMode: string, done: (err: Error) => void) { export default function(config: IConfig, isSubtitled: boolean, rtmpInputPath: string, filePath: string,
var subtitlePath = filePath + '.' + (subtitle.formats[config.format] ? config.format : 'ass'); streamMode: string, done: (err: Error) => void)
var videoPath = filePath; {
if (streamMode == "RTMP") const subtitlePath = filePath + '.' + (subtitle.formats[config.format] ? config.format : 'ass');
let videoPath = filePath;
if (streamMode === 'RTMP')
{ {
videoPath += path.extname(rtmpInputPath); videoPath += path.extname(rtmpInputPath);
} }
else else
{ {
videoPath += ".mp4"; videoPath += '.mp4';
} }
childProcess.exec(command() + ' ' + childProcess.exec(command() + ' ' +
'-o "' + filePath + '.mkv" ' + '-o "' + filePath + '.mkv" ' +
'"' + videoPath + '" ' + '"' + videoPath + '" ' +
(isSubtitled ? '"' + subtitlePath + '"' : ''), { (isSubtitled ? '"' + subtitlePath + '"' : ''), {
maxBuffer: Infinity maxBuffer: Infinity,
}, err => { }, (err) =>
if (err) return done(err); {
unlink(videoPath, subtitlePath, err => { if (err)
if (err) unlinkTimeout(videoPath, subtitlePath, 5000); {
done(null); return done(err);
}); }
unlink(videoPath, subtitlePath, (errin) =>
{
if (errin)
{
unlinkTimeout(videoPath, subtitlePath, 5000);
}
done(null);
}); });
});
} }
/** /**
* Determines the command for the operating system. * Determines the command for the operating system.
*/ */
function command(): string { function command(): string
if (os.platform() !== 'win32') return 'mkvmerge'; {
if (os.platform() !== 'win32')
{
return 'mkvmerge';
}
return '"' + path.join(__dirname, '../../bin/mkvmerge.exe') + '"'; return '"' + path.join(__dirname, '../../bin/mkvmerge.exe') + '"';
} }
@@ -45,9 +64,15 @@ function command(): string {
* Unlinks the video and subtitle. * Unlinks the video and subtitle.
* @private * @private
*/ */
function unlink(videoPath: string, subtitlePath: string, done: (err: Error) => void) { function unlink(videoPath: string, subtitlePath: string, done: (err: Error) => void)
fs.unlink(videoPath, err => { {
if (err) return done(err); fs.unlink(videoPath, (err) =>
{
if (err)
{
return done(err);
}
fs.unlink(subtitlePath, done); fs.unlink(subtitlePath, done);
}); });
} }
@@ -55,10 +80,16 @@ function unlink(videoPath: string, subtitlePath: string, done: (err: Error) => v
/** /**
* Attempts to unlink the video and subtitle with a timeout between each try. * Attempts to unlink the video and subtitle with a timeout between each try.
*/ */
function unlinkTimeout(videoPath: string, subtitlePath: string, timeout: number) { function unlinkTimeout(videoPath: string, subtitlePath: string, timeout: number)
setTimeout(() => { {
unlink(videoPath, subtitlePath, err => { setTimeout(() =>
if (err) unlinkTimeout(videoPath, subtitlePath, timeout); {
unlink(videoPath, subtitlePath, (err) =>
{
if (err)
{
unlinkTimeout(videoPath, subtitlePath, timeout);
}
}); });
}, timeout); }, timeout);
} }

View File

@@ -2,42 +2,49 @@
import childProcess = require('child_process'); import childProcess = require('child_process');
import path = require('path'); import path = require('path');
import os = require('os'); import os = require('os');
import log = require('../log');
/** /**
* Streams the video to disk. * Streams the video to disk.
*/ */
export default function(rtmpUrl: string, rtmpInputPath: string, swfUrl: string, filePath: string, fileExt: string, mode: string, done: (err: Error) => void) { export default function(rtmpUrl: string, rtmpInputPath: string, swfUrl: string, filePath: string,
if (mode == "RTMP") fileExt: string, mode: string, done: (err: Error) => void)
{
if (mode === 'RTMP')
{ {
childProcess.exec(command("rtmpdump") + ' ' + childProcess.exec(command('rtmpdump') + ' ' +
'-r "' + rtmpUrl + '" ' + '-r "' + rtmpUrl + '" ' +
'-y "' + rtmpInputPath + '" ' + '-y "' + rtmpInputPath + '" ' +
'-W "' + swfUrl + '" ' + '-W "' + swfUrl + '" ' +
'-o "' + filePath + fileExt + '"', { '-o "' + filePath + fileExt + '"', {
maxBuffer: Infinity maxBuffer: Infinity,
}, done); }, done);
} }
else if (mode == "HLS") else if (mode === 'HLS')
{ {
console.info("Experimental FFMPEG, MAY FAIL!!!"); const cmd = command('ffmpeg') + ' ' +
var cmd=command("ffmpeg") + ' ' + '-i "' + rtmpInputPath + '" ' +
'-i "' + rtmpInputPath + '" ' + '-c copy -bsf:a aac_adtstoasc ' +
'-c copy -bsf:a aac_adtstoasc ' +
'"' + filePath + '.mp4"'; '"' + filePath + '.mp4"';
childProcess.exec(cmd, { childProcess.exec(cmd, {
maxBuffer: Infinity maxBuffer: Infinity,
}, done); }, done);
} }
else else
{ {
console.error("No such mode: " + mode); log.error('No such mode: ' + mode);
} }
} }
/** /**
* Determines the command for the operating system. * Determines the command for the operating system.
*/ */
function command(exe: string): string { function command(exe: string): string
if (os.platform() !== 'win32') return exe; {
if (os.platform() !== 'win32')
{
return exe;
}
return '"' + path.join(__dirname, '../../bin/' + exe + '.exe') + '"'; return '"' + path.join(__dirname, '../../bin/' + exe + '.exe') + '"';
} }

View File

@@ -18,6 +18,7 @@
"src/cli.ts", "src/cli.ts",
"src/episode.ts", "src/episode.ts",
"src/index.ts", "src/index.ts",
"src/log.ts",
"src/interface/IConfig.d.ts", "src/interface/IConfig.d.ts",
"src/interface/IConfigLine.d.ts", "src/interface/IConfigLine.d.ts",
"src/interface/IConfigTask.d.ts", "src/interface/IConfigTask.d.ts",
@@ -40,13 +41,6 @@
"src/video/index.ts", "src/video/index.ts",
"src/video/merge.ts", "src/video/merge.ts",
"src/video/stream.ts", "src/video/stream.ts",
"typings/big-integer/big-integer.d.ts", "typings/index.d.ts"
"typings/cheerio/cheerio.d.ts",
"typings/commander/commander.d.ts",
"typings/form-data/form-data.d.ts",
"typings/mkdirp/mkdirp.d.ts",
"typings/node/node.d.ts",
"typings/request/request.d.ts",
"typings/xml2js/xml2js.d.ts"
] ]
} }

View File

@@ -1,33 +0,0 @@
{
"version": "v4",
"repo": "borisyankov/DefinitelyTyped",
"ref": "master",
"path": "typings",
"bundle": "typings/tsd.d.ts",
"installed": {
"node/node.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"commander/commander.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"xml2js/xml2js.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"cheerio/cheerio.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"mkdirp/mkdirp.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"request/request.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"big-integer/big-integer.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"form-data/form-data.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
}
}
}

View File

@@ -1,4 +1,5 @@
{ {
"extends": "tslint:latest",
"rules": { "rules": {
"ban": false, "ban": false,
"class-name": true, "class-name": true,
@@ -12,13 +13,13 @@
"interface-name": true, "interface-name": true,
"jsdoc-format": true, "jsdoc-format": true,
"label-position": true, "label-position": true,
"label-undefined": true,
"max-line-length": [true, 140], "max-line-length": [true, 140],
"member-ordering": [true, "member-ordering": [true,
"public-before-private", "public-before-private",
"static-before-instance", "static-before-instance",
"variables-before-functions" "variables-before-functions"
], ],
"array-type": [true, "array"],
"no-any": false, "no-any": false,
"no-arg": true, "no-arg": true,
"no-bitwise": true, "no-bitwise": true,
@@ -30,25 +31,18 @@
"trace" "trace"
], ],
"no-construct": true, "no-construct": true,
"no-constructor-vars": true,
"no-debugger": true, "no-debugger": true,
"no-duplicate-key": true,
"no-duplicate-variable": true, "no-duplicate-variable": true,
"no-empty": true, "no-empty": true,
"no-eval": true, "no-eval": true,
"no-string-literal": true, "no-string-literal": true,
"no-switch-case-fall-through": true, "no-switch-case-fall-through": true,
"no-trailing-comma": true,
"no-trailing-whitespace": true, "no-trailing-whitespace": true,
"no-unused-expression": true, "no-unused-expression": true,
"no-unused-variable": true,
"no-unreachable": true,
"no-use-before-declare": false, "no-use-before-declare": false,
"no-var-requires": true, "no-var-requires": true,
"one-line": [true, "one-line": [true,
"check-catch", "check-catch",
"check-else",
"check-open-brace",
"check-whitespace" "check-whitespace"
], ],
"quotemark": [true, "single"], "quotemark": [true, "single"],
@@ -66,7 +60,6 @@
"property-declaration": "nospace", "property-declaration": "nospace",
"variable-declaration": "nospace" "variable-declaration": "nospace"
}], }],
"use-strict": false,
"variable-name": false, "variable-name": false,
"whitespace": [true, "whitespace": [true,
"check-branch", "check-branch",

13
typings.json Normal file
View File

@@ -0,0 +1,13 @@
{
"name": "crunchy",
"globalDependencies": {
"node": "github:DefinitelyTyped/DefinitelyTyped/node/node.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"commander": "github:DefinitelyTyped/DefinitelyTyped/commander/commander.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"xml2js": "github:DefinitelyTyped/DefinitelyTyped/xml2js/xml2js.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"cheerio": "github:DefinitelyTyped/DefinitelyTyped/cheerio/cheerio.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"mkdirp": "github:DefinitelyTyped/DefinitelyTyped/mkdirp/mkdirp.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"request": "github:DefinitelyTyped/DefinitelyTyped/request/request.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"big-integer": "github:DefinitelyTyped/DefinitelyTyped/big-integer/big-integer.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"form-data": "github:DefinitelyTyped/DefinitelyTyped/form-data/form-data.d.ts#3882d337bb0808cde9fe4c08012508a48c135482"
}
}