34 Commits

Author SHA1 Message Date
Godzil
58f4dc61ff Fix login issue 2017-02-07 20:22:25 +00:00
Godzil
b96efacbd2 - Revert login using the token method
- Use the cloudscraper layer on top of request to pass through the cloudfare browser check
- switch from tsd to typings
2017-02-07 20:22:01 +00:00
Godzil
a346ab8854 1.1.14 2017-02-01 08:53:23 +00:00
Godzil
499530141e Disable debug message about ffmpeg 2017-02-01 08:53:14 +00:00
Godzil
d1457bb893 1.1.13 2017-01-28 13:38:20 +00:00
Godzil
8dfd1b447c Add a log objet to do some fancy output on the command line (not fully enabled under windows as it need some tests) 2017-01-28 13:38:14 +00:00
Godzil
ce63ae9a16 Upgrade to 1.1.12 to fix login issue 2017-01-23 21:13:13 +00:00
Godzil
70d80ccd17 Update dependency to more recent version, and correct a few warnings reported by ts 2017-01-23 21:06:34 +00:00
Manoël Trapier
7833fbe292 Merge pull request #13 from majewskim/master
Fix a login issue
2017-01-23 20:46:10 +00:00
Mateusz Majewski
fa6aa74442 Merge branch 'master' into master 2017-01-18 12:49:49 +02:00
Mateusz Majewski
fe2ed9fb76 Fixing login issue by bypassing the login form and making a request directly. 2017-01-18 11:19:41 +02:00
Mateusz Majewski
cc655b9e00 Fixing login issue by bypassing the login form and making a request directly. 2017-01-18 11:08:45 +02:00
Manoël Trapier
e1d2a55a01 Update README.md 2016-10-21 17:21:36 +01:00
Manoël Trapier
a31de0ef9d Remove .js from the name 2016-10-21 17:21:05 +01:00
Godzil
2853334d7f 1.1.11
- Update login mechanism to march CR september 2016 changes
2016-09-16 22:20:43 +01:00
Godzil
69dd28d31b Update login to match latest CR changes 2016-09-16 22:20:39 +01:00
Godzil
56afce02ea 1.1.10
- Change name format to follow Plex forvourite one.
- Remplace ":" in file name to prevent issue on Windows
2016-09-10 20:22:10 +01:00
Godzil
bc4697061e Remplace ':' in filename to make Windows happy 2016-09-10 20:17:19 +01:00
Godzil
55ffe85f77 Make the name to be more Plex friendly 2016-09-10 11:53:45 +01:00
Godzil
ec8c2c7716 1.1.9
New features:
 - Should correctly handle 10.5 or 1A episode numbers
 - Change characters in series name that are not allowed in filename (or could cause issues, like slash ( / ) or single quote ( ' )
 - Prevent overwrite of existing files (usefull when different season report the same number)
 - If an error during episode metadata scraping, it will set to Season 0, episode 0 (look at the logs!)
 - If an episode is not available yet, it will be skipped
2016-09-07 22:09:14 +01:00
Godzil
714a528f8b Prevent overwriting an existing output file 2016-09-07 21:55:14 +01:00
Godzil
8314d91bd7 Add functionality to ignore (instead of stopping) if an episode is not available yet 2016-09-07 21:51:36 +01:00
Godzil
5bd31f9e0b Add better episode numbering scheme 2016-09-07 21:34:29 +01:00
Godzil
95a93930f3 Merge branch 'master' of github.com:Godzil/crunchyroll.js 2016-08-22 13:10:02 +02:00
Godzil
4a9e1d0410 Update LICENSE 2016-08-22 12:03:05 +01:00
Godzil
1eacd0a5ca add license to npm 2016-08-22 13:03:02 +02:00
Godzil
3c32726745 1.1.8 2016-08-22 12:32:25 +02:00
Godzil
42ae0ae1fb Forget to rename main executable 2016-08-22 12:30:43 +02:00
Godzil
e4b3871919 1.1.7 2016-08-22 12:22:45 +02:00
Godzil
58e4a557e2 Update readme 2016-08-22 12:22:36 +02:00
Godzil
8371d68113 Correct errors 2016-08-22 12:21:17 +02:00
Godzil
b7d496fc9d Update README.md 2016-08-22 10:51:23 +01:00
Godzil
14260d04b3 Update README.md 2016-08-22 10:50:13 +01:00
Godzil
3d46b65d67 Update package.json 2016-08-22 10:47:18 +01:00
17 changed files with 314 additions and 123 deletions

View File

@@ -1,4 +1,5 @@
Copyright (c) 2015 Roel van Uden Copyright (c) 2015 Roel van Uden
Copyright (c) 2016 Manoel <Godzil> Trapier
Permission is hereby granted, free of charge, to any person obtaining a copy Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to of this software and associated documentation files (the "Software"), to

1
README Symbolic link
View File

@@ -0,0 +1 @@
README.md

View File

@@ -1,6 +1,6 @@
# CrunchyRoll.js # Crunchy: a fork of Deathspike/CrunchyRoll.js
*CrunchyRoll.js* is capable of downloading *anime* episodes from the popular *CrunchyRoll* streaming service. An episode is stored in the original video format (often H.264 in a MP4 container) and the configured subtitle format (ASS or SRT).The two output files are then merged into a single MKV file. *Crunchy* is capable of downloading *anime* episodes from the popular *CrunchyRoll* streaming service. An episode is stored in the original video format (often H.264 in a MP4 container) and the configured subtitle format (ASS or SRT).The two output files are then merged into a single MKV file.
## Motivation ## Motivation
@@ -10,6 +10,8 @@
This application is not endorsed or affliated with *CrunchyRoll*. The usage of this application enables episodes to be downloaded for offline convenience which may be forbidden by law in your country. Usage of this application may also cause a violation of the agreed *Terms of Service* between you and the stream provider. A tool is not responsible for your actions; please make an informed decision prior to using this application. This application is not endorsed or affliated with *CrunchyRoll*. The usage of this application enables episodes to be downloaded for offline convenience which may be forbidden by law in your country. Usage of this application may also cause a violation of the agreed *Terms of Service* between you and the stream provider. A tool is not responsible for your actions; please make an informed decision prior to using this application.
**PLEASE _ONLY_ USE THIS TOOL IF YOU HAVE A _PREMIUM ACCOUNT_**
## Configuration ## Configuration
It is recommended to enable authentication (`-p` and `-u`) so your account permissions and settings are available for use. It is not possible to download non-free material without an account and premium subscription. Furthermore, the default account settings are used when downloading. If you want the highest quality videos, configure these preferences at https://www.crunchyroll.com/acct/?action=video. It is recommended to enable authentication (`-p` and `-u`) so your account permissions and settings are available for use. It is not possible to download non-free material without an account and premium subscription. Furthermore, the default account settings are used when downloading. If you want the highest quality videos, configure these preferences at https://www.crunchyroll.com/acct/?action=video.
@@ -28,28 +30,28 @@ Use the applicable instructions to install. Is your operating system not listed?
1. Run in *Terminal*: `sudo apt-get install nodejs npm mkvtoolnix rtmpdump ffmpeg` 1. Run in *Terminal*: `sudo apt-get install nodejs npm mkvtoolnix rtmpdump ffmpeg`
2. Run in *Terminal*: `sudo ln -s /usr/bin/nodejs /usr/bin/node` 2. Run in *Terminal*: `sudo ln -s /usr/bin/nodejs /usr/bin/node`
3. Run in *Terminal*: `sudo npm install -g crunchyroll` 3. Run in *Terminal*: `sudo npm install -g crunchy`
### Mac OS X ### Mac OS X
1. Install *Homebrew* following the instructions at http://brew.sh/ 1. Install *Homebrew* following the instructions at http://brew.sh/
2. Run in *Terminal*: `brew install node mkvtoolnix rtmpdump ffmpeg` 2. Run in *Terminal*: `brew install node mkvtoolnix rtmpdump ffmpeg`
3. Run in *Terminal*: `npm install -g crunchyroll` 3. Run in *Terminal*: `npm install -g crunchy`
### Windows ### Windows
1. Install *NodeJS* following the instructions at http://nodejs.org/ 1. Install *NodeJS* following the instructions at http://nodejs.org/
3. Run in *Command Prompt*: `npm install -g crunchyroll` 3. Run in *Command Prompt*: `npm install -g crunchy`
## Instructions ## Instructions
Use the applicable instructions for the interface of your choice (currently limited to command-line). Use the applicable instructions for the interface of your choice (currently limited to command-line).
### Command-line Interface (`crunchyroll`) ### Command-line Interface (`crunchy`)
The [command-line interface](http://en.wikipedia.org/wiki/Command-line_interface) does not have a graphical component and is ideal for automation purposes and headless machines. The interface can run using a sequence of series addresses (the site address containing the episode listing), or with a batch-mode source file. The `crunchyroll --help` command will produce the following output: The [command-line interface](http://en.wikipedia.org/wiki/Command-line_interface) does not have a graphical component and is ideal for automation purposes and headless machines. The interface can run using a sequence of series addresses (the site address containing the episode listing), or with a batch-mode source file. The `crunchy --help` command will produce the following output:
Usage: crunchyroll [options] Usage: crunchy [options]
Options: Options:
@@ -74,15 +76,15 @@ When no sequence of series addresses is provided, the batch-mode source file wil
Download in batch-mode: Download in batch-mode:
crunchyroll crunchy
Download *Fairy Tail* to the current work directory: Download *Fairy Tail* to the current work directory:
crunchyroll http://www.crunchyroll.com/fairy-tail crunchy http://www.crunchyroll.com/fairy-tail
Download *Fairy Tail* to `C:\Anime`: Download *Fairy Tail* to `C:\Anime`:
crunchyroll --output C:\Anime http://www.crunchyroll.com/fairy-tail crunchy --output C:\Anime http://www.crunchyroll.com/fairy-tail
#### Switches #### Switches

View File

@@ -1,36 +1,38 @@
{ {
"author": "Roel van Uden", "author": "Godzil",
"description": "CrunchyRoll.js is capable of downloading anime episodes from the popular CrunchyRoll streaming service.", "description": "Crunchy is a fork of Crunchyroll.js, capable of downloading anime episodes from the popular CrunchyRoll streaming service.",
"license": "MIT",
"keywords": [ "keywords": [
"anime", "anime",
"download", "download",
"crunchyroll" "crunchyroll"
], ],
"name": "crunchyroll", "name": "crunchy",
"repository": { "repository": {
"type": "git", "type": "git",
"url": "git://github.com/Deathspike/crunchyroll.js.git" "url": "git://github.com/Godzil/crunchyroll.js.git"
}, },
"version": "1.1.5", "version": "1.1.15",
"bin": { "bin": {
"crunchyroll": "./bin/crunchyroll" "crunchy": "./bin/crunchy"
}, },
"dependencies": { "dependencies": {
"big-integer": "1.4.4", "big-integer": "1.4.4",
"cheerio": "0.18.0", "cheerio": "0.22.0",
"cloudscraper": "1.4.1",
"commander": "2.6.0", "commander": "2.6.0",
"mkdirp": "0.5.0", "mkdirp": "0.5.0",
"request": "2.53.0", "request": "2.74.0",
"xml2js": "0.4.5" "xml2js": "0.4.5"
}, },
"devDependencies": { "devDependencies": {
"tsd": "0.5.7", "typing": "2.1.0",
"tslint": "2.3.0-beta", "tslint": "2.3.0-beta",
"typescript": "1.5.0-beta" "typescript": "1.5.0-beta"
}, },
"scripts": { "scripts": {
"prepublish": "npm run tsd && tsc", "prepublish": "npm run types && tsc",
"test": "node ts --only-test", "test": "node ts --only-test",
"tsd": "tsd reinstall -o -s" "types": "typings install"
} }
} }

View File

@@ -7,15 +7,16 @@ import path = require('path');
import subtitle from './subtitle/index'; import subtitle from './subtitle/index';
import video from './video/index'; import video from './video/index';
import xml2js = require('xml2js'); import xml2js = require('xml2js');
import log = require('./log');
/** /**
* Streams the episode to disk. * Streams the episode to disk.
*/ */
export default function(config: IConfig, address: string, done: (err: Error) => void) { export default function(config: IConfig, address: string, done: (err: Error, ign: boolean) => void) {
scrapePage(config, address, (err, page) => { scrapePage(config, address, (err, page) => {
if (err) return done(err); if (err) return done(err, false);
scrapePlayer(config, address, page.id, (err, player) => { scrapePlayer(config, address, page.id, (err, player) => {
if (err) return done(err); if (err) return done(err, false);
download(config, page, player, done); download(config, page, player, done);
}); });
}); });
@@ -24,37 +25,72 @@ export default function(config: IConfig, address: string, done: (err: Error) =>
/** /**
* Completes a download and writes the message with an elapsed time. * Completes a download and writes the message with an elapsed time.
*/ */
function complete(message: string, begin: number, done: (err: Error) => void) { function complete(epName: string, message: string, begin: number, done: (err: Error, ign: boolean) => void) {
var timeInMs = Date.now() - begin; var timeInMs = Date.now() - begin;
var seconds = prefix(Math.floor(timeInMs / 1000) % 60, 2); var seconds = prefix(Math.floor(timeInMs / 1000) % 60, 2);
var minutes = prefix(Math.floor(timeInMs / 1000 / 60) % 60, 2); var minutes = prefix(Math.floor(timeInMs / 1000 / 60) % 60, 2);
var hours = prefix(Math.floor(timeInMs / 1000 / 60 / 60), 2); var hours = prefix(Math.floor(timeInMs / 1000 / 60 / 60), 2);
console.log(message + ' (' + hours + ':' + minutes + ':' + seconds + ')'); log.dispEpisode(epName, message + ' (' + hours + ':' + minutes + ':' + seconds + ')', true);
done(null); done(null, false);
}
/**
* Check if a file exist..
*/
function fileExist(path: string) {
try
{
fs.statSync(path);
return true;
}
catch (e) { }
return false;
} }
/** /**
* Downloads the subtitle and video. * Downloads the subtitle and video.
*/ */
function download(config: IConfig, page: IEpisodePage, player: IEpisodePlayer, done: (err: Error) => void) { function download(config: IConfig, page: IEpisodePage, player: IEpisodePlayer, done: (err: Error, ign: boolean) => void) {
var series = config.series || page.series; var series = config.series || page.series;
var fileName = name(config, page, series); series = series.replace("/","_").replace("'","_").replace(":","_");
var fileName = name(config, page, series, "").replace("/","_").replace("'","_").replace(":","_");
var filePath = path.join(config.output || process.cwd(), series, fileName); var filePath = path.join(config.output || process.cwd(), series, fileName);
if (fileExist(filePath + ".mkv"))
{
var count = 0;
log.warn("File '"+fileName+"' already exist...");
do
{
count = count + 1;
fileName = name(config, page, series, "-" + count).replace("/","_").replace("'","_").replace(":","_");
filePath = path.join(config.output || process.cwd(), series, fileName);
} while(fileExist(filePath + ".mkv"))
log.warn("Renaming to '"+fileName+"'...");
}
mkdirp(path.dirname(filePath), (err: Error) => { mkdirp(path.dirname(filePath), (err: Error) => {
if (err) return done(err); if (err) return done(err, false);
downloadSubtitle(config, player, filePath, err => { downloadSubtitle(config, player, filePath, err => {
if (err) return done(err); if (err) return done(err, false);
var now = Date.now(); var now = Date.now();
console.log('Fetching ' + fileName); if (player.video.file != undefined)
downloadVideo(config, page, player, filePath, err => { {
if (err) return done(err); log.dispEpisode(fileName, 'Fetching...', false);
if (config.merge) return complete('Finished ' + fileName, now, done); downloadVideo(config, page, player, filePath, err => {
var isSubtited = Boolean(player.subtitle); if (err) return done(err, false);
video.merge(config, isSubtited, player.video.file, filePath, player.mode, err => { if (config.merge) return complete(fileName, 'Finished!', now, done);
if (err) return done(err); var isSubtited = Boolean(player.subtitle);
complete('Finished ' + fileName, now, done); video.merge(config, isSubtited, player.video.file, filePath, player.video.mode, err => {
if (err) return done(err, false);
complete(fileName, 'Finished!', now, done);
});
}); });
}); }
else
{
log.dispEpisode(fileName, 'Ignoring: not released yet', true);
done(null, true);
}
}); });
}); });
} }
@@ -96,11 +132,13 @@ function downloadVideo(config: IConfig,
/** /**
* Names the file based on the config, page, series and tag. * Names the file based on the config, page, series and tag.
*/ */
function name(config: IConfig, page: IEpisodePage, series: string) { function name(config: IConfig, page: IEpisodePage, series: string, extra: string) {
var episode = (page.episode < 10 ? '0' : '') + page.episode; var episodeNum = parseInt(page.episode, 10);
var volume = (page.volume < 10 ? '0' : '') + page.volume; var volumeNum = parseInt(page.volume, 10);
var episode = (episodeNum < 10 ? '0' : '') + page.episode;
var volume = (volumeNum < 10 ? '0' : '') + page.volume;
var tag = config.tag || 'CrunchyRoll'; var tag = config.tag || 'CrunchyRoll';
return series + ' ' + volume + 'x' + episode + ' [' + tag + ']'; return series + ' - s' + volume + 'e' + episode +' - [' + tag + ']' + extra;
} }
/** /**
@@ -122,15 +160,29 @@ function scrapePage(config: IConfig, address: string, done: (err: Error, page?:
if (err) return done(err); if (err) return done(err);
var $ = cheerio.load(result); var $ = cheerio.load(result);
var swf = /^([^?]+)/.exec($('link[rel=video_src]').attr('href')); var swf = /^([^?]+)/.exec($('link[rel=video_src]').attr('href'));
var regexp = /-\s+(?:Watch\s+)?(.+?)(?:\s+Season\s+([0-9]+))?(?:\s+-)?\s+Episode\s+([0-9]+)/; var regexp = /\s*([^\n\r\t\f]+)\n?\s*[^0-9]*([0-9][0-9.]*)?,?\n?\s\s*[^0-9]*((PV )?[S0-9][P0-9.]*[a-fA-F]?)/;
var data = regexp.exec($('title').text()); var look = $('#showmedia_about_media').text();
if (!swf || !data) return done(new Error('Invalid page.')); var seasonTitle = $('span[itemprop="title"]').text();
var data = regexp.exec(look);
if (!swf || !data)
{
log.warn('Something wrong in the page at '+address+' (data are: '+look+')');
log.warn('Setting Season to 0 and episode to \0\...');
done(null, {
id: id,
episode: "0",
series: seasonTitle,
swf: swf[1],
volume: "0"
});
}
done(null, { done(null, {
id: id, id: id,
episode: parseInt(data[3], 10), episode: data[3],
series: data[1], series: data[1],
swf: swf[1], swf: swf[1],
volume: parseInt(data[2], 10) || 1 volume: data[2] || "1"
}); });
}); });
} }
@@ -165,7 +217,7 @@ function scrapePlayer(config: IConfig, address: string, id: number, done: (err:
data: player['default:preload'].subtitle.data data: player['default:preload'].subtitle.data
} : null, } : null,
video: { video: {
mode: streamMode; mode: streamMode,
file: player['default:preload'].stream_info.file, file: player['default:preload'].stream_info.file,
host: player['default:preload'].stream_info.host host: player['default:preload'].stream_info.host
} }

View File

@@ -1,7 +1,7 @@
interface IEpisodePage { interface IEpisodePage {
id: number; id: number;
episode: number; episode: string;
series: string; series: string;
volume: number; volume: string;
swf: string; swf: string;
} }

View File

@@ -5,6 +5,7 @@ interface IEpisodePlayer {
data: string; data: string;
}; };
video: { video: {
mode: string;
file: string; file: string;
host: string; host: string;
}; };

View File

@@ -1,5 +1,5 @@
interface ISeriesEpisode { interface ISeriesEpisode {
address: string; address: string;
episode: number; episode: string;
volume: number; volume: number;
} }

71
src/log.ts Normal file
View File

@@ -0,0 +1,71 @@
'use strict';
import os = require('os');
export function error(str: string)
{
if (os.platform() === 'win32')
{
console.log(' * ERROR: ' + str);
}
else
{
/* Do more fancy output */
console.error(' \x1B[1;31m* ERROR\x1B[0m: ' + str);
}
}
export function info(str: string)
{
if (os.platform() === 'win32')
{
console.log(' * INFO : ' + str);
}
else
{
/* Do more fancy output */
console.log(' \x1B[1;32m* INFO \x1B[0m: ' + str);
}
}
export function debug(str: string)
{
if (os.platform() === 'win32')
{
console.log(' * DEBUG: ' + str);
}
else
{
/* Do more fancy output */
console.log(' \x1B[1;35m* DEBUG\x1B[0m: ' + str);
}
}
export function warn(str: string)
{
if (os.platform() === 'win32')
{
console.log(' * WARN : ' + str);
}
else
{
/* Do more fancy output */
console.log(' \x1B[1;33m* WARN \x1B[0m: ' + str);
}
}
export function dispEpisode(name: string, status: string, addNL: boolean)
{
if (os.platform() === 'win32')
{
process.stdout.write(' > ' + name + ' : ' + status + '\x1B[0G');
}
else
{
/* Do more fancy output */
process.stdout.write(' \x1B[1;33m> \x1B[37m' + name + '\x1B[0m : \x1B[33m' + status + '\x1B[0m\x1B[0G');
}
if (addNL)
{
console.log('');
}
}

View File

@@ -1,6 +1,15 @@
'use strict'; 'use strict';
import request = require('request'); import request = require('request');
import cheerio = require('cheerio');
import log = require('./log');
var cloudscraper = require('cloudscraper');
var isAuthenticated = false; var isAuthenticated = false;
var isPremium = false;
var defaultHeaders: request.Headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.2; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0',
'Connection': 'keep-alive'
};
/** /**
* Performs a GET request for the resource. * Performs a GET request for the resource.
@@ -8,7 +17,7 @@ var isAuthenticated = false;
export function get(config: IConfig, options: request.Options, done: (err: Error, result?: string) => void) { export function get(config: IConfig, options: request.Options, done: (err: Error, result?: string) => void) {
authenticate(config, err => { authenticate(config, err => {
if (err) return done(err); if (err) return done(err);
request.get(modify(options), (err: Error, response: any, body: any) => { cloudscraper.request(modify(options, 'GET'), (err: Error, response: any, body: any) => {
if (err) return done(err); if (err) return done(err);
done(null, typeof body === 'string' ? body : String(body)); done(null, typeof body === 'string' ? body : String(body));
}); });
@@ -21,7 +30,7 @@ export function get(config: IConfig, options: request.Options, done: (err: Error
export function post(config: IConfig, options: request.Options, done: (err: Error, result?: string) => void) { export function post(config: IConfig, options: request.Options, done: (err: Error, result?: string) => void) {
authenticate(config, err => { authenticate(config, err => {
if (err) return done(err); if (err) return done(err);
request.post(modify(options), (err: Error, response: any, body: any) => { cloudscraper.request(modify(options, 'POST'), (err: Error, response: any, body: any) => {
if (err) return done(err); if (err) return done(err);
done(null, typeof body === 'string' ? body : String(body)); done(null, typeof body === 'string' ? body : String(body));
}); });
@@ -33,30 +42,92 @@ export function post(config: IConfig, options: request.Options, done: (err: Erro
*/ */
function authenticate(config: IConfig, done: (err: Error) => void) { function authenticate(config: IConfig, done: (err: Error) => void) {
if (isAuthenticated || !config.pass || !config.user) return done(null); if (isAuthenticated || !config.pass || !config.user) return done(null);
var options = {
form: { /* Bypass the login page and send a login request directly */
formname: 'RpcApiUser_Login', var options =
fail_url: 'https://www.crunchyroll.com/login', {
name: config.user, headers: defaultHeaders,
password: config.pass
},
jar: true, jar: true,
url: 'https://www.crunchyroll.com/?a=formhandler' gzip: false,
method: 'GET',
url: 'https://www.crunchyroll.com/login'
}; };
request.post(options, (err: Error) => {
// request(options, (err: Error, rep: string, body: string) =>
cloudscraper.request(options, (err: Error, rep: string, body: string) =>
{
if (err) return done(err); if (err) return done(err);
isAuthenticated = true;
done(null); var $ = cheerio.load(body);
/* Get the token from the login page */
var token = $('input[name="login_form[_token]"]').attr('value');
if (token === '') return done(new Error('Can`t find token!'));
var options =
{
headers: defaultHeaders,
form:
{
'login_form[redirect_url]': '/',
'login_form[name]': config.user,
'login_form[password]': config.pass,
'login_form[_token]': token
},
jar: true,
gzip: false,
method: 'POST',
url: 'https://www.crunchyroll.com/login'
};
// request.post(options, (err: Error, rep: string, body: string) =>
cloudscraper.request(options, (err: Error, rep: string, body: string) =>
{
if (err) return done(err);
/* The page return with a meta based redirection, as we wan't to check that everything is fine, reload
* the main page. A bit convoluted, but more sure.
*/
var options =
{
headers: defaultHeaders,
jar: true,
url: 'http://www.crunchyroll.com/',
method: 'GET'
};
cloudscraper.request(options, (err: Error, rep: string, body: string) =>
{
if (err) return done(err);
var $ = cheerio.load(body);
/* Check if auth worked */
var regexps = /ga\(\'set\', \'dimension[5-8]\', \'([^']*)\'\);/g;
var dims = regexps.exec($('script').text());
for (var i = 1; i < 5; i++)
{
if ((dims[i] !== undefined) && (dims[i] !== '') && (dims[i] !== 'not-registered')) { isAuthenticated = true; }
if ((dims[i] === 'premium') || (dims[i] === 'premiumplus')) { isPremium = true; }
}
if (isAuthenticated === false)
{
var error = $('ul.message, li.error').text();
return done(new Error('Authentication failed: ' + error));
}
if (isPremium === false) { log.warn('Do not use this app without a premium account.'); }
else { log.info('You have a premium account! Good!'); }
done(null);
});
});
}); });
} }
/** /**
* Modifies the options to use the authenticated cookie jar. * Modifies the options to use the authenticated cookie jar.
*/ */
function modify(options: string|request.Options): request.Options { function modify(options: string|request.Options, reqMethod: string): request.Options {
if (typeof options !== 'string') { if (typeof options !== 'string') {
options.jar = true; options.jar = true;
options.headers = defaultHeaders;
options.method = reqMethod;
return options; return options;
} }
return {jar: true, url: options.toString()}; return { jar: true, headers: defaultHeaders, url: options.toString(), method: reqMethod};
} }

View File

@@ -5,6 +5,7 @@ import fs = require('fs');
import request = require('./request'); import request = require('./request');
import path = require('path'); import path = require('path');
import url = require('url'); import url = require('url');
import log = require('./log');
var persistent = '.crpersistent'; var persistent = '.crpersistent';
/** /**
@@ -19,14 +20,22 @@ export default function(config: IConfig, address: string, done: (err: Error) =>
var i = 0; var i = 0;
(function next() { (function next() {
if (i >= page.episodes.length) return done(null); if (i >= page.episodes.length) return done(null);
download(cache, config, address, page.episodes[i], err => { download(cache, config, address, page.episodes[i], (err, ignored) => {
if (err) return done(err); if (err) return done(err);
var newCache = JSON.stringify(cache, null, ' '); if ((ignored == false) || (ignored == undefined))
fs.writeFile(persistentPath, newCache, err => { {
if (err) return done(err); var newCache = JSON.stringify(cache, null, ' ');
fs.writeFile(persistentPath, newCache, err => {
if (err) return done(err);
i += 1;
next();
});
}
else
{
i += 1; i += 1;
next(); next();
}); }
}); });
})(); })();
}); });
@@ -40,14 +49,14 @@ function download(cache: {[address: string]: number},
config: IConfig, config: IConfig,
baseAddress: string, baseAddress: string,
item: ISeriesEpisode, item: ISeriesEpisode,
done: (err: Error) => void) { done: (err: Error, ign: boolean) => void) {
if (!filter(config, item)) return done(null); if (!filter(config, item)) return done(null, false);
var address = url.resolve(baseAddress, item.address); var address = url.resolve(baseAddress, item.address);
if (cache[address]) return done(null); if (cache[address]) return done(null, false);
episode(config, address, err => { episode(config, address, (err, ignored) => {
if (err) return done(err); if (err) return done(err, false);
cache[address] = Date.now(); cache[address] = Date.now();
done(null); done(null, ignored);
}); });
} }
@@ -57,8 +66,8 @@ function download(cache: {[address: string]: number},
function filter(config: IConfig, item: ISeriesEpisode) { function filter(config: IConfig, item: ISeriesEpisode) {
// Filter on chapter. // Filter on chapter.
var episodeFilter = config.episode; var episodeFilter = config.episode;
if (episodeFilter > 0 && item.episode <= episodeFilter) return false; if (episodeFilter > 0 && parseInt(item.episode, 10) <= episodeFilter) return false;
if (episodeFilter < 0 && item.episode >= -episodeFilter) return false; if (episodeFilter < 0 && parseInt(item.episode, 10) >= -episodeFilter) return false;
// Filter on volume. // Filter on volume.
var volumeFilter = config.volume; var volumeFilter = config.volume;
@@ -75,18 +84,19 @@ function page(config: IConfig, address: string, done: (err: Error, result?: ISer
if (err) return done(err); if (err) return done(err);
var $ = cheerio.load(result); var $ = cheerio.load(result);
var title = $('span[itemprop=name]').text(); var title = $('span[itemprop=name]').text();
if (!title) return done(new Error('Invalid page.')); if (!title) return done(new Error('Invalid page.(' + address + ')'));
log.info("Checking availability for " + title);
var episodes: ISeriesEpisode[] = []; var episodes: ISeriesEpisode[] = [];
$('.episode').each((i, el) => { $('.episode').each((i, el) => {
if ($(el).children('img[src*=coming_soon]').length) return; if ($(el).children('img[src*=coming_soon]').length) return;
var volume = /([0-9]+)\s*$/.exec($(el).closest('ul').prev('a').text()); var volume = /([0-9]+)\s*$/.exec($(el).closest('ul').prev('a').text());
var regexp = /Episode\s+([0-9]+)\s*$/i; var regexp = /Episode\s+((PV )?[S0-9][P0-9.]*[a-fA-F]?)\s*$/i;
var episode = regexp.exec($(el).children('.series-title').text()); var episode = regexp.exec($(el).children('.series-title').text());
var address = $(el).attr('href'); var address = $(el).attr('href');
if (!address || !episode) return; if (!address || !episode) return;
episodes.push({ episodes.push({
address: address, address: address,
episode: parseInt(episode[0], 10), episode: episode[1],
volume: volume ? parseInt(volume[0], 10) : 1 volume: volume ? parseInt(volume[0], 10) : 1
}); });
}); });

View File

@@ -2,6 +2,7 @@
import childProcess = require('child_process'); import childProcess = require('child_process');
import path = require('path'); import path = require('path');
import os = require('os'); import os = require('os');
import log = require('../log');
/** /**
* Streams the video to disk. * Streams the video to disk.
@@ -19,7 +20,7 @@ import os = require('os');
} }
else if (mode == "HLS") else if (mode == "HLS")
{ {
console.info("Experimental FFMPEG, MAY FAIL!!!"); //log.debug("Experimental FFMPEG, MAY FAIL!!!");
var cmd=command("ffmpeg") + ' ' + var cmd=command("ffmpeg") + ' ' +
'-i "' + rtmpInputPath + '" ' + '-i "' + rtmpInputPath + '" ' +
'-c copy -bsf:a aac_adtstoasc ' + '-c copy -bsf:a aac_adtstoasc ' +
@@ -30,7 +31,7 @@ import os = require('os');
} }
else else
{ {
console.error("No such mode: " + mode); log.error("No such mode: " + mode);
} }
} }

View File

@@ -18,6 +18,7 @@
"src/cli.ts", "src/cli.ts",
"src/episode.ts", "src/episode.ts",
"src/index.ts", "src/index.ts",
"src/log.ts",
"src/interface/IConfig.d.ts", "src/interface/IConfig.d.ts",
"src/interface/IConfigLine.d.ts", "src/interface/IConfigLine.d.ts",
"src/interface/IConfigTask.d.ts", "src/interface/IConfigTask.d.ts",

View File

@@ -1,33 +0,0 @@
{
"version": "v4",
"repo": "borisyankov/DefinitelyTyped",
"ref": "master",
"path": "typings",
"bundle": "typings/tsd.d.ts",
"installed": {
"node/node.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"commander/commander.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"xml2js/xml2js.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"cheerio/cheerio.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"mkdirp/mkdirp.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"request/request.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"big-integer/big-integer.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
},
"form-data/form-data.d.ts": {
"commit": "3882d337bb0808cde9fe4c08012508a48c135482"
}
}
}

View File

@@ -47,8 +47,6 @@
"no-var-requires": true, "no-var-requires": true,
"one-line": [true, "one-line": [true,
"check-catch", "check-catch",
"check-else",
"check-open-brace",
"check-whitespace" "check-whitespace"
], ],
"quotemark": [true, "single"], "quotemark": [true, "single"],

13
typings.json Normal file
View File

@@ -0,0 +1,13 @@
{
"name": "crunchy",
"globalDependencies": {
"node": "github:DefinitelyTyped/DefinitelyTyped/node/node.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"commander": "github:DefinitelyTyped/DefinitelyTyped/commander/commander.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"xml2js": "github:DefinitelyTyped/DefinitelyTyped/xml2js/xml2js.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"cheerio": "github:DefinitelyTyped/DefinitelyTyped/cheerio/cheerio.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"mkdirp": "github:DefinitelyTyped/DefinitelyTyped/mkdirp/mkdirp.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"request": "github:DefinitelyTyped/DefinitelyTyped/request/request.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"big-integer": "github:DefinitelyTyped/DefinitelyTyped/big-integer/big-integer.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
"form-data": "github:DefinitelyTyped/DefinitelyTyped/form-data/form-data.d.ts#3882d337bb0808cde9fe4c08012508a48c135482"
}
}