1.2.2

Fix #58

.travis.yml

@@ -0,0 +1,10 @@
+language: node_js
+sudo: false
+node_js:
+ - 8
+ - 9
+before_install:
+ - npm install --only=dev
+script:
+ - npm run build
+ - npm test

README.md

@@ -1,5 +1,7 @@
 # Crunchy: a fork of Deathspike/CrunchyRoll.js
 
+[![Issue Stats](http://issuestats.com/github/Godzil/Crunchy/badge/issue)](http://issuestats.com/github/Godzil/Crunchy) [![Travis CI](https://travis-ci.org/Godzil/Crunchy.svg?branch=master)](https://travis-ci.org/Godzil/Crunchy) [![Maintainability](https://api.codeclimate.com/v1/badges/413c7ca11c0805b1ef3e/maintainability)](https://codeclimate.com/github/Godzil/Crunchy/maintainability) 
+
 *Crunchy* is capable of downloading *anime* episodes from the popular *CrunchyRoll* streaming service. An episode is stored in the original video format (often H.264 in a MP4 container) and the configured subtitle format (ASS or SRT).The two output files are then merged into a single MKV file.
 
 ## Motivation
@@ -19,7 +21,7 @@ It is recommended to enable authentication (`-p` and `-u`) so your account permi
 
 ## Prerequisites
 
-* NodeJS >= 0.12.x (http://nodejs.org/)
+* NodeJS >= 5.x (http://nodejs.org/)
 * NPM >= 2.5.x (https://www.npmjs.org/)
 
 ## Installation
@@ -93,6 +95,8 @@ Download *Fairy Tail* to `C:\Anime`:
 * `-p or --pass <s>` sets the password.
 * `-u or --user <s>` sets the e-mail address or username.
 
+ _Please remember that login has to be done for each call of Crunchy, as none of the credentials are stored_
+
 ##### Disables
 
 * `-c or --cache` disables the cache.
@@ -110,6 +114,36 @@ Download *Fairy Tail* to `C:\Anime`:
 * `-s or --series <s>` sets the series override.
 * `-t or --tag <s>` sets The subgroup. (Default: CrunchyRoll)
 
+## When things goes wrong
+
+First, make sure you have the latest version of Crunchy installed, if you run an older version, the issue you face may have been solved.
+
+Second thing to check, you have to give your credentials (-u and -p parameters) each time you run Crunchy. It does not actually store the token it receive when login and need to relog each time it is called. This may change in the future.
+
+Third, is it a recently released episode? If yes, sometimes CR have issues were the requested format is not available, and Crunchy is not able to get it. When in doubt, try to watch CR website, if it does not work there, Crunchy will not either. This is valid in all cases even on non recently released.
+
+Fourth, sometimes, CR website does weird things, and there are some transient errors, wait a couple of minutes (or hours) and try again. It often solved the issue on my side (yes I know that's not really a way of fixing, but if the error is on CR side, Crunchy can't do anything)
+
+If really nothing works or you find a problem with Crunchy, then you can go and fill an Issue, first read the already open and closed one to make sure you are not reporting an existing problem. If your problem has been already reported, what you can do is to either:
+- Add a comment saying you also have the same issue
+- Add a Thumbs Up reaction to the original entry in the issue, they will are used as a metric to know how many people are annoyed by that issue
+If you find one which correspond and is close, don't hesitate to add a comment, the issue may have not be fully solved.
+
+If there is no comparable opened or close issue, you can create a new one.
+
+### What to put in a bug report
+It is really important for me to know:
+- on which Operating System you are running Crunchy,
+- which anime you want to fetch if it is related to a specific one, 
+- The command line you use to run Crunchy
+- What message Crunchy is giving you if any
+
+**Please be careful to remove your real account login and password if they appear!**
+
+Also don't hesitate to add labels you feel apropriate on your report.
+
+_Note: You can also use a bug report for a feature requests._
+
 ## Developers
 
 More information will be added at a later point. For now the recommendations are:
@@ -117,3 +151,8 @@ More information will be added at a later point. For now the recommendations are
 * Atom with `atom-typescript` and `linter-tslint` (and dependencies).
 
 Since this project uses TypeScript, compile with `node ts` or `npm install`.
+
+#### A note about pull requests:
+If you want to help working on this project, Pull request are welcome, but please explain the goal of your changes, and do a pull request per change: you want to add support for _X_, _Y_ and _Z_, make a pull request for X, one for Y and one for Z. I'm not saying a pull request per commit that would be idiotic.
+The idea is if your pull request changes lots of thing at the same time, if just a single part can't be accepted, if will delay the acceptation of the whole pull request where some of the feature could be integrated quicker is they were requested alone.
+Also if for example the change _Y_ depends on _X_, you can wait for _X_ to be accepted before requesting for _Y_, if they are independant you can send a pull request for each at the same time.

bin/crunchy.sh

@@ -0,0 +1,10 @@
+#!/bin/bash
+PARAMS=$*
+for i in {1..20}; do
+	crunchy ${PARAMS}
+	if [ $? == 0 ]; then
+	 	break
+	fi
+	echo "Going to retry..."	
+	sleep 3
+done

package.json

@@ -12,32 +12,37 @@
     "type": "git",
     "url": "git://github.com/Godzil/crunchyroll.js.git"
   },
-  "version": "1.1.17",
+  "engines": {
+    "node": ">=5.0"
+  },
+  "version": "1.2.2",
   "bin": {
-    "crunchy": "./bin/crunchy"
+    "crunchy": "./bin/crunchy",
+    "crunchy.sh": "./bin/crunchy.sh"
   },
   "dependencies": {
-    "big-integer": "^1.4.4",
+    "big-integer": "^1.6.27",
     "cheerio": "^0.22.0",
-    "cloudscraper": "^1.4.1",
-    "commander": "^2.6.0",
-    "fs-extra": "^2.0.0",
+    "cloudscraper": "^1.5.0",
+    "commander": "^2.15.1",
+    "fs-extra": "^5.0.0",
     "mkdirp": "^0.5.0",
-    "request": "^2.74.0",
+    "request": "^2.85.0",
     "xml2js": "^0.4.5"
   },
   "devDependencies": {
+    "@types/cheerio": "^0.22.7",
+    "@types/mkdirp": "^0.5.2",
+    "@types/request": "^2.47.0",
+    "@types/xml2js": "^0.4.2",
     "tsconfig-lint": "^0.12.0",
-    "tslint": "^4.4.2",
-    "typescript": "^2.2.0",
-    "typings": "^2.1.0"
+    "tslint": "^5.9.1",
+    "typescript": "^2.8.1"
   },
   "scripts": {
-    "prepublish": "npm run types && tsc",
-    "compile": "tsc",
+    "prepublishOnly": "npm run build",
+    "build": "tsc",
     "test": "tslint -c ./tslint.json --project ./tsconfig.json ./src/**/*.ts",
-    "types": "typings install",
-    "reinstall": "tsd reinstall; npm run types",
     "start": "node ./bin/crunchy"
   },
   "bugs": {

src/batch.ts

@@ -3,6 +3,15 @@ import commander = require('commander');
 import fs = require('fs');
 import path = require('path');
 import series from './series';
+import log = require('./log');
+
+/* correspondances between resolution and value CR excpect */
+let resol_table: { [id: string]: IResolData; } = {
+    '360':  {quality:'60', format:'106'},
+    '480':  {quality:'61', format:'106'},
+    '720':  {quality:'62', format:'106'},
+    '1080': {quality:'80', format:'108'},
+};
 
 /**
  * Streams the batch of series to disk.
@@ -12,6 +21,28 @@ export default function(args: string[], done: (err?: Error) => void)
   const config = parse(args);
   const batchPath = path.join(config.output || process.cwd(), 'CrunchyRoll.txt');
 
+  // set resolution
+  if (config.resolution)
+  {
+    try
+    {
+      config.video_format = resol_table[config.resolution]['format'];
+      config.video_quality = resol_table[config.resolution]['quality'];
+    }
+    catch(e)
+    {
+      log.warn("Invalid resolution " + config.resolution + "p. Setting to 1080p")
+      config.video_format = resol_table['1080']['format'];
+      config.video_quality = resol_table['1080']['quality'];
+    }
+  }
+  else
+  {
+    /* 1080 by default */
+    config.video_format = resol_table['1080']['format'];
+    config.video_quality = resol_table['1080']['quality'];
+  }
+
   tasks(config, batchPath, (err, tasks) =>
   {
     if (err)
@@ -149,6 +180,8 @@ function parse(args: string[]): IConfigLine
     .option('-f, --format <s>', 'The subtitle format. (Default: ass)')
     .option('-o, --output <s>', 'The output path.')
     .option('-s, --series <s>', 'The series override.')
+    .option('-n, --filename <s>', 'The name override.')
     .option('-t, --tag <s>', 'The subgroup. (Default: CrunchyRoll)')
+    .option('-r, --resolution <s>', 'The video resolution. (Default: 1080 (360, 480, 720, 1080))')
     .parse(args);
 }

src/cli.ts

@@ -6,5 +6,8 @@ batch(process.argv, (err: any) =>
   if (err)
   {
     console.error(err.stack || err);
+    process.exit(-1)
   }
+  console.info("Done!")
+  process.exit(0)
 });

src/episode.ts

@@ -2,7 +2,7 @@
 import cheerio = require('cheerio');
 import fs = require('fs');
 import mkdirp = require('mkdirp');
-import request = require('./request');
+import my_request = require('./my_request');
 import path = require('path');
 import subtitle from './subtitle/index';
 import video from './video/index';
@@ -63,6 +63,11 @@ function fileExist(path: string)
   }
 }
 
+function sanitiseFileName(str: string)
+{
+  return str.replace(/[\/':\?\*"<>\.]/g, '_');
+}
+
 /**
  * Downloads the subtitle and video.
  */
@@ -70,8 +75,8 @@ function download(config: IConfig, page: IEpisodePage, player: IEpisodePlayer, d
 {
   let series = config.series || page.series;
 
-  series = series.replace('/', '_').replace('\'', '_').replace(':', '_');
-  let fileName = name(config, page, series, '').replace('/', '_').replace('\'', '_').replace(':', '_');
+  series = sanitiseFileName(series);
+  let fileName = sanitiseFileName(name(config, page, series, ''));
   let filePath = path.join(config.output || process.cwd(), series, fileName);
 
   if (fileExist(filePath + '.mkv'))
@@ -82,7 +87,7 @@ function download(config: IConfig, page: IEpisodePage, player: IEpisodePlayer, d
     do
     {
       count = count + 1;
-      fileName = name(config, page, series, '-' + count).replace('/', '_').replace('\'', '_').replace(':', '_');
+      fileName = sanitiseFileName(name(config, page, series, '-' + count));
       filePath = path.join(config.output || process.cwd(), series, fileName);
     } while (fileExist(filePath + '.mkv'));
 
@@ -96,6 +101,7 @@ function download(config: IConfig, page: IEpisodePage, player: IEpisodePlayer, d
       return done(errM, false);
     }
 
+    log.dispEpisode(fileName, 'Fetching...', false);
     downloadSubtitle(config, player, filePath, (errDS) =>
     {
       if (errDS)
@@ -106,7 +112,7 @@ function download(config: IConfig, page: IEpisodePage, player: IEpisodePlayer, d
       const now = Date.now();
       if (player.video.file !== undefined)
       {
-        log.dispEpisode(fileName, 'Fetching...', false);
+        log.dispEpisode(fileName, 'Fetching video...', false);
         downloadVideo(config, page, player, filePath, (errDV) =>
         {
           if (errDV)
@@ -121,6 +127,7 @@ function download(config: IConfig, page: IEpisodePage, player: IEpisodePlayer, d
 
           const isSubtited = Boolean(player.subtitle);
 
+          log.dispEpisode(fileName, 'Merging...', false);
           video.merge(config, isSubtited, player.video.file, filePath, player.video.mode, (errVM) =>
           {
             if (errVM)
@@ -196,7 +203,20 @@ function name(config: IConfig, page: IEpisodePage, series: string, extra: string
   const volume = (volumeNum < 10 ? '0' : '') + page.volume;
   const tag = config.tag || 'CrunchyRoll';
 
-  return series + ' - s' + volume + 'e' + episode + ' - [' + tag + ']' + extra;
+  if (!config.filename) {
+    return page.series + ' - s' + volume + 'e' + episode + ' - [' + tag + ']' + extra;
+  }
+
+  return config.filename
+      .replace(/{EPISODE_ID}/g, page.id.toString())
+      .replace(/{EPISODE_NUMBER}/g, episode)
+      .replace(/{SEASON_NUMBER}/g, volume)
+      .replace(/{VOLUME_NUMBER}/g, volume)
+      .replace(/{SEASON_TITLE}/g, page.season)
+      .replace(/{VOLUME_TITLE}/g, page.season)
+      .replace(/{SERIES_TITLE}/g, series)
+      .replace(/{EPISODE_TITLE}/g, page.title)
+      .replace(/{TAG}/g, tag) + extra;
 }
 
 /**
@@ -226,7 +246,7 @@ function scrapePage(config: IConfig, address: string, done: (err: Error, page?:
     return done(new Error('Invalid address.'));
   }
 
-  request.get(config, address, (err, result) =>
+  my_request.get(config, address, (err, result) =>
   {
     if (err)
     {
@@ -238,6 +258,7 @@ function scrapePage(config: IConfig, address: string, done: (err: Error, page?:
     const regexp = /\s*([^\n\r\t\f]+)\n?\s*[^0-9]*([0-9][\-0-9.]*)?,?\n?\s\s*[^0-9]*((PV )?[S0-9][P0-9.]*[a-fA-F]?)/;
     const look = $('#showmedia_about_media').text();
     const seasonTitle = $('span[itemprop="title"]').text();
+    let episodeTitle = $('#showmedia_about_name').text().replace(/[“”]/g, '');
     const data = regexp.exec(look);
 
     if (!swf || !data)
@@ -248,6 +269,8 @@ function scrapePage(config: IConfig, address: string, done: (err: Error, page?:
         episode: '0',
         id: epId,
         series: seasonTitle,
+        season: seasonTitle,
+        title: episodeTitle,
         swf: swf[1],
         volume: '0',
       });
@@ -258,6 +281,8 @@ function scrapePage(config: IConfig, address: string, done: (err: Error, page?:
         episode: data[3],
         id: epId,
         series: data[1],
+        season: seasonTitle,
+        title: episodeTitle,
         swf: swf[1],
         volume: data[2] || '1',
       });
@@ -277,8 +302,13 @@ function scrapePlayer(config: IConfig, address: string, id: number, done: (err:
     return done(new Error('Invalid address.'));
   }
 
-  request.post(config, {
-    form: {current_page: address},
+  my_request.post(config, {
+    form: {
+      current_page: address,
+      video_format: config.video_format,
+      video_quality: config.video_quality,
+      media_id: id
+    },
     url: url[1] + '/xml/?req=RpcApiVideoPlayer_GetStandardConfig&media_id=' + id,
   }, (err, result) =>
   {
@@ -303,9 +333,9 @@ function scrapePlayer(config: IConfig, address: string, id: number, done: (err:
         let streamMode = 'RTMP';
 
         if (player['default:preload'].stream_info.host === '')
-		{
-			streamMode = 'HLS';
-		}
+        {
+          streamMode = 'HLS';
+        }
 
         done(null, {
           subtitle: isSubtitled ? {

src/interface/IConfig.d.ts

@@ -12,5 +12,9 @@ interface IConfig {
   format?: string;
   output?: string;
   series?: string;
+  filename?: string;
   tag?: string;
+  resolution?: string;
+  video_format?: string;
+  video_quality?: string;
 }

src/interface/IEpisodePage.d.ts

@@ -3,5 +3,7 @@ interface IEpisodePage {
   episode: string;
   series: string;
   volume: string;
+  season: string;
+  title: string;
   swf: string;
 }

src/interface/IResolData.d.ts

@@ -0,0 +1,4 @@
+interface IResolData {
+  quality: string;
+  format: string;
+}

src/log.ts

@@ -28,10 +28,10 @@ export function warn(str: string)
 export function dispEpisode(name: string, status: string, addNL: boolean)
 {
   /* Do fancy output */
-  process.stdout.write(' \x1B[1;33m> \x1B[37m' + name + '\x1B[0m : \x1B[33m' + status + '\x1B[0m\x1B[0G');
+  process.stdout.write('\x1B[K \x1B[1;33m> \x1B[37m' + name + '\x1B[0m : \x1B[33m' + status + '\x1B[0m\x1B[0G');
 
   if (addNL)
   {
     console.log('');
   }
-}
+}

src/my_request.ts

@@ -9,14 +9,16 @@ let isPremium = false;
 
 const defaultHeaders: request.Headers =
 {
-  'User-Agent': 'Mozilla/5.0 (Windows NT 6.2; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0',
-  'Connection': 'keep-alive'
+  'User-Agent': 'Mozilla/5.0 (Windows NT 6.2; WOW64; x64; rv:58.0) Gecko/20100101 Firefox/58.0',
+  //             Mozilla/5.0 (Windows NT 6.2; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0',
+  'Connection': 'keep-alive',
+  'Referer': 'https://www.crunchyroll.com/login'
 };
 
 /**
  * Performs a GET request for the resource.
  */
-export function get(config: IConfig, options: request.Options, done: (err: Error, result?: string) => void)
+export function get(config: IConfig, options: string|request.Options, done: (err: Error, result?: string) => void)
 {
   authenticate(config, err =>
   {
@@ -99,9 +101,9 @@ function authenticate(config: IConfig, done: (err: Error) => void)
       headers: defaultHeaders,
       form:
       {
-        'login_form[redirect_url]': '/',
         'login_form[name]': config.user,
         'login_form[password]': config.pass,
+        'login_form[redirect_url]': '/',
         'login_form[_token]': token
       },
       jar: true,

src/series.ts

@@ -3,12 +3,27 @@ import cheerio = require('cheerio');
 import episode from './episode';
 import fs = require('fs');
 const fse = require('fs-extra');
-import request = require('./request');
+import my_request = require('./my_request');
 import path = require('path');
 import url = require('url');
 import log  = require('./log');
 const persistent = '.crpersistent';
 
+/**
+ * Check if a file exist..
+ */
+function fileExist(path: string)
+{
+  try
+  {
+    fs.statSync(path);
+    return true;
+  } catch (e)
+  {
+    return false;
+  }
+}
+
 /**
  * Streams the series to disk.
  */
@@ -17,7 +32,10 @@ export default function(config: IConfig, address: string, done: (err: Error) =>
   const persistentPath = path.join(config.output || process.cwd(), persistent);
 
   /* Make a backup of the persistent file in case of */
-  fse.copySync(persistentPath, persistentPath + '.backup');
+  if (fileExist(persistentPath))
+  {
+    fse.copySync(persistentPath, persistentPath + '.backup');
+  }
 
   fs.readFile(persistentPath, 'utf8', (err: Error, contents: string) =>
   {
@@ -140,7 +158,7 @@ function page(config: IConfig, address: string, done: (err: Error, result?: ISer
   else
   {
     let episodeCount = 0;
-    request.get(config, address, (err, result) => {
+    my_request.get(config, address, (err, result) => {
       if (err) {
         return done(err);
       }

src/subtitle/decode.ts

@@ -1,14 +1,14 @@
 /* tslint:disable:no-bitwise false */
 'use strict';
-import crypto = require('crypto');
 import bigInt = require('big-integer');
+import crypto = require('crypto');
 import zlib = require('zlib');
 
 /**
  * Decodes the data.
  */
- export default function(id: number, iv: Buffer|string, data: Buffer|string,
-                         done: (err?: Error, result?: Buffer) => void)
+export default function(id: number, iv: Buffer|string, data: Buffer|string,
+                        done: (err?: Error, result?: Buffer) => void)
 {
   try
   {

src/video/merge.ts

@@ -1,15 +1,16 @@
 'use strict';
 import childProcess = require('child_process');
 import fs = require('fs');
-import path = require('path');
 import os = require('os');
+import path = require('path');
+
 import subtitle from '../subtitle/index';
 
 /**
  * Merges the subtitle and video files into a Matroska Multimedia Container.
  */
- export default function(config: IConfig, isSubtitled: boolean, rtmpInputPath: string, filePath: string,
-                         streamMode: string, done: (err: Error) => void)
+export default function(config: IConfig, isSubtitled: boolean, rtmpInputPath: string, filePath: string,
+                        streamMode: string, done: (err: Error) => void)
 {
   const subtitlePath = filePath + '.' + (subtitle.formats[config.format] ? config.format : 'ass');
   let videoPath = filePath;

src/video/stream.ts

@@ -1,7 +1,8 @@
 'use strict';
 import childProcess = require('child_process');
-import path = require('path');
 import os = require('os');
+import path = require('path');
+
 import log  = require('../log');
 
 /**

tsconfig.json

@@ -1,46 +1,11 @@
 {
-    "version": "1.5.1-beta",
+    "target": "es6",
     "compilerOptions": {
         "declaration": true,
         "noImplicitAny": true,
         "removeComments": false,
         "module": "commonjs",
         "outDir": "dist",
-        "sourceMap": true,
-        "target": "es5"
-    },
-    "filesGlob": [
-        "src/**/*.ts",
-        "typings/**/*.ts"
-    ],
-    "files": [
-        "src/batch.ts",
-        "src/cli.ts",
-        "src/episode.ts",
-        "src/index.ts",
-        "src/log.ts",
-        "src/interface/IConfig.d.ts",
-        "src/interface/IConfigLine.d.ts",
-        "src/interface/IConfigTask.d.ts",
-        "src/interface/IEpisodePage.d.ts",
-        "src/interface/IEpisodePlayer.d.ts",
-        "src/interface/IEpisodePlayerConfig.d.ts",
-        "src/interface/IFormatterTable.d.ts",
-        "src/interface/ISeries.d.ts",
-        "src/interface/ISeriesEpisode.d.ts",
-        "src/interface/ISubtitle.d.ts",
-        "src/interface/ISubtitleEvent.d.ts",
-        "src/interface/ISubtitleStyle.d.ts",
-        "src/request.ts",
-        "src/series.ts",
-        "src/subtitle/decode.ts",
-        "src/subtitle/formats/ass.ts",
-        "src/subtitle/formats/index.ts",
-        "src/subtitle/formats/srt.ts",
-        "src/subtitle/index.ts",
-        "src/video/index.ts",
-        "src/video/merge.ts",
-        "src/video/stream.ts",
-        "typings/index.d.ts"
-    ]
+        "sourceMap": true
+    }
 }

typings.json

@@ -1,13 +0,0 @@
-{
-  "name": "crunchy",
-  "globalDependencies": {
-    "node": "github:DefinitelyTyped/DefinitelyTyped/node/node.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
-    "commander": "github:DefinitelyTyped/DefinitelyTyped/commander/commander.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
-    "xml2js": "github:DefinitelyTyped/DefinitelyTyped/xml2js/xml2js.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
-    "cheerio": "github:DefinitelyTyped/DefinitelyTyped/cheerio/cheerio.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
-    "mkdirp": "github:DefinitelyTyped/DefinitelyTyped/mkdirp/mkdirp.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
-    "request": "github:DefinitelyTyped/DefinitelyTyped/request/request.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
-    "big-integer": "github:DefinitelyTyped/DefinitelyTyped/big-integer/big-integer.d.ts#3882d337bb0808cde9fe4c08012508a48c135482",
-    "form-data": "github:DefinitelyTyped/DefinitelyTyped/form-data/form-data.d.ts#3882d337bb0808cde9fe4c08012508a48c135482"
-  }
-}