9.11.17

The argument sets the "cd" parameter on the request.
For both included providers, this flag disables validation ([1], [2]),
so doing the exact opposite of the described action.

This changes the label to the correct name and also flips the default
value to keep the old behavior.

[1] Google
<https://developers.google.com/speed/public-dns/docs/doh/json#supported_parameters>
[2] Cloudflare
<https://developers.cloudflare.com/1.1.1.1/dns-over-https/json-format/>

.eslintignore

@@ -1,2 +1 @@
-src/core/vendor/**
-src/web/static/clippy_assets/**
+src/core/vendor/**

.eslintrc.json

@@ -38,12 +38,16 @@
 
         // disable rules from base configurations
         "no-control-regex": "off",
+        "require-atomic-updates": "off",
+        "no-async-promise-executor": "off",
 
         // stylistic conventions
         "brace-style": ["error", "1tbs"],
+        "space-before-blocks": ["error", "always"],
         "block-spacing": "error",
         "array-bracket-spacing": "error",
         "comma-spacing": "error",
+        "spaced-comment": ["error", "always", { "exceptions": ["/"] } ],
         "comma-style": "error",
         "computed-property-spacing": "error",
         "no-trailing-spaces": "warn",
@@ -106,9 +110,6 @@
 
         "COMPILE_TIME": false,
         "COMPILE_MSG": false,
-        "PKG_VERSION": false,
-        "ENVIRONMENT_IS_WORKER": false,
-        "ENVIRONMENT_IS_NODE": false,
-        "ENVIRONMENT_IS_WEB": false
+        "PKG_VERSION": false
     }
 }

.github/CONTRIBUTING.md

@@ -6,7 +6,7 @@ There are lots of opportunities to contribute to CyberChef. If you want ideas, t
 
 Before your contributions can be accepted, you must:
 
- - Sign the [GCHQ Contributor Licence Agreement](https://github.com/gchq/Gaffer/wiki/GCHQ-OSS-Contributor-License-Agreement-V1.0)
+ - Sign the [GCHQ Contributor Licence Agreement](https://cla-assistant.io/gchq/CyberChef)
  - Push your changes to your fork.
  - Submit a pull request.
 
@@ -22,15 +22,15 @@ Before your contributions can be accepted, you must:
 * Line endings: UNIX style (\n)
 
 
-## Design Principals
+## Design Principles
 
 1. If at all possible, all operations and features should be client-side and not rely on connections to an external server. This increases the utility of CyberChef on closed networks and in virtual machines that are not connected to the Internet. Calls to external APIs may be accepted if there is no other option, but not for critical components.
-2. Latency should be kept to a minimum to enhance the user experience. This means that all operation code should sit on the client, rather than being loaded dynamically from a server.
-3. Use Vanilla JS if at all possible to reduce the number of libraries required and relied upon. Frameworks like jQuery, although included, should not be used unless absolutely necessary.
-4. Minimise the use of large libraries, especially for niche operations that won't be used very often - these will be downloaded by everyone using the app, whether they use that operation or not (due to principal 2).
+2. Latency should be kept to a minimum to enhance the user experience. This means that operation code should sit on the client and be executed there. However, as a trade-off between latency and bandwidth, operation code with large dependencies can be loaded in discrete modules in order to reduce the size of the initial download. The downloading of additional modules must remain entirely transparent so that the user is not inconvenienced.
+3. Large libraries should be kept in separate modules so that they are not downloaded by everyone who uses the app, just those who specifically require the relevant operations.
+4. Use Vanilla JS if at all possible to reduce the number of libraries required and relied upon. Frameworks like jQuery, although included, should not be used unless absolutely necessary.
 
 
-With these principals in mind, any changes or additions to CyberChef should keep it:
+With these principles in mind, any changes or additions to CyberChef should keep it:
 
  - Standalone
  - Efficient

.github/ISSUE_TEMPLATE/bug-report.md

@@ -7,19 +7,18 @@ assignees: ''
 
 ---
 
-<!-- Prefix the title above with 'Bug report:' -->
 **Describe the bug**
 A clear and concise description of what the bug is.
 
 **To Reproduce**
-Steps to reproduce the behavior or a link to the recipe / input used to cause the bug:
+Steps to reproduce the behaviour or a link to the recipe / input used to cause the bug:
 
 1. Go to '...'
 2. Click on '....'
 3. Scroll down to '....'
 4. See error
 
-**Expected behavior**
+**Expected behaviour**
 A clear and concise description of what you expected to happen.
 
 **Screenshots**
@@ -27,9 +26,8 @@ If applicable, add screenshots to help explain your problem.
 
 **Desktop (if relevant, please complete the following information):**
  - OS: [e.g. Windows] 
- - Browser [e.g. chrome, safari]
- - Version [e.g. 22]
+ - Browser: [e.g. chrome 72, firefox 60]
+ - CyberChef version: [e.g. 9.7.14]
 
 **Additional context**
 Add any other context about the problem here.
-

.github/ISSUE_TEMPLATE/feature-request.md

@@ -6,9 +6,9 @@ labels: feature
 assignees: ''
 
 ---
-<!-- Prefix the title above with 'Feature request:' -->
+
 **Is your feature request related to a problem? Please describe.**
-A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+A clear and concise description of what the problem is. E.g. I'm always frustrated when [...]
 
 **Describe the solution you'd like**
 A clear and concise description of what you want to happen.

.github/ISSUE_TEMPLATE/operation-request.md

@@ -7,8 +7,6 @@ assignees: ''
 
 ---
 
-<!-- Prefix the title above with 'Operation request:' -->
-
 ## Summary
 
 ### Example Input

.gitignore

@@ -2,14 +2,13 @@ node_modules
 npm-debug.log
 travis.log
 build
-docs/*
-!docs/*.conf.json
-!docs/*.ico
 .vscode
 .*.swp
-.DS_Store
 src/core/config/modules/*
 src/core/config/OperationConfig.json
 src/core/operations/index.mjs
+src/node/config/OperationConfig.json
+src/node/index.mjs
+**/*.DS_Store
 tests/browser/output/*
 

.npmignore

@@ -3,6 +3,5 @@ npm-debug.log
 travis.log
 build/*
 !build/node
-docs
 .vscode
 .github

.travis.yml

@@ -1,6 +1,7 @@
 language: node_js
 node_js:
-  - lts/*
+  - lts/dubnium
+cache: npm
 addons:
   chrome: stable
 install: npm install
@@ -10,8 +11,7 @@ before_script:
 script:
   - grunt lint
   - grunt test
-  - grunt docs
-  - grunt node
+  - grunt testnodeconsumer
   - grunt prod --msg="$COMPILE_MSG"
   - xvfb-run --server-args="-screen 0 1200x800x24" grunt testui
 before_deploy:
@@ -33,7 +33,7 @@ deploy:
     file_glob: true
     file:
       - build/prod/*.zip
-      - build/node/CyberChef.js
+      - src/node/cjs.js
     on:
       repo: gchq/CyberChef
       tags: true

CHANGELOG.md

@@ -2,6 +2,52 @@
 All major and minor version changes will be documented in this file. Details of patch-level version changes can be found in [commit messages](https://github.com/gchq/CyberChef/commits/master).
 
 
+### [9.11.0] - 2019-11-06
+- Implemented CFB, OFB, and CTR modes for Blowfish operations [@cbeuw] | [#653]
+
+### [9.10.0] - 2019-11-06
+- 'Lorenz' operation added [@VirtualColossus] | [#528]
+
+### [9.9.0] - 2019-11-01
+- Added support for 109 more character encodings [@n1474335]
+
+### [9.8.0] - 2019-10-31
+- 'Avro to JSON' operation added [@jarrodconnolly] | [#865]
+
+### [9.7.0] - 2019-09-13
+- 'Optical Character Recognition' operation added [@MShwed] [@n1474335] | [#632]
+
+### [9.6.0] - 2019-09-04
+- 'Bacon Cipher Encode' and 'Bacon Cipher Decode' operations added [@kassi] | [#500]
+
+### [9.5.0] - 2019-09-04
+- Various Steganography operations added: 'Extract LSB', 'Extract RGBA', 'Randomize Colour Palette', and 'View Bit Plane' [@Ge0rg3] | [#625]
+
+### [9.4.0] - 2019-08-30
+- 'Render Markdown' operation added [@j433866] | [#627]
+
+### [9.3.0] - 2019-08-30
+- 'Show on map' operation added [@j433866] | [#477]
+
+### [9.2.0] - 2019-08-23
+- 'Parse UDP' operation added [@h345983745] | [#614]
+
+### [9.1.0] - 2019-08-22
+- 'Parse SSH Host Key' operation added [@j433866] | [#595]
+- 'Defang IP Addresses' operation added [@h345983745] | [#556]
+
+## [9.0.0] - 2019-07-09
+- [Multiple inputs](https://github.com/gchq/CyberChef/wiki/Multiple-Inputs) are now supported in the main web UI, allowing you to upload and process multiple files at once [@j433866] | [#566]
+- A [Node.js API](https://github.com/gchq/CyberChef/wiki/Node-API) has been implemented, meaning that CyberChef can now be used as a library, either to provide specific operations, or an entire baking environment [@d98762625] | [#291]
+- A [read-eval-print loop (REPL)](https://github.com/gchq/CyberChef/wiki/Node-API#repl) is also included to enable prototyping and experimentation with the API [@d98762625] | [#291]
+- Light and dark Solarized themes added [@j433866] | [#566]
+
+<details>
+    <summary>Click to expand v8 minor versions</summary>
+    
+### [8.38.0] - 2019-07-03
+- 'Streebog' and 'GOST hash' operations added [@MShwed] [@n1474335] | [#530]
+
 ### [8.37.0] - 2019-07-03
 - 'CRC-8 Checksum' operation added [@MShwed] | [#591]
 
@@ -19,7 +65,7 @@ All major and minor version changes will be documented in this file. Details of
 - 'Bzip2 Compress' operation added and 'Bzip2 Decompress' operation greatly improved [@artemisbot] | [#531]
 
 ### [8.32.0] - 2019-06-27
-- 'Indec of Coincidence' operation added [@Ge0rg3] | [#571]
+- 'Index of Coincidence' operation added [@Ge0rg3] | [#571]
 
 ### [8.31.0] - 2019-04-12
 - The downloadable version of CyberChef is now a .zip file containing separate modules rather than a single .htm file. It is still completely standalone and will not make any external network requests. This change reduces the complexity of the build process significantly. [@n1474335]
@@ -120,6 +166,8 @@ All major and minor version changes will be documented in this file. Details of
 ### [8.1.0] - 2018-08-19
 - 'Dechunk HTTP response' operation added [@sevzero] | [#311]
 
+</details>
+
 ## [8.0.0] - 2018-08-05
 - Codebase rewritten using [ES modules](https://hacks.mozilla.org/2018/03/es-modules-a-cartoon-deep-dive/) and [classes](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Classes) [@n1474335] [@d98762625] [@artemisbot] [@picapi] | [#284]
 - Operation architecture restructured to make adding new operations a lot simpler [@n1474335] | [#284]
@@ -149,6 +197,19 @@ All major and minor version changes will be documented in this file. Details of
 
 
 
+[9.11.0]: https://github.com/gchq/CyberChef/releases/tag/v9.11.0
+[9.10.0]: https://github.com/gchq/CyberChef/releases/tag/v9.10.0
+[9.9.0]: https://github.com/gchq/CyberChef/releases/tag/v9.9.0
+[9.8.0]: https://github.com/gchq/CyberChef/releases/tag/v9.8.0
+[9.7.0]: https://github.com/gchq/CyberChef/releases/tag/v9.7.0
+[9.6.0]: https://github.com/gchq/CyberChef/releases/tag/v9.6.0
+[9.5.0]: https://github.com/gchq/CyberChef/releases/tag/v9.5.0
+[9.4.0]: https://github.com/gchq/CyberChef/releases/tag/v9.4.0
+[9.3.0]: https://github.com/gchq/CyberChef/releases/tag/v9.3.0
+[9.2.0]: https://github.com/gchq/CyberChef/releases/tag/v9.2.0
+[9.1.0]: https://github.com/gchq/CyberChef/releases/tag/v9.1.0
+[9.0.0]: https://github.com/gchq/CyberChef/releases/tag/v9.0.0
+[8.38.0]: https://github.com/gchq/CyberChef/releases/tag/v8.38.0
 [8.37.0]: https://github.com/gchq/CyberChef/releases/tag/v8.37.0
 [8.36.0]: https://github.com/gchq/CyberChef/releases/tag/v8.36.0
 [8.35.0]: https://github.com/gchq/CyberChef/releases/tag/v8.35.0
@@ -216,6 +277,10 @@ All major and minor version changes will be documented in this file. Details of
 [@masq]: https://github.com/masq
 [@Ge0rg3]: https://github.com/Ge0rg3
 [@MShwed]: https://github.com/MShwed
+[@kassi]: https://github.com/kassi
+[@jarrodconnolly]: https://github.com/jarrodconnolly
+[@VirtualColossus]: https://github.com/VirtualColossus
+[@cbeuw]: https://github.com/cbeuw
 
 [#95]: https://github.com/gchq/CyberChef/pull/299
 [#173]: https://github.com/gchq/CyberChef/pull/173
@@ -227,6 +292,7 @@ All major and minor version changes will be documented in this file. Details of
 [#277]: https://github.com/gchq/CyberChef/issues/277
 [#281]: https://github.com/gchq/CyberChef/pull/281
 [#284]: https://github.com/gchq/CyberChef/pull/284
+[#291]: https://github.com/gchq/CyberChef/pull/291
 [#294]: https://github.com/gchq/CyberChef/pull/294
 [#296]: https://github.com/gchq/CyberChef/pull/296
 [#298]: https://github.com/gchq/CyberChef/pull/298
@@ -253,15 +319,28 @@ All major and minor version changes will be documented in this file. Details of
 [#467]: https://github.com/gchq/CyberChef/pull/467
 [#468]: https://github.com/gchq/CyberChef/pull/468
 [#476]: https://github.com/gchq/CyberChef/pull/476
+[#477]: https://github.com/gchq/CyberChef/pull/477
 [#489]: https://github.com/gchq/CyberChef/pull/489
 [#496]: https://github.com/gchq/CyberChef/pull/496
+[#500]: https://github.com/gchq/CyberChef/pull/500
 [#506]: https://github.com/gchq/CyberChef/pull/506
 [#515]: https://github.com/gchq/CyberChef/pull/515
 [#516]: https://github.com/gchq/CyberChef/pull/516
 [#525]: https://github.com/gchq/CyberChef/pull/525
+[#528]: https://github.com/gchq/CyberChef/pull/528
+[#530]: https://github.com/gchq/CyberChef/pull/530
 [#531]: https://github.com/gchq/CyberChef/pull/531
 [#533]: https://github.com/gchq/CyberChef/pull/533
 [#535]: https://github.com/gchq/CyberChef/pull/535
+[#556]: https://github.com/gchq/CyberChef/pull/556
+[#566]: https://github.com/gchq/CyberChef/pull/566
 [#571]: https://github.com/gchq/CyberChef/pull/571
 [#585]: https://github.com/gchq/CyberChef/pull/585
 [#591]: https://github.com/gchq/CyberChef/pull/591
+[#595]: https://github.com/gchq/CyberChef/pull/595
+[#614]: https://github.com/gchq/CyberChef/pull/614
+[#625]: https://github.com/gchq/CyberChef/pull/625
+[#627]: https://github.com/gchq/CyberChef/pull/627
+[#632]: https://github.com/gchq/CyberChef/pull/632
+[#653]: https://github.com/gchq/CyberChef/pull/653
+[#865]: https://github.com/gchq/CyberChef/pull/865

Gruntfile.js

@@ -3,7 +3,6 @@
 const webpack = require("webpack");
 const HtmlWebpackPlugin = require("html-webpack-plugin");
 const BundleAnalyzerPlugin = require("webpack-bundle-analyzer").BundleAnalyzerPlugin;
-const NodeExternals = require("webpack-node-externals");
 const glob = require("glob");
 const path = require("path");
 
@@ -24,43 +23,58 @@ module.exports = function (grunt) {
         "A persistent task which creates a development build whenever source files are modified.",
         ["clean:dev", "clean:config", "exec:generateConfig", "concurrent:dev"]);
 
+    grunt.registerTask("prod",
+        "Creates a production-ready build. Use the --msg flag to add a compile message.",
+        [
+            "eslint", "clean:prod", "clean:config", "exec:generateConfig", "findModules", "webpack:web",
+            "copy:standalone", "zip:standalone", "clean:standalone", "chmod"
+        ]);
+
     grunt.registerTask("node",
         "Compiles CyberChef into a single NodeJS module.",
-        ["clean:node", "clean:config", "exec:generateConfig", "webpack:node", "chmod:build"]);
+        [
+            "clean:node", "clean:config", "clean:nodeConfig", "exec:generateConfig", "exec:generateNodeIndex"
+        ]);
 
     grunt.registerTask("test",
         "A task which runs all the operation tests in the tests directory.",
-        ["exec:generateConfig", "exec:opTests"]);
+        [
+            "clean:config", "clean:nodeConfig", "exec:generateConfig", "exec:generateNodeIndex",
+            "exec:nodeTests", "exec:opTests"
+        ]);
 
     grunt.registerTask("testui",
         "A task which runs all the UI tests in the tests directory. The prod task must already have been run.",
         ["connect:prod", "exec:browserTests"]);
 
-    grunt.registerTask("docs",
-        "Compiles documentation in the /docs directory.",
-        ["clean:docs", "jsdoc", "chmod:docs"]);
-
-    grunt.registerTask("prod",
-        "Creates a production-ready build. Use the --msg flag to add a compile message.",
-        [
-            "eslint", "clean:prod", "clean:config", "exec:generateConfig", "webpack:web",
-            "copy:standalone", "zip:standalone", "clean:standalone", "chmod"
-        ]);
+    grunt.registerTask("testnodeconsumer",
+        "A task which checks whether consuming CJS and ESM apps work with the CyberChef build",
+        ["exec:setupNodeConsumers", "exec:testCJSNodeConsumer", "exec:testESMNodeConsumer", "exec:testESMDeepImportNodeConsumer", "exec:teardownNodeConsumers"]);
 
     grunt.registerTask("default",
         "Lints the code base",
         ["eslint", "exec:repoSize"]);
 
-
-    grunt.registerTask("doc", "docs");
     grunt.registerTask("tests", "test");
     grunt.registerTask("lint", "eslint");
 
+    grunt.registerTask("findModules",
+        "Finds all generated modules and updates the entry point list for Webpack",
+        function(arg1, arg2) {
+            const moduleEntryPoints = listEntryModules();
+
+            grunt.log.writeln(`Found ${Object.keys(moduleEntryPoints).length} modules.`);
+
+            grunt.config.set("webpack.web.entry",
+                Object.assign({
+                    main: "./src/web/index.js"
+                }, moduleEntryPoints));
+        });
+
 
     // Load tasks provided by each plugin
     grunt.loadNpmTasks("grunt-eslint");
     grunt.loadNpmTasks("grunt-webpack");
-    grunt.loadNpmTasks("grunt-jsdoc");
     grunt.loadNpmTasks("grunt-contrib-clean");
     grunt.loadNpmTasks("grunt-contrib-copy");
     grunt.loadNpmTasks("grunt-contrib-watch");
@@ -80,17 +94,55 @@ module.exports = function (grunt) {
             COMPILE_TIME: JSON.stringify(compileTime),
             COMPILE_MSG: JSON.stringify(grunt.option("compile-msg") || grunt.option("msg") || ""),
             PKG_VERSION: JSON.stringify(pkg.version),
-            ENVIRONMENT_IS_WORKER: function() {
-                return typeof importScripts === "function";
-            },
-            ENVIRONMENT_IS_NODE: function() {
-                return typeof process === "object" && typeof require === "function";
-            },
-            ENVIRONMENT_IS_WEB: function() {
-                return typeof window === "object";
-            }
         },
-        moduleEntryPoints = listEntryModules();
+        moduleEntryPoints = listEntryModules(),
+        nodeConsumerTestPath = "~/tmp-cyberchef",
+        /**
+         * Configuration for Webpack production build. Defined as a function so that it
+         * can be recalculated when new modules are generated.
+         */
+        webpackProdConf = () => {
+            return {
+                mode: "production",
+                target: "web",
+                entry: Object.assign({
+                    main: "./src/web/index.js"
+                }, moduleEntryPoints),
+                output: {
+                    path: __dirname + "/build/prod",
+                    filename: chunkData => {
+                        return chunkData.chunk.name === "main" ? "assets/[name].js": "[name].js";
+                    },
+                    globalObject: "this"
+                },
+                resolve: {
+                    alias: {
+                        "./config/modules/OpModules.mjs": "./config/modules/Default.mjs"
+                    }
+                },
+                plugins: [
+                    new webpack.DefinePlugin(BUILD_CONSTANTS),
+                    new HtmlWebpackPlugin({
+                        filename: "index.html",
+                        template: "./src/web/html/index.html",
+                        chunks: ["main"],
+                        compileTime: compileTime,
+                        version: pkg.version,
+                        minify: {
+                            removeComments: true,
+                            collapseWhitespace: true,
+                            minifyJS: true,
+                            minifyCSS: true
+                        }
+                    }),
+                    new BundleAnalyzerPlugin({
+                        analyzerMode: "static",
+                        reportFilename: "BundleAnalyzerReport.html",
+                        openAnalyzer: false
+                    }),
+                ]
+            };
+        };
 
 
     /**
@@ -108,13 +160,33 @@ module.exports = function (grunt) {
         return entryModules;
     }
 
+    /**
+     * Detects the correct delimiter to use to chain shell commands together
+     * based on the current OS.
+     *
+     * @param {string[]} cmds
+     * @returns {string}
+     */
+    function chainCommands(cmds) {
+        const win = process.platform === "win32";
+        if (!win) {
+            return cmds.join(";");
+        }
+        return cmds
+            // && means that subsequent commands will not be executed if the
+            // previous one fails. & would coninue on a fail
+            .join("&&")
+            // Windows does not support \n properly
+            .replace("\n", "\\n");
+    }
+
     grunt.initConfig({
         clean: {
             dev: ["build/dev/*"],
             prod: ["build/prod/*"],
             node: ["build/node/*"],
             config: ["src/core/config/OperationConfig.json", "src/core/config/modules/*", "src/code/operations/index.mjs"],
-            docs: ["docs/*", "!docs/*.conf.json", "!docs/*.ico", "!docs/*.png"],
+            nodeConfig: ["src/node/index.mjs", "src/node/config/OperationConfig.json"],
             standalone: ["build/prod/CyberChef*.html"]
         },
         eslint: {
@@ -127,22 +199,6 @@ module.exports = function (grunt) {
             node: ["src/node/**/*.{js,mjs}"],
             tests: ["tests/**/*.{js,mjs}"],
         },
-        jsdoc: {
-            options: {
-                destination: "docs",
-                template: "node_modules/ink-docstrap/template",
-                recurse: true,
-                readme: "./README.md",
-                configure: "docs/jsdoc.conf.json"
-            },
-            all: {
-                src: [
-                    "src/**/*.js",
-                    "src/**/*.mjs",
-                    "!src/core/vendor/**/*"
-                ],
-            }
-        },
         accessibility: {
             options: {
                 accessibilityLevel: "WCAG2A",
@@ -157,64 +213,7 @@ module.exports = function (grunt) {
         },
         webpack: {
             options: webpackConfig,
-            web: () => {
-                return {
-                    mode: "production",
-                    target: "web",
-                    entry: Object.assign({
-                        main: "./src/web/index.js",
-                        sitemap: "./src/web/static/sitemap.js"
-                    }, moduleEntryPoints),
-                    output: {
-                        path: __dirname + "/build/prod",
-                        filename: chunkData => {
-                            return chunkData.chunk.name === "main" ? "assets/[name].js": "[name].js";
-                        },
-                        globalObject: "this"
-                    },
-                    resolve: {
-                        alias: {
-                            "./config/modules/OpModules": "./config/modules/Default"
-                        }
-                    },
-                    plugins: [
-                        new webpack.DefinePlugin(BUILD_CONSTANTS),
-                        new HtmlWebpackPlugin({
-                            filename: "index.html",
-                            template: "./src/web/html/index.html",
-                            chunks: ["main"],
-                            compileTime: compileTime,
-                            version: pkg.version,
-                            minify: {
-                                removeComments: true,
-                                collapseWhitespace: true,
-                                minifyJS: true,
-                                minifyCSS: true
-                            }
-                        }),
-                        new BundleAnalyzerPlugin({
-                            analyzerMode: "static",
-                            reportFilename: "BundleAnalyzerReport.html",
-                            openAnalyzer: false
-                        }),
-                    ]
-                };
-            },
-            node: {
-                mode: "production",
-                target: "node",
-                entry: "./src/node/index.mjs",
-                externals: [NodeExternals()],
-                output: {
-                    filename: "CyberChef.js",
-                    path: __dirname + "/build/node",
-                    library: "CyberChef",
-                    libraryTarget: "commonjs2"
-                },
-                plugins: [
-                    new webpack.DefinePlugin(BUILD_CONSTANTS)
-                ]
-            }
+            web: webpackProdConf(),
         },
         "webpack-dev-server": {
             options: {
@@ -246,12 +245,9 @@ module.exports = function (grunt) {
                     }, moduleEntryPoints),
                     resolve: {
                         alias: {
-                            "./config/modules/OpModules": "./config/modules/Default"
+                            "./config/modules/OpModules.mjs": "./config/modules/Default.mjs"
                         }
                     },
-                    output: {
-                        globalObject: "this",
-                    },
                     plugins: [
                         new webpack.DefinePlugin(BUILD_CONSTANTS),
                         new HtmlWebpackPlugin({
@@ -272,7 +268,6 @@ module.exports = function (grunt) {
                     "build/prod/**/*",
                     "!build/prod/index.html",
                     "!build/prod/BundleAnalyzerReport.html",
-                    "!build/prod/sitemap.js"
                 ],
                 dest: `build/prod/CyberChef_v${pkg.version}.zip`
             }
@@ -310,12 +305,7 @@ module.exports = function (grunt) {
                     {
                         src: "build/prod/index.html",
                         dest: "build/prod/index.html"
-                    },
-                    {
-                        expand: true,
-                        src: "docs/**",
-                        dest: "build/prod/"
-                    },
+                    }
                 ]
             },
             standalone: {
@@ -347,18 +337,12 @@ module.exports = function (grunt) {
                     mode: "755",
                 },
                 src: ["build/**/*", "build/"]
-            },
-            docs: {
-                options: {
-                    mode: "755",
-                },
-                src: ["docs/**/*", "docs/"]
             }
         },
         watch: {
             config: {
                 files: ["src/core/operations/**/*", "!src/core/operations/index.mjs"],
-                tasks: ["exec:generateConfig"]
+                tasks: ["exec:generateNodeIndex", "exec:generateConfig"]
             }
         },
         concurrent: {
@@ -369,33 +353,84 @@ module.exports = function (grunt) {
         },
         exec: {
             repoSize: {
-                command: [
+                command: chainCommands([
                     "git ls-files | wc -l | xargs printf '\n%b\ttracked files\n'",
                     "du -hs | egrep -o '^[^\t]*' | xargs printf '%b\trepository size\n'"
-                ].join(";"),
+                ]),
                 stderr: false
             },
             cleanGit: {
                 command: "git gc --prune=now --aggressive"
             },
             sitemap: {
-                command: "node build/prod/sitemap.js > build/prod/sitemap.xml"
+                command: "node --experimental-modules --no-warnings --no-deprecation src/web/static/sitemap.mjs > build/prod/sitemap.xml",
+                sync: true
             },
             generateConfig: {
-                command: [
+                command: chainCommands([
                     "echo '\n--- Regenerating config files. ---'",
                     "echo [] > src/core/config/OperationConfig.json",
                     "node --experimental-modules --no-warnings --no-deprecation src/core/config/scripts/generateOpsIndex.mjs",
                     "node --experimental-modules --no-warnings --no-deprecation src/core/config/scripts/generateConfig.mjs",
                     "echo '--- Config scripts finished. ---\n'"
-                ].join(";")
+                ]),
+                sync: true
+            },
+            generateNodeIndex: {
+                command: chainCommands([
+                    "echo '\n--- Regenerating node index ---'",
+                    "node --experimental-modules --no-warnings --no-deprecation src/node/config/scripts/generateNodeIndex.mjs",
+                    "echo '--- Node index generated. ---\n'"
+                ]),
+                sync: true
             },
             opTests: {
                 command: "node --experimental-modules --no-warnings --no-deprecation tests/operations/index.mjs"
             },
             browserTests: {
                 command: "./node_modules/.bin/nightwatch --env prod"
-            }
+            },
+            nodeTests: {
+                command: "node --experimental-modules --no-warnings --no-deprecation tests/node/index.mjs"
+            },
+            setupNodeConsumers: {
+                command: chainCommands([
+                    "echo '\n--- Testing node consumers ---'",
+                    "npm link",
+                    `mkdir ${nodeConsumerTestPath}`,
+                    `cp tests/node/consumers/* ${nodeConsumerTestPath}`,
+                    `cd ${nodeConsumerTestPath}`,
+                    "npm link cyberchef"
+                ]),
+                sync: true
+            },
+            teardownNodeConsumers: {
+                command: chainCommands([
+                    `rm -rf ${nodeConsumerTestPath}`,
+                    "echo '\n--- Node consumer tests complete ---'"
+                ]),
+            },
+            testCJSNodeConsumer: {
+                command: chainCommands([
+                    `cd ${nodeConsumerTestPath}`,
+                    "node --no-warnings cjs-consumer.js",
+                ]),
+                stdout: false,
+            },
+            testESMNodeConsumer: {
+                command: chainCommands([
+                    `cd ${nodeConsumerTestPath}`,
+                    "node --no-warnings --experimental-modules esm-consumer.mjs",
+                ]),
+                stdout: false,
+            },
+            testESMDeepImportNodeConsumer: {
+                command: chainCommands([
+                    `cd ${nodeConsumerTestPath}`,
+                    "node --no-warnings --experimental-modules esm-deep-import-consumer.mjs",
+                ]),
+                stdout: false,
+            },
         },
     });
 };

README.md

@@ -3,7 +3,6 @@
 [![Build Status](https://travis-ci.org/gchq/CyberChef.svg?branch=master)](https://travis-ci.org/gchq/CyberChef)
 [![dependencies Status](https://david-dm.org/gchq/CyberChef/status.svg)](https://david-dm.org/gchq/CyberChef)
 [![npm](https://img.shields.io/npm/v/cyberchef.svg)](https://www.npmjs.com/package/cyberchef)
-![](https://reposs.herokuapp.com/?path=gchq/CyberChef&color=blue)
 [![](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/gchq/CyberChef/blob/master/LICENSE)
 [![Gitter](https://badges.gitter.im/gchq/CyberChef.svg)](https://gitter.im/gchq/CyberChef?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
 
@@ -50,7 +49,7 @@ You can use as many operations as you like in simple or complex ways. Some examp
 
  - Drag and drop
      - Operations can be dragged in and out of the recipe list, or reorganised.
-     - Files up to 500MB can be dragged over the input box to load them directly into the browser.
+     - Files up to 2GB can be dragged over the input box to load them directly into the browser.
  - Auto Bake
      - Whenever you modify the input or the recipe, CyberChef will automatically "bake" for you and produce the output immediately.
      - This can be turned off and operated manually if it is affecting performance (if the input is very large, for instance).
@@ -67,19 +66,31 @@ You can use as many operations as you like in simple or complex ways. Some examp
  - Highlighting
      - When you highlight text in the input or output, the offset and length values will be displayed and, if possible, the corresponding data will be highlighted in the output or input respectively (example: [highlight the word 'question' in the input to see where it appears in the output][11]).
  - Save to file and load from file
-     - You can save the output to a file at any time or load a file by dragging and dropping it into the input field. Files up to around 500MB are supported (depending on your browser), however some operations may take a very long time to run over this much data.
+     - You can save the output to a file at any time or load a file by dragging and dropping it into the input field. Files up to around 2GB are supported (depending on your browser), however some operations may take a very long time to run over this much data.
  - CyberChef is entirely client-side
      - It should be noted that none of your recipe configuration or input (either text or files) is ever sent to the CyberChef web server - all processing is carried out within your browser, on your own computer.
-     - Due to this feature, CyberChef can be compiled into a single HTML file. You can download this file and drop it into a virtual machine, share it with other people, or use it independently on your local machine.
+     - Due to this feature, CyberChef can be downloaded and run locally. You can use the link in the top left corner of the app to download a full copy of CyberChef and drop it into a virtual machine, share it with other people, or host it in a closed network.
+
+
+## Deep linking
+
+By manipulation of CyberChef's URL hash, you can change the initial settings with which the page opens.
+The format is `https://gchq.github.io/CyberChef/#recipe=Operation()&input=...`
+
+Supported arguments are `recipe`, `input` (encoded in Base64), and `theme`. 
 
 
 ## Browser support
 
 CyberChef is built to support
 
- - Google Chrome 40+
- - Mozilla Firefox 35+
- - Microsoft Edge 14+
+ - Google Chrome 50+
+ - Mozilla Firefox 38+
+
+
+## Node.js support
+
+CyberChef is built to fully support Node.js `v10` and partially supports `v12`. Named imports using a deep import specifier does not work in `v12`. For more information, see the Node API page in the project [wiki pages](https://github.com/gchq/CyberChef/wiki)
 
 
 ## Contributing

SECURITY.md

@@ -0,0 +1,26 @@
+# Security Policy
+
+## Supported Versions
+
+CyberChef is supported on a best endeavours basis. Patches will be applied to
+the latest version rather than retroactively to older versions. To ensure you
+are using the most secure version of CyberChef, please make sure you have the
+[latest release](https://github.com/gchq/CyberChef/releases/latest). The
+official [live demo](https://gchq.github.io/CyberChef/) is always up to date.
+
+## Reporting a Vulnerability
+
+In most scenarios, the most appropriate way to report a vulnerability is to
+[raise a new issue](https://github.com/gchq/CyberChef/issues/new/choose)
+describing the problem in as much detail as possible, ideally with examples.
+This will obviously be public. If you feel that the vulnerability is
+significant enough to warrant a private disclosure, please email
+[oss@gchq.gov.uk](mailto:oss@gchq.gov.uk) and
+[n1474335@gmail.com](mailto:n1474335@gmail.com).
+
+Disclosures of vulnerabilities in CyberChef are always welcomed. Whilst we aim
+to write clean and secure code free from bugs, we recognise that this is an open
+source project written by analysts in their spare time, relying on dozens of
+open source libraries that are modified and updated on a regular basis. We hope
+that the community will continue to support us as we endeavour to maintain and
+develop this tool together.

babel.config.js

@@ -10,7 +10,7 @@ module.exports = function(api) {
             }]
         ],
         "plugins": [
-            "babel-plugin-syntax-dynamic-import",
+            "dynamic-import-node",
             [
                 "babel-plugin-transform-builtin-extend", {
                     "globals": ["Error"]

docs/jsdoc.conf.json

@@ -1,32 +0,0 @@
-{
-    "tags": {
-        "allowUnknownTags": true
-    },
-    "plugins": [
-        "plugins/markdown",
-        "node_modules/jsdoc-babel"
-    ],
-    "templates": {
-        "systemName": "CyberChef",
-        "footer": "",
-        "copyright": "© Crown Copyright 2017",
-        "navType": "inline",
-        "theme": "cerulean",
-        "linenums": true,
-        "collapseSymbols": false,
-        "inverseNav": true,
-        "outputSourceFiles": true,
-        "outputSourcePath": true,
-        "dateFormat": "ddd MMM Do YYYY",
-        "sort": false,
-        "logoFile": "cyberchef-32x32.png",
-        "cleverLinks": false,
-        "monospaceLinks": false,
-        "protocol": "html://",
-        "methodHeadingReturns": false
-    },
-    "markdown": {
-        "parser": "gfm",
-        "hardwrap": true
-    }
-}

package.json

@@ -1,6 +1,6 @@
 {
   "name": "cyberchef",
-  "version": "8.37.0",
+  "version": "9.11.17",
   "description": "The Cyber Swiss Army Knife for encryption, encoding, compression and data analysis.",
   "author": "n1474335 <n1474335@gmail.com>",
   "homepage": "https://gchq.github.io/CyberChef",
@@ -27,93 +27,93 @@
     "type": "git",
     "url": "https://github.com/gchq/CyberChef/"
   },
-  "main": "build/node/CyberChef.js",
+  "main": "src/node/cjs.js",
+  "module": "src/node/index.mjs",
   "bugs": "https://github.com/gchq/CyberChef/issues",
   "browserslist": [
-    "Chrome >= 40",
-    "Firefox >= 35",
-    "Edge >= 14",
-    "node >= 6.5"
+    "Chrome >= 50",
+    "Firefox >= 38",
+    "node >= 10"
   ],
   "devDependencies": {
-    "@babel/core": "^7.4.5",
-    "@babel/plugin-transform-runtime": "^7.4.4",
-    "@babel/preset-env": "^7.4.5",
-    "autoprefixer": "^9.6.0",
-    "babel-eslint": "^10.0.2",
+    "@babel/core": "^7.7.5",
+    "@babel/plugin-transform-runtime": "^7.7.6",
+    "@babel/preset-env": "^7.7.6",
+    "autoprefixer": "^9.7.3",
+    "babel-eslint": "^10.0.3",
     "babel-loader": "^8.0.6",
-    "babel-plugin-syntax-dynamic-import": "^6.18.0",
-    "chromedriver": "^75.0.0",
-    "colors": "^1.3.3",
-    "css-loader": "^3.0.0",
-    "eslint": "^6.0.1",
+    "babel-plugin-dynamic-import-node": "^2.3.0",
+    "chromedriver": "^78.0.1",
+    "colors": "^1.4.0",
+    "copy-webpack-plugin": "^5.0.5",
+    "css-loader": "^3.2.1",
+    "eslint": "^6.7.2",
     "exports-loader": "^0.7.0",
-    "file-loader": "^4.0.0",
+    "file-loader": "^5.0.2",
     "grunt": "^1.0.4",
     "grunt-accessibility": "~6.0.0",
     "grunt-chmod": "~1.1.1",
     "grunt-concurrent": "^3.0.0",
     "grunt-contrib-clean": "~2.0.0",
-    "grunt-contrib-connect": "^2.0.0",
+    "grunt-contrib-connect": "^2.1.0",
     "grunt-contrib-copy": "~1.0.0",
     "grunt-contrib-watch": "^1.1.0",
-    "grunt-eslint": "^21.1.0",
+    "grunt-eslint": "^22.0.0",
     "grunt-exec": "~3.0.0",
-    "grunt-jsdoc": "^2.4.0",
     "grunt-webpack": "^3.1.3",
     "grunt-zip": "^0.18.2",
     "html-webpack-plugin": "^3.2.0",
     "imports-loader": "^0.8.0",
-    "ink-docstrap": "^1.3.2",
-    "jsdoc-babel": "^0.5.0",
-    "mini-css-extract-plugin": "^0.7.0",
-    "nightwatch": "^1.1.12",
-    "node-sass": "^4.12.0",
-    "postcss-css-variables": "^0.13.0",
+    "mini-css-extract-plugin": "^0.8.0",
+    "nightwatch": "^1.3.2",
+    "node-sass": "^4.13.0",
+    "postcss-css-variables": "^0.14.0",
     "postcss-import": "^12.0.1",
     "postcss-loader": "^3.0.0",
     "prompt": "^1.0.0",
-    "sass-loader": "^7.1.0",
-    "sitemap": "^2.2.0",
-    "style-loader": "^0.23.1",
-    "svg-url-loader": "^2.3.3",
-    "url-loader": "^2.0.1",
-    "webpack": "^4.35.0",
-    "webpack-bundle-analyzer": "^3.3.2",
-    "webpack-dev-server": "^3.7.2",
+    "sass-loader": "^8.0.0",
+    "sitemap": "^5.1.0",
+    "style-loader": "^1.0.1",
+    "svg-url-loader": "^3.0.3",
+    "url-loader": "^3.0.0",
+    "webpack": "^4.41.2",
+    "webpack-bundle-analyzer": "^3.6.0",
+    "webpack-dev-server": "^3.9.0",
     "webpack-node-externals": "^1.7.2",
     "worker-loader": "^2.0.0"
   },
   "dependencies": {
-    "@babel/polyfill": "^7.4.4",
-    "@babel/runtime": "^7.4.5",
+    "@babel/polyfill": "^7.7.0",
+    "@babel/runtime": "^7.7.6",
     "arrive": "^2.4.1",
+    "avsc": "^5.4.16",
     "babel-plugin-transform-builtin-extend": "1.1.2",
     "bcryptjs": "^2.4.3",
     "bignumber.js": "^9.0.0",
     "blakejs": "^1.1.0",
-    "bootstrap": "4.3.1",
-    "bootstrap-colorpicker": "^2.5.3",
+    "bootstrap": "4.4.1",
+    "bootstrap-colorpicker": "^3.2.0",
     "bootstrap-material-design": "^4.1.2",
     "bson": "^4.0.2",
     "chi-squared": "^1.1.0",
-    "clippyjs": "0.0.3",
-    "core-js": "^3.1.4",
-    "crypto-api": "^0.8.3",
+    "codepage": "^1.14.0",
+    "core-js": "^3.4.8",
+    "crypto-api": "^0.8.5",
     "crypto-js": "^3.1.9-1",
     "ctph.js": "0.0.5",
-    "d3": "^5.9.4",
+    "d3": "^5.14.2",
     "d3-hexbin": "^0.2.2",
     "diff": "^4.0.1",
-    "es6-promisify": "^6.0.1",
-    "escodegen": "^1.11.1",
+    "es6-promisify": "^6.0.2",
+    "escodegen": "^1.12.0",
+    "esm": "^3.2.25",
     "esmangle": "^1.0.1",
     "esprima": "^4.0.1",
     "exif-parser": "^0.1.12",
     "file-saver": "^2.0.2",
     "geodesy": "^1.1.3",
-    "highlight.js": "^9.15.8",
-    "jimp": "^0.6.4",
+    "highlight.js": "^9.16.2",
+    "jimp": "^0.9.3",
     "jquery": "3.4.1",
     "js-crc": "^0.2.0",
     "js-sha3": "^0.8.0",
@@ -122,28 +122,30 @@
     "jsonwebtoken": "^8.5.1",
     "jsqr": "^1.2.0",
     "jsrsasign": "8.0.12",
-    "kbpgp": "2.1.2",
+    "kbpgp": "2.1.6",
     "libbzip2-wasm": "0.0.4",
-    "libyara-wasm": "0.0.12",
-    "lodash": "^4.17.11",
-    "loglevel": "^1.6.3",
+    "libyara-wasm": "^1.0.1",
+    "lodash": "^4.17.15",
+    "loglevel": "^1.6.6",
     "loglevel-message-prefix": "^3.0.0",
+    "markdown-it": "^10.0.0",
     "moment": "^2.24.0",
-    "moment-timezone": "^0.5.25",
+    "moment-timezone": "^0.5.27",
     "ngeohash": "^0.6.3",
-    "node-forge": "^0.8.5",
+    "node-forge": "^0.9.1",
     "node-md6": "^0.1.0",
-    "nodom": "^2.2.0",
+    "nodom": "^2.4.0",
     "notepack.io": "^2.2.0",
     "nwmatcher": "^1.4.4",
     "otp": "^0.1.3",
-    "popper.js": "^1.15.0",
+    "popper.js": "^1.16.0",
     "qr-image": "^3.2.0",
     "scryptsy": "^2.1.0",
     "snackbarjs": "^1.1.0",
-    "sortablejs": "^1.9.0",
+    "sortablejs": "^1.10.1",
     "split.js": "^1.5.11",
     "ssdeep.js": "0.0.2",
+    "tesseract.js": "^2.0.0-alpha.15",
     "ua-parser-js": "^0.7.20",
     "utf8": "^3.0.0",
     "vkbeautify": "^0.99.3",
@@ -155,9 +157,11 @@
   "scripts": {
     "start": "grunt dev",
     "build": "grunt prod",
+    "repl": "node src/node/repl.js",
     "test": "grunt test",
+    "test-node-consumer": "grunt testnodeconsumer",
     "testui": "grunt testui",
-    "docs": "grunt docs",
+    "testuidev": "npx nightwatch --env=dev",
     "lint": "grunt lint",
     "newop": "node --experimental-modules src/core/config/scripts/newOperation.mjs"
   }

src/core/Chef.mjs

@@ -4,9 +4,10 @@
  * @license Apache-2.0
  */
 
-import Dish from "./Dish";
-import Recipe from "./Recipe";
+import Dish from "./Dish.mjs";
+import Recipe from "./Recipe.mjs";
 import log from "loglevel";
+import { isWorkerEnvironment } from "./Utils.mjs";
 
 /**
  * The main controller for CyberChef.
@@ -28,8 +29,6 @@ class Chef {
      * @param {Object[]} recipeConfig - The recipe configuration object
      * @param {Object} options - The options object storing various user choices
      * @param {boolean} options.attempHighlight - Whether or not to attempt highlighting
-     * @param {number} progress - The position in the recipe to start from
-     * @param {number} [step] - Whether to only execute one operation in the recipe
      *
      * @returns {Object} response
      * @returns {string} response.result - The output of the recipe
@@ -38,46 +37,20 @@ class Chef {
      * @returns {number} response.duration - The number of ms it took to execute the recipe
      * @returns {number} response.error - The error object thrown by a failed operation (false if no error)
     */
-    async bake(input, recipeConfig, options, progress, step) {
+    async bake(input, recipeConfig, options) {
         log.debug("Chef baking");
         const startTime = new Date().getTime(),
             recipe      = new Recipe(recipeConfig),
             containsFc  = recipe.containsFlowControl(),
-            notUTF8     = options && options.hasOwnProperty("treatAsUtf8") && !options.treatAsUtf8;
-        let error = false;
-
-        if (containsFc && ENVIRONMENT_IS_WORKER()) self.setOption("attemptHighlight", false);
-
-        // Clean up progress
-        if (progress >= recipeConfig.length) {
+            notUTF8     = options && "treatAsUtf8" in options && !options.treatAsUtf8;
+        let error = false,
             progress = 0;
-        }
 
-        if (step) {
-            // Unset breakpoint on this step
-            recipe.setBreakpoint(progress, false);
-            // Set breakpoint on next step
-            recipe.setBreakpoint(progress + 1, true);
-        }
+        if (containsFc && isWorkerEnvironment()) self.setOption("attemptHighlight", false);
 
-        // If the previously run operation presented a different value to its
-        // normal output, we need to recalculate it.
-        if (recipe.lastOpPresented(progress)) {
-            progress = 0;
-        }
-
-        // If stepping with flow control, we have to start from the beginning
-        // but still want to skip all previous breakpoints
-        if (progress > 0 && containsFc) {
-            recipe.removeBreaksUpTo(progress);
-            progress = 0;
-        }
-
-        // If starting from scratch, load data
-        if (progress === 0) {
-            const type = input instanceof ArrayBuffer ? Dish.ARRAY_BUFFER : Dish.STRING;
-            this.dish.set(input, type);
-        }
+        // Load data
+        const type = input instanceof ArrayBuffer ? Dish.ARRAY_BUFFER : Dish.STRING;
+        this.dish.set(input, type);
 
         try {
             progress = await recipe.execute(this.dish, progress);
@@ -100,10 +73,10 @@ class Chef {
         // The threshold is specified in KiB.
         const threshold = (options.ioDisplayThreshold || 1024) * 1024;
         const returnType =
-            this.dish.size > threshold ?
-                Dish.ARRAY_BUFFER :
-                this.dish.type === Dish.HTML ?
-                    Dish.HTML :
+            this.dish.type === Dish.HTML ?
+                Dish.HTML :
+                this.dish.size > threshold ?
+                    Dish.ARRAY_BUFFER :
                     Dish.STRING;
 
         return {
@@ -196,6 +169,18 @@ class Chef {
         return await newDish.get(type);
     }
 
+    /**
+     * Gets the title of a dish and returns it
+     *
+     * @param {Dish} dish
+     * @param {number} [maxLength=100]
+     * @returns {string}
+     */
+    async getDishTitle(dish, maxLength=100) {
+        const newDish = new Dish(dish);
+        return await newDish.getTitle(maxLength);
+    }
+
 }
 
 export default Chef;

src/core/ChefWorker.js

@@ -6,9 +6,9 @@
  * @license Apache-2.0
  */
 
-import Chef from "./Chef";
+import Chef from "./Chef.mjs";
 import OperationConfig from "./config/OperationConfig.json";
-import OpModules from "./config/modules/OpModules";
+import OpModules from "./config/modules/OpModules.mjs";
 
 // Add ">" to the start of all log messages in the Chef Worker
 import loglevelMessagePrefix from "loglevel-message-prefix";
@@ -25,6 +25,8 @@ self.chef = new Chef();
 
 self.OpModules = OpModules;
 self.OperationConfig = OperationConfig;
+self.inputNum = -1;
+
 
 // Tell the app that the worker has loaded and is ready to operate
 self.postMessage({
@@ -35,6 +37,9 @@ self.postMessage({
 /**
  * Respond to message from parent thread.
  *
+ * inputNum is optional and only used for baking multiple inputs.
+ * Defaults to -1 when one isn't sent with the bake message.
+ *
  * Messages should have the following format:
  * {
  *     action: "bake" | "silentBake",
@@ -43,8 +48,9 @@ self.postMessage({
  *         recipeConfig: {[Object]},
  *         options: {Object},
  *         progress: {number},
- *         step: {boolean}
- *     } | undefined
+ *         step: {boolean},
+ *         [inputNum=-1]: {number}
+ *     }
  * }
  */
 self.addEventListener("message", function(e) {
@@ -62,6 +68,9 @@ self.addEventListener("message", function(e) {
         case "getDishAs":
             getDishAs(r.data);
             break;
+        case "getDishTitle":
+            getDishTitle(r.data);
+            break;
         case "docURL":
             // Used to set the URL of the current document so that scripts can be
             // imported into an inline worker.
@@ -91,30 +100,35 @@ self.addEventListener("message", function(e) {
 async function bake(data) {
     // Ensure the relevant modules are loaded
     self.loadRequiredModules(data.recipeConfig);
-
     try {
+        self.inputNum = (data.inputNum !== undefined) ? data.inputNum : -1;
         const response = await self.chef.bake(
             data.input,          // The user's input
             data.recipeConfig,   // The configuration of the recipe
-            data.options,        // Options set by the user
-            data.progress,       // The current position in the recipe
-            data.step            // Whether or not to take one step or execute the whole recipe
+            data.options         // Options set by the user
         );
 
+        const transferable = (data.input instanceof ArrayBuffer) ? [data.input] : undefined;
         self.postMessage({
             action: "bakeComplete",
             data: Object.assign(response, {
-                id: data.id
+                id: data.id,
+                inputNum: data.inputNum,
+                bakeId: data.bakeId
             })
-        });
+        }, transferable);
+
     } catch (err) {
         self.postMessage({
             action: "bakeError",
-            data: Object.assign(err, {
-                id: data.id
-            })
+            data: {
+                error: err.message || err,
+                id: data.id,
+                inputNum: data.inputNum
+            }
         });
     }
+    self.inputNum = -1;
 }
 
 
@@ -136,13 +150,33 @@ function silentBake(data) {
  */
 async function getDishAs(data) {
     const value = await self.chef.getDishAs(data.dish, data.type);
-
+    const transferable = (data.type === "ArrayBuffer") ? [value] : undefined;
     self.postMessage({
         action: "dishReturned",
         data: {
             value: value,
             id: data.id
         }
+    }, transferable);
+}
+
+
+/**
+ * Gets the dish title
+ *
+ * @param {object} data
+ * @param {Dish} data.dish
+ * @param {number} data.maxLength
+ * @param {number} data.id
+ */
+async function getDishTitle(data) {
+    const title = await self.chef.getDishTitle(data.dish, data.maxLength);
+    self.postMessage({
+        action: "dishReturned",
+        data: {
+            value: title,
+            id: data.id
+        }
     });
 }
 
@@ -175,7 +209,7 @@ self.loadRequiredModules = function(recipeConfig) {
     recipeConfig.forEach(op => {
         const module = self.OperationConfig[op.op].module;
 
-        if (!OpModules.hasOwnProperty(module)) {
+        if (!(module in OpModules)) {
             log.info(`Loading ${module} module`);
             self.sendStatusMessage(`Loading ${module} module`);
             self.importScripts(`${self.docURL}/modules/${module}.js`);
@@ -193,7 +227,28 @@ self.loadRequiredModules = function(recipeConfig) {
 self.sendStatusMessage = function(msg) {
     self.postMessage({
         action: "statusMessage",
-        data: msg
+        data: {
+            message: msg,
+            inputNum: self.inputNum
+        }
+    });
+};
+
+
+/**
+ * Send progress update to the app.
+ *
+ * @param {number} progress
+ * @param {number} total
+ */
+self.sendProgressMessage = function(progress, total) {
+    self.postMessage({
+        action: "progressMessage",
+        data: {
+            progress: progress,
+            total: total,
+            inputNum: self.inputNum
+        }
     });
 };
 

src/core/Dish.mjs

@@ -5,11 +5,22 @@
  * @license Apache-2.0
  */
 
-import Utils from "./Utils";
-import DishError from "./errors/DishError";
+import Utils, { isNodeEnvironment } from "./Utils.mjs";
+import DishError from "./errors/DishError.mjs";
 import BigNumber from "bignumber.js";
+import { detectFileType } from "./lib/FileType.mjs";
 import log from "loglevel";
 
+import DishByteArray from "./dishTypes/DishByteArray.mjs";
+import DishBigNumber from "./dishTypes/DishBigNumber.mjs";
+import DishFile from "./dishTypes/DishFile.mjs";
+import DishHTML from "./dishTypes/DishHTML.mjs";
+import DishJSON from "./dishTypes/DishJSON.mjs";
+import DishListFile from "./dishTypes/DishListFile.mjs";
+import DishNumber from "./dishTypes/DishNumber.mjs";
+import DishString from "./dishTypes/DishString.mjs";
+
+
 /**
  * The data being operated on by each operation.
  */
@@ -18,16 +29,27 @@ class Dish {
     /**
      * Dish constructor
      *
-     * @param {Dish} [dish=null] - A dish to clone
+     * @param {Dish || *} [dishOrInput=null] - A dish to clone OR an object
+     * literal to make into a dish
+     * @param {Enum} [type=null] (optional) - A type to accompany object
+     * literal input
      */
-    constructor(dish=null) {
+    constructor(dishOrInput=null, type = null) {
         this.value = new ArrayBuffer(0);
         this.type = Dish.ARRAY_BUFFER;
 
-        if (dish &&
-            dish.hasOwnProperty("value") &&
-            dish.hasOwnProperty("type")) {
-            this.set(dish.value, dish.type);
+        // Case: dishOrInput is dish object
+        if (dishOrInput &&
+            Object.prototype.hasOwnProperty.call(dishOrInput, "value") &&
+            Object.prototype.hasOwnProperty.call(dishOrInput, "type")) {
+            this.set(dishOrInput.value, dishOrInput.type);
+        // input and type defined separately
+        } else if (dishOrInput && type !== null) {
+            this.set(dishOrInput, type);
+        // No type declared, so infer it.
+        } else if (dishOrInput) {
+            const inferredType = Dish.typeEnum(dishOrInput.constructor.name);
+            this.set(dishOrInput, inferredType);
         }
     }
 
@@ -56,6 +78,7 @@ class Dish {
             case "big number":
                 return Dish.BIG_NUMBER;
             case "json":
+            case "object": // object constructor name. To allow JSON input in node.
                 return Dish.JSON;
             case "file":
                 return Dish.FILE;
@@ -99,6 +122,43 @@ class Dish {
     }
 
 
+    /**
+     * Returns the value of the data in the type format specified.
+     *
+     * If running in a browser, get is asynchronous.
+     *
+     * @param {number} type - The data type of value, see Dish enums.
+     * @param {boolean} [notUTF8=false] - Do not treat strings as UTF8.
+     * @returns {* | Promise} - (Browser) A promise | (Node) value of dish in given type
+     */
+    get(type, notUTF8=false) {
+        if (typeof type === "string") {
+            type = Dish.typeEnum(type);
+        }
+
+        if (this.type !== type) {
+
+            // Node environment => _translate is sync
+            if (isNodeEnvironment()) {
+                this._translate(type, notUTF8);
+                return this.value;
+
+            // Browser environment => _translate is async
+            } else {
+                return new Promise((resolve, reject) => {
+                    this._translate(type, notUTF8)
+                        .then(() => {
+                            resolve(this.value);
+                        })
+                        .catch(reject);
+                });
+            }
+        }
+
+        return this.value;
+    }
+
+
     /**
      * Sets the data value and type and then validates them.
      *
@@ -117,121 +177,86 @@ class Dish {
         this.type = type;
 
         if (!this.valid()) {
-            const sample = Utils.truncate(JSON.stringify(this.value), 13);
+            const sample = Utils.truncate(JSON.stringify(this.value), 25);
             throw new DishError(`Data is not a valid ${Dish.enumLookup(type)}: ${sample}`);
         }
     }
 
-
     /**
-     * Returns the value of the data in the type format specified.
+     * Returns the Dish as the given type, without mutating the original dish.
+     *
+     * If running in a browser, get is asynchronous.
+     *
+     * @Node
      *
      * @param {number} type - The data type of value, see Dish enums.
      * @param {boolean} [notUTF8=false] - Do not treat strings as UTF8.
-     * @returns {*} - The value of the output data.
+     * @returns {Dish | Promise} - (Browser) A promise | (Node) value of dish in given type
      */
-    async get(type, notUTF8=false) {
-        if (typeof type === "string") {
-            type = Dish.typeEnum(type);
-        }
-        if (this.type !== type) {
-            await this._translate(type, notUTF8);
-        }
-        return this.value;
+    presentAs(type, notUTF8=false) {
+        const clone = this.clone();
+        return clone.get(type, notUTF8);
     }
 
 
     /**
-     * Translates the data to the given type format.
-     *
-     * @param {number} toType - The data type of value, see Dish enums.
-     * @param {boolean} [notUTF8=false] - Do not treat strings as UTF8.
+     * Detects the MIME type of the current dish
+     * @returns {string}
      */
-    async _translate(toType, notUTF8=false) {
-        log.debug(`Translating Dish from ${Dish.enumLookup(this.type)} to ${Dish.enumLookup(toType)}`);
+    detectDishType() {
+        const data = new Uint8Array(this.value.slice(0, 2048)),
+            types = detectFileType(data);
 
-        // Convert data to intermediate ArrayBuffer type
-        try {
-            switch (this.type) {
-                case Dish.STRING:
-                    this.value = this.value ? Utils.strToArrayBuffer(this.value) : new ArrayBuffer;
-                    break;
-                case Dish.NUMBER:
-                    this.value = typeof this.value === "number" ? Utils.strToArrayBuffer(this.value.toString()) : new ArrayBuffer;
-                    break;
-                case Dish.HTML:
-                    this.value = this.value ? Utils.strToArrayBuffer(Utils.unescapeHtml(Utils.stripHtmlTags(this.value, true))) : new ArrayBuffer;
-                    break;
-                case Dish.BYTE_ARRAY:
-                    this.value = new Uint8Array(this.value).buffer;
-                    break;
-                case Dish.BIG_NUMBER:
-                    this.value = BigNumber.isBigNumber(this.value) ? Utils.strToArrayBuffer(this.value.toFixed()) : new ArrayBuffer;
-                    break;
-                case Dish.JSON:
-                    this.value = this.value ? Utils.strToArrayBuffer(JSON.stringify(this.value, null, 4)) : new ArrayBuffer;
-                    break;
-                case Dish.FILE:
-                    this.value = (await Utils.readFile(this.value)).buffer;
-                    break;
-                case Dish.LIST_FILE:
-                    this.value = await Promise.all(this.value.map(async f => Utils.readFile(f)));
-                    this.value = concatenateTypedArrays(...this.value).buffer;
-                    break;
-                default:
-                    break;
-            }
-        } catch (err) {
-            throw new DishError(`Error translating from ${Dish.enumLookup(this.type)} to ArrayBuffer: ${err}`);
-        }
-
-        this.type = Dish.ARRAY_BUFFER;
-
-        // Convert from ArrayBuffer to toType
-        try {
-            switch (toType) {
-                case Dish.STRING:
-                case Dish.HTML:
-                    this.value = this.value ? Utils.arrayBufferToStr(this.value, !notUTF8) : "";
-                    this.type = Dish.STRING;
-                    break;
-                case Dish.NUMBER:
-                    this.value = this.value ? parseFloat(Utils.arrayBufferToStr(this.value, !notUTF8)) : 0;
-                    this.type = Dish.NUMBER;
-                    break;
-                case Dish.BYTE_ARRAY:
-                    this.value = Array.prototype.slice.call(new Uint8Array(this.value));
-                    this.type = Dish.ARRAY_BUFFER;
-                    break;
-                case Dish.BIG_NUMBER:
-                    try {
-                        this.value = new BigNumber(Utils.arrayBufferToStr(this.value, !notUTF8));
-                    } catch (err) {
-                        this.value = new BigNumber(NaN);
-                    }
-                    this.type = Dish.BIG_NUMBER;
-                    break;
-                case Dish.JSON:
-                    this.value = JSON.parse(Utils.arrayBufferToStr(this.value, !notUTF8));
-                    this.type = Dish.JSON;
-                    break;
-                case Dish.FILE:
-                    this.value = new File(this.value, "unknown");
-                    this.type = Dish.FILE;
-                    break;
-                case Dish.LIST_FILE:
-                    this.value = [new File(this.value, "unknown")];
-                    this.type = Dish.LIST_FILE;
-                    break;
-                default:
-                    break;
-            }
-        } catch (err) {
-            throw new DishError(`Error translating from ArrayBuffer to ${Dish.enumLookup(toType)}: ${err}`);
+        if (!types.length || !types[0].mime || !types[0].mime === "text/plain") {
+            return null;
+        } else {
+            return types[0].mime;
         }
     }
 
 
+    /**
+     * Returns the title of the data up to the specified length
+     *
+     * @param {number} maxLength - The maximum title length
+     * @returns {string}
+     */
+    async getTitle(maxLength) {
+        let title = "";
+        let cloned;
+
+        switch (this.type) {
+            case Dish.FILE:
+                title = this.value.name;
+                break;
+            case Dish.LIST_FILE:
+                title = `${this.value.length} file(s)`;
+                break;
+            case Dish.JSON:
+                title = "application/json";
+                break;
+            case Dish.NUMBER:
+            case Dish.BIG_NUMBER:
+                title = this.value.toString();
+                break;
+            case Dish.ARRAY_BUFFER:
+            case Dish.BYTE_ARRAY:
+                title = this.detectDishType();
+                if (title !== null) break;
+                // fall through if no mime type was detected
+            default:
+                try {
+                    cloned = this.clone();
+                    cloned.value = cloned.value.slice(0, 256);
+                    title = await cloned.get(Dish.STRING);
+                } catch (err) {
+                    log.error(`${Dish.enumLookup(this.type)} cannot be sliced. ${err}`);
+                }
+        }
+
+        return title.slice(0, maxLength);
+    }
+
     /**
      * Validates that the value is the type that has been specified.
      * May have to disable parts of BYTE_ARRAY validation if it effects performance.
@@ -241,7 +266,7 @@ class Dish {
     valid() {
         switch (this.type) {
             case Dish.BYTE_ARRAY:
-                if (!(this.value instanceof Array)) {
+                if (!(this.value instanceof Uint8Array) && !(this.value instanceof Array)) {
                     return false;
                 }
 
@@ -262,7 +287,21 @@ class Dish {
             case Dish.ARRAY_BUFFER:
                 return this.value instanceof ArrayBuffer;
             case Dish.BIG_NUMBER:
-                return BigNumber.isBigNumber(this.value);
+                if (BigNumber.isBigNumber(this.value)) return true;
+                /*
+                    If a BigNumber is passed between WebWorkers it is serialised as a JSON
+                    object with a coefficient (c), exponent (e) and sign (s). We detect this
+                    and reinitialise it as a BigNumber object.
+                */
+                if (Object.keys(this.value).sort().equals(["c", "e", "s"])) {
+                    const temp = new BigNumber();
+                    temp.c = this.value.c;
+                    temp.e = this.value.e;
+                    temp.s = this.value.s;
+                    this.value = temp;
+                    return true;
+                }
+                return false;
             case Dish.JSON:
                 // All values can be serialised in some manner, so we return true in all cases
                 return true;
@@ -369,26 +408,109 @@ class Dish {
         return newDish;
     }
 
-}
+    /**
+     * Translates the data to the given type format.
+     *
+     * If running in the browser, _translate is asynchronous.
+     *
+     * @param {number} toType - The data type of value, see Dish enums.
+     * @param {boolean} [notUTF8=false] - Do not treat strings as UTF8.
+     * @returns {Promise || undefined}
+     */
+    _translate(toType, notUTF8=false) {
+        log.debug(`Translating Dish from ${Dish.enumLookup(this.type)} to ${Dish.enumLookup(toType)}`);
+
+        // Node environment => translate is sync
+        if (isNodeEnvironment()) {
+            this._toArrayBuffer();
+            this.type = Dish.ARRAY_BUFFER;
+            this._fromArrayBuffer(toType, notUTF8);
+
+        // Browser environment => translate is async
+        } else {
+            return new Promise((resolve, reject) => {
+                this._toArrayBuffer()
+                    .then(() => this.type = Dish.ARRAY_BUFFER)
+                    .then(() => {
+                        this._fromArrayBuffer(toType);
+                        resolve();
+                    })
+                    .catch(reject);
+            });
+        }
 
-/**
- * Concatenates a list of Uint8Arrays together
- *
- * @param {Uint8Array[]} arrays
- * @returns {Uint8Array}
- */
-function concatenateTypedArrays(...arrays) {
-    let totalLength = 0;
-    for (const arr of arrays) {
-        totalLength += arr.length;
     }
-    const result = new Uint8Array(totalLength);
-    let offset = 0;
-    for (const arr of arrays) {
-        result.set(arr, offset);
-        offset += arr.length;
+
+    /**
+     * Convert this.value to an ArrayBuffer
+     *
+     * If running in a browser, _toByteArray is asynchronous.
+     *
+     * @returns {Promise || undefined}
+     */
+    _toArrayBuffer() {
+        // Using 'bind' here to allow this.value to be mutated within translation functions
+        const toByteArrayFuncs = {
+            browser: {
+                [Dish.STRING]:          () => Promise.resolve(DishString.toArrayBuffer.bind(this)()),
+                [Dish.NUMBER]:          () => Promise.resolve(DishNumber.toArrayBuffer.bind(this)()),
+                [Dish.HTML]:            () => Promise.resolve(DishHTML.toArrayBuffer.bind(this)()),
+                [Dish.ARRAY_BUFFER]:    () => Promise.resolve(),
+                [Dish.BIG_NUMBER]:      () => Promise.resolve(DishBigNumber.toArrayBuffer.bind(this)()),
+                [Dish.JSON]:            () => Promise.resolve(DishJSON.toArrayBuffer.bind(this)()),
+                [Dish.FILE]:            () => DishFile.toArrayBuffer.bind(this)(),
+                [Dish.LIST_FILE]:       () => Promise.resolve(DishListFile.toArrayBuffer.bind(this)()),
+                [Dish.BYTE_ARRAY]:      () => Promise.resolve(DishByteArray.toArrayBuffer.bind(this)()),
+            },
+            node: {
+                [Dish.STRING]:          () => DishString.toArrayBuffer.bind(this)(),
+                [Dish.NUMBER]:          () => DishNumber.toArrayBuffer.bind(this)(),
+                [Dish.HTML]:            () => DishHTML.toArrayBuffer.bind(this)(),
+                [Dish.ARRAY_BUFFER]:    () => {},
+                [Dish.BIG_NUMBER]:      () => DishBigNumber.toArrayBuffer.bind(this)(),
+                [Dish.JSON]:            () => DishJSON.toArrayBuffer.bind(this)(),
+                [Dish.FILE]:            () => DishFile.toArrayBuffer.bind(this)(),
+                [Dish.LIST_FILE]:       () => DishListFile.toArrayBuffer.bind(this)(),
+                [Dish.BYTE_ARRAY]:      () => DishByteArray.toArrayBuffer.bind(this)(),
+            }
+        };
+
+        try {
+            return toByteArrayFuncs[isNodeEnvironment() && "node" || "browser"][this.type]();
+        } catch (err) {
+            throw new DishError(`Error translating from ${Dish.enumLookup(this.type)} to ArrayBuffer: ${err}`);
+        }
     }
-    return result;
+
+    /**
+     * Convert this.value to the given type from ArrayBuffer
+     *
+     * @param {number} toType - the Dish enum to convert to
+     * @param {boolean} [notUTF8=false] - Do not treat strings as UTF8.
+    */
+    _fromArrayBuffer(toType, notUTF8) {
+
+        // Using 'bind' here to allow this.value to be mutated within translation functions
+        const toTypeFunctions = {
+            [Dish.STRING]:          () => DishString.fromArrayBuffer.bind(this)(notUTF8),
+            [Dish.NUMBER]:          () => DishNumber.fromArrayBuffer.bind(this)(notUTF8),
+            [Dish.HTML]:            () => DishHTML.fromArrayBuffer.bind(this)(notUTF8),
+            [Dish.ARRAY_BUFFER]:    () => {},
+            [Dish.BIG_NUMBER]:      () => DishBigNumber.fromArrayBuffer.bind(this)(notUTF8),
+            [Dish.JSON]:            () => DishJSON.fromArrayBuffer.bind(this)(notUTF8),
+            [Dish.FILE]:            () => DishFile.fromArrayBuffer.bind(this)(),
+            [Dish.LIST_FILE]:       () => DishListFile.fromArrayBuffer.bind(this)(),
+            [Dish.BYTE_ARRAY]:      () => DishByteArray.fromArrayBuffer.bind(this)(),
+        };
+
+        try {
+            toTypeFunctions[toType]();
+            this.type = toType;
+        } catch (err) {
+            throw new DishError(`Error translating from ArrayBuffer to ${Dish.enumLookup(toType)}: ${err}`);
+        }
+    }
+
 }
 
 

src/core/Ingredient.mjs

@@ -4,8 +4,8 @@
  * @license Apache-2.0
  */
 
-import Utils from "./Utils";
-import {fromHex} from "./lib/Hex";
+import Utils from "./Utils.mjs";
+import {fromHex} from "./lib/Hex.mjs";
 
 /**
  * The arguments to operations.

src/core/Operation.mjs

@@ -4,8 +4,8 @@
  * @license Apache-2.0
  */
 
-import Dish from "./Dish";
-import Ingredient from "./Ingredient";
+import Dish from "./Dish.mjs";
+import Ingredient from "./Ingredient.mjs";
 
 /**
  * The Operation specified by the user to be run.

src/core/Recipe.mjs

@@ -5,10 +5,11 @@
  */
 
 import OperationConfig from "./config/OperationConfig.json";
-import OperationError from "./errors/OperationError";
-import Operation from "./Operation";
-import DishError from "./errors/DishError";
+import OperationError from "./errors/OperationError.mjs";
+import Operation from "./Operation.mjs";
+import DishError from "./errors/DishError.mjs";
 import log from "loglevel";
+import { isWorkerEnvironment } from "./Utils.mjs";
 
 // Cache container for modules
 let modules = null;
@@ -61,7 +62,7 @@ class Recipe  {
         if (!modules) {
             // Using Webpack Magic Comments to force the dynamic import to be included in the main chunk
             // https://webpack.js.org/api/module-methods/
-            modules = await import(/* webpackMode: "eager" */ "./config/modules/OpModules");
+            modules = await import(/* webpackMode: "eager" */ "./config/modules/OpModules.mjs");
             modules = modules.default;
         }
 
@@ -200,7 +201,12 @@ class Recipe  {
 
             try {
                 input = await dish.get(op.inputType);
-                log.debug("Executing operation");
+                log.debug(`Executing operation '${op.name}'`);
+
+                if (isWorkerEnvironment()) {
+                    self.sendStatusMessage(`Baking... (${i+1}/${this.opList.length})`);
+                    self.sendProgressMessage(i + 1, this.opList.length);
+                }
 
                 if (op.flowControl) {
                     // Package up the current state

src/core/Utils.mjs

@@ -5,11 +5,10 @@
  */
 
 import utf8 from "utf8";
-import {fromBase64, toBase64} from "./lib/Base64";
-import {fromHex} from "./lib/Hex";
-import {fromDecimal} from "./lib/Decimal";
-import {fromBinary} from "./lib/Binary";
-
+import {fromBase64, toBase64} from "./lib/Base64.mjs";
+import {fromHex} from "./lib/Hex.mjs";
+import {fromDecimal} from "./lib/Decimal.mjs";
+import {fromBinary} from "./lib/Binary.mjs";
 
 /**
  * Utility functions for use in operations, the core framework and the stage.
@@ -95,7 +94,7 @@ class Utils {
         const paddedBytes = new Array(numBytes);
         paddedBytes.fill(padByte);
 
-        Array.prototype.map.call(arr, function(b, i) {
+        [...arr].forEach((b, i) => {
             paddedBytes[i] = b;
         });
 
@@ -174,10 +173,11 @@ class Utils {
      * @returns {string}
      */
     static printable(str, preserveWs=false) {
-        if (ENVIRONMENT_IS_WEB() && window.app && !window.app.options.treatAsUtf8) {
+        if (isWebEnvironment() && window.app && !window.app.options.treatAsUtf8) {
             str = Utils.byteArrayToChars(Utils.strToByteArray(str));
         }
 
+        // eslint-disable-next-line no-misleading-character-class
         const re = /[\0-\x08\x0B-\x0C\x0E-\x1F\x7F-\x9F\xAD\u0378\u0379\u037F-\u0383\u038B\u038D\u03A2\u0528-\u0530\u0557\u0558\u0560\u0588\u058B-\u058E\u0590\u05C8-\u05CF\u05EB-\u05EF\u05F5-\u0605\u061C\u061D\u06DD\u070E\u070F\u074B\u074C\u07B2-\u07BF\u07FB-\u07FF\u082E\u082F\u083F\u085C\u085D\u085F-\u089F\u08A1\u08AD-\u08E3\u08FF\u0978\u0980\u0984\u098D\u098E\u0991\u0992\u09A9\u09B1\u09B3-\u09B5\u09BA\u09BB\u09C5\u09C6\u09C9\u09CA\u09CF-\u09D6\u09D8-\u09DB\u09DE\u09E4\u09E5\u09FC-\u0A00\u0A04\u0A0B-\u0A0E\u0A11\u0A12\u0A29\u0A31\u0A34\u0A37\u0A3A\u0A3B\u0A3D\u0A43-\u0A46\u0A49\u0A4A\u0A4E-\u0A50\u0A52-\u0A58\u0A5D\u0A5F-\u0A65\u0A76-\u0A80\u0A84\u0A8E\u0A92\u0AA9\u0AB1\u0AB4\u0ABA\u0ABB\u0AC6\u0ACA\u0ACE\u0ACF\u0AD1-\u0ADF\u0AE4\u0AE5\u0AF2-\u0B00\u0B04\u0B0D\u0B0E\u0B11\u0B12\u0B29\u0B31\u0B34\u0B3A\u0B3B\u0B45\u0B46\u0B49\u0B4A\u0B4E-\u0B55\u0B58-\u0B5B\u0B5E\u0B64\u0B65\u0B78-\u0B81\u0B84\u0B8B-\u0B8D\u0B91\u0B96-\u0B98\u0B9B\u0B9D\u0BA0-\u0BA2\u0BA5-\u0BA7\u0BAB-\u0BAD\u0BBA-\u0BBD\u0BC3-\u0BC5\u0BC9\u0BCE\u0BCF\u0BD1-\u0BD6\u0BD8-\u0BE5\u0BFB-\u0C00\u0C04\u0C0D\u0C11\u0C29\u0C34\u0C3A-\u0C3C\u0C45\u0C49\u0C4E-\u0C54\u0C57\u0C5A-\u0C5F\u0C64\u0C65\u0C70-\u0C77\u0C80\u0C81\u0C84\u0C8D\u0C91\u0CA9\u0CB4\u0CBA\u0CBB\u0CC5\u0CC9\u0CCE-\u0CD4\u0CD7-\u0CDD\u0CDF\u0CE4\u0CE5\u0CF0\u0CF3-\u0D01\u0D04\u0D0D\u0D11\u0D3B\u0D3C\u0D45\u0D49\u0D4F-\u0D56\u0D58-\u0D5F\u0D64\u0D65\u0D76-\u0D78\u0D80\u0D81\u0D84\u0D97-\u0D99\u0DB2\u0DBC\u0DBE\u0DBF\u0DC7-\u0DC9\u0DCB-\u0DCE\u0DD5\u0DD7\u0DE0-\u0DF1\u0DF5-\u0E00\u0E3B-\u0E3E\u0E5C-\u0E80\u0E83\u0E85\u0E86\u0E89\u0E8B\u0E8C\u0E8E-\u0E93\u0E98\u0EA0\u0EA4\u0EA6\u0EA8\u0EA9\u0EAC\u0EBA\u0EBE\u0EBF\u0EC5\u0EC7\u0ECE\u0ECF\u0EDA\u0EDB\u0EE0-\u0EFF\u0F48\u0F6D-\u0F70\u0F98\u0FBD\u0FCD\u0FDB-\u0FFF\u10C6\u10C8-\u10CC\u10CE\u10CF\u1249\u124E\u124F\u1257\u1259\u125E\u125F\u1289\u128E\u128F\u12B1\u12B6\u12B7\u12BF\u12C1\u12C6\u12C7\u12D7\u1311\u1316\u1317\u135B\u135C\u137D-\u137F\u139A-\u139F\u13F5-\u13FF\u169D-\u169F\u16F1-\u16FF\u170D\u1715-\u171F\u1737-\u173F\u1754-\u175F\u176D\u1771\u1774-\u177F\u17DE\u17DF\u17EA-\u17EF\u17FA-\u17FF\u180F\u181A-\u181F\u1878-\u187F\u18AB-\u18AF\u18F6-\u18FF\u191D-\u191F\u192C-\u192F\u193C-\u193F\u1941-\u1943\u196E\u196F\u1975-\u197F\u19AC-\u19AF\u19CA-\u19CF\u19DB-\u19DD\u1A1C\u1A1D\u1A5F\u1A7D\u1A7E\u1A8A-\u1A8F\u1A9A-\u1A9F\u1AAE-\u1AFF\u1B4C-\u1B4F\u1B7D-\u1B7F\u1BF4-\u1BFB\u1C38-\u1C3A\u1C4A-\u1C4C\u1C80-\u1CBF\u1CC8-\u1CCF\u1CF7-\u1CFF\u1DE7-\u1DFB\u1F16\u1F17\u1F1E\u1F1F\u1F46\u1F47\u1F4E\u1F4F\u1F58\u1F5A\u1F5C\u1F5E\u1F7E\u1F7F\u1FB5\u1FC5\u1FD4\u1FD5\u1FDC\u1FF0\u1FF1\u1FF5\u1FFF\u200B-\u200F\u202A-\u202E\u2060-\u206F\u2072\u2073\u208F\u209D-\u209F\u20BB-\u20CF\u20F1-\u20FF\u218A-\u218F\u23F4-\u23FF\u2427-\u243F\u244B-\u245F\u2700\u2B4D-\u2B4F\u2B5A-\u2BFF\u2C2F\u2C5F\u2CF4-\u2CF8\u2D26\u2D28-\u2D2C\u2D2E\u2D2F\u2D68-\u2D6E\u2D71-\u2D7E\u2D97-\u2D9F\u2DA7\u2DAF\u2DB7\u2DBF\u2DC7\u2DCF\u2DD7\u2DDF\u2E3C-\u2E7F\u2E9A\u2EF4-\u2EFF\u2FD6-\u2FEF\u2FFC-\u2FFF\u3040\u3097\u3098\u3100-\u3104\u312E-\u3130\u318F\u31BB-\u31BF\u31E4-\u31EF\u321F\u32FF\u4DB6-\u4DBF\u9FCD-\u9FFF\uA48D-\uA48F\uA4C7-\uA4CF\uA62C-\uA63F\uA698-\uA69E\uA6F8-\uA6FF\uA78F\uA794-\uA79F\uA7AB-\uA7F7\uA82C-\uA82F\uA83A-\uA83F\uA878-\uA87F\uA8C5-\uA8CD\uA8DA-\uA8DF\uA8FC-\uA8FF\uA954-\uA95E\uA97D-\uA97F\uA9CE\uA9DA-\uA9DD\uA9E0-\uA9FF\uAA37-\uAA3F\uAA4E\uAA4F\uAA5A\uAA5B\uAA7C-\uAA7F\uAAC3-\uAADA\uAAF7-\uAB00\uAB07\uAB08\uAB0F\uAB10\uAB17-\uAB1F\uAB27\uAB2F-\uABBF\uABEE\uABEF\uABFA-\uABFF\uD7A4-\uD7AF\uD7C7-\uD7CA\uD7FC-\uD7FF\uE000-\uF8FF\uFA6E\uFA6F\uFADA-\uFAFF\uFB07-\uFB12\uFB18-\uFB1C\uFB37\uFB3D\uFB3F\uFB42\uFB45\uFBC2-\uFBD2\uFD40-\uFD4F\uFD90\uFD91\uFDC8-\uFDEF\uFDFE\uFDFF\uFE1A-\uFE1F\uFE27-\uFE2F\uFE53\uFE67\uFE6C-\uFE6F\uFE75\uFEFD-\uFF00\uFFBF-\uFFC1\uFFC8\uFFC9\uFFD0\uFFD1\uFFD8\uFFD9\uFFDD-\uFFDF\uFFE7\uFFEF-\uFFFB\uFFFE\uFFFF]/g;
         const wsRe = /[\x09-\x10\x0D\u2028\u2029]/g;
 
@@ -201,9 +201,8 @@ class Utils {
      * Utils.parseEscapedChars("\\n");
      */
     static parseEscapedChars(str) {
-        return str.replace(/(\\)?\\([bfnrtv'"]|[0-3][0-7]{2}|[0-7]{1,2}|x[\da-fA-F]{2}|u[\da-fA-F]{4}|u\{[\da-fA-F]{1,6}\}|\\)/g, function(m, a, b) {
-            if (a === "\\") return "\\"+b;
-            switch (b[0]) {
+        return str.replace(/\\([bfnrtv'"]|[0-3][0-7]{2}|[0-7]{1,2}|x[\da-fA-F]{2}|u[\da-fA-F]{4}|u\{[\da-fA-F]{1,6}\}|\\)/g, function(m, a) {
+            switch (a[0]) {
                 case "\\":
                     return "\\";
                 case "0":
@@ -214,7 +213,7 @@ class Utils {
                 case "5":
                 case "6":
                 case "7":
-                    return String.fromCharCode(parseInt(b, 8));
+                    return String.fromCharCode(parseInt(a, 8));
                 case "b":
                     return "\b";
                 case "t":
@@ -232,12 +231,12 @@ class Utils {
                 case "'":
                     return "'";
                 case "x":
-                    return String.fromCharCode(parseInt(b.substr(1), 16));
+                    return String.fromCharCode(parseInt(a.substr(1), 16));
                 case "u":
-                    if (b[1] === "{")
-                        return String.fromCodePoint(parseInt(b.slice(2, -1), 16));
+                    if (a[1] === "{")
+                        return String.fromCodePoint(parseInt(a.slice(2, -1), 16));
                     else
-                        return String.fromCharCode(parseInt(b.substr(1), 16));
+                        return String.fromCharCode(parseInt(a.substr(1), 16));
             }
         });
     }
@@ -420,9 +419,9 @@ class Utils {
         const utf8Str = utf8.encode(str);
 
         if (str.length !== utf8Str.length) {
-            if (ENVIRONMENT_IS_WORKER()) {
+            if (isWorkerEnvironment()) {
                 self.setOption("attemptHighlight", false);
-            } else if (ENVIRONMENT_IS_WEB()) {
+            } else if (isWebEnvironment()) {
                 window.app.options.attemptHighlight = false;
             }
         }
@@ -475,9 +474,9 @@ class Utils {
         const utf8Str = utf8.encode(str);
 
         if (str.length !== utf8Str.length) {
-            if (ENVIRONMENT_IS_WORKER()) {
+            if (isWorkerEnvironment()) {
                 self.setOption("attemptHighlight", false);
-            } else if (ENVIRONMENT_IS_WEB()) {
+            } else if (isWebEnvironment()) {
                 window.app.options.attemptHighlight = false;
             }
         }
@@ -537,11 +536,10 @@ class Utils {
         const str = Utils.byteArrayToChars(byteArray);
         try {
             const utf8Str = utf8.decode(str);
-
             if (str.length !== utf8Str.length) {
-                if (ENVIRONMENT_IS_WORKER()) {
+                if (isWorkerEnvironment()) {
                     self.setOption("attemptHighlight", false);
-                } else if (ENVIRONMENT_IS_WEB()) {
+                } else if (isWebEnvironment()) {
                     window.app.options.attemptHighlight = false;
                 }
             }
@@ -593,6 +591,44 @@ class Utils {
         return utf8 ? Utils.byteArrayToUtf8(arr) : Utils.byteArrayToChars(arr);
     }
 
+    /**
+     * Calculates the Shannon entropy for a given set of data.
+     *
+     * @param {Uint8Array|ArrayBuffer} input
+     * @returns {number}
+     */
+    static calculateShannonEntropy(data) {
+        if (data instanceof ArrayBuffer) {
+            data = new Uint8Array(data);
+        }
+        const prob = [],
+            occurrences = new Array(256).fill(0);
+
+        // Count occurrences of each byte in the input
+        let i;
+        for (i = 0; i < data.length; i++) {
+            occurrences[data[i]]++;
+        }
+
+        // Store probability list
+        for (i = 0; i < occurrences.length; i++) {
+            if (occurrences[i] > 0) {
+                prob.push(occurrences[i] / data.length);
+            }
+        }
+
+        // Calculate Shannon entropy
+        let entropy = 0,
+            p;
+
+        for (i = 0; i < prob.length; i++) {
+            p = prob[i];
+            entropy += p * Math.log(p) / Math.log(2);
+        }
+
+        return -entropy;
+    }
+
 
     /**
      * Parses CSV data and returns it as a two dimensional array or strings.
@@ -760,15 +796,15 @@ class Utils {
             "%7E": "~",
             "%21": "!",
             "%24": "$",
-            //"%26": "&",
+            // "%26": "&",
             "%27": "'",
             "%28": "(",
             "%29": ")",
             "%2A": "*",
-            //"%2B": "+",
+            // "%2B": "+",
             "%2C": ",",
             "%3B": ";",
-            //"%3D": "=",
+            // "%3D": "=",
             "%3A": ":",
             "%40": "@",
             "%2F": "/",
@@ -994,7 +1030,7 @@ class Utils {
     /**
      * Reads a File and returns the data as a Uint8Array.
      *
-     * @param {File} file
+     * @param {File | for node: array|arrayBuffer|buffer|string} file
      * @returns {Uint8Array}
      *
      * @example
@@ -1002,33 +1038,57 @@ class Utils {
      * await Utils.readFile(new File(["hello"], "test"))
      */
     static readFile(file) {
-        return new Promise((resolve, reject) => {
-            const reader = new FileReader();
-            const data = new Uint8Array(file.size);
-            let offset = 0;
-            const CHUNK_SIZE = 10485760; // 10MiB
 
-            const seek = function() {
-                if (offset >= file.size) {
-                    resolve(data);
-                    return;
-                }
-                const slice = file.slice(offset, offset + CHUNK_SIZE);
-                reader.readAsArrayBuffer(slice);
-            };
+        if (isNodeEnvironment()) {
+            return Buffer.from(file).buffer;
+
+        } else {
+            return new Promise((resolve, reject) => {
+                const reader = new FileReader();
+                const data = new Uint8Array(file.size);
+                let offset = 0;
+                const CHUNK_SIZE = 10485760; // 10MiB
+
+                const seek = function() {
+                    if (offset >= file.size) {
+                        resolve(data);
+                        return;
+                    }
+                    const slice = file.slice(offset, offset + CHUNK_SIZE);
+                    reader.readAsArrayBuffer(slice);
+                };
+
+                reader.onload = function(e) {
+                    data.set(new Uint8Array(reader.result), offset);
+                    offset += CHUNK_SIZE;
+                    seek();
+                };
+
+                reader.onerror = function(e) {
+                    reject(reader.error.message);
+                };
 
-            reader.onload = function(e) {
-                data.set(new Uint8Array(reader.result), offset);
-                offset += CHUNK_SIZE;
                 seek();
-            };
+            });
+        }
+    }
 
-            reader.onerror = function(e) {
-                reject(reader.error.message);
-            };
+    /**
+     * Synchronously read the raw data from a File object.
+     *
+     * Only works in the Node environment
+     *
+     * @param {File} file - a File shim object (see src/node/File.mjs)
+     * @returns {ArrayBuffer} the data from the file in an ArrayBuffer
+     * @throws {TypeError} thrown if the method is called from a browser environment
+     */
+    static readFileSync(file) {
+        if (!isNodeEnvironment()) {
+            throw new TypeError("Browser environment cannot support readFileSync");
+        }
 
-            seek();
-        });
+        const arrayBuffer = Uint8Array.from(file.data);
+        return arrayBuffer.buffer;
     }
 
 
@@ -1130,6 +1190,30 @@ class Utils {
 
 }
 
+/**
+ * Check whether the code is running in a Node.js environment
+ * @returns {boolean}
+ */
+export function isNodeEnvironment() {
+    return typeof process !== "undefined" && process.versions != null && process.versions.node != null;
+}
+
+/**
+ * Check whether the code is running in a web environment
+ * @returns {boolean}
+*/
+export function isWebEnvironment() {
+    return typeof window === "object";
+}
+
+/**
+ * Check whether the code is running in a worker
+ * @returns {boolean}
+*/
+export function isWorkerEnvironment() {
+    return typeof importScripts === "function";
+}
+
 export default Utils;
 
 
@@ -1148,7 +1232,7 @@ export default Utils;
 Array.prototype.unique = function() {
     const u = {}, a = [];
     for (let i = 0, l = this.length; i < l; i++) {
-        if (u.hasOwnProperty(this[i])) {
+        if (Object.prototype.hasOwnProperty.call(u, this[i])) {
             continue;
         }
         a.push(this[i]);
@@ -1247,12 +1331,37 @@ String.prototype.count = function(chr) {
  * @param {string} msg
  */
 export function sendStatusMessage(msg) {
-    if (ENVIRONMENT_IS_WORKER())
+    if (isWorkerEnvironment())
         self.sendStatusMessage(msg);
-    else if (ENVIRONMENT_IS_WEB())
+    else if (isWebEnvironment())
         app.alert(msg, 10000);
-    else if (ENVIRONMENT_IS_NODE())
-        log.debug(msg);
+    else if (isNodeEnvironment())
+        // eslint-disable-next-line no-console
+        console.debug(msg);
+}
+
+const debounceTimeouts = {};
+
+/**
+ * Debouncer to stop functions from being executed multiple times in a
+ * short space of time
+ * https://davidwalsh.name/javascript-debounce-function
+ *
+ * @param {function} func - The function to be executed after the debounce time
+ * @param {number} wait - The time (ms) to wait before executing the function
+ * @param {string} id - Unique ID to reference the timeout for the function
+ * @param {object} scope - The object to bind to the debounced function
+ * @param {array} args - Array of arguments to be passed to func
+ * @returns {function}
+ */
+export function debounce(func, wait, id, scope, args) {
+    return function() {
+        const later = function() {
+            func.apply(scope, args);
+        };
+        clearTimeout(debounceTimeouts[id]);
+        debounceTimeouts[id] = setTimeout(later, wait);
+    };
 }
 
 
@@ -1264,14 +1373,14 @@ export function sendStatusMessage(msg) {
 // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/padStart
 if (!String.prototype.padStart) {
     String.prototype.padStart = function padStart(targetLength, padString) {
-        targetLength = targetLength>>0; //floor if number or convert non-number to 0;
+        targetLength = targetLength>>0; // floor if number or convert non-number to 0;
         padString = String((typeof padString !== "undefined" ? padString : " "));
         if (this.length > targetLength) {
             return String(this);
         } else {
             targetLength = targetLength-this.length;
             if (targetLength > padString.length) {
-                padString += padString.repeat(targetLength/padString.length); //append to original to ensure we are longer than needed
+                padString += padString.repeat(targetLength/padString.length); // append to original to ensure we are longer than needed
             }
             return padString.slice(0, targetLength) + String(this);
         }
@@ -1283,14 +1392,14 @@ if (!String.prototype.padStart) {
 // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/padEnd
 if (!String.prototype.padEnd) {
     String.prototype.padEnd = function padEnd(targetLength, padString) {
-        targetLength = targetLength>>0; //floor if number or convert non-number to 0;
+        targetLength = targetLength>>0; // floor if number or convert non-number to 0;
         padString = String((typeof padString !== "undefined" ? padString : " "));
         if (this.length > targetLength) {
             return String(this);
         } else {
             targetLength = targetLength-this.length;
             if (targetLength > padString.length) {
-                padString += padString.repeat(targetLength/padString.length); //append to original to ensure we are longer than needed
+                padString += padString.repeat(targetLength/padString.length); // append to original to ensure we are longer than needed
             }
             return String(this) + padString.slice(0, targetLength);
         }

src/core/config/Categories.json

@@ -59,7 +59,8 @@
             "From Braille",
             "Parse TLV",
             "CSV to JSON",
-            "JSON to CSV"
+            "JSON to CSV",
+            "Avro to JSON"
         ]
     },
     {
@@ -85,6 +86,8 @@
             "Vigenère Decode",
             "To Morse Code",
             "From Morse Code",
+            "Bacon Cipher Encode",
+            "Bacon Cipher Decode",
             "Bifid Cipher Encode",
             "Bifid Cipher Decode",
             "Affine Cipher Encode",
@@ -106,7 +109,8 @@
             "Enigma",
             "Bombe",
             "Multiple Bombe",
-            "Typex"
+            "Typex",
+            "Lorenz"
         ]
     },
     {
@@ -121,8 +125,10 @@
             "Generate PGP Key Pair",
             "PGP Encrypt",
             "PGP Decrypt",
+            "PGP Verify",
             "PGP Encrypt and Sign",
-            "PGP Decrypt and Verify"
+            "PGP Decrypt and Verify",
+            "Parse SSH Host Key"
         ]
     },
     {
@@ -166,6 +172,8 @@
             "Parse IP range",
             "Parse IPv6 address",
             "Parse IPv4 header",
+            "Parse UDP",
+            "Parse SSH Host Key",
             "Parse URI",
             "URL Encode",
             "URL Decode",
@@ -177,7 +185,8 @@
             "Group IP addresses",
             "Encode NetBIOS Name",
             "Decode NetBIOS Name",
-            "Defang URL"
+            "Defang URL",
+            "Defang IP Addresses"
         ]
     },
     {
@@ -197,8 +206,8 @@
             "Remove null bytes",
             "To Upper case",
             "To Lower case",
-			"To Case Insensitive Regex",
-			"From Case Insensitive Regex",
+            "To Case Insensitive Regex",
+            "From Case Insensitive Regex",
             "Add line numbers",
             "Remove line numbers",
             "To Table",
@@ -224,6 +233,7 @@
             "Convert speed",
             "Convert data units",
             "Convert co-ordinate format",
+            "Show on map",
             "Parse UNIX file permissions",
             "Swap endianness",
             "Parse colour code",
@@ -277,6 +287,7 @@
             "Zip",
             "Unzip",
             "Bzip2 Decompress",
+            "Bzip2 Compress",
             "Tar",
             "Untar"
         ]
@@ -302,6 +313,8 @@
             "Snefru",
             "BLAKE2b",
             "BLAKE2s",
+            "GOST hash",
+            "Streebog",
             "SSDEEP",
             "CTPH",
             "Compare SSDEEP hashes",
@@ -351,7 +364,8 @@
             "BSON serialise",
             "BSON deserialise",
             "To MessagePack",
-            "From MessagePack"
+            "From MessagePack",
+            "Render Markdown"
         ]
     },
     {
@@ -360,8 +374,13 @@
             "Detect File Type",
             "Scan for Embedded Files",
             "Extract Files",
+            "YARA Rules",
             "Remove EXIF",
-            "Extract EXIF"
+            "Extract EXIF",
+            "Extract RGBA",
+            "View Bit Plane",
+            "Randomize Colour Palette",
+            "Extract LSB"
         ]
     },
     {
@@ -369,6 +388,7 @@
         "ops": [
             "Render Image",
             "Play Media",
+            "Optical Character Recognition",
             "Remove EXIF",
             "Extract EXIF",
             "Split Colour Channels",
@@ -386,6 +406,7 @@
             "Cover Image",
             "Image Hue/Saturation/Lightness",
             "Sharpen Image",
+            "Normalise Image",
             "Convert Image Format",
             "Add Text To Image",
             "Hex Density chart",

src/core/config/scripts/generateConfig.mjs

@@ -9,12 +9,12 @@
  * @license Apache-2.0
  */
 
-/*eslint no-console: ["off"] */
+/* eslint no-console: ["off"] */
 
 import path from "path";
 import fs  from "fs";
 import process from "process";
-import * as Ops from "../../operations/index";
+import * as Ops from "../../operations/index.mjs";
 
 const dir = path.join(process.cwd() + "/src/core/config/");
 if (!fs.existsSync(dir)) {
@@ -45,11 +45,11 @@ for (const opObj in Ops) {
         args:        op.args
     };
 
-    if (op.hasOwnProperty("patterns")) {
+    if ("patterns" in op) {
         operationConfig[op.name].patterns = op.patterns;
     }
 
-    if (!modules.hasOwnProperty(op.module))
+    if (!(op.module in modules))
         modules[op.module] = {};
     modules[op.module][op.name] = opObj;
 }
@@ -84,7 +84,7 @@ for (const module in modules) {
 
     for (const opName in modules[module]) {
         const objName = modules[module][opName];
-        code += `import ${objName} from "../../operations/${objName}";\n`;
+        code += `import ${objName} from "../../operations/${objName}.mjs";\n`;
     }
 
     code += `
@@ -124,7 +124,7 @@ let opModulesCode = `/**
 `;
 
 for (const module in modules) {
-    opModulesCode += `import ${module}Module from "./${module}";\n`;
+    opModulesCode += `import ${module}Module from "./${module}.mjs";\n`;
 }
 
 opModulesCode += `

src/core/config/scripts/generateOpsIndex.mjs

@@ -7,7 +7,7 @@
  * @license Apache-2.0
  */
 
-/*eslint no-console: ["off"] */
+/* eslint no-console: ["off"] */
 
 import path from "path";
 import fs  from "fs";
@@ -39,7 +39,7 @@ let code = `/**
 `;
 
 opObjs.forEach(obj => {
-    code += `import ${obj} from "./${obj}";\n`;
+    code += `import ${obj} from "./${obj}.mjs";\n`;
 });
 
 code += `

src/core/config/scripts/newOperation.mjs

@@ -6,14 +6,14 @@
  * @license Apache-2.0
  */
 
-/*eslint no-console: ["off"] */
+/* eslint no-console: ["off"] */
 
 import prompt from "prompt";
 import colors from "colors";
 import process from "process";
 import fs from "fs";
 import path from "path";
-import EscapeString from "../../operations/EscapeString";
+import EscapeString from "../../operations/EscapeString.mjs";
 
 
 const dir = path.join(process.cwd() + "/src/core/operations/");
@@ -130,8 +130,8 @@ prompt.get(schema, (err, result) => {
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import OperationError from "../errors/OperationError";
+import Operation from "../Operation.mjs";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * ${result.opName} operation
@@ -208,7 +208,7 @@ ${result.highlight ? `
 export default ${moduleName};
 `;
 
-    //console.log(template);
+    // console.log(template);
 
     const filename = path.join(dir, `./${moduleName}.mjs`);
     if (fs.existsSync(filename)) {

src/core/dishTypes/DishBigNumber.mjs

@@ -0,0 +1,39 @@
+/**
+ * @author d98762625 [d98762625@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+import DishType from "./DishType.mjs";
+import Utils from "../Utils.mjs";
+import BigNumber from "bignumber.js";
+
+/**
+ * translation methods for BigNumber Dishes
+ */
+class DishBigNumber extends DishType {
+
+    /**
+     * convert the given value to a ArrayBuffer
+     * @param {BigNumber} value
+     */
+    static toArrayBuffer() {
+        DishBigNumber.checkForValue(this.value);
+        this.value = BigNumber.isBigNumber(this.value) ? Utils.strToArrayBuffer(this.value.toFixed()) : new ArrayBuffer;
+    }
+
+    /**
+     * convert the given value from a ArrayBuffer
+     * @param {boolean} notUTF8
+     */
+    static fromArrayBuffer(notUTF8) {
+        DishBigNumber.checkForValue(this.value);
+        try {
+            this.value = new BigNumber(Utils.arrayBufferToStr(this.value, !notUTF8));
+        } catch (err) {
+            this.value = new BigNumber(NaN);
+        }
+    }
+}
+
+export default DishBigNumber;

src/core/dishTypes/DishByteArray.mjs

@@ -0,0 +1,31 @@
+/**
+ * @author d98762625 [d98762625@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+import DishType from "./DishType.mjs";
+
+/**
+ * Translation methods for ArrayBuffer Dishes
+ */
+class DishByteArray extends DishType {
+
+    /**
+     * convert the given value to a ArrayBuffer
+     */
+    static toArrayBuffer() {
+        DishByteArray.checkForValue(this.value);
+        this.value = new Uint8Array(this.value).buffer;
+    }
+
+    /**
+     * convert the given value from a ArrayBuffer
+     */
+    static fromArrayBuffer() {
+        DishByteArray.checkForValue(this.value);
+        this.value = Array.prototype.slice.call(new Uint8Array(this.value));
+    }
+}
+
+export default DishByteArray;

src/core/dishTypes/DishFile.mjs

@@ -0,0 +1,42 @@
+/**
+ * @author d98762625 [d98762625@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+import DishType from "./DishType.mjs";
+import Utils, { isNodeEnvironment } from "../Utils.mjs";
+
+/**
+ * Translation methods for file Dishes
+ */
+class DishFile extends DishType {
+
+    /**
+     * convert the given value to an ArrayBuffer
+     * @param {File} value
+     */
+    static toArrayBuffer() {
+        DishFile.checkForValue(this.value);
+        if (isNodeEnvironment()) {
+            this.value = Utils.readFileSync(this.value);
+        } else {
+            return new Promise((resolve, reject) => {
+                Utils.readFile(this.value)
+                    .then(v => this.value = v.buffer)
+                    .then(resolve)
+                    .catch(reject);
+            });
+        }
+    }
+
+    /**
+     * convert the given value from an ArrayBuffer
+     */
+    static fromArrayBuffer() {
+        DishFile.checkForValue(this.value);
+        this.value = new File(this.value, "unknown");
+    }
+}
+
+export default DishFile;

src/core/dishTypes/DishHTML.mjs

@@ -0,0 +1,26 @@
+/**
+ * @author d98762625 [d98762625@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+import DishString from "./DishString.mjs";
+import Utils from "../Utils.mjs";
+
+/**
+ * Translation methods for HTML Dishes
+ */
+class DishHTML extends DishString {
+
+    /**
+     * convert the given value to a ArrayBuffer
+     * @param {String} value
+     */
+    static toArrayBuffer() {
+        DishHTML.checkForValue(this.value);
+        this.value = this.value ? Utils.strToArrayBuffer(Utils.unescapeHtml(Utils.stripHtmlTags(this.value, true))) : new ArrayBuffer;
+    }
+
+}
+
+export default DishHTML;

src/core/dishTypes/DishJSON.mjs

@@ -0,0 +1,33 @@
+/**
+ * @author d98762625 [d98762625@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+import DishType from "./DishType.mjs";
+import Utils from "../Utils.mjs";
+
+/**
+ * Translation methods for JSON dishes
+ */
+class DishJSON extends DishType {
+
+    /**
+     * convert the given value to a ArrayBuffer
+     */
+    static toArrayBuffer() {
+        DishJSON.checkForValue(this.value);
+        this.value = this.value ? Utils.strToArrayBuffer(JSON.stringify(this.value, null, 4)) : new ArrayBuffer;
+    }
+
+    /**
+     * convert the given value from a ArrayBuffer
+     * @param {boolean} notUTF8
+     */
+    static fromArrayBuffer(notUTF8) {
+        DishJSON.checkForValue(this.value);
+        this.value = JSON.parse(Utils.arrayBufferToStr(this.value, !notUTF8));
+    }
+}
+
+export default DishJSON;

src/core/dishTypes/DishListFile.mjs

@@ -0,0 +1,58 @@
+/**
+ * @author d98762625 [d98762625@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+import DishType from "./DishType.mjs";
+import { isNodeEnvironment } from "../Utils.mjs";
+
+
+/**
+ * Translation methods for ListFile Dishes
+ */
+class DishListFile extends DishType {
+
+    /**
+     * convert the given value to a ArrayBuffer
+     */
+    static toArrayBuffer() {
+        DishListFile.checkForValue(this.value);
+
+        if (isNodeEnvironment()) {
+            this.value = this.value.map(file => Uint8Array.from(file.data));
+        }
+        this.value = DishListFile.concatenateTypedArrays(...this.value).buffer;
+    }
+
+    /**
+     * convert the given value from a ArrayBuffer
+     */
+    static fromArrayBuffer() {
+        DishListFile.checkForValue(this.value);
+        this.value = [new File(this.value, "unknown")];
+    }
+
+
+    /**
+     * Concatenates a list of Uint8Arrays together
+     *
+     * @param {Uint8Array[]} arrays
+     * @returns {Uint8Array}
+     */
+    static concatenateTypedArrays(...arrays) {
+        let totalLength = 0;
+        for (const arr of arrays) {
+            totalLength += arr.length;
+        }
+        const result = new Uint8Array(totalLength);
+        let offset = 0;
+        for (const arr of arrays) {
+            result.set(arr, offset);
+            offset += arr.length;
+        }
+        return result;
+    }
+}
+
+export default DishListFile;

src/core/dishTypes/DishNumber.mjs

@@ -0,0 +1,34 @@
+/**
+ * @author d98762625 [d98762625@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+
+import DishType from "./DishType.mjs";
+import Utils from "../Utils.mjs";
+
+/**
+ * Translation methods for number dishes
+ */
+class DishNumber extends DishType {
+
+    /**
+     * convert the given value to a ArrayBuffer
+     */
+    static toArrayBuffer() {
+        DishNumber.checkForValue(this.value);
+        this.value = typeof this.value === "number" ? Utils.strToArrayBuffer(this.value.toString()) : new ArrayBuffer;
+    }
+
+    /**
+     * convert the given value from a ArrayBuffer
+     * @param {boolean} notUTF8
+     */
+    static fromArrayBuffer(notUTF8) {
+        DishNumber.checkForValue(this.value);
+        this.value = this.value ? parseFloat(Utils.arrayBufferToStr(this.value, !notUTF8)) : 0;
+    }
+}
+
+export default DishNumber;

src/core/dishTypes/DishString.mjs

@@ -0,0 +1,34 @@
+/**
+ * @author d98762625 [d98762625@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+
+import DishType from "./DishType.mjs";
+import Utils from "../Utils.mjs";
+
+/**
+ * Translation methods for string dishes
+ */
+class DishString extends DishType {
+
+    /**
+     * convert the given value to a ArrayBuffer
+     */
+    static toArrayBuffer() {
+        DishString.checkForValue(this.value);
+        this.value = this.value ? Utils.strToArrayBuffer(this.value) : new ArrayBuffer;
+    }
+
+    /**
+     * convert the given value from a ArrayBuffer
+     * @param {boolean} notUTF8
+     */
+    static fromArrayBuffer(notUTF8) {
+        DishString.checkForValue(this.value);
+        this.value = this.value ? Utils.arrayBufferToStr(this.value, !notUTF8) : "";
+    }
+}
+
+export default DishString;

src/core/dishTypes/DishType.mjs

@@ -0,0 +1,39 @@
+/**
+ * @author d98762625 [d98762625@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+
+/**
+ * Abstract class for dish translation methods
+ */
+class DishType {
+
+    /**
+     * Warn translations dont work without value from bind
+     */
+    static checkForValue(value) {
+        if (value === undefined) {
+            throw new Error("only use translation methods with .bind");
+        }
+    }
+
+    /**
+     * convert the given value to a ArrayBuffer
+     * @param {*} value
+     */
+    static toArrayBuffer() {
+        throw new Error("toArrayBuffer has not been implemented");
+    }
+
+    /**
+     * convert the given value from a ArrayBuffer
+     * @param {boolean} notUTF8
+     */
+    static fromArrayBuffer(notUTF8=undefined) {
+        throw new Error("fromArrayBuffer has not been implemented");
+    }
+}
+
+export default DishType;

src/core/dishTypes/index.mjs

@@ -0,0 +1,26 @@
+/**
+ * @author d98762625 [d98762625@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+
+import DishByteArray from "./DishByteArray.mjs";
+import DishBigNumber from "./DishBigNumber.mjs";
+import DishFile from "./DishFile.mjs";
+import DishHTML from "./DishHTML.mjs";
+import DishJSON from "./DishJSON.mjs";
+import DishListFile from "./DishListFile.mjs";
+import DishNumber from "./DishNumber.mjs";
+import DishString from "./DishString.mjs";
+
+export {
+    DishByteArray,
+    DishBigNumber,
+    DishFile,
+    DishHTML,
+    DishJSON,
+    DishListFile,
+    DishNumber,
+    DishString,
+};

src/core/errors/ExcludedOperationError.mjs

@@ -0,0 +1,25 @@
+/**
+ * Custom error type for handling operation that isnt included in node.js API
+ *
+ * @author d98762625 [d98762625@gmail.com]
+ * @copyright Crown Copyright 2018
+ * @license Apache-2.0
+ */
+class ExcludedOperationError extends Error {
+    /**
+     * Standard error constructor. Adds no new behaviour.
+     *
+     * @param args - Standard error args
+     */
+    constructor(...args) {
+        super(...args);
+
+        this.type = "ExcludedOperationError";
+
+        if (Error.captureStackTrace) {
+            Error.captureStackTrace(this, ExcludedOperationError);
+        }
+    }
+}
+
+export default ExcludedOperationError;

src/core/errors/index.mjs

@@ -0,0 +1,9 @@
+import OperationError from "./OperationError.mjs";
+import DishError from "./DishError.mjs";
+import ExcludedOperationError from "./ExcludedOperationError";
+
+export {
+    OperationError,
+    DishError,
+    ExcludedOperationError,
+};

src/core/lib/Arithmetic.mjs

@@ -5,7 +5,7 @@
  * @license Apache-2.0
  */
 
-import Utils from "../Utils";
+import Utils from "../Utils.mjs";
 import BigNumber from "bignumber.js";
 
 
@@ -109,7 +109,7 @@ export function mean(data) {
  */
 export function median(data) {
     if ((data.length % 2) === 0 && data.length > 0) {
-        data.sort(function(a, b){
+        data.sort(function(a, b) {
             return a.minus(b);
         });
         const first = data[Math.floor(data.length / 2)];

src/core/lib/BCD.mjs

@@ -22,7 +22,7 @@ export const ENCODING_SCHEME = [
 /**
  * Lookup table for the binary value of each digit representation.
  *
- * I wrote a very nice algorithm to generate 8 4 2 1 encoding programatically,
+ * I wrote a very nice algorithm to generate 8 4 2 1 encoding programmatically,
  * but unfortunately it's much easier (if less elegant) to use lookup tables
  * when supporting multiple encoding schemes.
  *

src/core/lib/Bacon.mjs

@@ -0,0 +1,66 @@
+/**
+ * Bacon Cipher resources.
+ *
+ * @author Karsten Silkenbäumer [github.com/kassi]
+ * @copyright Karsten Silkenbäumer 2019
+ * @license Apache-2.0
+ */
+
+/**
+ * Bacon definitions.
+ */
+export const BACON_ALPHABETS = {
+    "Standard (I=J and U=V)": {
+        alphabet: "ABCDEFGHIKLMNOPQRSTUWXYZ",
+        codes: [0, 1, 2, 3, 4, 5, 6, 7, 8, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 19, 20, 21, 22, 23]
+    },
+    "Complete": {
+        alphabet: "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+    }
+};
+export const BACON_TRANSLATION_01 = "0/1";
+export const BACON_TRANSLATION_AB = "A/B";
+export const BACON_TRANSLATION_CASE = "Case";
+export const BACON_TRANSLATION_AMNZ = "A-M/N-Z first letter";
+export const BACON_TRANSLATIONS = [
+    BACON_TRANSLATION_01,
+    BACON_TRANSLATION_AB,
+    BACON_TRANSLATION_CASE,
+    BACON_TRANSLATION_AMNZ,
+];
+export const BACON_TRANSLATIONS_FOR_ENCODING = [
+    BACON_TRANSLATION_01,
+    BACON_TRANSLATION_AB
+];
+export const BACON_CLEARER_MAP = {
+    [BACON_TRANSLATION_01]: /[^01]/g,
+    [BACON_TRANSLATION_AB]: /[^ABab]/g,
+    [BACON_TRANSLATION_CASE]: /[^A-Za-z]/g,
+};
+export const BACON_NORMALIZE_MAP = {
+    [BACON_TRANSLATION_AB]: {
+        "A": "0",
+        "B": "1",
+        "a": "0",
+        "b": "1"
+    },
+};
+
+/**
+ * Swaps zeros to ones and ones to zeros.
+ *
+ * @param {string} data
+ * @returns {string}
+ *
+ * @example
+ * // returns "11001 01010"
+ * swapZeroAndOne("00110 10101");
+ */
+export function swapZeroAndOne(string) {
+    return string.replace(/[01]/g, function (c) {
+        return {
+            "0": "1",
+            "1": "0"
+        }[c];
+    });
+}

src/core/lib/Base64.mjs

@@ -6,13 +6,13 @@
  * @license Apache-2.0
  */
 
-import Utils from "../Utils";
-
+import Utils from "../Utils.mjs";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * Base64's the input byte array using the given alphabet, returning a string.
  *
- * @param {byteArray|Uint8Array|string} data
+ * @param {byteArray|Uint8Array|ArrayBuffer|string} data
  * @param {string} [alphabet="A-Za-z0-9+/="]
  * @returns {string}
  *
@@ -25,11 +25,17 @@ import Utils from "../Utils";
  */
 export function toBase64(data, alphabet="A-Za-z0-9+/=") {
     if (!data) return "";
+    if (data instanceof ArrayBuffer) {
+        data = new Uint8Array(data);
+    }
     if (typeof data == "string") {
         data = Utils.strToByteArray(data);
     }
 
     alphabet = Utils.expandAlphRange(alphabet).join("");
+    if (alphabet.length !== 64 && alphabet.length !== 65) { // Allow for padding
+        throw new OperationError(`Invalid Base64 alphabet length (${alphabet.length}): ${alphabet}`);
+    }
 
     let output = "",
         chr1, chr2, chr3,
@@ -63,7 +69,7 @@ export function toBase64(data, alphabet="A-Za-z0-9+/=") {
 /**
  * UnBase64's the input string using the given alphabet, returning a byte array.
  *
- * @param {byteArray} data
+ * @param {string} data
  * @param {string} [alphabet="A-Za-z0-9+/="]
  * @param {string} [returnType="string"] - Either "string" or "byteArray"
  * @param {boolean} [removeNonAlphChars=true]
@@ -83,6 +89,9 @@ export function fromBase64(data, alphabet="A-Za-z0-9+/=", returnType="string", r
 
     alphabet = alphabet || "A-Za-z0-9+/=";
     alphabet = Utils.expandAlphRange(alphabet).join("");
+    if (alphabet.length !== 64 && alphabet.length !== 65) { // Allow for padding
+        throw new OperationError(`Invalid Base64 alphabet length (${alphabet.length}): ${alphabet}`);
+    }
 
     const output = [];
     let chr1, chr2, chr3,

src/core/lib/Base85.mjs

@@ -20,7 +20,7 @@ export const ALPHABET_OPTIONS = [
     },
     {
         name: "IPv6",
-        value: "0-9A-Za-z!#$%&()*+\\-;<=>?@^_`{|~}",
+        value: "0-9A-Za-z!#$%&()*+\\-;<=>?@^_`{|}~",
     }
 ];
 

src/core/lib/Binary.mjs

@@ -6,7 +6,7 @@
  * @license Apache-2.0
  */
 
-import Utils from "../Utils";
+import Utils from "../Utils.mjs";
 
 
 /**

src/core/lib/BitwiseOp.mjs

@@ -9,7 +9,7 @@
 /**
  * Runs bitwise operations across the input data.
  *
- * @param {byteArray} input
+ * @param {byteArray|Uint8Array} input
  * @param {byteArray} key
  * @param {function} func - The bitwise calculation to carry out
  * @param {boolean} nullPreserving

src/core/lib/Blowfish.mjs

@@ -0,0 +1,436 @@
+/**
+  Blowfish.js from Dojo Toolkit 1.8.1 (https://github.com/dojo/dojox/tree/1.8/encoding)
+  Extracted by Sladex (xslade@gmail.com)
+  Shoehorned into working with mjs for CyberChef by Matt C (matt@artemisbot.uk)
+  Refactored and implemented modes support by cbeuw (cbeuw.andy@gmail.com)
+
+  @license BSD
+  ========================================================================
+  The "New" BSD License:
+  **********************
+
+  Copyright (c) 2005-2016, The Dojo Foundation
+  All rights reserved.
+
+  Redistribution and use in source and binary forms, with or without
+  modification, are permitted provided that the following conditions are met:
+
+  * Redistributions of source code must retain the above copyright notice, this
+    list of conditions and the following disclaimer.
+  * Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in the documentation
+    and/or other materials provided with the distribution.
+  * Neither the name of the Dojo Foundation nor the names of its contributors
+    may be used to endorse or promote products derived from this software
+    without specific prior written permission.
+
+  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+  ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+  DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
+  FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+  SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+  CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+  OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+const crypto = {};
+
+import forge from "node-forge/dist/forge.min.js";
+
+
+/* dojo-release-1.8.1/dojo/_base/lang.js.uncompressed.js */
+
+const lang = {};
+lang.isString = function(it) {
+	// summary:
+	//		Return true if it is a String
+	// it: anything
+	//		Item to test.
+    return (typeof it == "string" || it instanceof String); // Boolean
+};
+
+
+/* dojo-release-1.8.1/dojo/_base/array.js.uncompressed.js */
+
+const arrayUtil = {};
+arrayUtil.map = function(arr, callback, thisObject, Ctr) {
+	// summary:
+	//		applies callback to each element of arr and returns
+	//		an Array with the results
+	// arr: Array|String
+	//		the array to iterate on. If a string, operates on
+	//		individual characters.
+	// callback: Function
+	//		a function is invoked with three arguments, (item, index,
+	//		array),	 and returns a value
+	// thisObject: Object?
+	//		may be used to scope the call to callback
+	// returns: Array
+	// description:
+	//		This function corresponds to the JavaScript 1.6 Array.map() method, with one difference: when
+	//		run over sparse arrays, this implementation passes the "holes" in the sparse array to
+	//		the callback function with a value of undefined. JavaScript 1.6's map skips the holes in the sparse array.
+	//		For more details, see:
+	//		https://developer.mozilla.org/en/Core_JavaScript_1.5_Reference/Objects/Array/map
+	// example:
+	//	| // returns [2, 3, 4, 5]
+	//	| array.map([1, 2, 3, 4], function(item){ return item+1 });
+
+	// TODO: why do we have a non-standard signature here? do we need "Ctr"?
+    let i = 0;
+    const l = arr && arr.length || 0, out = new (Ctr || Array)(l);
+    if (l && typeof arr == "string") arr = arr.split("");
+    if (thisObject) {
+        for (; i < l; ++i) {
+            out[i] = callback.call(thisObject, arr[i], i, arr);
+        }
+    } else {
+        for (; i < l; ++i) {
+            out[i] = callback(arr[i], i, arr);
+        }
+    }
+    return out; // Array
+};
+
+/* dojo-release-1.8.1/dojox/encoding/crypto/Blowfish.js.uncompressed.js */
+
+/*	Blowfish
+ *	Created based on the C# implementation by Marcus Hahn (http://www.hotpixel.net/)
+ *	Unsigned math based on Paul Johnstone and Peter Wood patches.
+ *	2005-12-08
+ */
+const boxes={
+    p: [
+        0x243f6a88, 0x85a308d3, 0x13198a2e, 0x03707344, 0xa4093822, 0x299f31d0, 0x082efa98, 0xec4e6c89,
+        0x452821e6, 0x38d01377, 0xbe5466cf, 0x34e90c6c, 0xc0ac29b7, 0xc97c50dd, 0x3f84d5b5, 0xb5470917,
+        0x9216d5d9, 0x8979fb1b
+    ],
+    s0: [
+        0xd1310ba6, 0x98dfb5ac, 0x2ffd72db, 0xd01adfb7, 0xb8e1afed, 0x6a267e96, 0xba7c9045, 0xf12c7f99,
+        0x24a19947, 0xb3916cf7, 0x0801f2e2, 0x858efc16, 0x636920d8, 0x71574e69, 0xa458fea3, 0xf4933d7e,
+        0x0d95748f, 0x728eb658, 0x718bcd58, 0x82154aee, 0x7b54a41d, 0xc25a59b5, 0x9c30d539, 0x2af26013,
+        0xc5d1b023, 0x286085f0, 0xca417918, 0xb8db38ef, 0x8e79dcb0, 0x603a180e, 0x6c9e0e8b, 0xb01e8a3e,
+        0xd71577c1, 0xbd314b27, 0x78af2fda, 0x55605c60, 0xe65525f3, 0xaa55ab94, 0x57489862, 0x63e81440,
+        0x55ca396a, 0x2aab10b6, 0xb4cc5c34, 0x1141e8ce, 0xa15486af, 0x7c72e993, 0xb3ee1411, 0x636fbc2a,
+        0x2ba9c55d, 0x741831f6, 0xce5c3e16, 0x9b87931e, 0xafd6ba33, 0x6c24cf5c, 0x7a325381, 0x28958677,
+        0x3b8f4898, 0x6b4bb9af, 0xc4bfe81b, 0x66282193, 0x61d809cc, 0xfb21a991, 0x487cac60, 0x5dec8032,
+        0xef845d5d, 0xe98575b1, 0xdc262302, 0xeb651b88, 0x23893e81, 0xd396acc5, 0x0f6d6ff3, 0x83f44239,
+        0x2e0b4482, 0xa4842004, 0x69c8f04a, 0x9e1f9b5e, 0x21c66842, 0xf6e96c9a, 0x670c9c61, 0xabd388f0,
+        0x6a51a0d2, 0xd8542f68, 0x960fa728, 0xab5133a3, 0x6eef0b6c, 0x137a3be4, 0xba3bf050, 0x7efb2a98,
+        0xa1f1651d, 0x39af0176, 0x66ca593e, 0x82430e88, 0x8cee8619, 0x456f9fb4, 0x7d84a5c3, 0x3b8b5ebe,
+        0xe06f75d8, 0x85c12073, 0x401a449f, 0x56c16aa6, 0x4ed3aa62, 0x363f7706, 0x1bfedf72, 0x429b023d,
+        0x37d0d724, 0xd00a1248, 0xdb0fead3, 0x49f1c09b, 0x075372c9, 0x80991b7b, 0x25d479d8, 0xf6e8def7,
+        0xe3fe501a, 0xb6794c3b, 0x976ce0bd, 0x04c006ba, 0xc1a94fb6, 0x409f60c4, 0x5e5c9ec2, 0x196a2463,
+        0x68fb6faf, 0x3e6c53b5, 0x1339b2eb, 0x3b52ec6f, 0x6dfc511f, 0x9b30952c, 0xcc814544, 0xaf5ebd09,
+        0xbee3d004, 0xde334afd, 0x660f2807, 0x192e4bb3, 0xc0cba857, 0x45c8740f, 0xd20b5f39, 0xb9d3fbdb,
+        0x5579c0bd, 0x1a60320a, 0xd6a100c6, 0x402c7279, 0x679f25fe, 0xfb1fa3cc, 0x8ea5e9f8, 0xdb3222f8,
+        0x3c7516df, 0xfd616b15, 0x2f501ec8, 0xad0552ab, 0x323db5fa, 0xfd238760, 0x53317b48, 0x3e00df82,
+        0x9e5c57bb, 0xca6f8ca0, 0x1a87562e, 0xdf1769db, 0xd542a8f6, 0x287effc3, 0xac6732c6, 0x8c4f5573,
+        0x695b27b0, 0xbbca58c8, 0xe1ffa35d, 0xb8f011a0, 0x10fa3d98, 0xfd2183b8, 0x4afcb56c, 0x2dd1d35b,
+        0x9a53e479, 0xb6f84565, 0xd28e49bc, 0x4bfb9790, 0xe1ddf2da, 0xa4cb7e33, 0x62fb1341, 0xcee4c6e8,
+        0xef20cada, 0x36774c01, 0xd07e9efe, 0x2bf11fb4, 0x95dbda4d, 0xae909198, 0xeaad8e71, 0x6b93d5a0,
+        0xd08ed1d0, 0xafc725e0, 0x8e3c5b2f, 0x8e7594b7, 0x8ff6e2fb, 0xf2122b64, 0x8888b812, 0x900df01c,
+        0x4fad5ea0, 0x688fc31c, 0xd1cff191, 0xb3a8c1ad, 0x2f2f2218, 0xbe0e1777, 0xea752dfe, 0x8b021fa1,
+        0xe5a0cc0f, 0xb56f74e8, 0x18acf3d6, 0xce89e299, 0xb4a84fe0, 0xfd13e0b7, 0x7cc43b81, 0xd2ada8d9,
+        0x165fa266, 0x80957705, 0x93cc7314, 0x211a1477, 0xe6ad2065, 0x77b5fa86, 0xc75442f5, 0xfb9d35cf,
+        0xebcdaf0c, 0x7b3e89a0, 0xd6411bd3, 0xae1e7e49, 0x00250e2d, 0x2071b35e, 0x226800bb, 0x57b8e0af,
+        0x2464369b, 0xf009b91e, 0x5563911d, 0x59dfa6aa, 0x78c14389, 0xd95a537f, 0x207d5ba2, 0x02e5b9c5,
+        0x83260376, 0x6295cfa9, 0x11c81968, 0x4e734a41, 0xb3472dca, 0x7b14a94a, 0x1b510052, 0x9a532915,
+        0xd60f573f, 0xbc9bc6e4, 0x2b60a476, 0x81e67400, 0x08ba6fb5, 0x571be91f, 0xf296ec6b, 0x2a0dd915,
+        0xb6636521, 0xe7b9f9b6, 0xff34052e, 0xc5855664, 0x53b02d5d, 0xa99f8fa1, 0x08ba4799, 0x6e85076a
+    ],
+    s1: [
+        0x4b7a70e9, 0xb5b32944, 0xdb75092e, 0xc4192623, 0xad6ea6b0, 0x49a7df7d, 0x9cee60b8, 0x8fedb266,
+        0xecaa8c71, 0x699a17ff, 0x5664526c, 0xc2b19ee1, 0x193602a5, 0x75094c29, 0xa0591340, 0xe4183a3e,
+        0x3f54989a, 0x5b429d65, 0x6b8fe4d6, 0x99f73fd6, 0xa1d29c07, 0xefe830f5, 0x4d2d38e6, 0xf0255dc1,
+        0x4cdd2086, 0x8470eb26, 0x6382e9c6, 0x021ecc5e, 0x09686b3f, 0x3ebaefc9, 0x3c971814, 0x6b6a70a1,
+        0x687f3584, 0x52a0e286, 0xb79c5305, 0xaa500737, 0x3e07841c, 0x7fdeae5c, 0x8e7d44ec, 0x5716f2b8,
+        0xb03ada37, 0xf0500c0d, 0xf01c1f04, 0x0200b3ff, 0xae0cf51a, 0x3cb574b2, 0x25837a58, 0xdc0921bd,
+        0xd19113f9, 0x7ca92ff6, 0x94324773, 0x22f54701, 0x3ae5e581, 0x37c2dadc, 0xc8b57634, 0x9af3dda7,
+        0xa9446146, 0x0fd0030e, 0xecc8c73e, 0xa4751e41, 0xe238cd99, 0x3bea0e2f, 0x3280bba1, 0x183eb331,
+        0x4e548b38, 0x4f6db908, 0x6f420d03, 0xf60a04bf, 0x2cb81290, 0x24977c79, 0x5679b072, 0xbcaf89af,
+        0xde9a771f, 0xd9930810, 0xb38bae12, 0xdccf3f2e, 0x5512721f, 0x2e6b7124, 0x501adde6, 0x9f84cd87,
+        0x7a584718, 0x7408da17, 0xbc9f9abc, 0xe94b7d8c, 0xec7aec3a, 0xdb851dfa, 0x63094366, 0xc464c3d2,
+        0xef1c1847, 0x3215d908, 0xdd433b37, 0x24c2ba16, 0x12a14d43, 0x2a65c451, 0x50940002, 0x133ae4dd,
+        0x71dff89e, 0x10314e55, 0x81ac77d6, 0x5f11199b, 0x043556f1, 0xd7a3c76b, 0x3c11183b, 0x5924a509,
+        0xf28fe6ed, 0x97f1fbfa, 0x9ebabf2c, 0x1e153c6e, 0x86e34570, 0xeae96fb1, 0x860e5e0a, 0x5a3e2ab3,
+        0x771fe71c, 0x4e3d06fa, 0x2965dcb9, 0x99e71d0f, 0x803e89d6, 0x5266c825, 0x2e4cc978, 0x9c10b36a,
+        0xc6150eba, 0x94e2ea78, 0xa5fc3c53, 0x1e0a2df4, 0xf2f74ea7, 0x361d2b3d, 0x1939260f, 0x19c27960,
+        0x5223a708, 0xf71312b6, 0xebadfe6e, 0xeac31f66, 0xe3bc4595, 0xa67bc883, 0xb17f37d1, 0x018cff28,
+        0xc332ddef, 0xbe6c5aa5, 0x65582185, 0x68ab9802, 0xeecea50f, 0xdb2f953b, 0x2aef7dad, 0x5b6e2f84,
+        0x1521b628, 0x29076170, 0xecdd4775, 0x619f1510, 0x13cca830, 0xeb61bd96, 0x0334fe1e, 0xaa0363cf,
+        0xb5735c90, 0x4c70a239, 0xd59e9e0b, 0xcbaade14, 0xeecc86bc, 0x60622ca7, 0x9cab5cab, 0xb2f3846e,
+        0x648b1eaf, 0x19bdf0ca, 0xa02369b9, 0x655abb50, 0x40685a32, 0x3c2ab4b3, 0x319ee9d5, 0xc021b8f7,
+        0x9b540b19, 0x875fa099, 0x95f7997e, 0x623d7da8, 0xf837889a, 0x97e32d77, 0x11ed935f, 0x16681281,
+        0x0e358829, 0xc7e61fd6, 0x96dedfa1, 0x7858ba99, 0x57f584a5, 0x1b227263, 0x9b83c3ff, 0x1ac24696,
+        0xcdb30aeb, 0x532e3054, 0x8fd948e4, 0x6dbc3128, 0x58ebf2ef, 0x34c6ffea, 0xfe28ed61, 0xee7c3c73,
+        0x5d4a14d9, 0xe864b7e3, 0x42105d14, 0x203e13e0, 0x45eee2b6, 0xa3aaabea, 0xdb6c4f15, 0xfacb4fd0,
+        0xc742f442, 0xef6abbb5, 0x654f3b1d, 0x41cd2105, 0xd81e799e, 0x86854dc7, 0xe44b476a, 0x3d816250,
+        0xcf62a1f2, 0x5b8d2646, 0xfc8883a0, 0xc1c7b6a3, 0x7f1524c3, 0x69cb7492, 0x47848a0b, 0x5692b285,
+        0x095bbf00, 0xad19489d, 0x1462b174, 0x23820e00, 0x58428d2a, 0x0c55f5ea, 0x1dadf43e, 0x233f7061,
+        0x3372f092, 0x8d937e41, 0xd65fecf1, 0x6c223bdb, 0x7cde3759, 0xcbee7460, 0x4085f2a7, 0xce77326e,
+        0xa6078084, 0x19f8509e, 0xe8efd855, 0x61d99735, 0xa969a7aa, 0xc50c06c2, 0x5a04abfc, 0x800bcadc,
+        0x9e447a2e, 0xc3453484, 0xfdd56705, 0x0e1e9ec9, 0xdb73dbd3, 0x105588cd, 0x675fda79, 0xe3674340,
+        0xc5c43465, 0x713e38d8, 0x3d28f89e, 0xf16dff20, 0x153e21e7, 0x8fb03d4a, 0xe6e39f2b, 0xdb83adf7
+    ],
+    s2: [
+        0xe93d5a68, 0x948140f7, 0xf64c261c, 0x94692934, 0x411520f7, 0x7602d4f7, 0xbcf46b2e, 0xd4a20068,
+        0xd4082471, 0x3320f46a, 0x43b7d4b7, 0x500061af, 0x1e39f62e, 0x97244546, 0x14214f74, 0xbf8b8840,
+        0x4d95fc1d, 0x96b591af, 0x70f4ddd3, 0x66a02f45, 0xbfbc09ec, 0x03bd9785, 0x7fac6dd0, 0x31cb8504,
+        0x96eb27b3, 0x55fd3941, 0xda2547e6, 0xabca0a9a, 0x28507825, 0x530429f4, 0x0a2c86da, 0xe9b66dfb,
+        0x68dc1462, 0xd7486900, 0x680ec0a4, 0x27a18dee, 0x4f3ffea2, 0xe887ad8c, 0xb58ce006, 0x7af4d6b6,
+        0xaace1e7c, 0xd3375fec, 0xce78a399, 0x406b2a42, 0x20fe9e35, 0xd9f385b9, 0xee39d7ab, 0x3b124e8b,
+        0x1dc9faf7, 0x4b6d1856, 0x26a36631, 0xeae397b2, 0x3a6efa74, 0xdd5b4332, 0x6841e7f7, 0xca7820fb,
+        0xfb0af54e, 0xd8feb397, 0x454056ac, 0xba489527, 0x55533a3a, 0x20838d87, 0xfe6ba9b7, 0xd096954b,
+        0x55a867bc, 0xa1159a58, 0xcca92963, 0x99e1db33, 0xa62a4a56, 0x3f3125f9, 0x5ef47e1c, 0x9029317c,
+        0xfdf8e802, 0x04272f70, 0x80bb155c, 0x05282ce3, 0x95c11548, 0xe4c66d22, 0x48c1133f, 0xc70f86dc,
+        0x07f9c9ee, 0x41041f0f, 0x404779a4, 0x5d886e17, 0x325f51eb, 0xd59bc0d1, 0xf2bcc18f, 0x41113564,
+        0x257b7834, 0x602a9c60, 0xdff8e8a3, 0x1f636c1b, 0x0e12b4c2, 0x02e1329e, 0xaf664fd1, 0xcad18115,
+        0x6b2395e0, 0x333e92e1, 0x3b240b62, 0xeebeb922, 0x85b2a20e, 0xe6ba0d99, 0xde720c8c, 0x2da2f728,
+        0xd0127845, 0x95b794fd, 0x647d0862, 0xe7ccf5f0, 0x5449a36f, 0x877d48fa, 0xc39dfd27, 0xf33e8d1e,
+        0x0a476341, 0x992eff74, 0x3a6f6eab, 0xf4f8fd37, 0xa812dc60, 0xa1ebddf8, 0x991be14c, 0xdb6e6b0d,
+        0xc67b5510, 0x6d672c37, 0x2765d43b, 0xdcd0e804, 0xf1290dc7, 0xcc00ffa3, 0xb5390f92, 0x690fed0b,
+        0x667b9ffb, 0xcedb7d9c, 0xa091cf0b, 0xd9155ea3, 0xbb132f88, 0x515bad24, 0x7b9479bf, 0x763bd6eb,
+        0x37392eb3, 0xcc115979, 0x8026e297, 0xf42e312d, 0x6842ada7, 0xc66a2b3b, 0x12754ccc, 0x782ef11c,
+        0x6a124237, 0xb79251e7, 0x06a1bbe6, 0x4bfb6350, 0x1a6b1018, 0x11caedfa, 0x3d25bdd8, 0xe2e1c3c9,
+        0x44421659, 0x0a121386, 0xd90cec6e, 0xd5abea2a, 0x64af674e, 0xda86a85f, 0xbebfe988, 0x64e4c3fe,
+        0x9dbc8057, 0xf0f7c086, 0x60787bf8, 0x6003604d, 0xd1fd8346, 0xf6381fb0, 0x7745ae04, 0xd736fccc,
+        0x83426b33, 0xf01eab71, 0xb0804187, 0x3c005e5f, 0x77a057be, 0xbde8ae24, 0x55464299, 0xbf582e61,
+        0x4e58f48f, 0xf2ddfda2, 0xf474ef38, 0x8789bdc2, 0x5366f9c3, 0xc8b38e74, 0xb475f255, 0x46fcd9b9,
+        0x7aeb2661, 0x8b1ddf84, 0x846a0e79, 0x915f95e2, 0x466e598e, 0x20b45770, 0x8cd55591, 0xc902de4c,
+        0xb90bace1, 0xbb8205d0, 0x11a86248, 0x7574a99e, 0xb77f19b6, 0xe0a9dc09, 0x662d09a1, 0xc4324633,
+        0xe85a1f02, 0x09f0be8c, 0x4a99a025, 0x1d6efe10, 0x1ab93d1d, 0x0ba5a4df, 0xa186f20f, 0x2868f169,
+        0xdcb7da83, 0x573906fe, 0xa1e2ce9b, 0x4fcd7f52, 0x50115e01, 0xa70683fa, 0xa002b5c4, 0x0de6d027,
+        0x9af88c27, 0x773f8641, 0xc3604c06, 0x61a806b5, 0xf0177a28, 0xc0f586e0, 0x006058aa, 0x30dc7d62,
+        0x11e69ed7, 0x2338ea63, 0x53c2dd94, 0xc2c21634, 0xbbcbee56, 0x90bcb6de, 0xebfc7da1, 0xce591d76,
+        0x6f05e409, 0x4b7c0188, 0x39720a3d, 0x7c927c24, 0x86e3725f, 0x724d9db9, 0x1ac15bb4, 0xd39eb8fc,
+        0xed545578, 0x08fca5b5, 0xd83d7cd3, 0x4dad0fc4, 0x1e50ef5e, 0xb161e6f8, 0xa28514d9, 0x6c51133c,
+        0x6fd5c7e7, 0x56e14ec4, 0x362abfce, 0xddc6c837, 0xd79a3234, 0x92638212, 0x670efa8e, 0x406000e0
+    ],
+    s3: [
+        0x3a39ce37, 0xd3faf5cf, 0xabc27737, 0x5ac52d1b, 0x5cb0679e, 0x4fa33742, 0xd3822740, 0x99bc9bbe,
+        0xd5118e9d, 0xbf0f7315, 0xd62d1c7e, 0xc700c47b, 0xb78c1b6b, 0x21a19045, 0xb26eb1be, 0x6a366eb4,
+        0x5748ab2f, 0xbc946e79, 0xc6a376d2, 0x6549c2c8, 0x530ff8ee, 0x468dde7d, 0xd5730a1d, 0x4cd04dc6,
+        0x2939bbdb, 0xa9ba4650, 0xac9526e8, 0xbe5ee304, 0xa1fad5f0, 0x6a2d519a, 0x63ef8ce2, 0x9a86ee22,
+        0xc089c2b8, 0x43242ef6, 0xa51e03aa, 0x9cf2d0a4, 0x83c061ba, 0x9be96a4d, 0x8fe51550, 0xba645bd6,
+        0x2826a2f9, 0xa73a3ae1, 0x4ba99586, 0xef5562e9, 0xc72fefd3, 0xf752f7da, 0x3f046f69, 0x77fa0a59,
+        0x80e4a915, 0x87b08601, 0x9b09e6ad, 0x3b3ee593, 0xe990fd5a, 0x9e34d797, 0x2cf0b7d9, 0x022b8b51,
+        0x96d5ac3a, 0x017da67d, 0xd1cf3ed6, 0x7c7d2d28, 0x1f9f25cf, 0xadf2b89b, 0x5ad6b472, 0x5a88f54c,
+        0xe029ac71, 0xe019a5e6, 0x47b0acfd, 0xed93fa9b, 0xe8d3c48d, 0x283b57cc, 0xf8d56629, 0x79132e28,
+        0x785f0191, 0xed756055, 0xf7960e44, 0xe3d35e8c, 0x15056dd4, 0x88f46dba, 0x03a16125, 0x0564f0bd,
+        0xc3eb9e15, 0x3c9057a2, 0x97271aec, 0xa93a072a, 0x1b3f6d9b, 0x1e6321f5, 0xf59c66fb, 0x26dcf319,
+        0x7533d928, 0xb155fdf5, 0x03563482, 0x8aba3cbb, 0x28517711, 0xc20ad9f8, 0xabcc5167, 0xccad925f,
+        0x4de81751, 0x3830dc8e, 0x379d5862, 0x9320f991, 0xea7a90c2, 0xfb3e7bce, 0x5121ce64, 0x774fbe32,
+        0xa8b6e37e, 0xc3293d46, 0x48de5369, 0x6413e680, 0xa2ae0810, 0xdd6db224, 0x69852dfd, 0x09072166,
+        0xb39a460a, 0x6445c0dd, 0x586cdecf, 0x1c20c8ae, 0x5bbef7dd, 0x1b588d40, 0xccd2017f, 0x6bb4e3bb,
+        0xdda26a7e, 0x3a59ff45, 0x3e350a44, 0xbcb4cdd5, 0x72eacea8, 0xfa6484bb, 0x8d6612ae, 0xbf3c6f47,
+        0xd29be463, 0x542f5d9e, 0xaec2771b, 0xf64e6370, 0x740e0d8d, 0xe75b1357, 0xf8721671, 0xaf537d5d,
+        0x4040cb08, 0x4eb4e2cc, 0x34d2466a, 0x0115af84, 0xe1b00428, 0x95983a1d, 0x06b89fb4, 0xce6ea048,
+        0x6f3f3b82, 0x3520ab82, 0x011a1d4b, 0x277227f8, 0x611560b1, 0xe7933fdc, 0xbb3a792b, 0x344525bd,
+        0xa08839e1, 0x51ce794b, 0x2f32c9b7, 0xa01fbac9, 0xe01cc87e, 0xbcc7d1f6, 0xcf0111c3, 0xa1e8aac7,
+        0x1a908749, 0xd44fbd9a, 0xd0dadecb, 0xd50ada38, 0x0339c32a, 0xc6913667, 0x8df9317c, 0xe0b12b4f,
+        0xf79e59b7, 0x43f5bb3a, 0xf2d519ff, 0x27d9459c, 0xbf97222c, 0x15e6fc2a, 0x0f91fc71, 0x9b941525,
+        0xfae59361, 0xceb69ceb, 0xc2a86459, 0x12baa8d1, 0xb6c1075e, 0xe3056a0c, 0x10d25065, 0xcb03a442,
+        0xe0ec6e0e, 0x1698db3b, 0x4c98a0be, 0x3278e964, 0x9f1f9532, 0xe0d392df, 0xd3a0342b, 0x8971f21e,
+        0x1b0a7441, 0x4ba3348c, 0xc5be7120, 0xc37632d8, 0xdf359f8d, 0x9b992f2e, 0xe60b6f47, 0x0fe3f11d,
+        0xe54cda54, 0x1edad891, 0xce6279cf, 0xcd3e7e6f, 0x1618b166, 0xfd2c1d05, 0x848fd2c5, 0xf6fb2299,
+        0xf523f357, 0xa6327623, 0x93a83531, 0x56cccd02, 0xacf08162, 0x5a75ebb5, 0x6e163697, 0x88d273cc,
+        0xde966292, 0x81b949d0, 0x4c50901b, 0x71c65614, 0xe6c6c7bd, 0x327a140a, 0x45e1d006, 0xc3f27b9a,
+        0xc9aa53fd, 0x62a80f00, 0xbb25bfe2, 0x35bdd2f6, 0x71126905, 0xb2040222, 0xb6cbcf7c, 0xcd769c2b,
+        0x53113ec0, 0x1640e3d3, 0x38abbd60, 0x2547adf0, 0xba38209c, 0xf746ce76, 0x77afa1c5, 0x20756060,
+        0x85cbfe4e, 0x8ae88dd8, 0x7aaaf9b0, 0x4cf9aa7e, 0x1948c25c, 0x02fb8a8c, 0x01c36ae4, 0xd6ebe1f9,
+        0x90d4f869, 0xa65cdea0, 0x3f09252d, 0xc208e69f, 0xb74e6132, 0xce77e25b, 0x578fdfe3, 0x3ac372e6
+    ]
+};
+
+
+////////////////////////////////////////////////////////////////////////////
+//	fixes based on patch submitted by Peter Wood (#5791)
+const xor = function(x, y) {
+    return (((x>>0x10)^(y>>0x10))<<0x10)|(((x&0xffff)^(y&0xffff))&0xffff);
+};
+
+
+const f = function(v, box) {
+    const d=box.s3[v&0xff]; v>>=8;
+    const c=box.s2[v&0xff]; v>>=8;
+    const b=box.s1[v&0xff]; v>>=8;
+    const a=box.s0[v&0xff];
+
+    let r = (((a>>0x10)+(b>>0x10)+(((a&0xffff)+(b&0xffff))>>0x10))<<0x10)|(((a&0xffff)+(b&0xffff))&0xffff);
+    r = (((r>>0x10)^(c>>0x10))<<0x10)|(((r&0xffff)^(c&0xffff))&0xffff);
+    return (((r>>0x10)+(d>>0x10)+(((r&0xffff)+(d&0xffff))>>0x10))<<0x10)|(((r&0xffff)+(d&0xffff))&0xffff);
+};
+
+
+const eb = function(o, box) {
+	//	TODO: see if this can't be made more efficient
+    let l=o.left;
+    let r=o.right;
+    l=xor(l, box.p[0]);
+    r=xor(r, xor(f(l, box), box.p[1]));
+    l=xor(l, xor(f(r, box), box.p[2]));
+    r=xor(r, xor(f(l, box), box.p[3]));
+    l=xor(l, xor(f(r, box), box.p[4]));
+    r=xor(r, xor(f(l, box), box.p[5]));
+    l=xor(l, xor(f(r, box), box.p[6]));
+    r=xor(r, xor(f(l, box), box.p[7]));
+    l=xor(l, xor(f(r, box), box.p[8]));
+    r=xor(r, xor(f(l, box), box.p[9]));
+    l=xor(l, xor(f(r, box), box.p[10]));
+    r=xor(r, xor(f(l, box), box.p[11]));
+    l=xor(l, xor(f(r, box), box.p[12]));
+    r=xor(r, xor(f(l, box), box.p[13]));
+    l=xor(l, xor(f(r, box), box.p[14]));
+    r=xor(r, xor(f(l, box), box.p[15]));
+    l=xor(l, xor(f(r, box), box.p[16]));
+    o.right=l;
+    o.left=xor(r, box.p[17]);
+};
+
+const db = function(o, box) {
+    let l=o.left;
+    let r=o.right;
+    l=xor(l, box.p[17]);
+    r=xor(r, xor(f(l, box), box.p[16]));
+    l=xor(l, xor(f(r, box), box.p[15]));
+    r=xor(r, xor(f(l, box), box.p[14]));
+    l=xor(l, xor(f(r, box), box.p[13]));
+    r=xor(r, xor(f(l, box), box.p[12]));
+    l=xor(l, xor(f(r, box), box.p[11]));
+    r=xor(r, xor(f(l, box), box.p[10]));
+    l=xor(l, xor(f(r, box), box.p[9]));
+    r=xor(r, xor(f(l, box), box.p[8]));
+    l=xor(l, xor(f(r, box), box.p[7]));
+    r=xor(r, xor(f(l, box), box.p[6]));
+    l=xor(l, xor(f(r, box), box.p[5]));
+    r=xor(r, xor(f(l, box), box.p[4]));
+    l=xor(l, xor(f(r, box), box.p[3]));
+    r=xor(r, xor(f(l, box), box.p[2]));
+    l=xor(l, xor(f(r, box), box.p[1]));
+    o.right=l;
+    o.left=xor(r, box.p[0]);
+};
+
+const encryptBlock=function(inblock, outblock, box) {
+    const o = {};
+    o.left=inblock[0];
+    o.right=inblock[1];
+    eb(o, box);
+    outblock[0] = o.left;
+    outblock[1] = o.right;
+};
+
+const decryptBlock=function(inblock, outblock, box) {
+    const o= {};
+    o.left=inblock[0];
+    o.right=inblock[1];
+    db(o, box);
+    outblock[0] = o.left;
+    outblock[1] = o.right;
+};
+
+crypto.Blowfish = new function() {
+    this.createCipher=function(key, modeName) {
+        return new forge.cipher.BlockCipher({
+            algorithm: new Blowfish.Algorithm(key, modeName),
+            key: key,
+            decrypt: false
+        });
+    };
+
+    this.createDecipher=function(key, modeName) {
+        return new forge.cipher.BlockCipher({
+            algorithm: new Blowfish.Algorithm(key, modeName),
+            key: key,
+            decrypt: true
+        });
+    };
+}();
+
+
+crypto.Blowfish.Algorithm=function(key, modeName) {
+    this.initialize({key: key});
+    const _box = this.box;
+    const modeOption = {
+        blockSize: 8,
+        cipher: {
+            encrypt: function(inblock, outblock) {
+                encryptBlock(inblock, outblock, _box);
+            },
+            decrypt: function(inblock, outblock) {
+                decryptBlock(inblock, outblock, _box);
+            },
+        }
+    };
+
+    switch (modeName.toLowerCase()) {
+        case "ecb":
+            this.mode=new forge.cipher.modes.ecb(modeOption);
+            break;
+        case "cbc":
+            this.mode=new forge.cipher.modes.cbc(modeOption);
+            break;
+        case "cfb":
+            this.mode=new forge.cipher.modes.cfb(modeOption);
+            break;
+        case "ofb":
+            this.mode=new forge.cipher.modes.ofb(modeOption);
+            break;
+        case "ctr":
+            this.mode=new forge.cipher.modes.ctr(modeOption);
+            break;
+        default:
+            this.mode=new forge.cipher.modes.ecb(modeOption);
+            break;
+    }
+
+};
+
+crypto.Blowfish.Algorithm.prototype.initialize=function(options) {
+    const POW8=Math.pow(2, 8);
+
+    let k=options.key;
+    if (lang.isString(k)) {
+        k = arrayUtil.map(k.split(""), function(item) {
+            return item.charCodeAt(0) & 0xff;
+        });
+    }
+
+	//	init the boxes
+    let pos=0, data=0;
+    const res={ left: 0, right: 0 };
+    const box = {
+        p: arrayUtil.map(boxes.p.slice(0), function(item) {
+            const l=k.length;
+            for (let j=0; j<4; j++) {
+                data=(data*POW8)|k[pos++ % l];
+            }
+            return (((item>>0x10)^(data>>0x10))<<0x10)|(((item&0xffff)^(data&0xffff))&0xffff);
+        }),
+        s0: boxes.s0.slice(0),
+        s1: boxes.s1.slice(0),
+        s2: boxes.s2.slice(0),
+        s3: boxes.s3.slice(0)
+    };
+
+	//	encrypt p and the s boxes
+    for (let i=0, l=box.p.length; i<l;) {
+        eb(res, box);
+        box.p[i++]=res.left;
+        box.p[i++]=res.right;
+    }
+    for (let i=0; i<4; i++) {
+        for (let j=0, l=box["s"+i].length; j<l;) {
+            eb(res, box);
+            box["s"+i][j++]=res.left;
+            box["s"+i][j++]=res.right;
+        }
+    }
+    this.box = box;
+};
+
+export const Blowfish = crypto.Blowfish;

src/core/lib/Bombe.mjs

@@ -7,9 +7,9 @@
  * @license Apache-2.0
  */
 
-import OperationError from "../errors/OperationError";
-import Utils from "../Utils";
-import {Rotor, Plugboard, a2i, i2a} from "./Enigma";
+import OperationError from "../errors/OperationError.mjs";
+import Utils from "../Utils.mjs";
+import {Rotor, Plugboard, a2i, i2a} from "./Enigma.mjs";
 
 /**
  * Convenience/optimisation subclass of Rotor

src/core/lib/Charts.mjs

@@ -5,7 +5,7 @@
  * @license Apache-2.0
  */
 
-import OperationError from "../errors/OperationError";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * @constant

src/core/lib/ChrEnc.mjs

@@ -12,16 +12,65 @@
 export const IO_FORMAT = {
     "UTF-8 (65001)": 65001,
     "UTF-7 (65000)": 65000,
-    "UTF16LE (1200)": 1200,
-    "UTF16BE (1201)": 1201,
-    "UTF16 (1201)": 1201,
+    "UTF-16LE (1200)": 1200,
+    "UTF-16BE (1201)": 1201,
+    "UTF-32LE (12000)": 12000,
+    "UTF-32BE (12001)": 12001,
     "IBM EBCDIC International (500)": 500,
     "IBM EBCDIC US-Canada (37)": 37,
+    "IBM EBCDIC Multilingual/ROECE (Latin 2) (870)": 870,
+    "IBM EBCDIC Greek Modern (875)": 875,
+    "IBM EBCDIC French (1010)": 1010,
+    "IBM EBCDIC Turkish (Latin 5) (1026)": 1026,
+    "IBM EBCDIC Latin 1/Open System (1047)": 1047,
+    "IBM EBCDIC Lao (1132/1133/1341)": 1132,
+    "IBM EBCDIC US-Canada (037 + Euro symbol) (1140)": 1140,
+    "IBM EBCDIC Germany (20273 + Euro symbol) (1141)": 1141,
+    "IBM EBCDIC Denmark-Norway (20277 + Euro symbol) (1142)": 1142,
+    "IBM EBCDIC Finland-Sweden (20278 + Euro symbol) (1143)": 1143,
+    "IBM EBCDIC Italy (20280 + Euro symbol) (1144)": 1144,
+    "IBM EBCDIC Latin America-Spain (20284 + Euro symbol) (1145)": 1145,
+    "IBM EBCDIC United Kingdom (20285 + Euro symbol) (1146)": 1146,
+    "IBM EBCDIC France (20297 + Euro symbol) (1147)": 1147,
+    "IBM EBCDIC International (500 + Euro symbol) (1148)": 1148,
+    "IBM EBCDIC Icelandic (20871 + Euro symbol) (1149)": 1149,
+    "IBM EBCDIC Germany (20273)": 20273,
+    "IBM EBCDIC Denmark-Norway (20277)": 20277,
+    "IBM EBCDIC Finland-Sweden (20278)": 20278,
+    "IBM EBCDIC Italy (20280)": 20280,
+    "IBM EBCDIC Latin America-Spain (20284)": 20284,
+    "IBM EBCDIC United Kingdom (20285)": 20285,
+    "IBM EBCDIC Japanese Katakana Extended (20290)": 20290,
+    "IBM EBCDIC France (20297)": 20297,
+    "IBM EBCDIC Arabic (20420)": 20420,
+    "IBM EBCDIC Greek (20423)": 20423,
+    "IBM EBCDIC Hebrew (20424)": 20424,
+    "IBM EBCDIC Korean Extended (20833)": 20833,
+    "IBM EBCDIC Thai (20838)": 20838,
+    "IBM EBCDIC Icelandic (20871)": 20871,
+    "IBM EBCDIC Cyrillic Russian (20880)": 20880,
+    "IBM EBCDIC Turkish (20905)": 20905,
+    "IBM EBCDIC Latin 1/Open System (1047 + Euro symbol) (20924)": 20924,
+    "IBM EBCDIC Cyrillic Serbian-Bulgarian (21025)": 21025,
+    "OEM United States (437)": 437,
+    "OEM Greek (formerly 437G); Greek (DOS) (737)": 737,
+    "OEM Baltic; Baltic (DOS) (775)": 775,
+    "OEM Russian; Cyrillic + Euro symbol (808)": 808,
+    "OEM Multilingual Latin 1; Western European (DOS) (850)": 850,
+    "OEM Latin 2; Central European (DOS) (852)": 852,
+    "OEM Cyrillic (primarily Russian) (855)": 855,
+    "OEM Turkish; Turkish (DOS) (857)": 857,
+    "OEM Multilingual Latin 1 + Euro symbol (858)": 858,
+    "OEM Portuguese; Portuguese (DOS) (860)": 860,
+    "OEM Icelandic; Icelandic (DOS) (861)": 861,
+    "OEM Hebrew; Hebrew (DOS) (862)": 862,
+    "OEM French Canadian; French Canadian (DOS) (863)": 863,
+    "OEM Arabic; Arabic (864) (864)": 864,
+    "OEM Nordic; Nordic (DOS) (865)": 865,
+    "OEM Russian; Cyrillic (DOS) (866)": 866,
+    "OEM Modern Greek; Greek, Modern (DOS) (869)": 869,
+    "OEM Cyrillic (primarily Russian) + Euro Symbol (872)": 872,
     "Windows-874 Thai (874)": 874,
-    "Japanese Shift-JIS (932)": 932,
-    "Simplified Chinese GBK (936)": 936,
-    "Korean (949)": 949,
-    "Traditional Chinese Big5 (950)": 950,
     "Windows-1250 Central European (1250)": 1250,
     "Windows-1251 Cyrillic (1251)": 1251,
     "Windows-1252 Latin (1252)": 1252,
@@ -31,10 +80,6 @@ export const IO_FORMAT = {
     "Windows-1256 Arabic (1256)": 1256,
     "Windows-1257 Baltic (1257)": 1257,
     "Windows-1258 Vietnam (1258)": 1258,
-    "US-ASCII (20127)": 20127,
-    "Simplified Chinese GB2312 (20936)": 20936,
-    "KOI8-R Russian Cyrillic (20866)": 20866,
-    "KOI8-U Ukrainian Cyrillic (21866)": 21866,
     "ISO-8859-1 Latin 1 Western European (28591)": 28591,
     "ISO-8859-2 Latin 2 Central European (28592)": 28592,
     "ISO-8859-3 Latin 3 South European (28593)": 28593,
@@ -43,6 +88,7 @@ export const IO_FORMAT = {
     "ISO-8859-6 Latin/Arabic (28596)": 28596,
     "ISO-8859-7 Latin/Greek (28597)": 28597,
     "ISO-8859-8 Latin/Hebrew (28598)": 28598,
+    "ISO 8859-8 Hebrew (ISO-Logical) (38598)": 38598,
     "ISO-8859-9 Latin 5 Turkish (28599)": 28599,
     "ISO-8859-10 Latin 6 Nordic (28600)": 28600,
     "ISO-8859-11 Latin/Thai (28601)": 28601,
@@ -50,9 +96,71 @@ export const IO_FORMAT = {
     "ISO-8859-14 Latin 8 Celtic (28604)": 28604,
     "ISO-8859-15 Latin 9 (28605)": 28605,
     "ISO-8859-16 Latin 10 (28606)": 28606,
-    "ISO-2022 JIS Japanese (50222)": 50222,
+    "ISO 2022 JIS Japanese with no halfwidth Katakana (50220)": 50220,
+    "ISO 2022 JIS Japanese with halfwidth Katakana (50221)": 50221,
+    "ISO 2022 Japanese JIS X 0201-1989 (1 byte Kana-SO/SI) (50222)": 50222,
+    "ISO 2022 Korean (50225)": 50225,
+    "ISO 2022 Simplified Chinese (50227)": 50227,
+    "ISO 6937 Non-Spacing Accent (20269)": 20269,
     "EUC Japanese (51932)": 51932,
+    "EUC Simplified Chinese (51936)": 51936,
     "EUC Korean (51949)": 51949,
+    "ISCII Devanagari (57002)": 57002,
+    "ISCII Bengali (57003)": 57003,
+    "ISCII Tamil (57004)": 57004,
+    "ISCII Telugu (57005)": 57005,
+    "ISCII Assamese (57006)": 57006,
+    "ISCII Oriya (57007)": 57007,
+    "ISCII Kannada (57008)": 57008,
+    "ISCII Malayalam (57009)": 57009,
+    "ISCII Gujarati (57010)": 57010,
+    "ISCII Punjabi (57011)": 57011,
+    "Japanese Shift-JIS (932)": 932,
+    "Simplified Chinese GBK (936)": 936,
+    "Korean (949)": 949,
+    "Traditional Chinese Big5 (950)": 950,
+    "US-ASCII (7-bit) (20127)": 20127,
+    "Simplified Chinese GB2312 (20936)": 20936,
+    "KOI8-R Russian Cyrillic (20866)": 20866,
+    "KOI8-U Ukrainian Cyrillic (21866)": 21866,
+    "Mazovia (Polish) MS-DOS (620)": 620,
+    "Arabic (ASMO 708) (708)": 708,
+    "Arabic (Transparent ASMO); Arabic (DOS) (720)": 720,
+    "Kamenický (Czech) MS-DOS (895)": 895,
+    "Korean (Johab) (1361)": 1361,
+    "MAC Roman (10000)": 10000,
+    "Japanese (Mac) (10001)": 10001,
+    "MAC Traditional Chinese (Big5) (10002)": 10002,
+    "Korean (Mac) (10003)": 10003,
+    "Arabic (Mac) (10004)": 10004,
+    "Hebrew (Mac) (10005)": 10005,
+    "Greek (Mac) (10006)": 10006,
+    "Cyrillic (Mac) (10007)": 10007,
+    "MAC Simplified Chinese (GB 2312) (10008)": 10008,
+    "Romanian (Mac) (10010)": 10010,
+    "Ukrainian (Mac) (10017)": 10017,
+    "Thai (Mac) (10021)": 10021,
+    "MAC Latin 2 (Central European) (10029)": 10029,
+    "Icelandic (Mac) (10079)": 10079,
+    "Turkish (Mac) (10081)": 10081,
+    "Croatian (Mac) (10082)": 10082,
+    "CNS Taiwan (Chinese Traditional) (20000)": 20000,
+    "TCA Taiwan (20001)": 20001,
+    "ETEN Taiwan (Chinese Traditional) (20002)": 20002,
+    "IBM5550 Taiwan (20003)": 20003,
+    "TeleText Taiwan (20004)": 20004,
+    "Wang Taiwan (20005)": 20005,
+    "Western European IA5 (IRV International Alphabet 5) (20105)": 20105,
+    "IA5 German (7-bit) (20106)": 20106,
+    "IA5 Swedish (7-bit) (20107)": 20107,
+    "IA5 Norwegian (7-bit) (20108)": 20108,
+    "T.61 (20261)": 20261,
+    "Japanese (JIS 0208-1990 and 0212-1990) (20932)": 20932,
+    "Korean Wansung (20949)": 20949,
+    "Extended/Ext Alpha Lowercase (21027)": 21027,
+    "Europa 3 (29001)": 29001,
+    "Atari ST/TT (47451)": 47451,
+    "HZ-GB2312 Simplified Chinese (52936)": 52936,
     "Simplified Chinese GB18030 (54936)": 54936,
 };
 

src/core/lib/Ciphers.mjs

@@ -9,7 +9,7 @@
  *
  */
 
-import OperationError from "../errors/OperationError";
+import OperationError from "../errors/OperationError.mjs";
 import CryptoJS from "crypto-js";
 
 /**

src/core/lib/ConvertCoordinates.mjs

@@ -6,9 +6,22 @@
  * @license Apache-2.0
  */
 
+import OperationError from "../errors/OperationError.mjs";
 import geohash from "ngeohash";
+/*
+Currently unable to update to geodesy v2 as we cannot load .js modules into a .mjs file.
+When we do update, imports will look like this:
+
+import LatLonEllipsoidal from "geodesy/latlon-ellipsoidal.js";
+import Mgrs from "geodesy/mgrs.js";
+import OsGridRef from "geodesy/osgridref.js";
+import Utm from "geodesy/utm.js";
+*/
 import geodesy from "geodesy";
-import OperationError from "../errors/OperationError";
+const LatLonEllipsoidal = geodesy.LatLonEllipsoidal,
+    Mgrs = geodesy.Mgrs,
+    OsGridRef = geodesy.OsGridRef,
+    Utm = geodesy.Utm;
 
 /**
  * Co-ordinate formats
@@ -116,22 +129,22 @@ export function convertCoordinates (input, inFormat, inDelim, outFormat, outDeli
     switch (inFormat) {
         case "Geohash":
             hash = geohash.decode(input.replace(/[^A-Za-z0-9]/g, ""));
-            latlon = new geodesy.LatLonEllipsoidal(hash.latitude, hash.longitude);
+            latlon = new LatLonEllipsoidal(hash.latitude, hash.longitude);
             break;
         case "Military Grid Reference System":
-            utm = geodesy.Mgrs.parse(input.replace(/[^A-Za-z0-9]/g, "")).toUtm();
+            utm = Mgrs.parse(input.replace(/[^A-Za-z0-9]/g, "")).toUtm();
             latlon = utm.toLatLonE();
             break;
         case "Ordnance Survey National Grid":
-            osng = geodesy.OsGridRef.parse(input.replace(/[^A-Za-z0-9]/g, ""));
-            latlon = geodesy.OsGridRef.osGridToLatLon(osng);
+            osng = OsGridRef.parse(input.replace(/[^A-Za-z0-9]/g, ""));
+            latlon = OsGridRef.osGridToLatLon(osng);
             break;
         case "Universal Transverse Mercator":
             // Geodesy needs a space between the first 2 digits and the next letter
             if (/^[\d]{2}[A-Za-z]/.test(input)) {
                 input = input.slice(0, 2) + " " + input.slice(2);
             }
-            utm = geodesy.Utm.parse(input);
+            utm = Utm.parse(input);
             latlon = utm.toLatLonE();
             break;
         case "Degrees Minutes Seconds":
@@ -143,7 +156,7 @@ export function convertCoordinates (input, inFormat, inDelim, outFormat, outDeli
                 if (splitLat.length >= 3 && splitLong.length >= 3) {
                     lat = convDMSToDD(splitLat[0], splitLat[1], splitLat[2], 10);
                     lon = convDMSToDD(splitLong[0], splitLong[1], splitLong[2], 10);
-                    latlon = new geodesy.LatLonEllipsoidal(lat.degrees, lon.degrees);
+                    latlon = new LatLonEllipsoidal(lat.degrees, lon.degrees);
                 } else {
                     throw new OperationError("Invalid co-ordinate format for Degrees Minutes Seconds");
                 }
@@ -152,7 +165,7 @@ export function convertCoordinates (input, inFormat, inDelim, outFormat, outDeli
                 splitLat = splitInput(split[0]);
                 if (splitLat.length >= 3) {
                     lat = convDMSToDD(splitLat[0], splitLat[1], splitLat[2]);
-                    latlon = new geodesy.LatLonEllipsoidal(lat.degrees, lat.degrees);
+                    latlon = new LatLonEllipsoidal(lat.degrees, lat.degrees);
                 } else {
                     throw new OperationError("Invalid co-ordinate format for Degrees Minutes Seconds");
                 }
@@ -168,7 +181,7 @@ export function convertCoordinates (input, inFormat, inDelim, outFormat, outDeli
                 // Convert to decimal degrees, and then convert to a geodesy object
                 lat = convDDMToDD(splitLat[0], splitLat[1], 10);
                 lon = convDDMToDD(splitLong[0], splitLong[1], 10);
-                latlon = new geodesy.LatLonEllipsoidal(lat.degrees, lon.degrees);
+                latlon = new LatLonEllipsoidal(lat.degrees, lon.degrees);
             } else {
                 // Not a pair, so only try to convert one set of co-ordinates
                 splitLat = splitInput(input);
@@ -176,7 +189,7 @@ export function convertCoordinates (input, inFormat, inDelim, outFormat, outDeli
                     throw new OperationError("Invalid co-ordinate format for Degrees Decimal Minutes.");
                 }
                 lat = convDDMToDD(splitLat[0], splitLat[1], 10);
-                latlon = new geodesy.LatLonEllipsoidal(lat.degrees, lat.degrees);
+                latlon = new LatLonEllipsoidal(lat.degrees, lat.degrees);
             }
             break;
         case "Decimal Degrees":
@@ -186,14 +199,14 @@ export function convertCoordinates (input, inFormat, inDelim, outFormat, outDeli
                 if (splitLat.length !== 1 || splitLong.length !== 1) {
                     throw new OperationError("Invalid co-ordinate format for Decimal Degrees.");
                 }
-                latlon = new geodesy.LatLonEllipsoidal(splitLat[0], splitLong[0]);
+                latlon = new LatLonEllipsoidal(splitLat[0], splitLong[0]);
             } else {
                 // Not a pair, so only try to convert one set of co-ordinates
                 splitLat = splitInput(split[0]);
                 if (splitLat.length !== 1) {
                     throw new OperationError("Invalid co-ordinate format for Decimal Degrees.");
                 }
-                latlon = new geodesy.LatLonEllipsoidal(splitLat[0], splitLat[0]);
+                latlon = new LatLonEllipsoidal(splitLat[0], splitLat[0]);
             }
             break;
         default:
@@ -260,7 +273,7 @@ export function convertCoordinates (input, inFormat, inDelim, outFormat, outDeli
             convLat = mgrs.toString(precision);
             break;
         case "Ordnance Survey National Grid":
-            osng = geodesy.OsGridRef.latLonToOsGrid(latlon);
+            osng = OsGridRef.latLonToOsGrid(latlon);
             if (osng.toString() === "") {
                 throw new OperationError("Could not convert co-ordinates to OS National Grid. Are the co-ordinates in range?");
             }
@@ -327,13 +340,13 @@ export function convertCoordinates (input, inFormat, inDelim, outFormat, outDeli
  * @param {string} input - The input data to be split
  * @returns {number[]} An array of the different items in the string, stored as floats
  */
-function splitInput (input){
+function splitInput (input) {
     const split = [];
 
     input.split(/\s+/).forEach(item => {
         // Remove any character that isn't a digit, decimal point or negative sign
         item = item.replace(/[^0-9.-]/g, "");
-        if (item.length > 0){
+        if (item.length > 0) {
             // Turn the item into a float
             split.push(parseFloat(item));
         }
@@ -350,7 +363,7 @@ function splitInput (input){
  * @param {number} precision - The precision the result should be rounded to
  * @returns {{string: string, degrees: number}} An object containing the raw converted value (obj.degrees), and a formatted string version (obj.string)
  */
-function convDMSToDD (degrees, minutes, seconds, precision){
+function convDMSToDD (degrees, minutes, seconds, precision) {
     const absDegrees = Math.abs(degrees);
     let conv = absDegrees + (minutes / 60) + (seconds / 3600);
     let outString = round(conv, precision) + "°";
@@ -566,7 +579,7 @@ export function findFormat (input, delim) {
     // Test DMS/DDM/DD formats
     if (testData !== undefined) {
         const split = splitInput(testData);
-        switch (split.length){
+        switch (split.length) {
             case 3:
                 return "Degrees Minutes Seconds";
             case 2:

src/core/lib/Decimal.mjs

@@ -6,7 +6,7 @@
  * @license Apache-2.0
  */
 
-import Utils from "../Utils";
+import Utils from "../Utils.mjs";
 
 
 /**

src/core/lib/Delim.mjs

@@ -32,7 +32,7 @@ export const WORD_DELIM_OPTIONS = ["Line feed", "CRLF", "Forward slash", "Backsl
 export const INPUT_DELIM_OPTIONS = ["Line feed", "CRLF", "Space", "Comma", "Semi-colon", "Colon", "Nothing (separate chars)"];
 
 /**
- * Armithmetic sequence delimiters
+ * Arithmetic sequence delimiters
  */
 export const ARITHMETIC_DELIM_OPTIONS = ["Line feed", "Space", "Comma", "Semi-colon", "Colon", "CRLF"];
 
@@ -72,3 +72,12 @@ export const JOIN_DELIM_OPTIONS = [
     {name: "Nothing (join chars)", value: ""}
 ];
 
+/**
+ * RGBA list delimiters.
+ */
+export const RGBA_DELIM_OPTIONS = [
+    {name: "Comma", value: ","},
+    {name: "Space", value: " "},
+    {name: "CRLF", value: "\\r\\n"},
+    {name: "Line Feed", value: "\n"}
+];

src/core/lib/Enigma.mjs

@@ -5,8 +5,8 @@
  * @copyright Crown Copyright 2019
  * @license Apache-2.0
  */
-import OperationError from "../errors/OperationError";
-import Utils from "../Utils";
+import OperationError from "../errors/OperationError.mjs";
+import Utils from "../Utils.mjs";
 
 /**
  * Provided default Enigma rotor set.
@@ -184,10 +184,10 @@ class PairMapBase {
                 // self-stecker
                 return;
             }
-            if (this.map.hasOwnProperty(a)) {
+            if (Object.prototype.hasOwnProperty.call(this.map, a)) {
                 throw new OperationError(`${name} connects ${pair[0]} more than once`);
             }
-            if (this.map.hasOwnProperty(b)) {
+            if (Object.prototype.hasOwnProperty.call(this.map, b)) {
                 throw new OperationError(`${name} connects ${pair[1]} more than once`);
             }
             this.map[a] = b;
@@ -203,7 +203,7 @@ class PairMapBase {
      * @returns {number}
      */
     transform(c) {
-        if (!this.map.hasOwnProperty(c)) {
+        if (!Object.prototype.hasOwnProperty.call(this.map, c)) {
             return c;
         }
         return this.map[c];

src/core/lib/FileType.mjs

@@ -6,8 +6,8 @@
  * @license Apache-2.0
  *
  */
-import {FILE_SIGNATURES} from "./FileSignatures";
-import {sendStatusMessage} from "../Utils";
+import {FILE_SIGNATURES} from "./FileSignatures.mjs";
+import {sendStatusMessage} from "../Utils.mjs";
 
 
 /**
@@ -75,7 +75,7 @@ function bytesMatch(sig, buf, offset=0) {
  * Given a buffer, detects magic byte sequences at specific positions and returns the
  * extension and mime type.
  *
- * @param {Uint8Array} buf
+ * @param {Uint8Array|ArrayBuffer} buf
  * @param {string[]} [categories=All] - Which categories of file to look for
  * @returns {Object[]} types
  * @returns {string} type.name - Name of file type
@@ -84,6 +84,10 @@ function bytesMatch(sig, buf, offset=0) {
  * @returns {string} [type.desc] - Description
  */
 export function detectFileType(buf, categories=Object.keys(FILE_SIGNATURES)) {
+    if (buf instanceof ArrayBuffer) {
+        buf = new Uint8Array(buf);
+    }
+
     if (!(buf && buf.length > 1)) {
         return [];
     }
@@ -174,7 +178,7 @@ export function scanForFileTypes(buf, categories=Object.keys(FILE_SIGNATURES)) {
  * @param {Uint8Array} buf - The buffer to search
  * @param {Object} sig - A single signature object (Not an array of signatures)
  * @param {number} offset - Where to start search from
- * @returs {number} The position of the match or -1 if one cannot be found.
+ * @returns {number} The position of the match or -1 if one cannot be found.
  */
 function locatePotentialSig(buf, sig, offset) {
     // Find values for first key and value in sig
@@ -203,7 +207,7 @@ function locatePotentialSig(buf, sig, offset) {
  * Detects whether the given buffer is a file of the type specified.
  *
  * @param {string|RegExp} type
- * @param {Uint8Array} buf
+ * @param {Uint8Array|ArrayBuffer} buf
  * @returns {string|false} The mime type or false if the type does not match
  */
 export function isType(type, buf) {
@@ -230,7 +234,7 @@ export function isType(type, buf) {
 /**
  * Detects whether the given buffer contains an image file.
  *
- * @param {Uint8Array} buf
+ * @param {Uint8Array|ArrayBuffer} buf
  * @returns {string|false} The mime type or false if the type does not match
  */
 export function isImage(buf) {

src/core/lib/Hash.mjs

@@ -7,8 +7,8 @@
  * @license Apache-2.0
  */
 
-import Utils from "../Utils";
-import CryptoApi from "crypto-api/src/crypto-api";
+import Utils from "../Utils.mjs";
+import CryptoApi from "crypto-api/src/crypto-api.mjs";
 
 
 /**

src/core/lib/Hex.mjs

@@ -6,13 +6,13 @@
  * @license Apache-2.0
  */
 
-import Utils from "../Utils";
+import Utils from "../Utils.mjs";
 
 
 /**
  * Convert a byte array into a hex string.
  *
- * @param {Uint8Array|byteArray} data
+ * @param {byteArray|Uint8Array|ArrayBuffer} data
  * @param {string} [delim=" "]
  * @param {number} [padding=2]
  * @returns {string}
@@ -26,6 +26,7 @@ import Utils from "../Utils";
  */
 export function toHex(data, delim=" ", padding=2) {
     if (!data) return "";
+    if (data instanceof ArrayBuffer) data = new Uint8Array(data);
 
     let output = "";
 
@@ -47,7 +48,7 @@ export function toHex(data, delim=" ", padding=2) {
 /**
  * Convert a byte array into a hex string as efficiently as possible with no options.
  *
- * @param {byteArray} data
+ * @param {byteArray|Uint8Array|ArrayBuffer} data
  * @returns {string}
  *
  * @example
@@ -56,6 +57,7 @@ export function toHex(data, delim=" ", padding=2) {
  */
 export function toHexFast(data) {
     if (!data) return "";
+    if (data instanceof ArrayBuffer) data = new Uint8Array(data);
 
     const output = [];
 

src/core/lib/IP.mjs

@@ -8,8 +8,8 @@
  * @license Apache-2.0
  */
 
-import Utils from "../Utils";
-import OperationError from "../errors/OperationError";
+import Utils from "../Utils.mjs";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * Parses an IPv4 CIDR range (e.g. 192.168.0.0/24) and displays information about it.
@@ -241,7 +241,7 @@ export function ipv6ListedRange(match, includeNetworkInfo) {
     ipv6List = ipv6List.filter(function(str) {
         return str.trim();
     });
-    for (let i =0; i < ipv6List.length; i++){
+    for (let i =0; i < ipv6List.length; i++) {
         ipv6List[i] = ipv6List[i].trim();
     }
     const ipv6CidrList = ipv6List.filter(function(a) {
@@ -502,8 +502,8 @@ export function ipv6Compare(a, b) {
     const a_ = strToIpv6(a),
         b_ = strToIpv6(b);
 
-    for (let i = 0; i < a_.length; i++){
-        if (a_[i] !== b_[i]){
+    for (let i = 0; i < a_.length; i++) {
+        if (a_[i] !== b_[i]) {
             return a_[i] - b_[i];
         }
     }

src/core/lib/ImageManipulation.mjs

@@ -6,7 +6,7 @@
  * @license Apache-2.0
  */
 
-import OperationError from "../errors/OperationError";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * Gaussian blurs an image.

src/core/lib/LoremIpsum.mjs

@@ -85,7 +85,7 @@ function getWords(length=3) {
     const words = [];
     let word;
     let previousWord;
-    while (words.length < length){
+    while (words.length < length) {
         do {
             word = wordList[Math.floor(Math.random() * wordList.length)];
         } while (previousWord === word);

src/core/lib/Magic.mjs

@@ -1,8 +1,8 @@
 import OperationConfig from "../config/OperationConfig.json";
-import Utils from "../Utils";
-import Recipe from "../Recipe";
-import Dish from "../Dish";
-import {detectFileType} from "./FileType";
+import Utils, { isWorkerEnvironment } from "../Utils.mjs";
+import Recipe from "../Recipe.mjs";
+import Dish from "../Dish.mjs";
+import {detectFileType} from "./FileType.mjs";
 import chiSquared from "chi-squared";
 
 /**
@@ -97,6 +97,7 @@ class Magic {
 
         if (!fileType.length) return null;
         return {
+            name: fileType[0].name,
             ext: fileType[0].extension,
             mime: fileType[0].mime,
             desc: fileType[0].description
@@ -338,7 +339,7 @@ class Magic {
         }
 
         // Prune branches that result in unhelpful outputs
-        results = results.filter(r =>
+        const prunedResults = results.filter(r =>
             (r.useful || r.data.length > 0) &&          // The operation resulted in ""
             (                                           // One of the following must be true
                 r.languageScores[0].probability > 0 ||    // Some kind of language was found
@@ -349,22 +350,22 @@ class Magic {
         );
 
         // Return a sorted list of possible recipes along with their properties
-        return results.sort((a, b) => {
+        return prunedResults.sort((a, b) => {
             // Each option is sorted based on its most likely language (lower is better)
             let aScore = a.languageScores[0].score,
                 bScore = b.languageScores[0].score;
 
-            // If a recipe results in a file being detected, it receives a relatively good score
-            if (a.fileType) aScore = 500;
-            if (b.fileType) bScore = 500;
-
             // If the result is valid UTF8, its score gets boosted (lower being better)
             if (a.isUTF8) aScore -= 100;
             if (b.isUTF8) bScore -= 100;
 
+            // If a recipe results in a file being detected, it receives a relatively good score
+            if (a.fileType && aScore > 500) aScore = 500;
+            if (b.fileType && bScore > 500) bScore = 500;
+
             // If the option is marked useful, give it a good score
-            if (a.useful) aScore = 100;
-            if (b.useful) bScore = 100;
+            if (a.useful && aScore > 100) aScore = 100;
+            if (b.useful && bScore > 100) bScore = 100;
 
             // Shorter recipes are better, so we add the length of the recipe to the score
             aScore += a.recipe.length;
@@ -395,7 +396,7 @@ class Magic {
         const dish = new Dish();
         dish.set(input, Dish.ARRAY_BUFFER);
 
-        if (ENVIRONMENT_IS_WORKER()) self.loadRequiredModules(recipeConfig);
+        if (isWorkerEnvironment()) self.loadRequiredModules(recipeConfig);
 
         const recipe = new Recipe(recipeConfig);
         try {
@@ -450,7 +451,7 @@ class Magic {
         const opPatterns = [];
 
         for (const op in OperationConfig) {
-            if (!OperationConfig[op].hasOwnProperty("patterns")) continue;
+            if (!("patterns" in OperationConfig[op])) continue;
 
             OperationConfig[op].patterns.forEach(pattern => {
                 opPatterns.push({
@@ -497,7 +498,7 @@ class Magic {
      * Taken from http://wikistats.wmflabs.org/display.php?t=wp
      *
      * @param {string} code - ISO 639 code
-     * @returns {string} The full name of the languge
+     * @returns {string} The full name of the language
      */
     static codeToLanguage(code) {
         return {

src/core/lib/PGP.mjs

@@ -10,7 +10,8 @@
  *
  */
 
-import OperationError from "../errors/OperationError";
+import OperationError from "../errors/OperationError.mjs";
+import { isWorkerEnvironment } from "../Utils.mjs";
 import kbpgp from "kbpgp";
 import * as es6promisify from "es6-promisify";
 const promisify = es6promisify.default ? es6promisify.default.promisify : es6promisify.promisify;
@@ -45,7 +46,7 @@ export const ASP = kbpgp.ASP({
                 msg = `Stage: ${info.what}`;
         }
 
-        if (ENVIRONMENT_IS_WORKER())
+        if (isWorkerEnvironment())
             self.sendStatusMessage(msg);
     }
 });

src/core/lib/Protobuf.mjs

@@ -1,4 +1,4 @@
-import Utils from "../Utils";
+import Utils from "../Utils.mjs";
 
 /**
  * Protobuf lib. Contains functions to decode protobuf serialised
@@ -16,14 +16,14 @@ class Protobuf {
     /**
      * Protobuf constructor
      *
-     * @param {byteArray} data
+     * @param {byteArray|Uint8Array} data
      */
     constructor(data) {
-        // Check we have a byteArray
-        if (data instanceof Array) {
+        // Check we have a byteArray or Uint8Array
+        if (data instanceof Array || data instanceof Uint8Array) {
             this.data = data;
         } else {
-            throw new Error("Protobuf input must be a byteArray");
+            throw new Error("Protobuf input must be a byteArray or Uint8Array");
         }
 
         // Set up masks
@@ -124,7 +124,7 @@ class Protobuf {
         // Get the field key/values
         const key = field.key;
         const value = field.value;
-        object[key] = object.hasOwnProperty(key) ?
+        object[key] = Object.prototype.hasOwnProperty.call(object, key) ?
             object[key] instanceof Array ?
                 object[key].concat([value]) :
                 [object[key], value] :
@@ -205,7 +205,7 @@ class Protobuf {
                     (this.data[this.offset] & this.VALUE) << shift :
                 (this.data[this.offset] & this.VALUE) * Math.pow(2, shift);
             shift += 7;
-        } while ((this.data[this.offset++] & this.MSD) === this.MSB);
+        } while ((this.data[this.offset++] & this.MSB) === this.MSB);
         return fieldNumber;
     }
 

src/core/lib/PublicKey.mjs

@@ -6,7 +6,7 @@
  * @license Apache-2.0
  */
 
-import { toHex, fromHex } from "./Hex";
+import { toHex, fromHex } from "./Hex.mjs";
 
 /**
  * Formats Distinguished Name (DN) strings.

src/core/lib/QRCode.mjs

@@ -6,11 +6,11 @@
  * @license Apache-2.0
  */
 
-import OperationError from "../errors/OperationError";
+import OperationError from "../errors/OperationError.mjs";
 import jsQR from "jsqr";
 import qr from "qr-image";
 import jimp from "jimp";
-import Utils from "../Utils";
+import Utils from "../Utils.mjs";
 
 /**
  * Parses a QR code image from an image
@@ -37,7 +37,7 @@ export async function parseQrCode(input, normalise) {
             image = await jimp.read(image);
         }
     } catch (err) {
-        throw new OperationError(`Error normalising iamge. (${err})`);
+        throw new OperationError(`Error normalising image. (${err})`);
     }
 
     const qrData = jsQR(image.bitmap.data, image.getWidth(), image.getHeight());

src/core/lib/Stream.mjs

@@ -155,19 +155,69 @@ export default class Stream {
         }
 
         // val is an array
-        let found = false;
-        while (!found && this.position < this.length) {
-            while (++this.position < this.length && this.bytes[this.position] !== val[0]) {
-                continue;
-            }
+
+        /**
+         * Builds the skip forward table from the value to be searched.
+         *
+         * @param {Uint8Array} val
+         * @param {Number} len
+         * @returns {Uint8Array}
+         */
+        function preprocess(val, len) {
+            const skiptable = new Array();
+            val.forEach((element, index) => {
+                skiptable[element] = len - index;
+            });
+            return skiptable;
+        }
+
+        const length = val.length;
+        const initial = val[length-1];
+        this.position = length;
+
+        // Get the skip table.
+        const skiptable = preprocess(val, length);
+        let found = true;
+
+        while (this.position < this.length) {
+            // Until we hit the final element of val in the stream.
+            while ((this.position < this.length) && (this.bytes[this.position++] !== initial));
+
             found = true;
-            for (let i = 1; i < val.length; i++) {
-                if (this.position + i > this.length || this.bytes[this.position + i] !== val[i])
+
+            // Loop through the elements comparing them to val.
+            for (let x = length-1; x >= 0; x--) {
+                if (this.bytes[this.position - length + x] !== val[x]) {
                     found = false;
+
+                    // If element is not equal to val's element then jump forward by the correct amount.
+                    this.position += skiptable[val[x]];
+                    break;
+                }
+            }
+            if (found) {
+                this.position -= length;
+                break;
             }
         }
     }
 
+
+    /**
+     * Consume bytes if they match the supplied value.
+     *
+     * @param {Number} val
+     */
+    consumeWhile(val) {
+        while (this.position < this.length) {
+            if (this.bytes[this.position] !== val) {
+                break;
+            }
+            this.position++;
+        }
+        this.bitPos = 0;
+    }
+
     /**
      * Consume the next byte if it matches the supplied value.
      *

src/core/lib/TLVParser.mjs

@@ -21,7 +21,7 @@ export default class TLVParser {
     /**
      * TLVParser constructor
      *
-     * @param {byteArray} input
+     * @param {byteArray|Uint8Array} input
      * @param {Object} options
      */
     constructor(input, options) {

src/core/lib/Typex.mjs

@@ -6,9 +6,9 @@
  * @copyright Crown Copyright 2019
  * @license Apache-2.0
  */
-import OperationError from "../errors/OperationError";
-import * as Enigma from "../lib/Enigma";
-import Utils from "../Utils";
+import OperationError from "../errors/OperationError.mjs";
+import * as Enigma from "../lib/Enigma.mjs";
+import Utils from "../Utils.mjs";
 
 /**
  * A set of example Typex rotors. No Typex rotor wirings are publicly available, so these are
@@ -98,14 +98,14 @@ export class TypexMachine extends Enigma.EnigmaBase {
                 if (x === " ") {
                     inputMod += "X";
                 } else if (mode) {
-                    if (KEYBOARD_REV.hasOwnProperty(x)) {
+                    if (Object.prototype.hasOwnProperty.call(KEYBOARD_REV, x)) {
                         inputMod += KEYBOARD_REV[x];
                     } else {
                         mode = false;
                         inputMod += "V" + x;
                     }
                 } else {
-                    if (KEYBOARD_REV.hasOwnProperty(x)) {
+                    if (Object.prototype.hasOwnProperty.call(KEYBOARD_REV, x)) {
                         mode = true;
                         inputMod += "Z" + KEYBOARD_REV[x];
                     } else {

src/core/lib/Zlib.mjs

@@ -6,7 +6,7 @@
  * @license Apache-2.0
  */
 
-import zlibAndGzip from "zlibjs/bin/zlib_and_gzip.min";
+import zlibAndGzip from "zlibjs/bin/zlib_and_gzip.min.js";
 
 const Zlib = zlibAndGzip.Zlib;
 

src/core/operations/A1Z26CipherDecode.mjs

@@ -4,10 +4,10 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import Utils from "../Utils";
-import {DELIM_OPTIONS} from "../lib/Delim";
-import OperationError from "../errors/OperationError";
+import Operation from "../Operation.mjs";
+import Utils from "../Utils.mjs";
+import {DELIM_OPTIONS} from "../lib/Delim.mjs";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * A1Z26 Cipher Decode operation

src/core/operations/A1Z26CipherEncode.mjs

@@ -4,9 +4,9 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import Utils from "../Utils";
-import {DELIM_OPTIONS} from "../lib/Delim";
+import Operation from "../Operation.mjs";
+import Utils from "../Utils.mjs";
+import {DELIM_OPTIONS} from "../lib/Delim.mjs";
 
 /**
  * A1Z26 Cipher Encode operation

src/core/operations/ADD.mjs

@@ -4,9 +4,9 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import Utils from "../Utils";
-import { bitOp, add, BITWISE_OP_DELIMS } from "../lib/BitwiseOp";
+import Operation from "../Operation.mjs";
+import Utils from "../Utils.mjs";
+import { bitOp, add, BITWISE_OP_DELIMS } from "../lib/BitwiseOp.mjs";
 
 /**
  * ADD operation

src/core/operations/AESDecrypt.mjs

@@ -4,10 +4,10 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import Utils from "../Utils";
+import Operation from "../Operation.mjs";
+import Utils from "../Utils.mjs";
 import forge from "node-forge/dist/forge.min.js";
-import OperationError from "../errors/OperationError";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * AES Decrypt operation
@@ -71,8 +71,8 @@ class AESDecrypt extends Operation {
      * @throws {OperationError} if cannot decrypt input or invalid key length
      */
     run(input, args) {
-        const key = Utils.convertToByteArray(args[0].string, args[0].option),
-            iv = Utils.convertToByteArray(args[1].string, args[1].option),
+        const key = Utils.convertToByteString(args[0].string, args[0].option),
+            iv = Utils.convertToByteString(args[1].string, args[1].option),
             mode = args[2],
             inputType = args[3],
             outputType = args[4],
@@ -91,7 +91,7 @@ The following algorithms will be used based on the size of the key:
 
         const decipher = forge.cipher.createDecipher("AES-" + mode, key);
         decipher.start({
-            iv: iv,
+            iv: iv.length === 0 ? "" : iv,
             tag: gcmTag
         });
         decipher.update(forge.util.createBuffer(input));

src/core/operations/AESEncrypt.mjs

@@ -4,10 +4,10 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import Utils from "../Utils";
+import Operation from "../Operation.mjs";
+import Utils from "../Utils.mjs";
 import forge from "node-forge/dist/forge.min.js";
-import OperationError from "../errors/OperationError";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * AES Encrypt operation

src/core/operations/AND.mjs

@@ -4,9 +4,9 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import Utils from "../Utils";
-import { bitOp, and, BITWISE_OP_DELIMS } from "../lib/BitwiseOp";
+import Operation from "../Operation.mjs";
+import Utils from "../Utils.mjs";
+import { bitOp, and, BITWISE_OP_DELIMS } from "../lib/BitwiseOp.mjs";
 
 /**
  * AND operation

src/core/operations/AddLineNumbers.mjs

@@ -4,7 +4,7 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
+import Operation from "../Operation.mjs";
 
 /**
  * Add line numbers operation

src/core/operations/AddTextToImage.mjs

@@ -4,10 +4,11 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import OperationError from "../errors/OperationError";
-import { isImage } from "../lib/FileType";
-import { toBase64 } from "../lib/Base64";
+import Operation from "../Operation.mjs";
+import OperationError from "../errors/OperationError.mjs";
+import { isImage } from "../lib/FileType.mjs";
+import { toBase64 } from "../lib/Base64.mjs";
+import { isWorkerEnvironment } from "../Utils.mjs";
 import jimp from "jimp";
 
 /**
@@ -120,7 +121,7 @@ class AddTextToImage extends Operation {
         let xPos = args[3],
             yPos = args[4];
 
-        if (!isImage(new Uint8Array(input))) {
+        if (!isImage(input)) {
             throw new OperationError("Invalid file type.");
         }
 
@@ -131,7 +132,7 @@ class AddTextToImage extends Operation {
             throw new OperationError(`Error loading image. (${err})`);
         }
         try {
-            if (ENVIRONMENT_IS_WORKER())
+            if (isWorkerEnvironment())
                 self.sendStatusMessage("Adding text to image...");
 
             const fontsMap = {};

src/core/operations/Adler32Checksum.mjs

@@ -4,8 +4,8 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import Utils from "../Utils";
+import Operation from "../Operation.mjs";
+import Utils from "../Utils.mjs";
 
 /**
  * Adler-32 Checksum operation
@@ -22,13 +22,13 @@ class Adler32Checksum extends Operation {
         this.module = "Crypto";
         this.description = "Adler-32 is a checksum algorithm which was invented by Mark Adler in 1995, and is a modification of the Fletcher checksum. Compared to a cyclic redundancy check of the same length, it trades reliability for speed (preferring the latter).<br><br>Adler-32 is more reliable than Fletcher-16, and slightly less reliable than Fletcher-32.";
         this.infoURL = "https://wikipedia.org/wiki/Adler-32";
-        this.inputType = "byteArray";
+        this.inputType = "ArrayBuffer";
         this.outputType = "string";
         this.args = [];
     }
 
     /**
-     * @param {byteArray} input
+     * @param {ArrayBuffer} input
      * @param {Object[]} args
      * @returns {string}
      */
@@ -36,6 +36,7 @@ class Adler32Checksum extends Operation {
         const MOD_ADLER = 65521;
         let a = 1,
             b = 0;
+        input = new Uint8Array(input);
 
         for (let i = 0; i < input.length; i++) {
             a += input[i];

src/core/operations/AffineCipherDecode.mjs

@@ -4,9 +4,9 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import Utils from "../Utils";
-import OperationError from "../errors/OperationError";
+import Operation from "../Operation.mjs";
+import Utils from "../Utils.mjs";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * Affine Cipher Decode operation

src/core/operations/AffineCipherEncode.mjs

@@ -4,8 +4,8 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import { affineEncode } from "../lib/Ciphers";
+import Operation from "../Operation.mjs";
+import { affineEncode } from "../lib/Ciphers.mjs";
 
 /**
  * Affine Cipher Encode operation

src/core/operations/AnalyseHash.mjs

@@ -4,8 +4,8 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import OperationError from "../errors/OperationError";
+import Operation from "../Operation.mjs";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * Analyse hash operation

src/core/operations/AtbashCipher.mjs

@@ -4,8 +4,8 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import { affineEncode } from "../lib/Ciphers";
+import Operation from "../Operation.mjs";
+import { affineEncode } from "../lib/Ciphers.mjs";
 
 /**
  * Atbash Cipher operation

src/core/operations/AvroToJSON.mjs

@@ -0,0 +1,76 @@
+/**
+ * @author jarrodconnolly [jarrod@nestedquotes.ca]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+import Operation from "../Operation.mjs";
+import OperationError from "../errors/OperationError.mjs";
+import avro from "avsc";
+
+/**
+ * Avro to JSON operation
+ */
+class AvroToJSON extends Operation {
+
+    /**
+     * AvroToJSON constructor
+     */
+    constructor() {
+        super();
+
+        this.name = "Avro to JSON";
+        this.module = "Serialise";
+        this.description = "Converts Avro encoded data into JSON.";
+        this.infoURL = "https://wikipedia.org/wiki/Apache_Avro";
+        this.inputType = "ArrayBuffer";
+        this.outputType = "string";
+        this.args = [
+            {
+                name: "Force Valid JSON",
+                type: "boolean",
+                value: true
+            }
+        ];
+    }
+
+    /**
+     * @param {ArrayBuffer} input
+     * @param {Object[]} args
+     * @returns {string}
+     */
+    run(input, args) {
+        if (input.byteLength <= 0) {
+            throw new OperationError("Please provide an input.");
+        }
+
+        const forceJSON = args[0];
+
+        return new Promise((resolve, reject) => {
+            const result = [];
+            const inpArray = new Uint8Array(input);
+            const decoder = new avro.streams.BlockDecoder();
+
+            decoder
+                .on("data", function (obj) {
+                    result.push(obj);
+                })
+                .on("error", function () {
+                    reject(new OperationError("Error parsing Avro file."));
+                })
+                .on("end", function () {
+                    if (forceJSON) {
+                        resolve(result.length === 1 ? JSON.stringify(result[0], null, 4) : JSON.stringify(result, null, 4));
+                    } else {
+                        const data = result.reduce((result, current) => result + JSON.stringify(current) + "\n", "");
+                        resolve(data);
+                    }
+                });
+
+            decoder.write(inpArray);
+            decoder.end();
+        });
+    }
+}
+
+export default AvroToJSON;

src/core/operations/BLAKE2b.mjs

@@ -4,11 +4,11 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
+import Operation from "../Operation.mjs";
 import blakejs from "blakejs";
-import OperationError from "../errors/OperationError";
-import Utils from "../Utils";
-import { toBase64 } from "../lib/Base64";
+import OperationError from "../errors/OperationError.mjs";
+import Utils from "../Utils.mjs";
+import { toBase64 } from "../lib/Base64.mjs";
 
 /**
  * BLAKE2b operation
@@ -50,7 +50,7 @@ class BLAKE2b extends Operation {
     /**
      * @param {ArrayBuffer} input
      * @param {Object[]} args
-     * @returns {string} The input having been hashed with BLAKE2b in the encoding format speicifed.
+     * @returns {string} The input having been hashed with BLAKE2b in the encoding format specified.
      */
     run(input, args) {
         const [outSize, outFormat] = args;

src/core/operations/BLAKE2s.mjs

@@ -4,11 +4,11 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
+import Operation from "../Operation.mjs";
 import blakejs from "blakejs";
-import OperationError from "../errors/OperationError";
-import Utils from "../Utils";
-import { toBase64 } from "../lib/Base64";
+import OperationError from "../errors/OperationError.mjs";
+import Utils from "../Utils.mjs";
+import { toBase64 } from "../lib/Base64.mjs";
 
 /**
  * BLAKE2s Operation
@@ -51,7 +51,7 @@ class BLAKE2s extends Operation {
     /**
      * @param {ArrayBuffer} input
      * @param {Object[]} args
-     * @returns {string} The input having been hashed with BLAKE2s in the encoding format speicifed.
+     * @returns {string} The input having been hashed with BLAKE2s in the encoding format specified.
      */
     run(input, args) {
         const [outSize, outFormat] = args;

src/core/operations/BSONDeserialise.mjs

@@ -4,9 +4,9 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
+import Operation from "../Operation.mjs";
 import bson from "bson";
-import OperationError from "../errors/OperationError";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * BSON deserialise operation
@@ -20,7 +20,7 @@ class BSONDeserialise extends Operation {
         super();
 
         this.name = "BSON deserialise";
-        this.module = "BSON";
+        this.module = "Serialise";
         this.description = "BSON is a computer data interchange format used mainly as a data storage and network transfer format in the MongoDB database. It is a binary form for representing simple data structures, associative arrays (called objects or documents in MongoDB), and various data types of specific interest to MongoDB. The name 'BSON' is based on the term JSON and stands for 'Binary JSON'.<br><br>Input data should be in a raw bytes format.";
         this.infoURL = "https://wikipedia.org/wiki/BSON";
         this.inputType = "ArrayBuffer";

src/core/operations/BSONSerialise.mjs

@@ -4,9 +4,9 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
+import Operation from "../Operation.mjs";
 import bson from "bson";
-import OperationError from "../errors/OperationError";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * BSON serialise operation
@@ -20,7 +20,7 @@ class BSONSerialise extends Operation {
         super();
 
         this.name = "BSON serialise";
-        this.module = "BSON";
+        this.module = "Serialise";
         this.description = "BSON is a computer data interchange format used mainly as a data storage and network transfer format in the MongoDB database. It is a binary form for representing simple data structures, associative arrays (called objects or documents in MongoDB), and various data types of specific interest to MongoDB. The name 'BSON' is based on the term JSON and stands for 'Binary JSON'.<br><br>Input data should be valid JSON.";
         this.infoURL = "https://wikipedia.org/wiki/BSON";
         this.inputType = "string";

src/core/operations/BaconCipherDecode.mjs

@@ -0,0 +1,107 @@
+/**
+ * @author Karsten Silkenbäumer [github.com/kassi]
+ * @copyright Karsten Silkenbäumer 2019
+ * @license Apache-2.0
+ */
+
+import Operation from "../Operation.mjs";
+import {
+    BACON_ALPHABETS,
+    BACON_TRANSLATION_CASE, BACON_TRANSLATION_AMNZ, BACON_TRANSLATIONS, BACON_CLEARER_MAP, BACON_NORMALIZE_MAP,
+    swapZeroAndOne
+} from "../lib/Bacon.mjs";
+
+/**
+ * Bacon Cipher Decode operation
+ */
+class BaconCipherDecode extends Operation {
+    /**
+     * BaconCipherDecode constructor
+     */
+    constructor() {
+        super();
+
+        this.name = "Bacon Cipher Decode";
+        this.module = "Default";
+        this.description = "Bacon's cipher or the Baconian cipher is a method of steganography devised by Francis Bacon in 1605. A message is concealed in the presentation of text, rather than its content.";
+        this.infoURL = "https://wikipedia.org/wiki/Bacon%27s_cipher";
+        this.inputType = "string";
+        this.outputType = "string";
+        this.args = [
+            {
+                "name": "Alphabet",
+                "type": "option",
+                "value": Object.keys(BACON_ALPHABETS)
+            },
+            {
+                "name": "Translation",
+                "type": "option",
+                "value": BACON_TRANSLATIONS
+            },
+            {
+                "name": "Invert Translation",
+                "type": "boolean",
+                "value": false
+            }
+        ];
+    }
+
+    /**
+     * @param {string} input
+     * @param {Object[]} args
+     * @returns {string}
+     */
+    run(input, args) {
+        const [alphabet, translation, invert] = args;
+        const alphabetObject = BACON_ALPHABETS[alphabet];
+
+        // remove invalid characters
+        input = input.replace(BACON_CLEARER_MAP[translation], "");
+
+        // normalize to unique alphabet
+        if (BACON_NORMALIZE_MAP[translation] !== undefined) {
+            input = input.replace(/./g, function (c) {
+                return BACON_NORMALIZE_MAP[translation][c];
+            });
+        } else if (translation === BACON_TRANSLATION_CASE) {
+            const codeA = "A".charCodeAt(0);
+            const codeZ = "Z".charCodeAt(0);
+            input = input.replace(/./g, function (c) {
+                const code = c.charCodeAt(0);
+                if (code >= codeA && code <= codeZ) {
+                    return "1";
+                } else {
+                    return "0";
+                }
+            });
+        } else if (translation === BACON_TRANSLATION_AMNZ) {
+            const words = input.split(/\s+/);
+            const letters = words.map(function (e) {
+                if (e) {
+                    const code = e[0].toUpperCase().charCodeAt(0);
+                    return code >= "N".charCodeAt(0) ? "1" : "0";
+                } else {
+                    return "";
+                }
+            });
+            input = letters.join("");
+        }
+
+        if (invert) {
+            input = swapZeroAndOne(input);
+        }
+
+        // group into 5
+        const inputArray = input.match(/(.{5})/g) || [];
+
+        let output = "";
+        for (let i = 0; i < inputArray.length; i++) {
+            const code = inputArray[i];
+            const number = parseInt(code, 2);
+            output += number < alphabetObject.alphabet.length ? alphabetObject.alphabet[number] : "?";
+        }
+        return output;
+    }
+}
+
+export default BaconCipherDecode;

src/core/operations/BaconCipherEncode.mjs

@@ -0,0 +1,101 @@
+/**
+ * @author Karsten Silkenbäumer [github.com/kassi]
+ * @copyright Karsten Silkenbäumer 2019
+ * @license Apache-2.0
+ */
+
+import Operation from "../Operation.mjs";
+import {
+    BACON_ALPHABETS,
+    BACON_TRANSLATIONS_FOR_ENCODING, BACON_TRANSLATION_AB,
+    swapZeroAndOne
+} from "../lib/Bacon.mjs";
+
+/**
+ * Bacon Cipher Encode operation
+ */
+class BaconCipherEncode extends Operation {
+    /**
+     * BaconCipherEncode constructor
+     */
+    constructor() {
+        super();
+
+        this.name = "Bacon Cipher Encode";
+        this.module = "Default";
+        this.description = "Bacon's cipher or the Baconian cipher is a method of steganography devised by Francis Bacon in 1605. A message is concealed in the presentation of text, rather than its content.";
+        this.infoURL = "https://wikipedia.org/wiki/Bacon%27s_cipher";
+        this.inputType = "string";
+        this.outputType = "string";
+        this.args = [
+            {
+                "name": "Alphabet",
+                "type": "option",
+                "value": Object.keys(BACON_ALPHABETS)
+            },
+            {
+                "name": "Translation",
+                "type": "option",
+                "value": BACON_TRANSLATIONS_FOR_ENCODING
+            },
+            {
+                "name": "Keep extra characters",
+                "type": "boolean",
+                "value": false
+            },
+            {
+                "name": "Invert Translation",
+                "type": "boolean",
+                "value": false
+            }
+        ];
+    }
+
+    /**
+     * @param {string} input
+     * @param {Object[]} args
+     * @returns {string}
+     */
+    run(input, args) {
+        const [alphabet, translation, keep, invert] = args;
+
+        const alphabetObject = BACON_ALPHABETS[alphabet];
+        const charCodeA = "A".charCodeAt(0);
+        const charCodeZ = "Z".charCodeAt(0);
+
+        let output = input.replace(/./g, function (c) {
+            const charCode = c.toUpperCase().charCodeAt(0);
+            if (charCode >= charCodeA && charCode <= charCodeZ) {
+                let code = charCode - charCodeA;
+                if (alphabetObject.codes !== undefined) {
+                    code = alphabetObject.codes[code];
+                }
+                const bacon = ("00000" + code.toString(2)).substr(-5, 5);
+                return bacon;
+            } else {
+                return c;
+            }
+        });
+
+        if (invert) {
+            output = swapZeroAndOne(output);
+        }
+        if (!keep) {
+            output = output.replace(/[^01]/g, "");
+            const outputArray = output.match(/(.{5})/g) || [];
+            output = outputArray.join(" ");
+        }
+        if (translation === BACON_TRANSLATION_AB) {
+            output = output.replace(/[01]/g, function (c) {
+                return {
+                    "0": "A",
+                    "1": "B"
+                }[c];
+            });
+        }
+
+        return output;
+    }
+}
+
+export default BaconCipherEncode;

src/core/operations/Bcrypt.mjs

@@ -4,8 +4,9 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
+import Operation from "../Operation.mjs";
 import bcrypt from "bcryptjs";
+import { isWorkerEnvironment } from "../Utils.mjs";
 
 /**
  * Bcrypt operation
@@ -44,7 +45,7 @@ class Bcrypt extends Operation {
 
         return await bcrypt.hash(input, salt, null, p => {
             // Progress callback
-            if (ENVIRONMENT_IS_WORKER())
+            if (isWorkerEnvironment())
                 self.sendStatusMessage(`Progress: ${(p * 100).toFixed(0)}%`);
         });
 

src/core/operations/BcryptCompare.mjs

@@ -4,8 +4,10 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
+import Operation from "../Operation.mjs";
 import bcrypt from "bcryptjs";
+import { isWorkerEnvironment } from "../Utils.mjs";
+
 
 /**
  * Bcrypt compare operation
@@ -43,7 +45,7 @@ class BcryptCompare extends Operation {
 
         const match = await bcrypt.compare(input, hash, null, p => {
             // Progress callback
-            if (ENVIRONMENT_IS_WORKER())
+            if (isWorkerEnvironment())
                 self.sendStatusMessage(`Progress: ${(p * 100).toFixed(0)}%`);
         });
 

src/core/operations/BcryptParse.mjs

@@ -4,8 +4,8 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import OperationError from "../errors/OperationError";
+import Operation from "../Operation.mjs";
+import OperationError from "../errors/OperationError.mjs";
 import bcrypt from "bcryptjs";
 
 /**

src/core/operations/BifidCipherDecode.mjs

@@ -4,9 +4,9 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import { genPolybiusSquare } from "../lib/Ciphers";
-import OperationError from "../errors/OperationError";
+import Operation from "../Operation.mjs";
+import { genPolybiusSquare } from "../lib/Ciphers.mjs";
+import OperationError from "../errors/OperationError.mjs";
 
 /**
  * Bifid Cipher Decode operation

src/core/operations/BifidCipherEncode.mjs

@@ -4,9 +4,9 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
-import OperationError from "../errors/OperationError";
-import { genPolybiusSquare } from "../lib/Ciphers";
+import Operation from "../Operation.mjs";
+import OperationError from "../errors/OperationError.mjs";
+import { genPolybiusSquare } from "../lib/Ciphers.mjs";
 
 /**
  * Bifid Cipher Encode operation

src/core/operations/BitShiftLeft.mjs

@@ -4,7 +4,7 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
+import Operation from "../Operation.mjs";
 
 /**
  * Bit shift left operation
@@ -21,8 +21,8 @@ class BitShiftLeft extends Operation {
         this.module = "Default";
         this.description = "Shifts the bits in each byte towards the left by the specified amount.";
         this.infoURL = "https://wikipedia.org/wiki/Bitwise_operation#Bit_shifts";
-        this.inputType = "byteArray";
-        this.outputType = "byteArray";
+        this.inputType = "ArrayBuffer";
+        this.outputType = "ArrayBuffer";
         this.args = [
             {
                 "name": "Amount",
@@ -33,16 +33,17 @@ class BitShiftLeft extends Operation {
     }
 
     /**
-     * @param {byteArray} input
+     * @param {ArrayBuffer} input
      * @param {Object[]} args
-     * @returns {byteArray}
+     * @returns {ArrayBuffer}
      */
     run(input, args) {
         const amount = args[0];
+        input = new Uint8Array(input);
 
         return input.map(b => {
             return (b << amount) & 0xff;
-        });
+        }).buffer;
     }
 
     /**

src/core/operations/BitShiftRight.mjs

@@ -4,7 +4,7 @@
  * @license Apache-2.0
  */
 
-import Operation from "../Operation";
+import Operation from "../Operation.mjs";
 
 /**
  * Bit shift right operation
@@ -21,8 +21,8 @@ class BitShiftRight extends Operation {
         this.module = "Default";
         this.description = "Shifts the bits in each byte towards the right by the specified amount.<br><br><i>Logical shifts</i> replace the leftmost bits with zeros.<br><i>Arithmetic shifts</i> preserve the most significant bit (MSB) of the original byte keeping the sign the same (positive or negative).";
         this.infoURL = "https://wikipedia.org/wiki/Bitwise_operation#Bit_shifts";
-        this.inputType = "byteArray";
-        this.outputType = "byteArray";
+        this.inputType = "ArrayBuffer";
+        this.outputType = "ArrayBuffer";
         this.args = [
             {
                 "name": "Amount",
@@ -38,18 +38,19 @@ class BitShiftRight extends Operation {
     }
 
     /**
-     * @param {byteArray} input
+     * @param {ArrayBuffer} input
      * @param {Object[]} args
-     * @returns {byteArray}
+     * @returns {ArrayBuffer}
      */
     run(input, args) {
         const amount = args[0],
             type = args[1],
             mask = type === "Logical shift" ? 0 : 0x80;
+        input = new Uint8Array(input);
 
         return input.map(b => {
             return (b >>> amount) ^ (b & mask);
-        });
+        }).buffer;
     }
 
     /**