9.2.0

Updated tests to match new results.
Included a Python script to generate AES-GCM tests

CHANGELOG.md

@@ -2,6 +2,13 @@
 All major and minor version changes will be documented in this file. Details of patch-level version changes can be found in [commit messages](https://github.com/gchq/CyberChef/commits/master).
 
 
+### [9.2.0] - 2019-08-23
+- 'Parse UDP' operation added [@h345983745] | [#614]
+
+### [9.1.0] - 2019-08-22
+- 'Parse SSH Host Key' operation added [@j433866] | [#595]
+- 'Defang IP Addresses' operation added [@h345983745] | [#556]
+
 ## [9.0.0] - 2019-07-09
 - [Multiple inputs](https://github.com/gchq/CyberChef/wiki/Multiple-Inputs) are now supported in the main web UI, allowing you to upload and process multiple files at once [@j433866] | [#566]
 - A [Node.js API](https://github.com/gchq/CyberChef/wiki/Node-API) has been implemented, meaning that CyberChef can now be used as a library, either to provide specific operations, or an entire baking environment [@d98762625] | [#291]
@@ -158,6 +165,8 @@ All major and minor version changes will be documented in this file. Details of
 
 
 
+[9.2.0]: https://github.com/gchq/CyberChef/releases/tag/v9.2.0
+[9.1.0]: https://github.com/gchq/CyberChef/releases/tag/v9.1.0
 [9.0.0]: https://github.com/gchq/CyberChef/releases/tag/v9.0.0
 [8.38.0]: https://github.com/gchq/CyberChef/releases/tag/v8.38.0
 [8.37.0]: https://github.com/gchq/CyberChef/releases/tag/v8.37.0
@@ -275,7 +284,10 @@ All major and minor version changes will be documented in this file. Details of
 [#531]: https://github.com/gchq/CyberChef/pull/531
 [#533]: https://github.com/gchq/CyberChef/pull/533
 [#535]: https://github.com/gchq/CyberChef/pull/535
+[#556]: https://github.com/gchq/CyberChef/pull/556
 [#566]: https://github.com/gchq/CyberChef/pull/566
 [#571]: https://github.com/gchq/CyberChef/pull/571
 [#585]: https://github.com/gchq/CyberChef/pull/585
 [#591]: https://github.com/gchq/CyberChef/pull/591
+[#595]: https://github.com/gchq/CyberChef/pull/595
+[#614]: https://github.com/gchq/CyberChef/pull/614

README.md

@@ -3,7 +3,6 @@
 [![Build Status](https://travis-ci.org/gchq/CyberChef.svg?branch=master)](https://travis-ci.org/gchq/CyberChef)
 [![dependencies Status](https://david-dm.org/gchq/CyberChef/status.svg)](https://david-dm.org/gchq/CyberChef)
 [![npm](https://img.shields.io/npm/v/cyberchef.svg)](https://www.npmjs.com/package/cyberchef)
-![](https://reposs.herokuapp.com/?path=gchq/CyberChef&color=blue)
 [![](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/gchq/CyberChef/blob/master/LICENSE)
 [![Gitter](https://badges.gitter.im/gchq/CyberChef.svg)](https://gitter.im/gchq/CyberChef?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
 

SECURITY.md

@@ -0,0 +1,26 @@
+# Security Policy
+
+## Supported Versions
+
+CyberChef is supported on a best endeavours basis. Patches will be applied to
+the latest version rather than retroactively to older versions. To ensure you
+are using the most secure version of CyberChef, please make sure you have the
+[latest release](https://github.com/gchq/CyberChef/releases/latest). The
+official [live demo](https://gchq.github.io/CyberChef/) is always up to date.
+
+## Reporting a Vulnerability
+
+In most scenarios, the most appropriate way to report a vulnerability is to
+[raise a new issue](https://github.com/gchq/CyberChef/issues/new/choose)
+describing the problem in as much detail as possible, ideally with examples.
+This will obviously be public. If you feel that the vulnerability is
+significant enough to warrant a private disclosure, please email
+[oss@gchq.gov.uk](mailto:oss@gchq.gov.uk) and
+[n1474335@gmail.com](mailto:n1474335@gmail.com).
+
+Disclosures of vulnerabilities in CyberChef are always welcomed. Whilst we aim
+to write clean and secure code free from bugs, we recognise that this is an open
+source project written by analysts in their spare time, relying on dozens of
+open source libraries that are modified and updated on a regular basis. We hope
+that the community will continue to support us as we endeavour to maintain and
+develop this tool together.

package-lock.json

@@ -1,6 +1,6 @@
 {
   "name": "cyberchef",
-  "version": "9.0.5",
+  "version": "9.2.0",
   "lockfileVersion": 1,
   "requires": true,
   "dependencies": {

package.json

@@ -1,6 +1,6 @@
 {
   "name": "cyberchef",
-  "version": "9.0.5",
+  "version": "9.2.0",
   "description": "The Cyber Swiss Army Knife for encryption, encoding, compression and data analysis.",
   "author": "n1474335 <n1474335@gmail.com>",
   "homepage": "https://gchq.github.io/CyberChef",

src/core/config/Categories.json

@@ -122,7 +122,8 @@
             "PGP Encrypt",
             "PGP Decrypt",
             "PGP Encrypt and Sign",
-            "PGP Decrypt and Verify"
+            "PGP Decrypt and Verify",
+            "Parse SSH Host Key"
         ]
     },
     {
@@ -166,6 +167,8 @@
             "Parse IP range",
             "Parse IPv6 address",
             "Parse IPv4 header",
+            "Parse UDP",
+            "Parse SSH Host Key",
             "Parse URI",
             "URL Encode",
             "URL Decode",
@@ -177,7 +180,8 @@
             "Group IP addresses",
             "Encode NetBIOS Name",
             "Decode NetBIOS Name",
-            "Defang URL"
+            "Defang URL",
+            "Defang IP Addresses"
         ]
     },
     {

src/core/lib/Protobuf.mjs

@@ -205,7 +205,7 @@ class Protobuf {
                     (this.data[this.offset] & this.VALUE) << shift :
                 (this.data[this.offset] & this.VALUE) * Math.pow(2, shift);
             shift += 7;
-        } while ((this.data[this.offset++] & this.MSD) === this.MSB);
+        } while ((this.data[this.offset++] & this.MSB) === this.MSB);
         return fieldNumber;
     }
 

src/core/operations/AESDecrypt.mjs

@@ -71,8 +71,8 @@ class AESDecrypt extends Operation {
      * @throws {OperationError} if cannot decrypt input or invalid key length
      */
     run(input, args) {
-        const key = Utils.convertToByteArray(args[0].string, args[0].option),
-            iv = Utils.convertToByteArray(args[1].string, args[1].option),
+        const key = Utils.convertToByteString(args[0].string, args[0].option),
+            iv = Utils.convertToByteString(args[1].string, args[1].option),
             mode = args[2],
             inputType = args[3],
             outputType = args[4],
@@ -91,7 +91,7 @@ The following algorithms will be used based on the size of the key:
 
         const decipher = forge.cipher.createDecipher("AES-" + mode, key);
         decipher.start({
-            iv: iv,
+            iv: iv.length === 0 ? "" : iv,
             tag: gcmTag
         });
         decipher.update(forge.util.createBuffer(input));

src/core/operations/DefangIPAddresses.mjs

@@ -0,0 +1,61 @@
+/**
+ * @author h345983745
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+import Operation from "../Operation";
+
+
+/**
+ * Defang IP Addresses operation
+ */
+class DefangIPAddresses extends Operation {
+
+    /**
+     * DefangIPAddresses constructor
+     */
+    constructor() {
+        super();
+
+        this.name = "Defang IP Addresses";
+        this.module = "Default";
+        this.description = "Takes a IPv4 or IPv6 address and 'Defangs' it, meaning the IP becomes invalid, removing the risk of accidentally utilising it as an IP address.";
+        this.infoURL = "https://isc.sans.edu/forums/diary/Defang+all+the+things/22744/";
+        this.inputType = "string";
+        this.outputType = "string";
+        this.args = [];
+
+    }
+
+    /**
+     * @param {string} input
+     * @param {Object[]} args
+     * @returns {string}
+     */
+    run(input, args) {
+        input = input.replace(IPV4_REGEX, x => {
+            return x.replace(/\./g, "[.]");
+        });
+
+        input = input.replace(IPV6_REGEX, x => {
+            return x.replace(/:/g, "[:]");
+        });
+
+        return input;
+    }
+}
+
+export default DefangIPAddresses;
+
+
+/**
+ * IPV4 regular expression
+ */
+const IPV4_REGEX = new RegExp("(?:(?:\\d|[01]?\\d\\d|2[0-4]\\d|25[0-5])\\.){3}(?:25[0-5]|2[0-4]\\d|[01]?\\d\\d|\\d)(?:\\/\\d{1,2})?", "g");
+
+
+/**
+ * IPV6 regular expression
+ */
+const IPV6_REGEX = new RegExp("((?=.*::)(?!.*::.+::)(::)?([\\dA-Fa-f]{1,4}:(:|\\b)|){5}|([\\dA-Fa-f]{1,4}:){6})((([\\dA-Fa-f]{1,4}((?!\\3)::|:\\b|(?![\\dA-Fa-f])))|(?!\\2\\3)){2}|(((2[0-4]|1\\d|[1-9])?\\d|25[0-5])\\.?\\b){4})", "g");

src/core/operations/ParseColourCode.mjs

@@ -113,7 +113,7 @@ CMYK: ${cmyk}
     }).on('colorpickerChange', function(e) {
         var color = e.color.string('rgba');
         document.getElementById('input-text').value = color;
-        window.app.autoBake();
+        window.app.manager.input.debounceInputChange(new Event("keyup"));
     });
 </script>`;
     }

src/core/operations/ParseSSHHostKey.mjs

@@ -0,0 +1,150 @@
+/**
+ * @author j433866 [j433866@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+import Operation from "../Operation";
+import OperationError from "../errors/OperationError";
+import Utils from "../Utils";
+import { fromBase64 } from "../lib/Base64";
+import { fromHex, toHexFast } from "../lib/Hex";
+
+/**
+ * Parse SSH Host Key operation
+ */
+class ParseSSHHostKey extends Operation {
+
+    /**
+     * ParseSSHHostKey constructor
+     */
+    constructor() {
+        super();
+
+        this.name = "Parse SSH Host Key";
+        this.module = "Default";
+        this.description = "Parses a SSH host key and extracts fields from it.<br>The key type can be:<ul><li>ssh-rsa</li><li>ssh-dss</li><li>ecdsa-sha2</li></ul>The key format can be either Hex or Base64.";
+        this.infoURL = "https://wikipedia.org/wiki/Secure_Shell";
+        this.inputType = "string";
+        this.outputType = "string";
+        this.args = [
+            {
+                name: "Input Format",
+                type: "option",
+                value: [
+                    "Auto",
+                    "Base64",
+                    "Hex"
+                ]
+            }
+        ];
+    }
+
+    /**
+     * @param {string} input
+     * @param {Object[]} args
+     * @returns {string}
+     */
+    run(input, args) {
+        const [inputFormat] = args,
+            inputKey = this.convertKeyToBinary(input.trim(), inputFormat),
+            fields = this.parseKey(inputKey),
+            keyType = Utils.byteArrayToChars(fromHex(fields[0]), "");
+
+        let output = `Key type: ${keyType}`;
+
+        if (keyType === "ssh-rsa") {
+            output += `\nExponent: 0x${fields[1]}`;
+            output += `\nModulus: 0x${fields[2]}`;
+        } else if (keyType === "ssh-dss") {
+            output += `\np: 0x${fields[1]}`;
+            output += `\nq: 0x${fields[2]}`;
+            output += `\ng: 0x${fields[3]}`;
+            output += `\ny: 0x${fields[4]}`;
+        } else if (keyType.startsWith("ecdsa-sha2")) {
+            output += `\nCurve: ${Utils.byteArrayToChars(fromHex(fields[1]))}`;
+            output += `\nPoint: 0x${fields.slice(2)}`;
+        } else {
+            output += "\nUnsupported key type.";
+            output += `\nParameters: ${fields.slice(1)}`;
+        }
+
+        return output;
+    }
+
+    /**
+     * Converts the key to binary format from either hex or base64
+     *
+     * @param {string} inputKey
+     * @param {string} inputFormat
+     * @returns {byteArray}
+     */
+    convertKeyToBinary(inputKey, inputFormat) {
+        const keyPattern = new RegExp(/^(?:[ssh]|[ecdsa-sha2])\S+\s+(\S*)/),
+            keyMatch = inputKey.match(keyPattern);
+
+        if (keyMatch) {
+            inputKey = keyMatch[1];
+        }
+
+        if (inputFormat === "Auto") {
+            inputFormat = this.detectKeyFormat(inputKey);
+        }
+        if (inputFormat === "Hex") {
+            return fromHex(inputKey);
+        } else if (inputFormat === "Base64") {
+            return fromBase64(inputKey, null, "byteArray");
+        } else {
+            throw new OperationError("Invalid input format.");
+        }
+    }
+
+
+    /**
+     * Detects if the key is base64 or hex encoded
+     *
+     * @param {string} inputKey
+     * @returns {string}
+     */
+    detectKeyFormat(inputKey) {
+        const hexPattern = new RegExp(/^(?:[\dA-Fa-f]{2}[ ,;:]?)+$/);
+        const b64Pattern = new RegExp(/^\s*(?:[A-Za-z\d+/]{4})+(?:[A-Za-z\d+/]{2}==|[A-Za-z\d+/]{3}=)?\s*$/);
+
+        if (hexPattern.test(inputKey)) {
+            return "Hex";
+        } else if (b64Pattern.test(inputKey)) {
+            return "Base64";
+        } else {
+            throw new OperationError("Unable to detect input key format.");
+        }
+    }
+
+
+    /**
+     * Parses fields from the key
+     *
+     * @param {byteArray} key
+     */
+    parseKey(key) {
+        const fields = [];
+        while (key.length > 0) {
+            const lengthField = key.slice(0, 4);
+            let decodedLength = 0;
+            for (let i = 0; i < lengthField.length; i++) {
+                decodedLength += lengthField[i];
+                decodedLength = decodedLength << 8;
+            }
+            decodedLength = decodedLength >> 8;
+            // Break if length wasn't decoded correctly
+            if (decodedLength <= 0) break;
+
+            fields.push(toHexFast(key.slice(4, 4 + decodedLength)));
+            key = key.slice(4 + decodedLength);
+        }
+
+        return fields;
+    }
+
+}
+
+export default ParseSSHHostKey;

src/core/operations/ParseUDP.mjs

@@ -0,0 +1,84 @@
+/**
+ * @author h345983745 []
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+
+import Operation from "../Operation.mjs";
+import Stream from "../lib/Stream.mjs";
+import {toHex} from "../lib/Hex.mjs";
+import OperationError from "../errors/OperationError.mjs";
+
+/**
+ * Parse UDP operation
+ */
+class ParseUDP extends Operation {
+
+    /**
+     * ParseUDP constructor
+     */
+    constructor() {
+        super();
+
+        this.name = "Parse UDP";
+        this.module = "Default";
+        this.description = "Parses a UDP header and payload (if present).";
+        this.infoURL = "https://wikipedia.org/wiki/User_Datagram_Protocol";
+        this.inputType = "ArrayBuffer";
+        this.outputType = "json";
+        this.presentType = "html";
+        this.args = [];
+    }
+
+    /**
+     * @param {ArrayBuffer} input
+     * @returns {Object}
+     */
+    run(input, args) {
+        if (input.byteLength < 8) {
+            throw new OperationError("Need 8 bytes for a UDP Header");
+        }
+
+        const s = new Stream(new Uint8Array(input));
+        // Parse Header
+        const UDPPacket = {
+            "Source port": s.readInt(2),
+            "Destination port": s.readInt(2),
+            "Length": s.readInt(2),
+            "Checksum": toHex(s.getBytes(2), "")
+        };
+        // Parse data if present
+        if (s.hasMore()) {
+            UDPPacket.Data = toHex(s.getBytes(UDPPacket.Length - 8), "");
+        }
+
+        return UDPPacket;
+    }
+
+    /**
+     * Displays the UDP Packet in a table style
+     * @param {Object} data
+     * @returns {html}
+     */
+    present(data) {
+        const html = [];
+        html.push("<table class='table table-hover table-sm table-bordered table-nonfluid' style='table-layout: fixed'>");
+        html.push("<tr>");
+        html.push("<th>Field</th>");
+        html.push("<th>Value</th>");
+        html.push("</tr>");
+
+        for (const key in data) {
+            html.push("<tr>");
+            html.push("<td style=\"word-wrap:break-word\">" + key + "</td>");
+            html.push("<td>" + data[key] + "</td>");
+            html.push("</tr>");
+        }
+        html.push("</table>");
+        return html.join("");
+    }
+
+}
+
+
+export default ParseUDP;

src/web/App.mjs

@@ -670,18 +670,22 @@ class App {
      *
      * @param {string} title - The title of the box
      * @param {string} body - The question (HTML supported)
+     * @param {string} accept - The text of the accept button
+     * @param {string} reject - The text of the reject button
      * @param {function} callback - A function accepting one boolean argument which handles the
      *   response e.g. function(answer) {...}
      * @param {Object} [scope=this] - The object to bind to the callback function
      *
      * @example
      * // Pops up a box asking if the user would like a cookie. Prints the answer to the console.
-     * this.confirm("Question", "Would you like a cookie?", function(answer) {console.log(answer);});
+     * this.confirm("Question", "Would you like a cookie?", "Yes", "No", function(answer) {console.log(answer);});
      */
-    confirm(title, body, callback, scope) {
+    confirm(title, body, accept, reject, callback, scope) {
         scope = scope || this;
         document.getElementById("confirm-title").innerHTML = title;
         document.getElementById("confirm-body").innerHTML = body;
+        document.getElementById("confirm-yes").innerText = accept;
+        document.getElementById("confirm-no").innerText = reject;
         document.getElementById("confirm-modal").style.display = "block";
 
         this.confirmClosed = false;
@@ -694,9 +698,14 @@ class App {
                 callback.bind(scope)(true);
                 $("#confirm-modal").modal("hide");
             }.bind(this))
-            .one("hide.bs.modal", function(e) {
-                if (!this.confirmClosed)
+            .one("click", "#confirm-no", function() {
+                this.confirmClosed = true;
                 callback.bind(scope)(false);
+            }.bind(this))
+            .one("hide.bs.modal", function(e) {
+                if (!this.confirmClosed) {
+                    callback.bind(scope)(undefined);
+                }
                 this.confirmClosed = true;
             }.bind(this));
     }

src/web/Manager.mjs

@@ -224,7 +224,7 @@ class Manager {
         document.getElementById("options").addEventListener("click", this.options.optionsClick.bind(this.options));
         document.getElementById("reset-options").addEventListener("click", this.options.resetOptionsClick.bind(this.options));
         this.addDynamicListener(".option-item input[type=checkbox]", "change", this.options.switchChange, this.options);
-        this.addDynamicListener(".option-item input[type=checkbox]", "change", this.options.setWordWrap, this.options);
+        this.addDynamicListener(".option-item input[type=checkbox]#wordWrap", "change", this.options.setWordWrap, this.options);
         this.addDynamicListener(".option-item input[type=checkbox]#useMetaKey", "change", this.bindings.updateKeybList, this.bindings);
         this.addDynamicListener(".option-item input[type=number]", "keyup", this.options.numberChange, this.options);
         this.addDynamicListener(".option-item input[type=number]", "change", this.options.numberChange, this.options);

src/web/html/index.html

@@ -382,7 +382,7 @@
             </div>
         </div>
 
-        <div class="modal" id="save-modal" tabindex="-1" role="dialog">
+        <div class="modal fade" id="save-modal" tabindex="-1" role="dialog">
             <div class="modal-dialog modal-lg" role="document">
                 <div class="modal-content">
                     <div class="modal-header">
@@ -442,7 +442,7 @@
             </div>
         </div>
 
-        <div class="modal" id="load-modal" tabindex="-1" role="dialog">
+        <div class="modal fade" id="load-modal" tabindex="-1" role="dialog">
             <div class="modal-dialog modal-lg" role="document">
                 <div class="modal-content">
                     <div class="modal-header">
@@ -469,7 +469,7 @@
             </div>
         </div>
 
-        <div class="modal" id="options-modal" tabindex="-1" role="dialog">
+        <div class="modal fade" id="options-modal" tabindex="-1" role="dialog">
             <div class="modal-dialog modal-lg" role="document">
                 <div class="modal-content">
                     <div class="modal-header">
@@ -563,14 +563,21 @@
                         <div class="checkbox option-item">
                             <label for="imagePreview">
                                 <input type="checkbox" option="imagePreview" id="imagePreview">
-                                Render a preview of the input if it's detected to be an image.
+                                Render a preview of the input if it's detected to be an image
                             </label>
                         </div>
 
                         <div class="checkbox option-item">
                             <label for="syncTabs">
                                 <input type="checkbox" option="syncTabs" id="syncTabs">
-                                    Keep the current tab in sync between the input and output.
+                                Keep the current tab in sync between the input and output
+                            </label>
+                        </div>
+
+                        <div class="checkbox option-item">
+                            <label for="preserveCR" data-toggle="tooltip" data-placement="right" data-html="true" title="As HTML textareas don't support carriage returns, editing input must be turned off to preserve them.<br><br>When this option is enabled, editing is disabled for pasted text that contains carriage returns. Otherwise, editing will remain enabled but carriage returns will not be preserved.">
+                                <input type="checkbox" option="preserveCR" id="preserveCR">
+                                Preserve carriage returns when pasting an input
                             </label>
                         </div>
                     </div>
@@ -582,7 +589,7 @@
             </div>
         </div>
 
-        <div class="modal" id="favourites-modal" tabindex="-1" role="dialog">
+        <div class="modal fade" id="favourites-modal" tabindex="-1" role="dialog">
             <div class="modal-dialog modal-lg" role="document">
                 <div class="modal-content">
                     <div class="modal-header">
@@ -608,7 +615,7 @@
             </div>
         </div>
 
-        <div class="modal" id="support-modal" tabindex="-1" role="dialog">
+        <div class="modal fade" id="support-modal" tabindex="-1" role="dialog">
             <div class="modal-dialog modal-lg" role="document">
                 <div class="modal-content">
                     <div class="modal-header">
@@ -744,7 +751,7 @@
             </div>
         </div>
 
-        <div class="modal" id="confirm-modal" tabindex="-1" role="dialog">
+        <div class="modal fade" id="confirm-modal" tabindex="-1" role="dialog">
             <div class="modal-dialog modal-lg" role="document">
                 <div class="modal-content">
                     <div class="modal-header">
@@ -763,7 +770,7 @@
             </div>
         </div>
 
-        <div class="modal" id="input-tab-modal" tabindex="-1" role="dialog">
+        <div class="modal fade" id="input-tab-modal" tabindex="-1" role="dialog">
             <div class="modal-dialog modal-lg" role="document">
                 <div class="modal-content">
                     <div class="modal-header">
@@ -822,7 +829,7 @@
             </div>
         </div>
 
-        <div class="modal" id="output-tab-modal" tabindex="-1" role="dialog">
+        <div class="modal fade" id="output-tab-modal" tabindex="-1" role="dialog">
             <div class="modal-dialog modal-lg" role="document">
                 <div class="modal-content">
                     <div class="modal-header">

src/web/index.js

@@ -53,7 +53,9 @@ function main() {
         logLevel:            "info",
         autoMagic:           true,
         imagePreview:        true,
-        syncTabs:            true
+        syncTabs:            true,
+        preserveCR:          true,
+        userSetCR:           false
     };
 
     document.removeEventListener("DOMContentLoaded", main, false);

src/web/waiters/InputWaiter.mjs

@@ -222,8 +222,6 @@ class InputWaiter {
         if (Object.prototype.hasOwnProperty.call(r, "progress") &&
             Object.prototype.hasOwnProperty.call(r, "inputNum")) {
             this.manager.tabs.updateInputTabProgress(r.inputNum, r.progress, 100);
-        } else if (Object.prototype.hasOwnProperty.call(r, "fileBuffer")) {
-            this.manager.tabs.updateInputTabProgress(r.inputNum, 100, 100);
         }
 
         const transferable = Object.prototype.hasOwnProperty.call(r, "fileBuffer") ? [r.fileBuffer] : undefined;
@@ -305,6 +303,9 @@ class InputWaiter {
             case "removeChefWorker":
                 this.removeChefWorker();
                 break;
+            case "fileLoaded":
+                this.fileLoaded(r.data.inputNum);
+                break;
             default:
                 log.error(`Unknown action ${r.action}.`);
         }
@@ -331,7 +332,7 @@ class InputWaiter {
      * @param {number} inputData.size - The size in bytes of the input file
      * @param {string} inputData.type - The MIME type of the input file
      * @param {number} inputData.progress - The load progress of the input file
-     * @param {boolean} [silent=false] - If true, fires the manager statechange event
+     * @param {boolean} [silent=false] - If false, fires the manager statechange event
      */
     async set(inputData, silent=false) {
         return new Promise(function(resolve, reject) {
@@ -373,7 +374,7 @@ class InputWaiter {
 
                 if (!silent) window.dispatchEvent(this.manager.statechange);
             } else {
-                this.setFile(inputData);
+                this.setFile(inputData, silent);
             }
 
         }.bind(this));
@@ -389,8 +390,9 @@ class InputWaiter {
      * @param {number} inputData.size - The size in bytes of the input file
      * @param {string} inputData.type - The MIME type of the input file
      * @param {number} inputData.progress - The load progress of the input file
+     * @param {boolean} [silent=true] - If false, fires the manager statechange event
      */
-    setFile(inputData) {
+    setFile(inputData, silent=true) {
         const activeTab = this.manager.tabs.getActiveInputTab();
         if (inputData.inputNum !== activeTab) return;
 
@@ -414,6 +416,30 @@ class InputWaiter {
 
         this.setInputInfo(inputData.size, null);
         this.displayFilePreview(inputData);
+
+        if (!silent) window.dispatchEvent(this.manager.statechange);
+    }
+
+    /**
+     * Update file details when a file completes loading
+     *
+     * @param {number} inputNum - The inputNum of the input which has finished loading
+     */
+    fileLoaded(inputNum) {
+        this.manager.tabs.updateInputTabProgress(inputNum, 100, 100);
+
+        const activeTab = this.manager.tabs.getActiveInputTab();
+        if (activeTab !== inputNum) return;
+
+        this.inputWorker.postMessage({
+            action: "setInput",
+            data: {
+                inputNum: inputNum,
+                silent: false
+            }
+        });
+
+        this.updateFileProgress(inputNum, 100);
     }
 
     /**
@@ -495,19 +521,6 @@ class InputWaiter {
             fileLoaded.textContent = progress + "%";
             fileLoaded.style.color = "";
         }
-
-        if (progress === 100 && progress !== oldProgress) {
-            // Don't set the input if the progress hasn't changed
-            this.inputWorker.postMessage({
-                action: "setInput",
-                data: {
-                    inputNum: inputNum,
-                    silent: false
-                }
-            });
-            window.dispatchEvent(this.manager.statechange);
-
-        }
     }
 
     /**
@@ -711,33 +724,50 @@ class InputWaiter {
      *
      * @param {event} e
      */
-    inputPaste(e) {
-        const pastedData = e.clipboardData.getData("Text");
-        if (pastedData.length < (this.app.options.ioDisplayThreshold * 1024)) {
-            // Pasting normally fires the inputChange() event before
-            // changing the value, so instead change it here ourselves
-            // and manually fire inputChange()
-            e.preventDefault();
-            const inputText = document.getElementById("input-text");
-            const selStart = inputText.selectionStart;
-            const selEnd = inputText.selectionEnd;
-            const startVal = inputText.value.slice(0, selStart);
-            const endVal = inputText.value.slice(selEnd);
-
-            inputText.value = startVal + pastedData + endVal;
-            inputText.setSelectionRange(selStart + pastedData.length, selStart + pastedData.length);
-            this.debounceInputChange(e);
-        } else {
+    async inputPaste(e) {
         e.preventDefault();
         e.stopPropagation();
 
+        const self = this;
+        /**
+         * Triggers the input file/binary data overlay
+         *
+         * @param {string} pastedData
+         */
+        function triggerOverlay(pastedData) {
             const file = new File([pastedData], "PastedData", {
                 type: "text/plain",
                 lastModified: Date.now()
             });
 
-            this.loadUIFiles([file]);
+            self.loadUIFiles([file]);
+        }
+
+        const pastedData = e.clipboardData.getData("Text");
+        const inputText = document.getElementById("input-text");
+        const selStart = inputText.selectionStart;
+        const selEnd = inputText.selectionEnd;
+        const startVal = inputText.value.slice(0, selStart);
+        const endVal = inputText.value.slice(selEnd);
+        const val = startVal + pastedData + endVal;
+
+        if (val.length >= (this.app.options.ioDisplayThreshold * 1024)) {
+            // Data too large to display, use overlay
+            triggerOverlay(val);
             return false;
+        } else if (await this.preserveCarriageReturns(val)) {
+            // Data contains a carriage return and the user doesn't wish to edit it, use overlay
+            // We check this in a separate condition to make sure it is not run unless absolutely
+            // necessary.
+            triggerOverlay(val);
+            return false;
+        } else {
+            // Pasting normally fires the inputChange() event before
+            // changing the value, so instead change it here ourselves
+            // and manually fire inputChange()
+            inputText.value = val;
+            inputText.setSelectionRange(selStart + pastedData.length, selStart + pastedData.length);
+            this.debounceInputChange(e);
         }
     }
 
@@ -815,6 +845,46 @@ class InputWaiter {
         }
     }
 
+    /**
+     * Checks if an input contains carriage returns.
+     * If a CR is detected, checks if the preserve CR option has been set,
+     * and if not, asks the user for their preference.
+     *
+     * @param {string} input - The input to be checked
+     * @returns {boolean} - If true, the input contains a CR which should be
+     *      preserved, so display an overlay so it can't be edited
+     */
+    async preserveCarriageReturns(input) {
+        if (input.indexOf("\r") < 0) return false;
+
+        const optionsStr = "This behaviour can be changed in the <a href='#' onclick='document.getElementById(\"options\").click()'>Options pane</a>";
+        if (!this.app.options.userSetCR) {
+            // User has not set a CR preference yet
+            let preserve = await new Promise(function(resolve, reject) {
+                this.app.confirm(
+                    "Carriage Return Detected",
+                    "A <a href='https://wikipedia.org/wiki/Carriage_return'>carriage return</a> (<code>\\r</code>, <code>0x0d</code>) was detected in your input. As HTML textareas <a href='https://html.spec.whatwg.org/multipage/form-elements.html#the-textarea-element'>can't display carriage returns</a>, editing must be turned off to preserve them. <br>Alternatively, you can enable editing but your carriage returns will not be preserved.<br><br>This preference will be saved but can be toggled in the options pane.",
+                    "Preserve Carriage Returns",
+                    "Enable Editing", resolve, this);
+            }.bind(this));
+            if (preserve === undefined) {
+                // The confirm pane was closed without picking a specific choice
+                this.app.alert(`Not preserving carriage returns.\n${optionsStr}`, 5000);
+                preserve = false;
+            }
+            this.manager.options.updateOption("preserveCR", preserve);
+            this.manager.options.updateOption("userSetCR", true);
+        } else {
+            if (this.app.options.preserveCR) {
+                this.app.alert(`A carriage return (\\r, 0x0d) was detected in your input, so editing has been disabled to preserve it.<br>${optionsStr}`, 10000);
+            } else {
+                this.app.alert(`A carriage return (\\r, 0x0d) was detected in your input. Editing is remaining enabled, but carriage returns will not be preserved.<br>${optionsStr}`, 10000);
+            }
+        }
+
+        return this.app.options.preserveCR;
+    }
+
     /**
      * Load files from the UI into the inputWorker
      *

src/web/waiters/OptionsWaiter.mjs

@@ -1,25 +1,31 @@
 /**
- * Waiter to handle events related to the CyberChef options.
- *
  * @author n1474335 [n1474335@gmail.com]
  * @copyright Crown Copyright 2016
  * @license Apache-2.0
- *
- * @constructor
- * @param {App} app - The main view object for CyberChef.
  */
-const OptionsWaiter = function(app, manager) {
+
+/**
+ * Waiter to handle events related to the CyberChef options.
+ */
+class OptionsWaiter {
+
+    /**
+     * OptionsWaiter constructor.
+     *
+     * @param {App} app - The main view object for CyberChef.
+     * @param {Manager} manager - The CyberChef event manager.
+     */
+    constructor(app, manager) {
         this.app = app;
         this.manager = manager;
-};
-
+    }
 
     /**
      * Loads options and sets values of switches and inputs to match them.
      *
      * @param {Object} options
      */
-OptionsWaiter.prototype.load = function(options) {
+    load(options) {
         for (const option in options) {
             this.app.options[option] = options[option];
         }
@@ -47,7 +53,7 @@ OptionsWaiter.prototype.load = function(options) {
                 selects[i].selectedIndex = 0;
             }
         }
-};
+    }
 
 
     /**
@@ -56,81 +62,80 @@ OptionsWaiter.prototype.load = function(options) {
      *
      * @param {event} e
      */
-OptionsWaiter.prototype.optionsClick = function(e) {
+    optionsClick(e) {
         e.preventDefault();
         $("#options-modal").modal();
-};
+    }
 
 
     /**
      * Handler for reset options click events.
      * Resets options back to their default values.
      */
-OptionsWaiter.prototype.resetOptionsClick = function() {
+    resetOptionsClick() {
         this.load(this.app.doptions);
-};
+    }
 
 
     /**
      * Handler for switch change events.
- * Modifies the option state and saves it to local storage.
      *
      * @param {event} e
      */
-OptionsWaiter.prototype.switchChange = function(e) {
+    switchChange(e) {
         const el = e.target;
         const option = el.getAttribute("option");
         const state = el.checked;
 
-    log.debug(`Setting ${option} to ${state}`);
-    this.app.options[option] = state;
-
-    if (this.app.isLocalStorageAvailable())
-        localStorage.setItem("options", JSON.stringify(this.app.options));
-};
+        this.updateOption(option, state);
+    }
 
 
     /**
      * Handler for number change events.
- * Modifies the option value and saves it to local storage.
      *
      * @param {event} e
      */
-OptionsWaiter.prototype.numberChange = function(e) {
+    numberChange(e) {
         const el = e.target;
         const option = el.getAttribute("option");
         const val = parseInt(el.value, 10);
 
-    log.debug(`Setting ${option} to ${val}`);
-    this.app.options[option] = val;
-
-    if (this.app.isLocalStorageAvailable())
-        localStorage.setItem("options", JSON.stringify(this.app.options));
-};
+        this.updateOption(option, val);
+    }
 
 
     /**
      * Handler for select change events.
- * Modifies the option value and saves it to local storage.
      *
      * @param {event} e
      */
-OptionsWaiter.prototype.selectChange = function(e) {
+    selectChange(e) {
         const el = e.target;
         const option = el.getAttribute("option");
 
-    log.debug(`Setting ${option} to ${el.value}`);
-    this.app.options[option] = el.value;
+        this.updateOption(option, el.value);
+    }
+
+    /**
+     * Modifies an option value and saves it to local storage.
+     *
+     * @param {string} option - The option to be updated
+     * @param {string|number|boolean} value - The new value of the option
+     */
+    updateOption(option, value) {
+        log.debug(`Setting ${option} to ${value}`);
+        this.app.options[option] = value;
 
         if (this.app.isLocalStorageAvailable())
             localStorage.setItem("options", JSON.stringify(this.app.options));
-};
+    }
 
 
     /**
      * Sets or unsets word wrap on the input and output depending on the wordWrap option value.
      */
-OptionsWaiter.prototype.setWordWrap = function() {
+    setWordWrap() {
         document.getElementById("input-text").classList.remove("word-wrap");
         document.getElementById("output-text").classList.remove("word-wrap");
         document.getElementById("output-html").classList.remove("word-wrap");
@@ -144,7 +149,7 @@ OptionsWaiter.prototype.setWordWrap = function() {
             document.getElementById("input-highlighter").classList.add("word-wrap");
             document.getElementById("output-highlighter").classList.add("word-wrap");
         }
-};
+    }
 
 
     /**
@@ -152,11 +157,11 @@ OptionsWaiter.prototype.setWordWrap = function() {
      *
      * @param {Event} e
      */
-OptionsWaiter.prototype.themeChange = function (e) {
+    themeChange(e) {
         const themeClass = e.target.value;
 
         document.querySelector(":root").className = themeClass;
-};
+    }
 
 
     /**
@@ -164,11 +169,12 @@ OptionsWaiter.prototype.themeChange = function (e) {
      *
      * @param {Event} e
      */
-OptionsWaiter.prototype.logLevelChange = function (e) {
+    logLevelChange(e) {
         const level = e.target.value;
         log.setLevel(level, false);
         this.manager.worker.setLogLevel();
         this.manager.input.setLogLevel();
-};
+    }
+}
 
 export default OptionsWaiter;

src/web/waiters/OutputWaiter.mjs

@@ -217,6 +217,9 @@ class OutputWaiter {
      */
     removeAllOutputs() {
         this.outputs = {};
+
+        this.resetSwitch();
+
         const tabsList = document.getElementById("output-tabs");
         const tabsListChildren = tabsList.children;
 
@@ -516,9 +519,10 @@ class OutputWaiter {
             this.app.alert("Could not find any output data to download. Has this output been baked?", 3000);
             return;
         }
-        let fileName = window.prompt("Please enter a filename: ", "download.dat");
+        const fileName = window.prompt("Please enter a filename: ", "download.dat");
 
-        if (fileName === null) fileName = "download.dat";
+        // Assume if the user clicks cancel they don't want to download
+        if (fileName === null) return;
 
         const data = await dish.get(Dish.ARRAY_BUFFER),
             file = new File([data], fileName);
@@ -529,14 +533,24 @@ class OutputWaiter {
      * Handler for save all click event
      * Saves all outputs to a single archvie file
      */
-    saveAllClick() {
+    async saveAllClick() {
         const downloadButton = document.getElementById("save-all-to-file");
         if (downloadButton.firstElementChild.innerHTML === "archive") {
             this.downloadAllFiles();
-        } else if (window.confirm("Cancel zipping of outputs?")) {
+        } else {
+            const cancel = await new Promise(function(resolve, reject) {
+                this.app.confirm(
+                    "Cancel zipping?",
+                    "The outputs are currently being zipped for download.<br>Cancel zipping?",
+                    "Continue zipping",
+                    "Cancel zipping",
+                    resolve, this);
+            }.bind(this));
+            if (!cancel) {
                 this.terminateZipWorker();
             }
         }
+    }
 
 
     /**
@@ -544,13 +558,18 @@ class OutputWaiter {
      * be zipped for download
      */
     async downloadAllFiles() {
-        return new Promise(resolve => {
         const inputNums = Object.keys(this.outputs);
         for (let i = 0; i < inputNums.length; i++) {
             const iNum = inputNums[i];
             if (this.outputs[iNum].status !== "baked" ||
             this.outputs[iNum].bakeId !== this.manager.worker.bakeId) {
-                    if (window.confirm("Not all outputs have been baked yet. Continue downloading outputs?")) {
+                const continueDownloading = await new Promise(function(resolve, reject) {
+                    this.app.confirm(
+                        "Incomplete outputs",
+                        "Not all outputs have been baked yet. Continue downloading outputs?",
+                        "Download", "Cancel", resolve, this);
+                }.bind(this));
+                if (continueDownloading) {
                     break;
                 } else {
                     return;
@@ -594,7 +613,6 @@ class OutputWaiter {
             fileExtension: fileExt
         });
         this.zipWorker.addEventListener("message", this.handleZipWorkerMessage.bind(this));
-        });
     }
 
     /**
@@ -1213,14 +1231,39 @@ class OutputWaiter {
      * Moves the current output into the input textarea.
      */
     async switchClick() {
-        const active = await this.getDishBuffer(this.getOutputDish(this.manager.tabs.getActiveOutputTab()));
+        const activeTab = this.manager.tabs.getActiveOutputTab();
+        const transferable = [];
+
+        const switchButton = document.getElementById("switch");
+        switchButton.classList.add("spin");
+        switchButton.disabled = true;
+        switchButton.firstElementChild.innerHTML = "autorenew";
+        $(switchButton).tooltip("hide");
+
+        let active = await this.getDishBuffer(this.getOutputDish(activeTab));
+
+        if (!this.outputExists(activeTab)) {
+            this.resetSwitchButton();
+            return;
+        }
+
+        if (this.outputs[activeTab].data.type === "string" &&
+            active.byteLength <= this.app.options.ioDisplayThreshold * 1024) {
+            const dishString = await this.getDishStr(this.getOutputDish(activeTab));
+            if (!await this.manager.input.preserveCarriageReturns(dishString)) {
+                active = dishString;
+            }
+        } else {
+            transferable.push(active);
+        }
+
         this.manager.input.inputWorker.postMessage({
             action: "inputSwitch",
             data: {
-                inputNum: this.manager.tabs.getActiveInputTab(),
+                inputNum: activeTab,
                 outputData: active
             }
-        }, [active]);
+        }, transferable);
     }
 
     /**
@@ -1238,6 +1281,9 @@ class OutputWaiter {
     inputSwitch(switchData) {
         this.switchOrigData = switchData;
         document.getElementById("undo-switch").disabled = false;
+
+        this.resetSwitchButton();
+
     }
 
     /**
@@ -1246,17 +1292,35 @@ class OutputWaiter {
      */
     undoSwitchClick() {
         this.manager.input.updateInputObj(this.switchOrigData.inputNum, this.switchOrigData.data);
+
+        this.manager.input.fileLoaded(this.switchOrigData.inputNum);
+
+        this.resetSwitch();
+    }
+
+    /**
+     * Removes the switch data and resets the switch buttons
+     */
+    resetSwitch() {
+        if (this.switchOrigData !== undefined) {
+            delete this.switchOrigData;
+        }
+
         const undoSwitch = document.getElementById("undo-switch");
         undoSwitch.disabled = true;
         $(undoSwitch).tooltip("hide");
 
-        this.manager.input.inputWorker.postMessage({
-            action: "setInput",
-            data: {
-                inputNum: this.switchOrigData.inputNum,
-                silent: false
+        this.resetSwitchButton();
     }
-        });
+
+    /**
+     * Resets the switch button to its usual state
+     */
+    resetSwitchButton() {
+        const switchButton = document.getElementById("switch");
+        switchButton.classList.remove("spin");
+        switchButton.disabled = false;
+        switchButton.firstElementChild.innerHTML = "open_in_browser";
     }
 
     /**

src/web/workers/InputWorker.mjs

@@ -202,6 +202,7 @@ self.bakeInput = function(inputNum, bakeId) {
     if (inputObj === null ||
         inputObj === undefined ||
         inputObj.status !== "loaded") {
+
         self.postMessage({
             action: "queueInputError",
             data: {
@@ -441,7 +442,7 @@ self.updateTabHeader = function(inputNum) {
  *
  * @param {object} inputData
  * @param {number} inputData.inputNum - The input to get the data for
- * @param {boolean} inputData.silent - If false, the manager statechange event won't be fired
+ * @param {boolean} inputData.silent - If false, the manager statechange event will be fired
  */
 self.setInput = function(inputData) {
     const inputNum = inputData.inputNum;
@@ -590,7 +591,7 @@ self.updateInputObj = function(inputData) {
     const inputNum = inputData.inputNum;
     const data = inputData.data;
 
-    if (self.getInputObj(inputNum) === -1) return;
+    if (self.getInputObj(inputNum) === undefined) return;
 
     self.inputs[inputNum].data = data;
 };
@@ -663,11 +664,19 @@ self.handleLoaderMessage = function(r) {
     if ("fileBuffer" in r) {
         log.debug(`Input file ${inputNum} loaded.`);
         self.loadingInputs--;
+
         self.updateInputValue({
             inputNum: inputNum,
             value: r.fileBuffer
         });
 
+        self.postMessage({
+            action: "fileLoaded",
+            data: {
+                inputNum: inputNum
+            }
+        });
+
         const idx = self.getLoaderWorkerIdx(r.id);
         self.loadNextFile(idx);
     } else if ("progress" in r) {
@@ -782,7 +791,7 @@ self.loadFiles = function(filesData) {
     }
 
     self.getLoadProgress();
-    self.setInput({inputNum: activeTab, silent: false});
+    self.setInput({inputNum: activeTab, silent: true});
 };
 
 /**
@@ -1025,7 +1034,7 @@ self.inputSwitch = function(switchData) {
     const currentData = currentInput.data;
     if (currentInput === undefined || currentInput === null) return;
 
-    if (typeof switchData.outputData === "object") {
+    if (typeof switchData.outputData !== "string") {
         const output = new Uint8Array(switchData.outputData),
             types = detectFileType(output);
         let type = "unknown",
@@ -1036,15 +1045,22 @@ self.inputSwitch = function(switchData) {
         }
 
         // ArrayBuffer
-        currentInput.data = {
+        self.updateInputObj({
+            inputNum: switchData.inputNum,
+            data: {
                 fileBuffer: switchData.outputData,
                 name: `output.${ext}`,
                 size: switchData.outputData.byteLength.toLocaleString(),
                 type: type
-        };
+            }
+        });
     } else {
         // String
-        currentInput.data = switchData.outputData;
+        self.updateInputValue({
+            inputNum: switchData.inputNum,
+            value: switchData.outputData,
+            force: true
+        });
     }
 
     self.postMessage({
@@ -1055,6 +1071,11 @@ self.inputSwitch = function(switchData) {
         }
     });
 
-    self.setInput({inputNum: switchData.inputNum, silent: false});
+    self.postMessage({
+        action: "fileLoaded",
+        data: {
+            inputNum: switchData.inputNum
+        }
+    });
 
 };

tests/lib/TestRegister.mjs

@@ -48,6 +48,7 @@ class TestRegister {
      * Runs all the tests in the register.
      */
     runTests () {
+        console.log("Running tests...");
         return Promise.all(
             this.tests.map(function(test, i) {
                 const chef = new Chef();
@@ -103,6 +104,8 @@ class TestRegister {
      * Run all api related tests and wrap results in report format
      */
     runApiTests() {
+        console.log("Running tests...");
+
         return Promise.all(this.apiTests.map(async function(test, i) {
             const result = {
                 test: test,

tests/lib/utils.mjs

@@ -15,28 +15,48 @@
  * @param {string} status
  * @returns {string}
  */
-const statusToIcon = function statusToIcon(status) {
-    const icons = {
+function statusToIcon(status) {
+    return {
         erroring: "🔥",
         failing: "❌",
         passing: "✔️️",
-    };
-    return icons[status] || "?";
-};
-
+    }[status] || "?";
+}
 
 /**
- * Displays a given test result in the console.
  * Counts test statuses.
  *
- * @param {Object} testStatusCounts
+ * @param {Object} testStatus
  * @param {Object} testResult
  */
 function handleTestResult(testStatus, testResult) {
     testStatus.allTestsPassing = testStatus.allTestsPassing && testResult.status === "passing";
-    const newCount = (testStatus.counts[testResult.status] || 0) + 1;
-    testStatus.counts[testResult.status] = newCount;
+    testStatus.counts[testResult.status] = (testStatus.counts[testResult.status] || 0) + 1;
     testStatus.counts.total += 1;
+}
+
+/**
+ * Log each test result, count tests and failures.
+ *
+ * @param {Object} testStatus - object describing test run data
+ * @param {Object[]} results - results from TestRegister
+ */
+export function logTestReport(testStatus, results) {
+    console.log("Tests completed.");
+
+    results.forEach(r => handleTestResult(testStatus, r));
+
+    console.log();
+    for (const testStatusCount in testStatus.counts) {
+        const count = testStatus.counts[testStatusCount];
+        if (count > 0) {
+            console.log(testStatusCount.toUpperCase() + "\t" + count);
+        }
+    }
+    console.log();
+
+    // Print error messages for tests that didn't pass
+    results.filter(res => res.status !== "passing").forEach(testResult => {
         console.log([
             statusToIcon(testResult.status),
             testResult.test.name
@@ -50,24 +70,8 @@ function handleTestResult(testStatus, testResult) {
                     .replace(/\n/g, "\n\t")
             );
         }
-}
-
-/**
- * Log each test result, count tests and failures. Log test suite run duration.
- *
- * @param {Object} testStatus - object describing test run data
- * @param {Object[]} results - results from TestRegister
- */
-export function logTestReport(testStatus, results) {
-    results.forEach(r => handleTestResult(testStatus, r));
-    console.log("\n");
-
-    for (const testStatusCount in testStatus.counts) {
-        const count = testStatus.counts[testStatusCount];
-        if (count > 0) {
-            console.log(testStatusCount.toUpperCase(), count);
-        }
-    }
+    });
+    console.log();
 
     process.exit(testStatus.allTestsPassing ? 0 : 1);
 }
@@ -81,4 +85,3 @@ export function setLongTestFailure() {
         process.exit(1);
     }, 60 * 1000);
 }
-

tests/node/tests/nodeApi.mjs

@@ -136,7 +136,7 @@ TestRegister.addApiTests([
 
     it("chef.help: returns multiple results", () => {
         const result = chef.help("base 64");
-        assert.strictEqual(result.length, 10);
+        assert.strictEqual(result.length, 11);
     }),
 
     it("chef.help: looks in description for matches too", () => {

tests/node/tests/operations.mjs

@@ -906,7 +906,7 @@ smothering ampersand abreast
     }),
 
     it("to unix timestamp", () => {
-        assert.strictEqual(chef.toUNIXTimestamp("04-01-2001").toString(), "986083200 (Sun 1 April 2001 00:00:00 UTC)");
+        assert.strictEqual(chef.toUNIXTimestamp("2001-04-01").toString(), "986083200 (Sun 1 April 2001 00:00:00 UTC)");
     }),
 
     it("Translate DateTime format", () => {

tests/operations/index.mjs

@@ -86,6 +86,9 @@ import "./tests/Typex";
 import "./tests/BLAKE2b";
 import "./tests/BLAKE2s";
 import "./tests/Protobuf";
+import "./tests/ParseSSHHostKey";
+import "./tests/DefangIP";
+import "./tests/ParseUDP";
 
 // Cannot test operations that use the File type yet
 //import "./tests/SplitColourChannels";

tests/operations/tests/Crypt.mjs

@@ -18,6 +18,42 @@ TestRegister.addTests([
      *
      * All random data blocks (binary input, keys and IVs) were generated from /dev/urandom using dd:
      * > dd if=/dev/urandom of=key.txt bs=16 count=1
+     *
+     *
+     * The following is a Python script used to generate the AES-GCM tests.
+     * It uses PyCryptodome (https://www.pycryptodome.org) to handle the AES encryption and decryption.
+     *
+     * from Crypto.Cipher import AES
+     * import binascii
+
+     * input_data = "0123456789ABCDEF"
+     * key = binascii.unhexlify("00112233445566778899aabbccddeeff")
+     * iv = binascii.unhexlify("ffeeddccbbaa99887766554433221100")
+     *
+     * cipher = AES.new(key, AES.MODE_GCM, nonce=iv)
+     * cipher_text, tag = cipher.encrypt_and_digest(binascii.unhexlify(input_data))
+     *
+     * cipher = AES.new(key, AES.MODE_GCM, nonce=iv)
+     * decrypted = cipher.decrypt_and_verify(cipher_text, tag)
+     *
+     * key = binascii.hexlify(key).decode("UTF-8")
+     * iv = binascii.hexlify(iv).decode("UTF-8")
+     * cipher_text = binascii.hexlify(cipher_text).decode("UTF-8")
+     * tag = binascii.hexlify(tag).decode("UTF-8")
+     * decrypted = binascii.hexlify(decrypted).decode("UTF-8")
+     *
+     * print("Key: {}\nIV : {}\nInput data: {}\n\nEncrypted ciphertext: {}\nGCM tag: {}\n\nDecrypted plaintext : {}".format(key, iv, input_data, cipher_text, tag, decrypted))
+     *
+     *
+     * Outputs:
+     * Key: 00112233445566778899aabbccddeeff
+     * IV : ffeeddccbbaa99887766554433221100
+     * Input data: 0123456789ABCDEF
+     *
+     * Encrypted ciphertext: 8feeafedfdb2f6f9
+     * GCM tag: 654ef4957c6e2b0cc6501d8f9bcde032
+     *
+     * Decrypted plaintext : 0123456789abcdef
      */
     {
         name: "AES Encrypt: no key",
@@ -54,6 +90,21 @@ The following algorithms will be used based on the size of the key:
             }
         ],
     },
+    {
+        name: "AES Encrypt: AES-128-CTR, no IV, ASCII",
+        input: "The quick brown fox jumps over the lazy dog.",
+        expectedOutput: "a98c9e8e3b7c894384d740e4f0f4ed0be2bbb1e0e13a255812c3c6b0a629e4ad759c075b2469c6f4fb2c0cf9",
+        recipeConfig: [
+            {
+                "op": "AES Encrypt",
+                "args": [
+                    {"option": "Hex", "string": "00112233445566778899aabbccddeeff"},
+                    {"option": "Hex", "string": ""},
+                    "CTR", "Raw", "Hex"
+                ]
+            }
+        ],
+    },
     {
         name: "AES Encrypt: AES-128-CBC with IV, ASCII",
         input: "The quick brown fox jumps over the lazy dog.",
@@ -645,6 +696,22 @@ The following algorithms will be used based on the size of the key:
             }
         ],
     },
+    {
+        name: "AES Decrypt: AES-128-CTR, no IV, ASCII",
+        input: "a98c9e8e3b7c894384d740e4f0f4ed0be2bbb1e0e13a255812c3c6b0a629e4ad759c075b2469c6f4fb2c0cf9",
+        expectedOutput: "The quick brown fox jumps over the lazy dog.",
+        recipeConfig: [
+            {
+                "op": "AES Decrypt",
+                "args": [
+                    {"option": "Hex", "string": "00112233445566778899aabbccddeeff"},
+                    {"option": "Hex", "string": ""},
+                    "CTR", "Hex", "Raw",
+                    {"option": "Hex", "string": ""}
+                ]
+            }
+        ],
+    },
     {
         name: "AES Decrypt: AES-128-CBC with IV, ASCII",
         input: "4fa077d50cc71a57393e7b542c4e3aea0fb75383b97083f2f568ffc13c0e7a47502ec6d9f25744a061a3a5e55fe95e8d",
@@ -807,7 +874,7 @@ The following algorithms will be used based on the size of the key:
     },
     {
         name: "AES Decrypt: AES-128-GCM, Binary",
-        input: "fa17fcbf5e8763322c1b0c8562e1512ed9d702ef70c1643572b9de3e34ae6b535e6c1b992432aa6d06fb6f80c861262aef66e7c26035afe77bd3861261e4e092b523f058f8ebef2143db21bc16d02f7a011efb07419300cb41c3b884d1d8d6a766b8963c",
+        input: "5a29debb5c5f38cdf8aee421bd94dbbf3399947faddf205f88b3ad8ecb0c51214ec0e28bf78942dfa212d7eb15259bbdcac677b4c05f473eeb9331d74f31d441d97d56eb5c73b586342d72128ca528813543dc0fc7eddb7477172cc9194c18b2e1383e4e",
         expectedOutput: "7a0e643132750e96d805d11e9e48e281fa39a41039286423cc1c045e5442b40bf1c3f2822bded3f9c8ef11cb25da64dda9c7ab87c246bd305385150c98f31465c2a6180fe81d31ea289b916504d5a12e1de26cb10adba84a0cb0c86f94bc14bc554f3018",
         recipeConfig: [
             {
@@ -816,7 +883,7 @@ The following algorithms will be used based on the size of the key:
                     {"option": "Hex", "string": "51e201d463698ef5f717f71f5b4712af"},
                     {"option": "Hex", "string": "1748e7179bd56570d51fa4ba287cc3e5"},
                     "GCM", "Hex", "Hex",
-                    {"option": "Hex", "string": "fa6bbb34c8cde65a3d7b93fb094fc84f"}
+                    {"option": "Hex", "string": "70fad2ca19412c20f40fd06918736e56"}
                 ]
             }
         ],
@@ -903,7 +970,7 @@ The following algorithms will be used based on the size of the key:
     },
     {
         name: "AES Decrypt: AES-192-GCM, Binary",
-        input: "ed22946f96964d300b45f5ce2d9601ba87682da1a603c90e6d4f7738729b0602f613ee392c9bfc7792594474f1213fb99185851f02ece4df0e93995e49f97aa4d0a337d7a80d83e4219dae5a3d36658f8659cdd5ed7c32707f98656fab7fb43f7a61e37c",
+        input: "318b479d919d506f0cd904f2676fab263a7921b6d7e0514f36e03ae2333b77fa66ef5600babcb2ee9718aeb71fc357412343c1f2cb351d8715bb0aedae4a6468124f9c4aaf6a721b306beddbe63a978bec8baeeba4b663be33ee5bc982746bd4aed1c38b",
         expectedOutput: "7a0e643132750e96d805d11e9e48e281fa39a41039286423cc1c045e5442b40bf1c3f2822bded3f9c8ef11cb25da64dda9c7ab87c246bd305385150c98f31465c2a6180fe81d31ea289b916504d5a12e1de26cb10adba84a0cb0c86f94bc14bc554f3018",
         recipeConfig: [
             {
@@ -912,7 +979,7 @@ The following algorithms will be used based on the size of the key:
                     {"option": "Hex", "string": "6801ed503c9d96ee5f9d78b07ab1b295dba3c2adf81c7816"},
                     {"option": "Hex", "string": "1748e7179bd56570d51fa4ba287cc3e5"},
                     "GCM", "Hex", "Hex",
-                    {"option": "Hex", "string": "be17cb31edb77f648b9d1032b235b33d"}
+                    {"option": "Hex", "string": "86db597d5302595223cadbd990f1309b"}
                 ]
             }
         ],
@@ -999,7 +1066,7 @@ The following algorithms will be used based on the size of the key:
     },
     {
         name: "AES Decrypt: AES-256-GCM, Binary",
-        input: "e3f1b236eaf3b9df69df8133a1b417fa42b242d8ad49e4d2f3469aca7e2a41737e4f2c8a0d212143287088fad51743577dc6dfa8ed328ca90113cbeb9b137926b2168cc037bdc371777e6ee02b9d9c017b6054fd83d43b4885fbe9c044a8574f1491a893",
+        input: "1287f188ad4d7ab0d9ff69b3c29cb11f861389532d8cb9337181da2e8cfc74a84927e8c0dd7a28a32fd485afe694259a63c199b199b95edd87c7aa95329feac340f2b78b72956a85f367044d821766b1b7135815571df44900695f1518cf3ae38ecb650f",
         expectedOutput: "7a0e643132750e96d805d11e9e48e281fa39a41039286423cc1c045e5442b40bf1c3f2822bded3f9c8ef11cb25da64dda9c7ab87c246bd305385150c98f31465c2a6180fe81d31ea289b916504d5a12e1de26cb10adba84a0cb0c86f94bc14bc554f3018",
         recipeConfig: [
             {
@@ -1008,7 +1075,7 @@ The following algorithms will be used based on the size of the key:
                     {"option": "Hex", "string": "2d767f6e9333d1c77581946e160b2b7368c2cdd5e2b80f04ca09d64e02afbfe1"},
                     {"option": "Hex", "string": "1748e7179bd56570d51fa4ba287cc3e5"},
                     "GCM", "Hex", "Hex",
-                    {"option": "Hex", "string": "23ddbd3ee4de33f98a9ea9a170bdf268"}
+                    {"option": "Hex", "string": "821b1e5f32dad052e502775a523d957a"}
                 ]
             }
         ],

tests/operations/tests/DefangIP.mjs

@@ -0,0 +1,43 @@
+/**
+ * DefangIP tests.
+ *
+ * @author h345983745
+ *
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+import TestRegister from "../../lib/TestRegister.mjs";
+
+TestRegister.addTests([
+    {
+        name: "Defang IP: Valid IPV4",
+        input: "192.168.1.1",
+        expectedOutput: "192[.]168[.]1[.]1",
+        recipeConfig: [
+            {
+                op: "Defang IP Addresses",
+                args: [],
+            },
+        ],
+    }, {
+        name: "Defang IP: Valid IPV6",
+        input: "2001:0db8:85a3:0000:0000:8a2e:0370:7343",
+        expectedOutput: "2001[:]0db8[:]85a3[:]0000[:]0000[:]8a2e[:]0370[:]7343",
+        recipeConfig: [
+            {
+                op: "Defang IP Addresses",
+                args: [],
+            },
+        ],
+    }, {
+        name: "Defang IP: Valid IPV6 Shorthand",
+        input: "2001:db8:3c4d:15::1a2f:1a2b",
+        expectedOutput: "2001[:]db8[:]3c4d[:]15[:][:]1a2f[:]1a2b",
+        recipeConfig: [
+            {
+                op: "Defang IP Addresses",
+                args: [],
+            },
+        ],
+    },
+]);

tests/operations/tests/ParseSSHHostKey.mjs

@@ -0,0 +1,65 @@
+/**
+ * Parse SSH Host Key tests
+ *
+ * @author j433866 [j433866@gmail.com]
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+import TestRegister from "../../lib/TestRegister.mjs";
+
+TestRegister.addTests([
+    {
+        name: "SSH Host Key: RSA",
+        input: "AAAAB3NzaC1yc2EAAAADAQABAAABAQDiJZ/9W9Ix/Dk9b+K4E+RGCug1AtkGXaJ9vNIY0YHFHLpWsB8DAuh/cGEI9TLbL1gzR2wG+RJNQ2EAQVWe6ypkK63Jm4zw4re+vhEiszpnP889J0h5N9yzyTndesrl4d3cQtv861FcKDPxUJbRALdtl6gwOB7BCL8gsXJLLVLO4EesrbPXD454qpVt7CgJXEXByOFjcIm3XwkdOnXMPHHnMSD7EIN1SvQMD6PfIDrbDd6KQt5QXW/Rc/BsfX5cbUIV1QW5A/GbepXHHKmWRtLC2J/mH3hW2Zq/hITPEaJdG1CtIilQmJaZGXpfGIwFeb0Av9pSL926arZZ6vDi9ctF",
+        expectedOutput: `Key type: ssh-rsa
+Exponent: 0x010001
+Modulus: 0x00e2259ffd5bd231fc393d6fe2b813e4460ae83502d9065da27dbcd218d181c51cba56b01f0302e87f706108f532db2f5833476c06f9124d43610041559eeb2a642badc99b8cf0e2b7bebe1122b33a673fcf3d27487937dcb3c939dd7acae5e1dddc42dbfceb515c2833f15096d100b76d97a830381ec108bf20b1724b2d52cee047acadb3d70f8e78aa956dec28095c45c1c8e1637089b75f091d3a75cc3c71e73120fb1083754af40c0fa3df203adb0dde8a42de505d6fd173f06c7d7e5c6d4215d505b903f19b7a95c71ca99646d2c2d89fe61f7856d99abf8484cf11a25d1b50ad222950989699197a5f188c0579bd00bfda522fddba6ab659eaf0e2f5cb45`,
+        recipeConfig: [
+            {
+                op: "Parse SSH Host Key",
+                args: ["Base64"]
+            }
+        ]
+    },
+    {
+        name: "SSH Host Key: DSA",
+        input: "AAAAB3NzaC1kc3MAAACBAMnoZCOzvaQqs//9mxK2USZvJBc7b1dFJiBcV80abN6maE+203pTRPIPCpPt0deQxv4YN3dSHoodEcArWxs1QRAIuRsQIvsUP7chovzGnxP84XWK5sbfrseD0vxZ7UR0NaAFPcSgeXcWC1SG9uvrAJQlyp4DBy+fKuqiYmwaz0bHAAAAFQCXNJ4yiE1V7LpCU2V1JKbqDvICMwAAAIB/5aR1iBOeyCVpj0dP3YZmoxd9R7FCC/0UuOf0lx4E6WHT6Z2QuPBhc2mpNDq2M0VF9oJfVWgcfG8r1rlXaCYODSacGcbnW5VKQ+LKkkALmg4h8jFCHReUC+Hmia/v8LyDwPO1wK6ETn7a3m80yM7gAU5ZNurVIVVP2lB65mjEsQAAAIA3ct9YRB6iUCvOD45sZM1C9oTC24Ttmaou0GcpWx3h0/iZ8mbil1cjaO9frRNZ/vSSVWEhEDNG8gwkjZWlvnJL3y1XUxbMll4WbmI/Q1kzKwopceaFwMbYTPKDg6L1RtCMUxSUyKsFk1c4SpEPlDS7DApZs5PgmWgMd/u6vwMXyg==",
+        expectedOutput: `Key type: ssh-dss
+p: 0x00c9e86423b3bda42ab3fffd9b12b651266f24173b6f574526205c57cd1a6cdea6684fb6d37a5344f20f0a93edd1d790c6fe183777521e8a1d11c02b5b1b35411008b91b1022fb143fb721a2fcc69f13fce1758ae6c6dfaec783d2fc59ed447435a0053dc4a07977160b5486f6ebeb009425ca9e03072f9f2aeaa2626c1acf46c7
+q: 0x0097349e32884d55ecba4253657524a6ea0ef20233
+g: 0x7fe5a47588139ec825698f474fdd8666a3177d47b1420bfd14b8e7f4971e04e961d3e99d90b8f0617369a9343ab6334545f6825f55681c7c6f2bd6b95768260e0d269c19c6e75b954a43e2ca92400b9a0e21f231421d17940be1e689afeff0bc83c0f3b5c0ae844e7edade6f34c8cee0014e5936ead521554fda507ae668c4b1
+y: 0x3772df58441ea2502bce0f8e6c64cd42f684c2db84ed99aa2ed067295b1de1d3f899f266e297572368ef5fad1359fef492556121103346f20c248d95a5be724bdf2d575316cc965e166e623f4359332b0a2971e685c0c6d84cf28383a2f546d08c531494c8ab059357384a910f9434bb0c0a59b393e099680c77fbbabf0317ca`,
+        recipeConfig: [
+            {
+                op: "Parse SSH Host Key",
+                args: ["Base64"]
+            }
+        ]
+    },
+    {
+        name: "SSH Host Key: ECDSA",
+        input: "AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGxZWSAGJyJQoVBwFCpr420eRUZDE/kw2YWm5vDro8050DZ1ZzqIuYaNl0BGzMcRTeasGtJuI8G84ZQQSgca3C4=",
+        expectedOutput: `Key type: ecdsa-sha2-nistp256
+Curve: nistp256
+Point: 0x046c59592006272250a15070142a6be36d1e45464313f930d985a6e6f0eba3cd39d03675673a88b9868d974046ccc7114de6ac1ad26e23c1bce194104a071adc2e`,
+        recipeConfig: [
+            {
+                op: "Parse SSH Host Key",
+                args: ["Base64"]
+            }
+        ]
+    },
+    {
+        name: "SSH Host Key: Extract key",
+        input: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDiJZ/9W9Ix/Dk9b+K4E+RGCug1AtkGXaJ9vNIY0YHFHLpWsB8DAuh/cGEI9TLbL1gzR2wG+RJNQ2EAQVWe6ypkK63Jm4zw4re+vhEiszpnP889J0h5N9yzyTndesrl4d3cQtv861FcKDPxUJbRALdtl6gwOB7BCL8gsXJLLVLO4EesrbPXD454qpVt7CgJXEXByOFjcIm3XwkdOnXMPHHnMSD7EIN1SvQMD6PfIDrbDd6KQt5QXW/Rc/BsfX5cbUIV1QW5A/GbepXHHKmWRtLC2J/mH3hW2Zq/hITPEaJdG1CtIilQmJaZGXpfGIwFeb0Av9pSL926arZZ6vDi9ctF test@test",
+        expectedOutput: `Key type: ssh-rsa
+Exponent: 0x010001
+Modulus: 0x00e2259ffd5bd231fc393d6fe2b813e4460ae83502d9065da27dbcd218d181c51cba56b01f0302e87f706108f532db2f5833476c06f9124d43610041559eeb2a642badc99b8cf0e2b7bebe1122b33a673fcf3d27487937dcb3c939dd7acae5e1dddc42dbfceb515c2833f15096d100b76d97a830381ec108bf20b1724b2d52cee047acadb3d70f8e78aa956dec28095c45c1c8e1637089b75f091d3a75cc3c71e73120fb1083754af40c0fa3df203adb0dde8a42de505d6fd173f06c7d7e5c6d4215d505b903f19b7a95c71ca99646d2c2d89fe61f7856d99abf8484cf11a25d1b50ad222950989699197a5f188c0579bd00bfda522fddba6ab659eaf0e2f5cb45`,
+        recipeConfig: [
+            {
+                op: "Parse SSH Host Key",
+                args: ["Base64"]
+            }
+        ]
+    }
+]);

tests/operations/tests/ParseUDP.mjs

@@ -0,0 +1,68 @@
+/**
+ * Parse UDP tests.
+ *
+ * @author h345983745
+ *
+ * @copyright Crown Copyright 2019
+ * @license Apache-2.0
+ */
+import TestRegister from "../../lib/TestRegister.mjs";
+
+TestRegister.addTests([
+    {
+        name: "Parse UDP: No Data - JSON",
+        input: "04 89 00 35 00 2c 01 01",
+        expectedOutput: "{\"Source port\":1161,\"Destination port\":53,\"Length\":44,\"Checksum\":\"0101\"}",
+        recipeConfig: [
+            {
+                op: "From Hex",
+                args: ["Auto"],
+            },
+            {
+                op: "Parse UDP",
+                args: [],
+            },
+            {
+                op: "JSON Minify",
+                args: [],
+            },
+        ],
+    }, {
+        name: "Parse UDP: With Data - JSON",
+        input: "04 89 00 35 00 2c 01 01 02 02",
+        expectedOutput: "{\"Source port\":1161,\"Destination port\":53,\"Length\":44,\"Checksum\":\"0101\",\"Data\":\"0202\"}",
+        recipeConfig: [
+            {
+                op: "From Hex",
+                args: ["Auto"],
+            },
+            {
+                op: "Parse UDP",
+                args: [],
+            },
+            {
+                op: "JSON Minify",
+                args: [],
+            },
+        ],
+    },
+    {
+        name: "Parse UDP: Not Enough Bytes",
+        input: "04 89 00",
+        expectedOutput: "Need 8 bytes for a UDP Header",
+        recipeConfig: [
+            {
+                op: "From Hex",
+                args: ["Auto"],
+            },
+            {
+                op: "Parse UDP",
+                args: [],
+            },
+            {
+                op: "JSON Minify",
+                args: [],
+            },
+        ],
+    }
+]);

webpack.config.js

@@ -103,11 +103,17 @@ module.exports = {
                     "sass-loader",
                 ]
             },
+            /**
+             * The limit for these files has been increased to 60,000 (60KB)
+             * to ensure the material icons font is inlined.
+             *
+             * See: https://github.com/gchq/CyberChef/issues/612
+             */
             {
                 test: /\.(ico|eot|ttf|woff|woff2)$/,
                 loader: "url-loader",
                 options: {
-                    limit: 10000,
+                    limit: 60000,
                     name: "[hash].[ext]",
                     outputPath: "assets"
                 }