JavaScript logo

Back when I first wrote my unzip implementation in pure JS using Web Workers (code here), JavaScript runtimes were a very new thing (NodeJS had been released less than a year before). Ok, I had played with C++ bindings to the V8 JS engine for a hobby video game engine I had been writing, but that was it for me when it came to "JavaScript outside of the browser".

Well over a decade later and JavaScript/Typescript runtimes are all the rage in this continuously fractious software world. Even so, it hadn't really ever occurred to me that the unzip/unrar/untar implementations in BitJS might be useful in NodeJS or other runtimes (Deno, Bun) until someone opened a bug.

Anyway, the way unzip/unrar worked was pretty straightforward: The host code passes bytes into the unzip/unrar implementation via a postMessage() call, the implementation does some bits and bobs as a WebWorker (aka not on the UI thread), crawling through bytes of the archive and emitting interesting events that the host code listens for (like "here's a file I extracted").

sequenceDiagram participant Host participant Worker box Worker JavaScript Context participant WorkerGlobalScope participant unrar.js end Host->>Worker: postMessage<br/>(rar bytes) Worker-->>WorkerGlobalScope: WorkerGlobalScope->>unrar.js: onmessage<br/>(rar bytes) Note right of unrar.js: unrar<br/>the thing unrar.js->>WorkerGlobalScope: postMessage<br/>(an extracted file) WorkerGlobalScope-->>Worker: Worker->>Host: onmessage<br/>(an extracted file) unrar.js->>WorkerGlobalScope: postMessage<br/>(2nd extracted file) WorkerGlobalScope-->>Worker: Worker->>Host: onmessage<br/>(2nd extracted file)

Unfortunately, Node still has not adopted Web Workers (though eventually they may); they even have their own different thing called Worker Threads - confusing. Anyway, it left me wondering how I should approach supporting Node... until I learned about MessageChannel / MessagePort, which are now supported nearly universally (as of Node 15).

So in the end, it continues to be pretty simple. The MessageChannel becomes the abstraction through which messages are passed, the host code owns one MessagePort, the unzip implementation owns the other, and the implementation no longer assumes it lives in a WebWorker (oh and thanks Dynamic Imports!).

sequenceDiagram participant Host Code participant Port1 box Any JavaScript Context (could be a Web Worker) participant Port2 participant unrar.js end Host Code->>Port1: postMessage(rar bytes) Port1-->>Port2: (MessageChannel) Port2->>unrar.js: onmessage(rar bytes) Note right of unrar.js: unrar<br/>the thing unrar.js->>Port2: postMessage(an extracted file) Port2-->>Port1: (MessageChannel) Port1->>Host Code: onmessage(an extracted file) unrar.js->>Port2: postMessage(2nd extracted file) Port2-->>Port1: (MessageChannel) Port1->>Host Code: onmessage(2nd extracted file)

This allows environments that support Web Workers to keep their Web Worker implementation and the NodeJS version to have the implementation in its main thread. If someone wants to make it more performant using Node's Worker Threads send pull requests!

It seems like all JS libraries that do intensive computations (like training ML models or mining teh bitcoinz) and then emit a series of events, should probably think of MessageChannel as the means of communication with the host software going forward so that the implementation can be ported to more environments. What? WebAssembly? ... oh shhhh!

This little weekend hack also let me write some decent automated unit tests for BitJS decompression, so hurrah for that too!

§1357 · December 27, 2023 · Uncategorized · (No comments) ·

JavaScript logo

The esteemed Dr. Axel Rauschmayer has written a blog post about a new proposal for JavaScript: Types as Comments. It definitely has got me thinking.

On the one hand, I'm a huge fan of JavaScript evolution over the last decade or so. Arrow functions, const/var, classes, modules - that's all good stuff that has improved the language.

Typescript logo

Seemingly on the same side of this argument: I'm also a huge fan of Typescript. It changed how I do large-scale frontend development four or so years ago - static typing in a language helps me write clear code, catches a huge number of bugs at compile-time, and the tooling and overall ecosystem around it is top-notch. Kudos to Microsoft on this one.

But I'm not convinced the value of this proposal nets out positive. The proposal itself says that the primary motivation is to inch JavaScript evolution towards eventually supporting static types:

Does JavaScript need static type-checking?

"Given how much effort organizations and teams have put into building type-checkers and adopting them, the answer is yes."



You have to accept that JavaScript is the language of the runtime. We shouldn't be caring about types at runtime - that's the job of the toolchain prior to deploying. That's the way every other language works (feel free to tell me how wrong I am in the comments - I am no language expert!).

Static typing only helps developers, not users.

Think about it this way: what level of performance degradation are you willing to accept for full support of static types? Is it ok for the JavaScript parser and runtime to be 5% slower for every web page or 15% larger in code size? Of course I'm pulling these numbers out of my ass.

When I was first fanboying out on TS, I used to think "gee, wouldn't it be really cool for browsers to support Typescript natively". Wouldn't that be an awesome way to learn and view source, etc. But you know what's good for that instead? Super-fast and small JavaScript for best performance and if you want to share your developer brilliance, have an optional compile-mode and source maps that point to your awesome Typescript.

Patrick the Star

Another thing I don't like about the proposal is that, while they are clearly heavily influenced by Typescript, they hedge:

How does this proposal relate to TypeScript?

"This proposal is a balancing act: trying to be as TypeScript compatible as possible while still allowing other type systems"

You are JavaScript - if you are adamant about eventually supporting static types, why not boldly point towards a "north star" of a language that has already proven itself? Is it that you don't want to admit Typescript won? Is it lingering anti-Microsoft bias?

My opinion: If we are going to add static typing for web apps, we should just use Typescript as a new script type. <script type="application/x-typescript"> seems a better option to me so that the browser can choose an appropriate parser, etc.

§1296 · March 10, 2022 · JavaScript, Software, Technology, Uncategorized, Web · 1 comment ·

Well, two posts in a year - that's better than only one! Let's see, what did I accomplish this year, hack-wise?

  • bitjs, Binary Tools for JavaScript:
    • Added a Zipper to create a zip file in JS from byte arrays (issue #29). No compression, store-only for now.
  • kthoom, a Comic Book Reader:
    • Added a Metadata Viewer (issue #18) and Editor (issue #49) using the aforementioned Zipper.
  • TNO, a turn-based strategy game:
    • Finished rewriting an old, DOS-based game as a web-based game for its 25th anniversary 🙂
  • Carve, a vector graphics editor:
    • Started creating a rudimentary SVG editor in the open. Maybe this was irresponsible of me, given SVG-edit exists and I have lots of history there, but I wanted fresh infrastructure (TypeScript, mainly) and a chance to try and get the architecture right in the beginning..
  • Music playlist service and player
    • Private project where I assembled all my offline music, created a microservice to arrange, list, edit music into playlists stored in the Cloud (Firestore), and accessed via a web music player.

All of these little hack projects are strictly in the service of scratching various itches I have. Any fun projects you worked on this year?

Happy New Year, all!

§1292 · January 1, 2022 · Uncategorized · Comments Off on Hack Scratch 2021 ·

Logo for the kthoom comic book reader

Decided to pluck at one of my open source projects on the weekend. The latest idea was to use the Google Drive API to let users load up their comic book files (cbr/cbz/cbt) into kthoom (a comic book reader).

This should help those users storing their comic books on Google Drive to be able to access them on Chromebooks. Haven't tried it on a tablet yet - I wonder if that works...

I'm always accepting patches if someone wants to write the equivalent code for Microsoft OneDrive.

§1048 · August 26, 2014 · Uncategorized · 3 comments ·

I want to believe. SVG as an image format.After finding some time to go through and mass-delete spam comments on this blog, I realized that it's time to turn on the "Automatically close comments on post older than XX days" checkbox in WordPress. This means that posts older than 90 days will no longer be able to be commented on.

In case it's not obvious, I haven't been writing that much here these days. I've been too busy at my day job as a Google+ engineer and at my night job being a husband and father. If you really want to "follow me" these days, the best bet is Google+ for now. Let me know if you need an invite.

I'm not saying I won't pick up the ol' blogging pen from time to time. In fact, writing this very post has made me realize how much the UI has improved in WordPress. Also, I might have a couple new things to talk about in a little while, which I'll likely post here and reference on Google+.

And yes, I've obviously been fully absorbed into the collective at this point 🙂

§1021 · September 11, 2011 · Uncategorized · 3 comments ·