Channeling a Communication Breakdown

JavaScript logo

Back when I first wrote my unzip implementation in pure JS using Web Workers (code here), JavaScript runtimes were a very new thing (NodeJS had been released less than a year before). Ok, I had played with C++ bindings to the V8 JS engine for a hobby video game engine I had been writing, but that was it for me when it came to "JavaScript outside of the browser".

Well over a decade later and JavaScript/Typescript runtimes are all the rage in this continuously fractious software world. Even so, it hadn't really ever occurred to me that the unzip/unrar/untar implementations in BitJS might be useful in NodeJS or other runtimes (Deno, Bun) until someone opened a bug.

Anyway, the way unzip/unrar worked was pretty straightforward: The host code passes bytes into the unzip/unrar implementation via a postMessage() call, the implementation does some bits and bobs as a WebWorker (aka not on the UI thread), crawling through bytes of the archive and emitting interesting events that the host code listens for (like "here's a file I extracted").

sequenceDiagram participant Host participant Worker box Worker JavaScript Context participant WorkerGlobalScope participant unrar.js end Host->>Worker: postMessage<br/>(rar bytes) Worker-->>WorkerGlobalScope: WorkerGlobalScope->>unrar.js: onmessage<br/>(rar bytes) Note right of unrar.js: unrar<br/>the thing unrar.js->>WorkerGlobalScope: postMessage<br/>(an extracted file) WorkerGlobalScope-->>Worker: Worker->>Host: onmessage<br/>(an extracted file) unrar.js->>WorkerGlobalScope: postMessage<br/>(2nd extracted file) WorkerGlobalScope-->>Worker: Worker->>Host: onmessage<br/>(2nd extracted file)

Unfortunately, Node still has not adopted Web Workers (though eventually they may); they even have their own different thing called Worker Threads - confusing. Anyway, it left me wondering how I should approach supporting Node... until I learned about MessageChannel / MessagePort, which are now supported nearly universally (as of Node 15).

So in the end, it continues to be pretty simple. The MessageChannel becomes the abstraction through which messages are passed, the host code owns one MessagePort, the unzip implementation owns the other, and the implementation no longer assumes it lives in a WebWorker (oh and thanks Dynamic Imports!).

sequenceDiagram participant Host Code participant Port1 box Any JavaScript Context (could be a Web Worker) participant Port2 participant unrar.js end Host Code->>Port1: postMessage(rar bytes) Port1-->>Port2: (MessageChannel) Port2->>unrar.js: onmessage(rar bytes) Note right of unrar.js: unrar<br/>the thing unrar.js->>Port2: postMessage(an extracted file) Port2-->>Port1: (MessageChannel) Port1->>Host Code: onmessage(an extracted file) unrar.js->>Port2: postMessage(2nd extracted file) Port2-->>Port1: (MessageChannel) Port1->>Host Code: onmessage(2nd extracted file)

This allows environments that support Web Workers to keep their Web Worker implementation and the NodeJS version to have the implementation in its main thread. If someone wants to make it more performant using Node's Worker Threads send pull requests!

It seems like all JS libraries that do intensive computations (like training ML models or mining teh bitcoinz) and then emit a series of events, should probably think of MessageChannel as the means of communication with the host software going forward so that the implementation can be ported to more environments. What? WebAssembly? ... oh shhhh!

This little weekend hack also let me write some decent automated unit tests for BitJS decompression, so hurrah for that too!

§1357 · December 27, 2023 · Uncategorized · (No comments) ·

All We Need is Just a Little Patience

JavaScript

Oh, remember that time six years ago when I said one day soon we'll be able to use ES Modules everywhere, even inside Web Workers and Service Workers? Well that day is next week when the last browser ships support for ES Modules in Workers - finally!

Yes, I know that ES Modules are terrible, actually - but the fact is that they are the future of JavaScript / TypeScript / Node / Deno development so we all might as well accept it. I actually think they're great. I've been using them for hobby projects forever - though I rarely use transpilers, bundlers, etc.

Celebrate

Sure, it will be some time before CommonJS modules are resigned to the ash heap of history, until then https://esm.sh/#docs can help... but I for one welcome our Isomorphic JavaScript overlords.

§1350 · May 31, 2023 · JavaScript, Software, Technology, Web · Comments Off on All We Need is Just a Little Patience ·

The Story of The Veldt

Roll the dice

I'll try to keep this one short - it's not exactly tech-related, but it kinda is. A tiny story about a tiny story that you might enjoy, in three parts.

Part One

I got into some speculative short story reading during the pandemic and Ray Bradbury was one author I got into. I think it started with some of my comic book research from the 1950s. Now I won't go as far as Rachel Bloom on ol' Ray, but the man definitely knew how to spin a yarn. In terms of sheer volume of interesting ideas, he beats out Isaac Asimov for me big time.

Anyway, in 1950 Ray Bradbury wrote this great short story called "The Veldt". It's only 13 pages and, rather than read a synopsis about the story, I think you should read it. Here's a free copy I found, though I noticed some typos. G'wan read it, I'll wait.

Back? I thought it was great, what about you? I think it's got a lot to say about how tech has invaded our lives, de-sensitized us from each other, maybe other things that are way over my head. And it holds up today - nice and dark.

Part Two

Ever hear of deadmau5? He's an electronic music artist from my homeland. Not sure if you're into progressive house or the techno scene, but the guy has earned a lot of awards for his work. Anyway, he wrote a song called The Veldt in 2012 (the year Ray Bradbury died). Here you should watch the video:

Rolling Stone thought it was one of the 50 best songs of 2012. My musical wheelhouse is more the 1990s and early aughts, but I really love the tune.

Part Three

And how did this come to be? Apparently deadmau5 created the music in a "22-hour live streaming session" back in March 2012. Crazy, but what's even crazier is that one day later, deadmau5 discovered that a fan named Chris James had posted vocals for the track. He was ready to shred the guy on his live stream but instead this happened:

So yeah, that music video you watched, the song that officially landed on the album, the one that Rolling Stone thought so highly of, is the combination of deadmau5 and Chris James' vocals weaved together over the course of 48 hours.

In some way, I find this a satisfying balance to Ray's meditations on technology... Anyway, I'm off to play in Africa!

§1343 · May 17, 2023 · Entertainment, Music, Technology · Comments Off on The Story of The Veldt ·

Dead Simple: TSX, ES Modules, Preact

Logo for JSON

I decided to learn a bit of React again for some toy projects (my day job JS framework is Angular). Most of the tutorials for setting up a React project "from scratch" ultimately result in using the create-react-app library to hide a lot of the complexity (transpilers, bundlers), but I'm not interested in Webpack or Babel - primarily because I know browsers don't need them anymore. Also, I like to learn about how the bits and bobs work together for small projects like this. Here are the 3 things I want: Typescript, JSX, and compile/deploy to ES modules. Can this actually be done? Yes, and it's not that difficult.

Read the rest of this entry ...
§1318 · April 8, 2023 · JavaScript, React, Software, Technology, Web · Comments Off on Dead Simple: TSX, ES Modules, Preact ·

3 Wishes for JSON

Logo for JSON

The elder days of my blog are littered with useless musings on XML and JSON. So why not throw another one on the pile? 🙂

I wish JSON supported:

  1. Multi-line strings (but no string interpolation)
  2. Trailing commas (in arrays and object property lists)
  3. Comments

Of course none of these will probably ever happen, but having any of these three would keep JSON wonderfully simple but vastly improve the user experience of those who have to hand-tweak JSON files (which is a lot of people!). I kept these feature requests to only things supported by ECMAScript itself. I also note that several parsers seem to accept JSON with #2 and #3.

YAML has all these, but of course it has many other features that, I assume, make it burdensome to write a fully compliant parser.

[Update Jan 2023: I finally filed a bug on VS Code for this idea of back-ticked string support in their tasks.json. Maybe it's crazy, I haven't decided... but feel free to vote it up!]

§1306 · October 26, 2022 · JavaScript, Software, Technology · Comments Off on 3 Wishes for JSON · Tags: , ,

← Older entries
Newer entries →