dice

Once upon a time, I ported a turn-based strategy / tactical combat game to the web. The average play time for the game is quite long (many hours), so saving games is an important feature.

PC/Console games have two ways to do "game saves":

  1. Save on the machine, keyed to the user thanks to the operating system's file system and
  2. Save on the cloud, keyed to the user by some identity provider.

1) is easy to implement (writing and reading a file to disk). 2) is much harder to implement, costs money to run, and requires the gaming machine to have network connectivity.

Web games can do 2) just as easily as PC games. Since this is a hobby project with near-zero players, I don't plan to build my own Cloud service, and I don't trust any Cloud-hosted "gaming" API to be stable enough or not get killed in the future. So that leaves me with saving and loading games locally.

Unfortunately, browsers have a major road block: they do not allow native file system access. Back a decade ago when I started to write the game, it looked to me like browsers would eventually implement this: progress on browser compatibility and "native app" capabilities were pretty breathtaking at that time.

So I put up with some jankiness: To load a savegame, pick a file from your computer. To save a game, the web app downloads the savegame file to your computer. That kind of sucks and doesn't allow a save-as-you-play experience, but good enough for now. I threw in more jank with the option to use Google Drive API as well for a cheap-o version of a Cloud service. (Why not Play Games API? Because they eventually killed their Web API. See "stable forever" above.)

All this jank was an acceptable workaround until browsers caught up, or so I thought.

dice

Here we are a decade later. Though Chrome desktop has proposed and implemented the File System Access API for native file access, it's not quite enough when compared to what native apps can do - and other browser vendors have adamantly refused to implement true native file access, making this capability a sad bloated corpse floating in a lake. I'm saying native file access is dead in the water.

Why didn't I just use localStorage or IndexedDb for this? Well, for one, it was stubbornness and naïvety. For two, I worried about how much data I could store reliably. For three, I thought that users would expect their saved games in Chrome to be playable in Firefox or Safari on the same machine or want ways of passing their saved games around.

The fact my savegames are something like 35k means that storage was never really a concern.

The fact that most users only ever use a single browser, means I don't need to really worry that much about portability.

It's actually pretty straightforward to implement a virtual file system in IndexedDb that works good enough. Heck, Apple did it as some weird peace offering to people who wanted File System Access. [As an aside: as far as I can tell, an origin-private file system offers zero user-facing benefits over IndexedDb, you're just trading one API for another... so it's a devex offering. True native file access offers user benefits, such as backing up your save games, hacking, modding, sharing outside of the browser]

So in the end, I just wrote my own storage service that loads/saves games to an IndexedDB store in the browser. These days, superior user experience trumps web specification idealism for me every time.

apple

P.S. I'm no security expert. I'm sure Mozilla and Apple have good reasons for rejecting the File System Access API. I'm just sitting here loving the technology of the web (zero install, the APIs, JavaScript/Typescript) and the developer experience but wishing I had all the perks that native apps do.

P.P.S. I was surprised to learn that an IndexedDB instance for a page at foo.net is different than the IndexedDB instance of an iframe of foo.net hosted at bar.net. That's a subtlety I hadn't thought through.

§1394 · April 19, 2024 · Apple, Entertainment, Firefox, Games, Google, Software, Technology, Web · (No comments) ·


dice

I am a big BIG fan of Bethesda's Elder Scrolls and I consider Morrowind one of the best games ever made (my top 2 are Morrowind and Doom in no particular order). I have put in only a few hours into Starfield (so far), and I have to say I like it less and less. Primarily my complaints lie with all the loading scenes that break the illusion of an immersive, seamless galaxy to explore. It feels like Skyrim with even more disconnected regions.

In Morrowind you could walk from the wilderness into a city with no observable loading, the only loading screens were upon fast-travel (which was not necessary) or going indoors. As you leveled up, you could learn levitate spells that let you fly over seemingly insurmountable mountain ranges or water breathing spells that let you explore underwater. You could learn water walking and even walk across the ocean from the mainland to the new island included in the Bloodmoon expansion, if you were patient enough and didn't want to fast-travel via ship.

Oblivion did away with some of the crazier things (like Levitation and Water Walking), and added loading screens as you move into cities and this trend sadly continued into Skyrim. And yet, I still loved those games and devoured them, spending hours. Something about the ability to look at the horizon and say "I'm going to walk to there" was very captivating.

In Starfield, you get loading screen after loading screen, moving from location to location, outdoor to indoor, in and out of spaceships, launching from the surface, grav-jumping, etc. I know this criticism is not incredibly insightful or novel; many folks have already complained loudly about this online and contrasted it against the illusion of seamlessness in No Man's Sky. I know I'm just adding to the din, but I can't help myself, my disappointment is as vast as the regions in Starfield are not.

There are no enormous planets floating in endless space. You're just an ant hopping from leaf to pretty leaf trying to cross a river. I expected so much more.

I will give it a few more hours, but sadly it's been a dud for me so far. Anybody have a recommendation for a modern open world game that knows how to maintain the illusion?

§1376 · January 18, 2024 · Entertainment, Games, Microsoft · 1 comment ·


music

Apropos of nothing, to the person who once said to me that the 1990s had no good music, I offer a list of 40 albums worth owning. This list has a good drubbing of obvious Alternative Rock and Brit Pop picks, but banks and dips into Shoe Gaze, Noise Pop, Trip Hop, and Electronica, which are more recent musical interests of mine. I own other albums from the nineties that are Canadian Rock-weighted (Tragically Hip, Tea Party, Odds):

  • Heaven or Las Vegas by The Cocteau Twins, 1990
  • Gala by Lush, 1990
  • Achtung Baby! by U2, 1991
  • Ten by Pearl Jam, 1991
  • Nevermind by Nirvana, 1991
  • Loveless by My Bloody Valentine, 1991
  • Blood Sugar Sex Magik by Red Hot Chili Peppers, 1991
  • Screamadelica by Primal Scream, 1991
  • Peng! by Stereolab, 1992
  • Lazer-Guided Melodies by Spiritualized, 1992
  • So Tonight That I Might See by Mazzy Star, 1993
  • Siamese Dream by The Smashing Pumpkins, 1993
  • Debut by Björk, 1993
  • G-Stoned by Kruder & Dorfmeister, 1993
  • Zooropa by U2, 1993
  • Dummy by Portishead, 1994
  • Superunknown by Soundgarden, 1994
  • Weezer by Weezer, 1994
  • Post by Björk, 1995
  • (What's the Story) Morning Glory? by Oasis, 1995
  • Mellon Collie and the Infinite Sadness by The Smashing Pumpkins, 1995
  • The Bends by Radiohead, 1995
  • Exit Planet Dust by The Chemical Brothers, 1995
  • Electr-O-Pura by Yo La Tengo, 1995
  • Odelay by Beck, 1996
  • Emperor Tomato Ketchup by Stereolab, 1996
  • Among My Swan by Mazzy Star, 1996
  • Ok, Computer by Radiohead, 1997
  • Urban Hymns by The Verve, 1997
  • Ladies and Gentlemen We Are Floating in Space by Spiritualized, 1997
  • Homogenic by Björk, 1997
  • Opera by Tosca, 1997
  • I Can Hear the Heart Beating as One by Yo La Tengo, 1997
  • Northern Star by Groove Armada, 1998
  • Mezzanine by Massive Attack, 1998
  • Moon Safari by Air, 1998
  • Music Has the Right to Children by Boards of Canada, 1998
  • Glee by Bran Van 3000, 1998
  • Vertigo by Groove Armada, 1999
  • Surrender by The Chemical Brothers, 1999

That 1997 tho.

And since I want to keep the party going, here's an extension of this list into the first half of the 2000's, 20 more albums too good to leave off:

  • Since I Left You by The Avalanches, 2000
  • Felt Mountain by Goldfrapp, 2000
  • Kid A by Radiohead, 2000
  • And Then Nothing Turned Itself Inside-Out by Yo La Tengo, 2000
  • Simple Things by Zero 7, 2001
  • Amnesiac by Radiohead, 2001
  • Comfort Eagle by Cake, 2001
  • Goodbye Country (Hello Nightclub) by Groove Armada, 2001
  • Charango by Morcheeba, 2002
  • Machine Says Yes by FC Kahuna, 2002
  • Lost Horizons by Lemon Jelly, 2002
  • Details by Frou Frou, 2002
  • Elephant by The White Stripes, 2003
  • Black Cherry by Goldfrapp, 2003
  • Talkie Walkie by Air, 2004
  • When It Falls by Zero 7, 2004
  • Lemon Jelly.ky by Lemon Jelly, 2004
  • The Campfire Headphase by Boards of Canada, 2005
  • Haughty Melodic by Mike Doughty, 2005
  • The Cosmic Game by Thievery Corporation, 2005

I guess it's kind of easy to look back twenty years into the past and peer at "the best of" with the rosy hue of critical acclaim and record sales, but man, do I enjoy the music. Please drop me notes on what I've missed or what I should pick up from the next twenty years!

§1286 · January 3, 2024 · Entertainment, Music · (No comments) ·


JavaScript logo

Back when I first wrote my unzip implementation in pure JS using Web Workers (code here), JavaScript runtimes were a very new thing (NodeJS had been released less than a year before). Ok, I had played with C++ bindings to the V8 JS engine for a hobby video game engine I had been writing, but that was it for me when it came to "JavaScript outside of the browser".

Well over a decade later and JavaScript/Typescript runtimes are all the rage in this continuously fractious software world. Even so, it hadn't really ever occurred to me that the unzip/unrar/untar implementations in BitJS might be useful in NodeJS or other runtimes (Deno, Bun) until someone opened a bug.

Anyway, the way unzip/unrar worked was pretty straightforward: The host code passes bytes into the unzip/unrar implementation via a postMessage() call, the implementation does some bits and bobs as a WebWorker (aka not on the UI thread), crawling through bytes of the archive and emitting interesting events that the host code listens for (like "here's a file I extracted").

sequenceDiagram participant Host participant Worker box Worker JavaScript Context participant WorkerGlobalScope participant unrar.js end Host->>Worker: postMessage<br/>(rar bytes) Worker-->>WorkerGlobalScope: WorkerGlobalScope->>unrar.js: onmessage<br/>(rar bytes) Note right of unrar.js: unrar<br/>the thing unrar.js->>WorkerGlobalScope: postMessage<br/>(an extracted file) WorkerGlobalScope-->>Worker: Worker->>Host: onmessage<br/>(an extracted file) unrar.js->>WorkerGlobalScope: postMessage<br/>(2nd extracted file) WorkerGlobalScope-->>Worker: Worker->>Host: onmessage<br/>(2nd extracted file)

Unfortunately, Node still has not adopted Web Workers (though eventually they may); they even have their own different thing called Worker Threads - confusing. Anyway, it left me wondering how I should approach supporting Node... until I learned about MessageChannel / MessagePort, which are now supported nearly universally (as of Node 15).

So in the end, it continues to be pretty simple. The MessageChannel becomes the abstraction through which messages are passed, the host code owns one MessagePort, the unzip implementation owns the other, and the implementation no longer assumes it lives in a WebWorker (oh and thanks Dynamic Imports!).

sequenceDiagram participant Host Code participant Port1 box Any JavaScript Context (could be a Web Worker) participant Port2 participant unrar.js end Host Code->>Port1: postMessage(rar bytes) Port1-->>Port2: (MessageChannel) Port2->>unrar.js: onmessage(rar bytes) Note right of unrar.js: unrar<br/>the thing unrar.js->>Port2: postMessage(an extracted file) Port2-->>Port1: (MessageChannel) Port1->>Host Code: onmessage(an extracted file) unrar.js->>Port2: postMessage(2nd extracted file) Port2-->>Port1: (MessageChannel) Port1->>Host Code: onmessage(2nd extracted file)

This allows environments that support Web Workers to keep their Web Worker implementation and the NodeJS version to have the implementation in its main thread. If someone wants to make it more performant using Node's Worker Threads send pull requests!

It seems like all JS libraries that do intensive computations (like training ML models or mining teh bitcoinz) and then emit a series of events, should probably think of MessageChannel as the means of communication with the host software going forward so that the implementation can be ported to more environments. What? WebAssembly? ... oh shhhh!

This little weekend hack also let me write some decent automated unit tests for BitJS decompression, so hurrah for that too!

§1357 · December 27, 2023 · Uncategorized · (No comments) ·


JavaScript

Oh, remember that time six years ago when I said one day soon we'll be able to use ES Modules everywhere, even inside Web Workers and Service Workers? Well that day is next week when the last browser ships support for ES Modules in Workers - finally!

Yes, I know that ES Modules are terrible, actually - but the fact is that they are the future of JavaScript / TypeScript / Node / Deno development so we all might as well accept it. I actually think they're great. I've been using them for hobby projects forever - though I rarely use transpilers, bundlers, etc.

Celebrate

Sure, it will be some time before CommonJS modules are resigned to the ash heap of history, until then https://esm.sh/#docs can help... but I for one welcome our Isomorphic JavaScript overlords.

§1350 · May 31, 2023 · JavaScript, Software, Technology, Web · (No comments) ·