1BRC in Dart and JavaScript

After receiving some encouraging words, I just wanted to provide a super quick follow-up to the previous article. If you haven’t read it, it’s worth reading it first. And thanks to everyone who made it through and provided feedback 🙏.

One of the points I found personally most intriguing and hard to quantify, was the amortization of JIT overhead and the reverse-complement JIT dominance. As such, I thought it would be contemporary and fun to look at some One-Billion-Row-Challenge (1BRC) results. In a nutshell: the challenge’s goal is to parse 1 billion temperature measurements from a ~14GB file and calculate the avg/min/max for all weather stations as quickly as possible. The main bottlenecks tend to be:

• How to best read the file, ideally in parallel
• How to best parse each measurement, also in parallel
• And efficient accumulation, e.g. hash map access.

Based on the attention gathered in other communities, I was a bit surprised to see so few solutions for JavaScript or Dart. Ultimately, I did find one decently fast JS implementation but nothing even remotely comparable in Dart. Well then, I did a quick conversion of the JS solution with a few smaller tweaks. If nothing else, it’s probably a more apples to apples comparison this way. Also, I promise that this time around it’s actually much more of a language performance comparison, since both implementations only defer to their native runtimes for file I/O:

It turns out that given the nature of the problem, large and tight loops, Dart JIT takes the 🎂. I guess if one is Ok with a 64% memory penalty for a 21% performance improvement, JITs are not entirely w/o merit, at least in Dart. That said and if performance is of utmost importance to you, this kind of problem lends itself to lower-level languages with some clocking in at under 1s (C, C++, Rust, …). A lot of the gains can be attributed to memory mapped IO and cache-line-friendly hash maps. Some solutions went over board severely compromising on hash collisions.

It’s also interesting to see on a tangent that the fastest Java implementations used GraalVM native image, i.e. AOT compiled Java 🤯.

Keeping it short, we saw that Dart gave us a slight edge over Node.js. Dart JIT took the shortest amount of time, while Dart AOT provided easily the best performance by footprint. Furthering my confirmation bias, it was fun to see GraalVM native image beating out the JVM 😈¹.