SYNX 3.8 is faster everywhere, and every parser finally agrees

SYNX 3.8.0 is out, and if all you do is write config files, you won't see anything new in the files themselves, because the grammar hasn't moved at all. What moved is everything underneath it: the Rust core, the TypeScript package, the Python and Node bindings and all seven native ports were measured against 3.7.4 on the same machine and the same set of files, and then rebuilt wherever the time was actually going.

How much faster

Here's the short version, written in the format itself:

# 3.7.4 -> 3.8.0, same machine, same files
rust
  config_2_5kb     38.6us -> 17.6us
  static_4mb       60.1ms -> 35.3ms
  active_1_2mb     81.0ms -> 22.5ms
typescript_node
  config_2_5kb     34.0us -> 20.4us
  active_1_2mb    297.5ms -> 31.1ms
python
  config_2_5kb     50.3us -> 17.6us
  synxl_100k      173.7ms -> 83.2ms

The biggest jump is TypeScript on large documents with markers, and the reason is a little embarrassing: every :calc key used to rescan its expression against every other key in the document, so the work grew with the square of the file. In Rust most of the win came from memory. With the Windows system allocator a plain parse spent close to half its time asking for memory, and 3.8.0 asks about a third as often on static files and a fifth as often in active mode. The binary .synxb format got the same treatment: the strings a document uses most now get the shortest indexes, so files are 3 to 20 percent smaller in Rust and up to 40 percent smaller in TypeScript, and the same document now always compiles to the same bytes, which 3.7.4 didn't promise.

The parsers didn't actually agree

This part we didn't go looking for. To make sure the faster code gave the same results, we started feeding the same randomly generated documents to the Rust core and to the TypeScript package and comparing what came out, and 17 percent of the !active documents came out different. Some of that was TypeScript being wrong and some was Rust, and some of it was Rust being wrong differently on every run: it walked objects in hash-map order, so the tax:calc / total:calc example straight from the spec failed with CALC_ERR in about half of all runs.

And once we started looking, there was more of it. Markers inside list items were never applied in Rust at all. Synx.stringify and .synxb changed data on the way back, so "007" came back as the number 7 and |+ code blocks lost their indentation, and 1,777 of 3,000 random documents didn't survive the round trip. A few documents could even take the whole process down: :clamp:nan:5 made Rust panic, and a file that included itself overflowed the stack. All 55 of these are fixed, and on the final code the Rust core and TypeScript agree on every one of 70,000 generated documents.

Seven ports, all on 3.8.0

SYNX also has native parsers for Go, Java, .NET, Dart, C++, Swift and Godot, and they had been falling a few releases behind. All of them are on 3.8.0 now, checked the same way against the Rust core on thousands of generated documents each, with nothing left that differs. Some changed a lot more than others: the C++ and Swift ports needed 88 and 47 seconds for a 1.2 MB active document and now need 33 and 95 milliseconds, and the Godot addon didn't even compile on Godot 4.7 before this.

Updating

Your .synx files don't need to change. The packages are @aperturesyndicate/synx-format on npm, synx-core on crates.io and synx-format on PyPI, and the ports come through their usual registries. If you use synx-core from Rust directly, objects are now an ordered IndexMap instead of a HashMap, so code that builds its own HashMap and wraps it needs a .collect(), while everything that reads through get and iteration compiles as before.

There's one thing the format still can't express: a list inside a list item, followed by another item, doesn't survive writing and reading back, because SYNX keeps one open list at a time. That one is planned for 4.0, together with compact strings, which in a lab copy of the core made parsing another 1.2 to 1.5 times faster.

Every change, with the numbers, is in the changelog and in the 3.8.0 release on GitHub.