Why Signed JSON Breaks and How a Canonical Encoding Fixes It
Here’s a bug that shows up in almost every system that signs or hashes structured data. Service A builds a record, serializes it as JSON, hashes it and signs the hash. Service B receives the record, parses it, does some work, serializes it again and checks the signature. It fails. Nobody changed a single field.
What changed was the bytes. Service B’s JSON library wrote the keys in a different order. Or it wrote 1.0 as 1. Or it escaped é as é where A wrote the character itself. All three are valid JSON for the same data. A hash doesn’t care about data, though. It cares about bytes, and there are many ways to write the same JSON.
The same thing bites caches keyed by a hash of a request, deduplication that compares records by hash, and audit logs that chain each entry to the hash of the last one. Anywhere “same data” has to mean “same hash”, a format with several spellings is a trap.
The usual fixes, and where they leak
The common answer is canonicalization: before hashing, rewrite the JSON by a fixed set of rules. Sort the keys, strip whitespace, settle on one way to write numbers and strings. There’s a standard for it, RFC 8785, and libraries in most languages.
It works, as long as every implementation gets every rule exactly right. That’s the leak. Number formatting alone is subtle, since the shortest text that reads back as the same float is hard to produce, and two libraries that each pass their own tests can still disagree on a value nobody thought to try. Canonicalization is also a step you have to remember. Forget it once, in one service, and you’re back to two spellings.
Binary formats have the same problem in a different shape. Protocol Buffers make no promise that two encoders produce the same bytes. CBOR has a deterministic mode, but a decoder in normal mode happily accepts the other encodings too, so a non-canonical message can travel through a system unnoticed until it’s hashed.
What changes when the format itself has one spelling
BXM Canon makes the canonical form the only form. Map keys are always written in byte order. Numbers always use the shortest length prefix. Integers and floats are separate types, so 1 and 1.0 are different values with different bytes, and nothing quietly converts one into the other. NaN, infinity and negative zero aren’t values at all, because each one invites a second opinion.
Then the decoder enforces it. A map with keys out of order isn’t “a map, reordered on the way in”. It’s an error, with the byte offset where it went wrong. The same goes for a number padded to a longer length, or a key that appears twice. So the encoder can’t produce two spellings, and the decoder won’t accept two spellings. That’s the property that matters: if bytes decode at all, encoding the result gives back exactly those bytes.
The practical effect is that canonicalization stops being a step. You don’t rewrite before hashing. The bytes you have are already the only bytes the value can have, and checking a file is canonical is a quick scan that builds nothing in memory.
One implementation instead of many
The other half of the problem is cross-language agreement. Canon is written once, in Rust, and offered to other languages through a C API and, from version 0.2.0, as a JavaScript package compiled from the same core to WebAssembly, with Python next. When the Node service and the Go service call the same code, they can’t disagree about a corner case.
For anyone who does write another implementation, the spec ships with conformance vectors: 30 values with their exact bytes and 30 byte strings that must be rejected, each labelled with the reason. Passing both files is the bar.
When you don’t need it
If your data is only read by people, or only ever parsed and never hashed or compared byte for byte, plain JSON is fine. Canon keeps JSON as its text form for exactly that reason. You can write Canon as JSON, read it back as JSON, and reach for the bytes only when the bytes matter.