A length prefix tells a parser where a region ends. It does not prove that the value inside used every byte.
That distinction caused a quiet canonical-encoding problem in snarkVM. A decoder could accept a valid nested value, stop when that value was complete, and ignore bytes still sitting inside its declared region. The parsed object looked fine. Its wire representation was not.
PR #3409 closes that gap for length-delimited sub-objects in Future, Plaintext, and Record. The change is small: after a nested decoder returns, the enclosing decoder verifies that the temporary slice is empty. Any remainder makes the whole object invalid.
Small patch. Good teeth.
A frame is not a proof
Suppose a nested value has one canonical encoding, C, whose length is four bytes. Its enclosing field uses a little-endian u16 length:
04 00 | C0 C1 C2 C3
A permissive decoder can also accept this:
05 00 | C0 C1 C2 C3 AA
The outer parser reads the length 5, copies five bytes into a bounded region, and invokes the nested parser. The nested parser consumes the first four bytes and returns a valid value. If the caller never inspects the remaining AA, decoding succeeds.
Both byte strings now produce the same semantic value:
Decode(04 00 || C) = V
Decode(05 00 || C || AA) = V
Serialization goes the other way:
Encode(V) = 04 00 || C
The second accepted input changes when decoded and encoded again. In algebraic form, the codec has violated the property that matters here:
Encode(Decode(bytes)) = bytes
A round trip does not always need to preserve bytes. Text formats often normalize whitespace, for example. Cryptographic binary formats need a much stronger reason before permitting aliases, because raw bytes routinely become hash inputs, database keys, signed messages, or consensus payloads.
The extra byte need not contain a second valid object. It can be zero. It can be a fragment of another encoding. The nested parser has already returned, so the byte has no meaning under the grammar.
Calling it padding would not fix the issue. Padding must be specified: where it is allowed, how long it may be, which byte values are legal, and whether the encoder must emit it. snarkVM's regions did not define such padding. Unconsumed data was garbage that happened to survive parsing.
Boundaries and canonical bytes
Length-delimited parsing solves two useful problems. First, the outer decoder can isolate a nested object without trusting that object to find the next sibling. Second, a malformed nested length cannot make that decoder read indefinitely through the enclosing structure.
Neither property demands complete consumption.
Think of the length as a fence around the parser. The fence prevents escape. It does not require the parser to inspect the whole yard.
Canonical encoding adds a separate rule: every accepted byte string must be the encoder's chosen representation of the resulting value. A practical decoder can enforce that rule in either of two ways:
- Decode the region, then require an empty remainder.
- Decode, re-encode, and compare with the original region.
Exact consumption is cheaper and clearer when each nested decoder is already responsible for canonical field encodings. Re-encoding can catch a wider class of normalization, but it allocates more work and may hide which layer accepted the bad representation.
PR #3409 takes the right local approach. The length-delimited caller owns the region, so that caller must check the remainder. A nested read_le_internal function cannot know whether it was asked to parse one complete region or the first value in a stream containing several values.
Rust APIs make that distinction easy to miss. A decoder taking Read is often designed to consume one object and leave the reader positioned after it. That behavior is useful for sequential streams. Once a caller has carved out a slice declared to contain exactly one object, however, leaving bytes behind is an error.
Bincode exposes the same design choice. Slice-based decoding can reject trailing bytes because the decoder knows the input boundary. Reader-based decoding may intentionally leave data for the next object. The nom parser library is similarly explicit: ordinary parsers return the unconsumed input, while all_consuming turns any remainder into failure.
The format layer must choose. Silence is a choice too, and usually the wrong one near cryptographic data.
Where snarkVM went loose
The vulnerable pattern in PR #3409 looked like this:
let num_bytes = u16::read_le(&mut reader)?;
let mut bytes = Vec::new();
(&mut reader)
.take(num_bytes as u64)
.read_to_end(&mut bytes)?;
let entry = Argument::read_le_internal(
&mut bytes.as_slice(),
depth,
)?;
The take call bounds the read to num_bytes. Good. The temporary vector isolates the nested region. Also good.
Then the code creates a temporary slice and hands it directly to read_le_internal. The nested parser advances that temporary reader, returns entry, and drops the reader. Any remainder disappears with it.
The strict shape keeps the slice alive long enough to inspect it:
let num_bytes = u16::read_le(&mut reader)?;
let mut bytes = Vec::new();
(&mut reader)
.take(num_bytes as u64)
.read_to_end(&mut bytes)?;
let mut region = bytes.as_slice();
let entry = Argument::read_le_internal(
&mut region,
depth,
)?;
if !region.is_empty() {
return Err(invalid_data());
}
That snippet is schematic, but the ownership rule is the point. The caller retains the cursor and checks what the child left behind.
Future, Plaintext, and Record use this pattern for different nested objects. A future contains encoded arguments. Structured plaintext contains child values. A record contains entries with visibility and value information. In each case, the surrounding format declares a u16-sized region and expects one nested object inside it.
Recursive depth checks do another job. They stop a hostile value from nesting indefinitely or blowing the call stack. A depth-safe parser can still accept trailing garbage, while an exact parser can still recurse too deeply. Both checks belong in the decoder because they block different input classes.
One subtle point matters for reviewers: checking the outer reader after the whole Future or Record is not enough. The unwanted byte sits inside a valid declared field length. From the outer parser's view, the field consumed exactly five bytes. Only the parser that owns the five-byte sub-slice can see that its child used four.
That is why exact consumption must be enforced at every length-delimited nesting boundary, not merely at the top-level API.
How aliases escape
Malleability here means an attacker can alter accepted bytes without altering the decoded value. No proof system has been broken. No private key falls out of a parser. The risk comes from two components assigning identity at different representation layers.
Consider a gateway that receives raw bytes and computes an object ID:
raw_id = H(received_bytes)
A worker decodes the object, serializes it canonically, and computes another ID:
value_id = H(Encode(Decode(received_bytes)))
For a canonical input, the IDs agree. For an accepted alias with a trailing byte, they usually differ:
H(05 00 || C || AA) != H(04 00 || C)
Now the same semantic object has two identities. That can produce duplicate cache entries, failed deduplication, mismatched database indexes, or disagreement about which bytes a signature covered.
Consensus risk needs careful wording. A permissive parser does not automatically create a chain split. Every validator may run the same implementation and hash the same representation at every relevant point. The dangerous condition appears when one path hashes received bytes while another hashes reconstructed bytes, or when implementations disagree about whether the alias is valid.
Strict decoding removes that condition near the source. Auditing every later hash and storage operation is still useful, but relying on global consistency is brittle. New callers appear. SDKs get rewritten. Agents pass values between Rust, WebAssembly, TypeScript, JSON, and binary envelopes.
An agent pipeline makes a concrete example. An agent fetches an encoded record from a service, indexes the original bytes, decodes it through an SDK, then sends the reconstructed object to a signer. The index and signer may now refer to different byte strings even though both display the same record fields. Rejection at the first decoder prevents that history from forking.
The PHP project recently addressed a related behavior in unserialize(): a valid serialized object followed by ignored bytes could succeed. Its proposed rule describes the bug neatly. If bytes can be removed from the end without changing the returned value, the parser should report trailing data.
Aleo has already met this bug family elsewhere. snarkOS PR #4344 tightened BFT transmission decoding so payloads could not carry ignored suffixes. snarkVM PR #3397 rejected compressed infinity points with a nonzero coordinate payload. Different data types, same rule: the decoder should not accept more representations than the serializer emits.
The recent review of hard limits on untrusted edges covered numeric bounds such as capped allocations. Exact consumption is a semantic bound. It limits an encoded value to bytes that its grammar can explain.
Testing the fixed contract
A happy-path round trip will not catch this bug. The normal serializer never emits the unwanted suffix, so ordinary generated values stay canonical.
Regression tests need to construct malformed encodings deliberately:
- Serialize a valid nested value, increase its enclosing
u16length by one, append one byte inside the region, and require decoding to fail. - Repeat the mutation for
Future, structuredPlaintext, andRecord, including a nested value below the first recursion level. - Try suffixes such as
00,FF, a valid type tag, and the prefix of another value. Every one must fail for the same reason. - Preserve positive tests for empty containers, maximum permitted legal values, ordinary nested objects, and byte-for-byte round trips.
Property testing can state the canonical contract directly:
For every byte string b accepted by Decode:
Encode(Decode(b)) == b
Fuzzers should mutate delimiters and payloads together. Appending a byte without increasing the inner length only tests top-level trailing-data handling. The interesting case increments the length so the garbage remains inside a field that the outer parser believes it consumed correctly.
Cross-implementation tests matter too. Feed the same corpus to native snarkVM, SDK bindings, indexer codecs, and any independent implementation. Each decoder should accept the canonical corpus and reject every suffixed variant. Matching decoded JSON is not enough; acceptance itself must match.
Developers building on high-level Leo normally will not touch read_le_internal. The rule still reaches their software. Wallet imports, record scanners, transaction relays, proving services, and agent tools all cross serialization boundaries. If an API accepts bytes, parse one value and demand the end of its declared region.
Rejecting aliases may be less friendly than silently normalizing them. Good. Cryptographic parsers are the wrong place for guesswork. A producer emitting surplus bytes has a broken encoder, and an immediate error is far easier to diagnose than a hash mismatch several services later.
PR #3409 puts the check where the evidence still exists: beside the cursor, before the remaining bytes are dropped.