Compact RowBinary array and map decoding stacks

[ruslandoga]

Summary

Compact RowBinary array and map decoding so collection length no longer expands the type stack before values are read.
Fixed-width primitive arrays use generated counter loops sourced from the existing scalar decoder table, while parameterized arrays and nested collections use compact accumulator continuations.
Maps carry key/value types, remaining entries, and accumulated values in one frame instead of appending two type entries per map item.
Validated with formatting, 479 tests including focused byte-by-byte streaming cases, and comparative Benchee runs against current master.

[ruslandoga]

Updated final design and benchmark notes after testing the broader RowBinary redesigns.

Root cause

The old decoder materialized parser work from the collection length before reading collection values:

• Array(T) built List.duplicate(T, size) and appended it to the type stack.
• Map(K, V) recursively built [K, V, K, V, ...] with 2 * size entries.

That made both eager decoding and streaming continuation state proportional to the declared collection length. A length prefix for one million values allocated a large type list even when no value bytes had arrived.

Final implementation

Arrays now carry remaining and acc as recursive function arguments. Fixed-width primitive loops are generated from the existing simple_types decoder metadata; the same entry owns byte width, binary pattern, and decoded value, so there is no UInt8-only path or second fixed-array type catalog. Adding a primitive scalar decoder automatically supplies its array decoder.

When all fixed-width bytes are available, the loop slices that exact chunk and decodes it without constructing parser operations. When a chunk ends partway through an array, the continuation stores one {:array_items, type, remaining, acc, original_row} entry. Variable-width and parameterized scalar arrays use the same counter loop through decode_one/2; complex nested elements use one compact array_acc structural frame.

Maps now keep one map_acc frame containing key type, value type, remaining pair count, and the parent row. Completed key/value values are accumulated in wire order and folded directly into a map, without building a 2 * size type list or chunking/reversing temporary pairs.

This follows the useful parts of both references:

• msgpax carries collection progress as index/count arguments and keeps only parent collection state in outer.
• Erlang json.erl stores structural array/object frames and resumes into the current parser function instead of expanding future parser operations.

RowBinary has one important difference: the schema type is already known and constant for every array element. Keeping generated primitive loops lets BEAM select that parser once and avoids paying runtime type dispatch for every value.

Rejected prototypes

I implemented and measured a full reusable type-cycle state machine for rows, arrays, maps, and tuples, with index/count arguments and only structural outer frames. It was correct across all byte-by-byte tests, but on Apple M2 / Elixir 1.20.2 / OTP 29 it regressed Array(UInt8) x 100_000 from 1.77 ms / 5.70 MB on master to 15.04 ms / 12.31 MB, and Map(String, UInt8) x 10_000 from 6.53 ms / 4.14 MB to 11.41 ms / 5.05 MB.

I also tested one runtime-generic decode_one/2 collection loop and a direct runtime type case. Both paid type dispatch or large generated decision trees on every element; observed Array(UInt8) x 100_000 times were roughly 5-10 ms, so neither was retained. The final source-level generator is generic across primitive types while preserving one selected hot loop at runtime.

Final comparative benchmark

Benchee configuration: Apple M2, Elixir 1.20.2, OTP 29.0.3 JIT, clean dev compilation, 2 s warmup, 5 s run, and 2 s memory measurement. master is current origin/master (eafa097); the PR is rebased on that commit. The table uses medians because scheduler outliers made averages materially noisier.

Scenario	master median / memory	PR median / memory	Result
Array(UInt8) x 10_000	179.38 us / 376.67 KB	62.92 us / 170.80 KB	2.85x faster, 55% less memory
Array(UInt8) x 100_000	1.74 ms / 5.70 MB	0.63 ms / 2.65 MB	2.76x faster, 54% less memory
Map(String, UInt8) x 10_000	3.72 ms / 4.14 MB	2.59 ms / 1.96 MB	1.44x faster, 53% less memory

The 10k result is a complete-payload benchmark: every run decodes all 10,000 element bytes and returns the materialized array. It is not the length-prefix-only continuation case. A short diagnostic run once made one PR build appear slower than an equivalent PR build; their executed hot path was identical and the run had high timing variance. The clean, longer comparison above is the relevant result. Absolute timing still varies with scheduler load, while the measured allocation difference was stable across runs.

The wider 100k primitive-array run on the final code measured 591.90 us / 2.46 MB for UInt64, 608.92 us / 2.46 MB for Int64, 1.69 ms / 3.93 MB for Float64, and 6.38 ms / 14.32 MB for Date.

Length-prefix-only continuation allocation remains constant-size: 224 bytes for arrays and 288 bytes for maps in the earlier one-million-element run, rather than growing with the declared length. The nanosecond timing in that microbenchmark is noisy; allocation shape is the meaningful result.

Validation

• MIX_ENV=test mix format --check-formatted
• mix test test/ch/row_binary_test.exs: 67 passed
• mix test --exclude dynamic: 472 passed, including 171 doctests and 40 properties; 7 Dynamic tests excluded
• RowBinary line coverage in that non-Dynamic run: 96.7% (up from 92.7% before the added continuation-path tests)
• mix test: 479 passed, including 171 doctests and 40 properties