Latest Results
VarBin: build views straight into a VarBinViewBuilder
Appending a `VarBin` array to a `VarBinViewBuilder` canonicalized it first, which
built the views, wrapped them in a `VarBinViewArray` the builder immediately
unwrapped, and then had `append_varbinview_array` rewrite every view to rebase
its buffer index onto the builder's.
Number the buffer up front instead: `varbin_decode_views` takes the index the
pushed buffer will land at, so the views come out already correct and the append
is one view per row plus a buffer push. `varbin_to_canonical` now shares that
helper, which also shrinks what `match_each_integer_ptype!` duplicates.
A builder configured to compact still goes the canonical route — it chooses per
buffer whether to keep, slice or rewrite it by measuring the finished views, and
`push_buffer_and_adjusted_views` would bypass that.
chunk_array_builder, fastest of 100 samples, 3 runs each:
| benchmark (rows x chunks) | before | after | speedup |
| ------------------------------------- | ------- | ------- | ------- |
| varbin_to_varbinview_builder 10x1000 | 313 µs | 204 µs | 1.53x |
| varbin_to_varbinview_builder 100x100 | 87.3 µs | 69.5 µs | 1.26x |
| varbin_to_varbinview_builder 1000x10 | 58.6 µs | 54.8 µs | 1.07x |
| varbin_opt_to_varbinview_bldr 10x1000 | 378 µs | 266 µs | 1.42x |
| varbin_into_canonical 10x1000 | 357 µs | 253 µs | 1.41x |
The last row is the control: it never touches the builder, so its gain is from
the shared-helper extraction alone. Relative to it, the append itself drops from
0.88x to 0.81x of a canonicalization.
Signed-off-by: Robert Kruszewski <github@robertk.io>rk/varbin-to-varbinview-builder perf(sparse): build the fixed-size-list fill tile once, not once per gap
`append_fill` handed the fill scalar to `Constant::append_to_builder` per gap,
which materialized the scalar's elements into an array every time. Hoist that
out of the loop the way the list path already does: the fill's elements become
an array once, up front, and every gap tiles that same array through
`FixedSizeListBuilder::append_array_as_repeated_list`.
A fixed-size list holds its elements back to back, so it cannot point a run of
rows at one shared range the way a list view can - but it need not copy the
tile per row either. The run goes in as a `ChunkedArray` of `n` clones of the
tile, which costs `n` reference bumps and no element data at all. The child
keeps that whole run as a single chunk: unpacking it would spill a chunk per
row into the child's chunk list, which every later append and the final
`finish` walk, and that list is what makes the difference - unpacking measured
36.0 us against 26.0 us on `(512, 7, 4)`.
Elements that are all the same scalar do better still, collapsing to a single
constant chunk however many rows they cover.
`canonicalize_sparse_fixed_size_list` medians, against develop:
(512, 7, 4) 21.1 us -> 26.0 us (was 79.0 us)
(1024, 17, 8) 29.1 us -> 22.7 us (was 139.5 us)
(8192, 1024, 4) 150.1 us -> 34.5 us (was 375.9 us)
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>claude/constant-fast-paths-9ze0t6 perf(sparse): build the fixed-size-list fill tile once, not once per gap
`append_fill` handed the fill scalar to `Constant::append_to_builder` per gap,
which materialized the scalar's elements into an array every time. Hoist that
out of the loop the way the list path already does: the fill's elements become
an array once, up front, and every gap tiles that same array through
`FixedSizeListBuilder::append_array_as_repeated_list`.
Elements that are all the same scalar stay a constant array, so tiling them
over a gap costs nothing at all however many rows it covers. Anything else is
copied in per row, because a fixed-size list holds its elements back to back
and cannot point its rows at one shared copy.
`canonicalize_sparse_fixed_size_list` medians, against develop:
(512, 7, 4) 21.1 us -> 25.6 us (was 55.0 us)
(1024, 17, 8) 29.1 us -> 32.9 us (was 63.9 us)
(8192, 1024, 4) 150.1 us -> 140.1 us (was 142.3 us)
What is left of the gap against develop is the per-row copy: each row pays its
own append for `list_size` elements, where tiling a run could double the region
already written instead.
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>claude/constant-fast-paths-9ze0t6 Store i128 assembly into a pre-sized buffer, and dedupe part-wise kernels
Re-running `benches/decimal_assemble.rs` after the review corrected a
claim the previous commit made. Specializing the part count is worth
1.85x on `i256`, as reported, but on `i128` it is worth only ~1.04x — the
1.25x figure did not reproduce. What actually costs on `i128` is the
write: pushing into a reserved buffer instead of storing into a pre-sized
one is the whole difference at 16 bytes per row.
A new `i128_row_write` variant isolates it, holding the loop shape fixed
and changing only the output buffer. Over 65,536 rows, `fastest` of three
runs each:
| shape | i128 | i256 |
| ---------------------------- | ------- | ------- |
| row, runtime part count | 143 µs | 351 µs |
| row, const part count, push | 138 µs | 190 µs |
| row, const part count, write | 83 µs | 196 µs |
| column, lane writes | 103 µs | 438 µs |
So the columnar shape was never the interesting axis: it beats the
*pushing* row loop on `i128` but still loses to the single-pass write,
and the second pass buys nothing once the push is gone. On `i256` the
write shape ties the push shape, because 32 bytes of stores per row
dominate either way, so only `assemble_i128` changes. Through the array
API, one lower part goes 138 µs -> 83 µs (1.6x); three parts is unchanged
at ~209 µs.
The rest is cleanup from the same review.
Seven kernels open-coded "map every part, rebuild the array", and two of
them had already been fixed in this branch for dropping the lower parts
on the floor. `map_parts`, `with_msp` and `decimal_dtype` replace all
seven, so a part-wise op cannot silently lose a part, and the argument
for why an MSP-only rebuild is sound lives in one doc comment instead of
being restated or omitted per site.
Dead code: `DecimalBytePartsDataParts` had exactly one reference in the
repository — its own definition — and this branch had been growing it a
field and doc comments. The `[first]` arm of the `i256` dispatch is
unreachable, since one lower part under a <=64-bit MSP always lands in an
`i128`; a comment now says so where the arm was.
Visibility: `assemble_decimal`, `assembled_values_type` and
`LOWER_PART_DTYPE` had no callers outside the crate and are now
crate-private. `assemble_decimal` was public only so the benchmark could
call it, but `canonicalize_byte_parts` already measures the same assembly
through the array API, so the two `*_assemble_shipped` benches go with
it. As public API it could also panic rather than error on an unsigned
MSP, since signedness is only checked on the zero-parts path.
The metadata accessor `lower_parts()` returned a count while the
generated slots accessor of the same name returns the arrays, both in
scope in the same module; it is now `lower_part_count()`. The btrblocks
scheme spelled the child layout as `1 + MAX_LOWER_PARTS` and `idx + 1`
where the encoding crate has named slot constants; it now uses them.
Three hand-rolled LCGs become `StdRng::seed_from_u64`, matching the rest
of the repo. Four one-line rejection tests became one `rstest` in the
previous commit; the two removed columnar bench variants are recorded in
the module doc with their numbers rather than kept as dead code.
Signed-off-by: "Joe Isaacs" <joe.isaacs@live.co.uk>claude/decimal-byte-parts-pr-p0ugog Latest Branches
-48%
rk/varbin-to-varbinview-builder 0%
rk/onpair-decode-into-builder -32%
claude/chunked-canonical-via-builder-9ze0t6 © 2026 CodSpeed Technology