Latest Results
Store i128 assembly into a pre-sized buffer, and dedupe part-wise kernels
Re-running `benches/decimal_assemble.rs` after the review corrected a
claim the previous commit made. Specializing the part count is worth
1.85x on `i256`, as reported, but on `i128` it is worth only ~1.04x — the
1.25x figure did not reproduce. What actually costs on `i128` is the
write: pushing into a reserved buffer instead of storing into a pre-sized
one is the whole difference at 16 bytes per row.
A new `i128_row_write` variant isolates it, holding the loop shape fixed
and changing only the output buffer. Over 65,536 rows, `fastest` of three
runs each:
| shape | i128 | i256 |
| ---------------------------- | ------- | ------- |
| row, runtime part count | 143 µs | 351 µs |
| row, const part count, push | 138 µs | 190 µs |
| row, const part count, write | 83 µs | 196 µs |
| column, lane writes | 103 µs | 438 µs |
So the columnar shape was never the interesting axis: it beats the
*pushing* row loop on `i128` but still loses to the single-pass write,
and the second pass buys nothing once the push is gone. On `i256` the
write shape ties the push shape, because 32 bytes of stores per row
dominate either way, so only `assemble_i128` changes. Through the array
API, one lower part goes 138 µs -> 83 µs (1.6x); three parts is unchanged
at ~209 µs.
The rest is cleanup from the same review.
Seven kernels open-coded "map every part, rebuild the array", and two of
them had already been fixed in this branch for dropping the lower parts
on the floor. `map_parts`, `with_msp` and `decimal_dtype` replace all
seven, so a part-wise op cannot silently lose a part, and the argument
for why an MSP-only rebuild is sound lives in one doc comment instead of
being restated or omitted per site.
Dead code: `DecimalBytePartsDataParts` had exactly one reference in the
repository — its own definition — and this branch had been growing it a
field and doc comments. The `[first]` arm of the `i256` dispatch is
unreachable, since one lower part under a <=64-bit MSP always lands in an
`i128`; a comment now says so where the arm was.
Visibility: `assemble_decimal`, `assembled_values_type` and
`LOWER_PART_DTYPE` had no callers outside the crate and are now
crate-private. `assemble_decimal` was public only so the benchmark could
call it, but `canonicalize_byte_parts` already measures the same assembly
through the array API, so the two `*_assemble_shipped` benches go with
it. As public API it could also panic rather than error on an unsigned
MSP, since signedness is only checked on the zero-parts path.
The metadata accessor `lower_parts()` returned a count while the
generated slots accessor of the same name returns the arrays, both in
scope in the same module; it is now `lower_part_count()`. The btrblocks
scheme spelled the child layout as `1 + MAX_LOWER_PARTS` and `idx + 1`
where the encoding crate has named slot constants; it now uses them.
Three hand-rolled LCGs become `StdRng::seed_from_u64`, matching the rest
of the repo. Four one-line rejection tests became one `rstest` in the
previous commit; the two removed columnar bench variants are recorded in
the module doc with their numbers rather than kept as dead code.
Signed-off-by: "Joe Isaacs" <joe.isaacs@live.co.uk>claude/decimal-byte-parts-pr-p0ugog perf(sparse): build the fixed-size-list fill tile once, not once per gap
`append_fill` handed the fill scalar to `Constant::append_to_builder` per gap,
which materialized the scalar's elements into an array every time. Hoist that
out of the loop the way the list path already does: the fill's elements become
an array once, up front, and every gap tiles that same array through
`FixedSizeListBuilder::append_array_as_repeated_list`.
Elements that are all the same scalar stay a constant array, so tiling them
over a gap costs nothing at all however many rows it covers. Anything else is
copied in per row, because a fixed-size list holds its elements back to back
and cannot point its rows at one shared copy.
`canonicalize_sparse_fixed_size_list` medians, against develop:
(512, 7, 4) 21.1 us -> 25.6 us (was 55.0 us)
(1024, 17, 8) 29.1 us -> 32.9 us (was 63.9 us)
(8192, 1024, 4) 150.1 us -> 140.1 us (was 142.3 us)
What is left of the gap against develop is the per-row copy: each row pays its
own append for `list_size` elements, where tiling a run could double the region
already written instead.
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>claude/constant-fast-paths-9ze0t6 perf(array): accumulate nested builder validity without a null buffer
A nested builder learns about validity from two sources: one row at a time
as scalars are appended, and a whole array's worth at a time as arrays are.
Only the first needs a null buffer, but `LazyBitBufferBuilder` treated both
the same, so every appended array had its validity executed into a `Mask`
and its bits copied.
`ValidityBuilder` keeps a whole array's validity as a run and concatenates
the runs at the end, the way `Validity::concat` already does for
`StructArray::try_concat`. `AllValid` and `AllInvalid` runs cost nothing,
array-backed runs are bool arrays that are already built, and a builder that
only ever saw uniform validity still answers from its nullability rather
than producing a bool array. However few values a run covers, it is kept as
it arrived, so a builder's validity is split on exactly the boundaries its
children are.
`StructBuilder`, `ListBuilder`, `ListViewBuilder` and `FixedSizeListBuilder`
use it; the leaf builders keep `LazyBitBufferBuilder`.
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>claude/builders-lazy-validity-9ze0t6 feat(array): builders no longer canonicalize their children
Nested builders used to push every appended child array through
`append_to_builder`, which decoded it into the child's canonical builder.
That work is wasted: `Canonical` only promises a canonical *top level*, so
struct fields, list elements and extension storage are free to stay
compressed.
Introduce `ChildBuilder`, which accumulates a child as a `Vec<ArrayRef>` of
chunks plus a scalar builder for the values that cannot come from an array,
and stitches them into a `ChunkedArray` on `finish` when more than one chunk
accumulated. `StructBuilder`, `ListBuilder`, `ListViewBuilder`,
`FixedSizeListBuilder` and `ExtensionBuilder` now hold their children this
way.
However short the appended array, it becomes a chunk. Deciding on the caller's
behalf that its values are cheaper copied than referenced would be guessing at
a boundary only the caller can see, and a caller that wants them copied has
`append_scalar`. A child is therefore chunked on exactly the boundaries it was
appended on.
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>claude/builders-canonical-children-9ze0t6 perf(array): append repeated and viewed lists in bulk, not a value at a time
Four callers walked a builder one list at a time where a whole run of them was
available up front.
`Sparse` canonicalization filled the gaps between patches by appending the fill
value once per row, and appended the patches themselves one list at a time. A
gap is a run of one value, so it goes in as a single `ConstantArray`:
canonicalizing a constant list array points every view at one copy of the
value, so a gap now costs the fill value's elements once however many rows it
covers. Appending a 10,000-row gap of a three-element fill produced 30,000
elements; it now produces 3. Patches landing on consecutive rows go in as one
slice of the patch array, so the builder sees an append per gap rather than one
per patch.
The patch values are flattened once up front instead. That is what lets a run
be sliced out of them, and it also means a null patch carries a zero-size view
rather than the elements the old code skipped, so the appended run holds
exactly the elements the patches reference.
`ListBuilder::append_listview_array` sliced the elements array and appended the
slice once per list, which is what its `ListViewBuilder` twin stopped doing.
`ListArray` offsets can only describe contiguous, in-order lists, so flatten
the incoming views to that layout - a no-op when they are laid out that way
already - and then append the referenced elements in one go, walking only the
metadata to rebase the offsets.
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>claude/builders-bulk-loop-callers-9ze0t6 Latest Branches
-11%
claude/decimal-byte-parts-pr-p0ugog -31%
claude/chunked-canonical-via-builder-9ze0t6 ×2.8
claude/constant-fast-paths-9ze0t6 © 2026 CodSpeed Technology