Latest Results
test: trace reductions and executions of scan ops over complex and compressed arrays (#8828)
Stacked on #7814 (the `trace_op` execution/optimization tracing
harness).
Builds out trace-test coverage for how take/filter/compare/like reduce
and execute over complex arrays, using the harness from the base PR:
**vortex-array** (promotes `test_harness/trace.rs` to a directory module
`trace/mod.rs` + `trace/tests.rs`; harness source unchanged):
- take (`DictArray`) over a chunked array — no reduce rule fires;
execution canonicalizes the chunked values chunk-by-chunk before the
dict take kernel runs.
- filter over a struct with complex children (dict-of-strings + chunked
fields) — `FilterStructRule` pushes the filter into each field,
`FilterReduceAdaptor(Dict)` rewrites the dict field at optimize time.
- compare (`Binary` Eq vs constant) over a dict —
`DictionaryScalarFnValuesPushDownRule` reduces the compare into the
dictionary values.
- like over a dict-of-strings — `LikeReduceAdaptor(Dict)` reduces into
the dictionary values.
**vortex-runend** (new `trace_tests.rs`, `insta` added as
dev-dependency):
- compare vs constant reduces via `RunEndScalarFnRule` at optimize time;
filter and take execute via the
`FilterExecuteAdaptor`/`TakeExecuteAdaptor` parent kernels.
**vortex-btrblocks** (new `trace_tests.rs`; `insta`, `tpchgen`,
`tpchgen-arrow`, `arrow-array` added as dev-dependencies): generates
TPC-H lineitem (SF 0.001, 4096 rows, deterministic — tpchgen and the
compressor's sampling seed are both fixed), compresses it with
`BtrBlocksCompressor`, and traces TPC-H-style scan predicates over the
resulting encodings:
- `l_shipdate >= const` over `ext(date) → for → bitpacked` — extension
compare kernel at execution time, with
`CastReduceAdaptor(FoR)/(BitPacked)` reductions inside the arrow
fallback.
- `l_quantity < const` over `decimal_byte_parts → dict → bitpacked` —
the decimal_byte_parts compare kernel pushes into the byte-parts
dictionary, where `DictionaryScalarFnValuesPushDownRule` reduces the
compare to the 50 dictionary values before decode.
- `l_shipmode = const` over dict-of-FSST — dict pushdown at optimize
time, FSST compare kernel + dict decode at execution time.
- `l_comment LIKE` over FSST — the FSST like kernel matches in
compressed space.
- filter over a struct of compressed columns — `FilterStructRule` plus
per-encoding pushdown via `DecimalBytePartsFilterPushDownRule`,
`ExtensionFilterPushDownRule`, `FoRFilterPushDownRule`, and
`FilterReduceAdaptor(Dict)`, leaving execution a no-op.
- take over a struct of compressed columns — `TakeReduceAdaptor(Struct)`
absorbs the take entirely at optimize time.
Each btrblocks test also asserts the canonical result matches running
the same operation over the uncompressed column. A 1MiB TPC-H text pool
replaces the spec-default 300MiB one, cutting per-test setup from ~9s to
~0.5s without changing the traces.
This PR is tests-only; no production code changes.
- `cargo nextest run -p vortex-array` — 2961 passed (includes the 8
trace tests, run in parallel to confirm the thread-local recorder is
deterministic).
- `cargo nextest run -p vortex-runend` — 61 passed.
- `cargo nextest run -p vortex-btrblocks` — 43 passed.
- `cargo clippy --all-targets` on the three touched crates — clean.
- `cargo +nightly fmt --all` and `git diff --check` — clean.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01RMBcCiwpC3dbGLXRbK7UuR
---
_Generated by [Claude
Code](https://claude.ai/code/session_01RMBcCiwpC3dbGLXRbK7UuR)_
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Latest Branches
0%
0%
0%
rk/onpair-decode-into-builder © 2026 CodSpeed Technology