Avatar for the vortex-data user
vortex-data
vortex
BlogDocsChangelog

Performance History

Latest Results

feat(vortex-buffer): arch-specialized SIMD bool packing for collect_bool Replace the bit-at-a-time shift-OR loop behind collect_bool_word / collect_bool_words with byte-to-bit packing kernels specialized per architecture, without changing any public signature: - x86-64 AVX-512BW: one vptestmb packs 64 bools into a u64 - x86-64 AVX2: two vpcmpeqb + vpmovmskb halves - x86-64 SSE2 (baseline): four pcmpeqb + pmovmskb quarters - aarch64 NEON (baseline): ushl by bit position + addp reduction tree - other targets and Miri: branch-free SWAR multiply fallback The API has two tiers. The default (collect_bool / collect_bool_words, backed by collect_bool_words_inline) compiles the word loop once with the widest kernel enabled at compile time (SSE2/NEON on stock targets; AVX2/AVX-512 under -C target-cpu=native) so the predicate, the [bool; 64] materialization, and the pack all inline into the caller. The opt-in tier (collect_bool_multiversioned / collect_bool_words_multiversioned) compiles the loop with the predicate inside once per CPU feature level and selects a clone by runtime feature detection; its docs state the caller's assertion โ€” the predicate must be trivially cheap, because the #[target_feature] call boundary deoptimizes non-trivial predicates (an early all-multiversioned draft regressed FSST fsst_prefix 13-63% this way; the pack kernel choice itself moves that DFA-bound workload by at most ~4%). Route the call sites whose predicates are bounds-check-free gathers or comparisons through the multiversioned tier: From<&[bool]> / From<&[u8]>, ByteBool boolean decoding, and the primitive and decimal between kernels (now reading via get_unchecked, sound because the predicate only receives indices 0..len โ€” a contract now documented on every entry point together with the bounds-check performance note). Measured on AVX-512 hardware (divan, 64Ki bits, medians): the fused word loop with a bool-gather predicate runs 14-24x faster than the old scalar loop per SIMD level (SSE2 1.7us, AVX2 1.2us, AVX-512 1.1us vs 26.5us); BitBufferMut::from(&[bool]) improves ~43x; a between-shaped predicate doubles under the multiversioned tier (5.5us vs 11.2us); fsst_prefix is unchanged or slightly better than before the change. Generated asm verified per kernel (the fused AVX-512 loop is vmovdqu64 + vptestmb + kmovq per 64 bools; NEON is 14 instructions, its addp tree additionally validated against a reference model on 10,004 random inputs). New benches cover each kernel and both tiers; under CodSpeed only the shipped entry points are tracked (from_bool_slice, collect_bool_u32_gt). Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcsiAHXQc4e11yYcP3a3NX
claude/collect-bool-arch-optimizations-bo09ws
7 minutes ago
feat: add CpuKernel for one-time CPU-feature kernel dispatch Hot kernels re-ran cached CPU-feature checks on every call: each is_x86_feature_detected! is an atomic load + bit test per call, and sites like select_in_chunk chained three of them per 64-byte chunk. aarch64 paths were separate cfg blocks at every call site. Add vortex_buffer::CpuKernel<F>, a LazyLock-like slot for function pointers: the selector passed to CpuKernel::new runs once on the first get(), and every later call is a relaxed atomic load, a never-taken predicted branch, and an indirect call. No macros; the two transmute_copy calls between F and the atomic pointer are contained in the type, whose constructor statically asserts F is pointer-sized. Selectors model both dispatch dimensions in plain code: one cfg(target_arch) block per architecture that returns early (runtime feature probes as an if-chain inside), then the portable default as the plain tail - no cfg(not(any(...))) negation blocks anywhere. An arm that returns unconditionally (aarch64 NEON needs no probe) makes the tail unreachable there, silenced by an allow(unreachable_code) scoped to just the tail block. Kernel types are unsafe fn pointers so #[target_feature] kernels coerce directly by name with a single narrow unsafe call per site. Tiny inputs are gated before the dispatch so they keep the old fully inlined code path: count_ones_aligned calls the scalar kernel directly below 32 bytes (where SIMD never engaged anyway), and scan_chunks calls the per-architecture unconditional kernel directly for two chunks or fewer. Without these gates the dispatch indirection dominates sub-SIMD-threshold workloads: CodSpeed flagged true_count[128] (~-30%) and rank_single[(1024, 0.1)]; cachegrind measured the rank regression at +27 instructions per call and parity after the gates (37.70M vs 37.71M baseline instructions for 100k rank calls). Converted sites: fastlanes transpose_bits/untranspose_bits, vortex-buffer scan_chunks/select_in_chunk/select_in_word/count_ones_aligned, vortex-array filter_bitbuffer_by_mask, and vortex-mask intersect_bit_buffers_dispatch/intersect_rank_indices_dispatch. Intentionally left: primitive take (already one-time via LazyLock dyn kernel), intersect_mask_driven_dispatch (generic over an iterator type, so its instantiations cannot share one fn-pointer slot), and the compile-time IS_CONST_LANE_WIDTH constant. The count_ones any-length wrappers carry #[target_feature] so the SIMD kernels inline into them, avoiding an extra call level, and the vortex_bitbuffer and rank bench mains pre-resolve the kernel slots (matching the existing CPUID pre-warm precedent) so no single-shot benchmark measurement includes first-call selection. benches/cpu_dispatch.rs compares dispatch overhead per call against the same #[inline(never)] kernel. Fastest-of-3 per 1024 calls: direct 2.44us, CpuKernel 2.44us (indistinguishable, confirmed by disassembly: load + fused null test + indirect jmp), LazyLock<fn> 3.04us, and three cached feature checks per call 4.54us. All touched crates also cargo check clean on aarch64-unknown-linux-gnu. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DUFxXZLv4xxEqEYcoGWhfJ
claude/vortex-aarch-pattern-abstraction-s3javb
11 minutes ago
feat(vortex-buffer): arch-specialized SIMD bool packing for collect_bool Replace the bit-at-a-time shift-OR loop behind collect_bool_word / collect_bool_words with byte-to-bit packing kernels specialized per architecture, without changing any public signature: - x86-64 AVX-512BW: one vptestmb packs 64 bools into a u64 - x86-64 AVX2: two vpcmpeqb + vpmovmskb halves - x86-64 SSE2 (baseline): four pcmpeqb + pmovmskb quarters - aarch64 NEON (baseline): ushl by bit position + addp reduction tree - other targets and Miri: branch-free SWAR multiply fallback The API has two tiers. The default (collect_bool / collect_bool_words, backed by collect_bool_words_inline) compiles the word loop once with the widest kernel enabled at compile time (SSE2/NEON on stock targets; AVX2/AVX-512 under -C target-cpu=native) so the predicate, the [bool; 64] materialization, and the pack all inline into the caller. The opt-in tier (collect_bool_multiversioned / collect_bool_words_multiversioned) compiles the loop with the predicate inside once per CPU feature level and selects a clone by runtime feature detection; its docs state the caller's assertion โ€” the predicate must be trivially cheap, because the #[target_feature] call boundary deoptimizes non-trivial predicates (an early all-multiversioned draft regressed FSST fsst_prefix 13-63% this way; the pack kernel choice itself moves that DFA-bound workload by at most ~4%). Route the call sites whose predicates are bounds-check-free gathers or comparisons through the multiversioned tier: From<&[bool]> / From<&[u8]>, ByteBool boolean decoding, and the primitive and decimal between kernels (now reading via get_unchecked, sound because the predicate only receives indices 0..len โ€” a contract now documented on every entry point together with the bounds-check performance note). Measured on AVX-512 hardware (divan, 64Ki bits, medians): the fused word loop with a bool-gather predicate runs 14-24x faster than the old scalar loop per SIMD level (SSE2 1.7us, AVX2 1.2us, AVX-512 1.1us vs 26.5us); BitBufferMut::from(&[bool]) improves ~43x; a between-shaped predicate doubles under the multiversioned tier (5.5us vs 11.2us); fsst_prefix is unchanged or slightly better than before the change. Generated asm verified per kernel (the fused AVX-512 loop is vmovdqu64 + vptestmb + kmovq per 64 bools; NEON is 14 instructions, its addp tree additionally validated against a reference model on 10,004 random inputs). New benches cover each kernel and both tiers; under CodSpeed only the shipped entry points are tracked (from_bool_slice, collect_bool_u32_gt). Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcsiAHXQc4e11yYcP3a3NX
claude/collect-bool-arch-optimizations-bo09ws
55 minutes ago
feat(vortex-buffer): arch-specialized SIMD bool packing for collect_bool Replace the bit-at-a-time shift-OR loop behind collect_bool_word / collect_bool_words with byte-to-bit packing kernels specialized per architecture, without changing any public signature: - x86-64 AVX-512BW: one vptestmb packs 64 bools into a u64 - x86-64 AVX2: two vpcmpeqb + vpmovmskb halves - x86-64 SSE2 (baseline): four pcmpeqb + pmovmskb quarters - aarch64 NEON (baseline): ushl by bit position + addp reduction tree - other targets and Miri: branch-free SWAR multiply fallback The API has two tiers. The default (collect_bool / collect_bool_words, backed by collect_bool_words_inline) compiles the word loop once with the widest kernel enabled at compile time (SSE2/NEON on stock targets; AVX2/AVX-512 under -C target-cpu=native) so the predicate, the [bool; 64] materialization, and the pack all inline into the caller. The opt-in tier (collect_bool_multiversioned / collect_bool_words_multiversioned) compiles the loop with the predicate inside once per CPU feature level and selects a clone by runtime feature detection; its docs state the caller's assertion โ€” the predicate must be trivially cheap, because the #[target_feature] call boundary deoptimizes non-trivial predicates (an early all-multiversioned draft regressed FSST fsst_prefix 13-63% this way; the pack kernel choice itself moves that DFA-bound workload by at most ~4%). Route the call sites whose predicates are bounds-check-free gathers or comparisons through the multiversioned tier: From<&[bool]> / From<&[u8]>, ByteBool boolean decoding, and the primitive and decimal between kernels (now reading via get_unchecked, sound because the predicate only receives indices 0..len โ€” a contract now documented on every entry point together with the bounds-check performance note). Measured on AVX-512 hardware (divan, 64Ki bits, medians): the fused word loop with a bool-gather predicate runs 14-24x faster than the old scalar loop per SIMD level (SSE2 1.7us, AVX2 1.2us, AVX-512 1.1us vs 26.5us); BitBufferMut::from(&[bool]) improves ~43x; a between-shaped predicate doubles under the multiversioned tier (5.5us vs 11.2us); fsst_prefix is unchanged or slightly better than before the change. Generated asm verified per kernel (the fused AVX-512 loop is vmovdqu64 + vptestmb + kmovq per 64 bools; NEON is 14 instructions, its addp tree additionally validated against a reference model on 10,004 random inputs). New benches cover each kernel and both tiers; under CodSpeed only the shipped entry points are tracked (from_bool_slice, collect_bool_u32_gt). Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcsiAHXQc4e11yYcP3a3NX
claude/collect-bool-arch-optimizations-bo09ws
57 minutes ago
feat(vortex-buffer): arch-specialized SIMD bool packing for collect_bool Replace the bit-at-a-time shift-OR loop behind collect_bool_word / collect_bool_words with byte-to-bit packing kernels specialized per architecture, without changing any public signature: - x86-64 AVX-512BW: one vptestmb packs 64 bools into a u64 - x86-64 AVX2: two vpcmpeqb + vpmovmskb halves - x86-64 SSE2 (baseline): four pcmpeqb + pmovmskb quarters - aarch64 NEON (baseline): ushl by bit position + addp reduction tree - other targets and Miri: branch-free SWAR multiply fallback The API has two tiers. The default (collect_bool / collect_bool_words, backed by collect_bool_words_inline) compiles the word loop once with the widest kernel enabled at compile time (SSE2/NEON on stock targets; AVX2/AVX-512 under -C target-cpu=native) so the predicate, the [bool; 64] materialization, and the pack all inline into the caller. The opt-in tier (collect_bool_multiversioned / collect_bool_words_multiversioned) compiles the loop with the predicate inside once per CPU feature level and selects a clone by runtime feature detection; its docs state the caller's assertion โ€” the predicate must be trivially cheap, because the #[target_feature] call boundary deoptimizes non-trivial predicates (an early all-multiversioned draft regressed FSST fsst_prefix 13-63% this way; the pack kernel choice itself moves that DFA-bound workload by at most ~4%). Route the call sites whose predicates are bounds-check-free gathers or comparisons through the multiversioned tier: From<&[bool]> / From<&[u8]>, ByteBool boolean decoding, and the primitive and decimal between kernels (now reading via get_unchecked, sound because the predicate only receives indices 0..len โ€” a contract now documented on every entry point together with the bounds-check performance note). Measured on AVX-512 hardware (divan, 64Ki bits, medians): the fused word loop with a bool-gather predicate runs 14-24x faster than the old scalar loop per SIMD level (SSE2 1.7us, AVX2 1.2us, AVX-512 1.1us vs 26.5us); BitBufferMut::from(&[bool]) improves ~43x; a between-shaped predicate doubles under the multiversioned tier (5.5us vs 11.2us); fsst_prefix is unchanged or slightly better than before the change. Generated asm verified per kernel (the fused AVX-512 loop is vmovdqu64 + vptestmb + kmovq per 64 bools; NEON is 14 instructions, its addp tree additionally validated against a reference model on 10,004 random inputs). New benches cover each kernel and both tiers; under CodSpeed only the shipped entry points are tracked (from_bool_slice, collect_bool_u32_gt). Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcsiAHXQc4e11yYcP3a3NX
claude/collect-bool-arch-optimizations-bo09ws
1 hour ago

Latest Branches

CodSpeed Performance Gauge
+17%
feat[buffer]: specialized `collect_bool` per arch#8749
1 hour ago
e64d06d
claude/collect-bool-arch-optimizations-bo09ws
CodSpeed Performance Gauge
+14%
3 hours ago
d67d634
claude/vortex-aarch-pattern-abstraction-s3javb
CodSpeed Performance Gauge
+10%
ยฉ 2026 CodSpeed Technology
Home Terms Privacy Docs