Latest Results
Evaluate join dynamic filters in V2 scans before projecting
A hash join publishes a dynamic filter on its probe side's keys once its table is built, but
Vortex rejected it, so the probe scan decoded, converted, repartitioned and probed every row.
The V2 scan now takes such filters as a planner stage: `RowFilter` runs after the scan's own
conjuncts, reads only the key columns over the rows they kept, and the projection decodes
only what it keeps. V1 is unchanged; `VortexSource` accepts the filters only for V2 scans.
Most of a partitioned join's filter, `CASE hash % N WHEN i THEN bounds_i AND lookup_i`, was
DataFusion's generic CASE machinery: per-branch filtering, a bit-by-bit result scatter, and
bounds a key found in the table always satisfies. `DynamicRowFilter` hashes each row once,
looks each partition's rows up in its table, and drops bounds beside a membership test.
Filters are advisory, so one that keeps most rows stops being evaluated, and none is taken
when the scan outputs only the filter's keys, where it would cost what the join's probe does.
Hot geomean V2/V1 on 4 cores, interleaved runs: SF1 0.90 -> 0.74, SF10 0.80 -> 0.70
(q7 0.30, q17 0.33, q20 0.37, q18 0.56 at SF10). Row counts match on every query.
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0111zwMS5rJDMbRPaAc6UVxS Latest Branches
-3%
-28%
+2%
rk/write-canonicalize-once © 2026 CodSpeed Technology