Latest Results
fix(optimizer): only push limit onto the Left/Right join outer side
Pushing a Limit onto an input of a full Outer join is unsound: dropping rows
from one input turns rows on the other input that used to match into
unmatched rows, so the join emits null-extended rows the original plan never
produces. With l = [1, 2], r = [2] and LIMIT 1, pushing onto the left input
can return (a=2, b=null, c=r2), a row that is absent from the unoptimized
result. This holds for a one-sided push as well, so Outer joins are now a
no-op and only Left (push onto left) and Right (push onto right) are pushed
into.
Mark the pushed-down limit as a local (per-partition) limit and lower it in
the distributed executor to a plain per-partition limit instead of the
coordinated DistributedLimit, which hangs when a shuffle is scheduled
directly above it - the join repartitions the very side the limit was pushed
onto. The retained outer global limit still performs the exact final cut, so
a per-partition limit below the join is sufficient.
Make the idempotency guard offset-agnostic: a child Limit emits at most
`limit` rows regardless of its offset, so an input that is already
`Limit{limit, offset}` is no longer re-wrapped. Previously such an input was
re-wrapped on every pass and folded back by the Limit-Limit rule, making the
rule oscillate instead of reaching a fixed point.
Tests: Rust unit tests for the Left/Right push, the offset case, both
idempotency guards and the Inner/Outer/Anti/Semi no-ops, plus Python
regression tests that run a multi-partition Left/Right join with an outer
limit.
Closes #2275hello-peter-tang:push-down-limit-outer-join Latest Branches
0%
hello-peter-tang:push-down-limit-outer-join 0%
hello-peter-tang:startswith-stats-pruning 0%
© 2026 CodSpeed Technology