Avatar for the Eventual-Inc user
Eventual-Inc
Daft
BlogDocsChangelog

Performance History

Latest Results

fix(parquet): account for deleted rows in limit pushdown prune_row_groups subtracted each row group's pre-delete num_rows() from the limit budget, so a heavily-deleted row group exhausted it and later row groups with matching rows were never decoded. Subtract the deletes falling in each row group's visible range instead. Adds a test module to helpers.rs, which had none.
jackylee-ch:parquet-limit-deleted-rows
14 minutes ago
Merge branch 'main' into bugfix/joinable-stream-background-error
zhouwenjiajia:bugfix/joinable-stream-background-error
5 hours ago
refactor(optimizer): drop the local-limit machinery from the join limit pushdown Reverts the `Limit.local` plumbing (logical `Limit` field, `with_local`, the builder test helper and the distributed `LimitNode` per-partition lowering) that was added to work around a panic on the ray runner. It did not fix that panic - the pushed limit still tripped it - and it spread a new plan-level concept plus a field that every `with_new_children` call has to carry by hand across two crates. The panic is a pre-existing defect in the distributed shuffle, not something this rule introduces: `transpose_materialized_outputs` requires every upstream output to carry exactly `num_partitions` partitions, but a task whose input was cut to zero rows by a limit below the shuffle emits no partitions at all. It reproduces on main with plain user code and no optimizer change: left = daft.range(0, 1000, partitions=8).with_column("k", col("id") % 100) right = daft.range(0, 100, partitions=8).select(col("id").alias("k")) left.limit(20).join(right, on="k", how="left").limit(20).to_pydict() # DAFT_RUNNER=ray -> panicked: "Expected all outputs to have 8 partitions, # got 0, 8, 0, 0, 0, 0, 0, 8" That is fixed separately in the shuffle transpose helper; with that fix in place this rule needs no special limit kind, so the change is back to a single file. The rule itself is unchanged: push onto the left input of a Left join and the right input of a Right join, no-op for Inner/Outer/Anti/Semi. Partially addresses #2275 - the Outer case requested there cannot be implemented by bounding either input, because limiting one side turns rows of the other side that used to match into unmatched rows and the join then emits null-extended rows the original plan never produces.
hello-peter-tang:push-down-limit-outer-join
9 hours ago
fix(distributed): satisfy clippy for joinset polling
zhouwenjiajia:bugfix/joinable-stream-background-error
10 hours ago
Merge branch 'main' into fix/escape-sql-string-literals
chuqCTC:fix/escape-sql-string-literals
15 hours ago

Latest Branches

CodSpeed Performance Gauge
0%
fix(parquet): account for deleted rows in limit pushdown#7444
31 minutes ago
293e763
jackylee-ch:parquet-limit-deleted-rows
CodSpeed Performance Gauge
0%
3 hours ago
86640af
jackylee-ch:iceberg-partition-repr-parity
CodSpeed Performance Gauge
0%
5 hours ago
13fc70a
zhouwenjiajia:bugfix/joinable-stream-background-error
© 2026 CodSpeed Technology
Home Terms Privacy Docs