Avatar for the Eventual-Inc user
Eventual-Inc
Daft
BlogDocsChangelog

Performance History

Latest Results

fix(asof): make join_asof output partition-invariant on (by, on) ties join_asof's result depended on partition count and execution order when several right rows shared the same (by, on) key: the operator kept whichever row it saw first during the concurrent probe/merge, which varies with how right batches split across workers and with shuffle order. The same query could return different right-side values at different partition counts. Resolve equal-on-key ties by value instead of arrival order: keep the greatest right row for backward/nearest and the least for forward, compared lexicographically over the on-key and the right output columns. Being a per-key max/min over a total order, the choice is independent of execution order, so the output is partition-invariant. This matches pandas merge_asof (backward keeps the last row, forward the first). - daft-local-execution: the tie-break folds into the existing per-batch-pair comparator (nearest breaks only exact (distance, on-key) ties). It compares the pruned output columns, which are already retained for output, so it pins no extra memory. - daft-distributed: the no-by carryover keeps every row tied at the boundary on-key (whole-frame window + filter), mirroring the with-by path, so the local operator stays the single place ties are broken. Regression test test_no_by_duplicate_keys_partition_invariant asserts the result is identical at 1/2/4/8 partitions across all three strategies. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
chuqCTC:asof-deterministic-tie-break
2 hours ago
Merge branch 'main' into fix/escape-sql-string-literals
chuqCTC:fix/escape-sql-string-literals
4 hours ago
Merge branch 'main' into fix/escape-sql-string-literals
chuqCTC:fix/escape-sql-string-literals
4 hours ago
Merge branch 'main' into codex/add-to-map-function
dwallener:codex/add-to-map-function
7 hours ago

Latest Branches

CodSpeed Performance Gauge
0%
fix(asof): make join_asof output partition-invariant on (by, on) ties#7281
3 hours ago
4b8169c
chuqCTC:asof-deterministic-tie-break
CodSpeed Performance Gauge
0%
4 hours ago
fcc50ec
chuqCTC:fix/escape-sql-string-literals
CodSpeed Performance Gauge
0%
7 hours ago
15db5fa
dwallener:codex/add-to-map-function
© 2026 CodSpeed Technology
Home Terms Privacy Docs