Eventual-Inc/Daft

Eventual-Inc

Daft

Blog Docs Changelog

Performance History

Latest Results

fix(io): use checked_shl and saturating_mul in retry jitter calculation

ARDA7787:patch-1

25 seconds ago

refactor(flotilla): remove unused `cancel_plan` `NativeExecutor::cancel_plan` (and its `PyNativeExecutor` binding) had no callers — orphaned from an earlier design. `cancel_input` is what the flotilla cancel path actually uses; per-plan teardown happens naturally when `try_finish` removes the last `active_input_id`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

colin/fix-distributed-limit-cancel-noise

10 hours ago

fix(flotilla): silence noisy logs from distributed-limit cancellation PR #6942 surfaced three categories of confusing output during limit cancellation. None changed query results; all are removed here. - `ray.cancel(result_handle)` on every non-contributor task marked them FAILED in Ray's task table. Replaced with `actor.cancel_input.remote(...)`, which closes the per-input result channel inside the worker's `NativeExecutor` so `run_plan` drains and returns FINISHED. - `claim` / `start_task` on the torn-down counter actor used to raise `ActorDiedError`, propagate out of `DistributedLimitSink::execute`, and tear down the worker's run loop. They now swallow that error as a residual-race safety net (`LimitNode` drains task cancels first, so this only fires when a sink call is mid-flight). - "Plan execution task has died; cannot enqueue new input" was the downstream symptom of the previous two on concurrent inputs sharing a plan; no longer reproducible. Mechanism: 1. `NativeExecutor` exposes `cancel_input(fingerprint, input_id)`. It sends through a new unbounded channel; `run_execution_loop` drops the per-input result sender, ending the Python iteration cleanly. 2. `TaskResultHandle::cancel_callback` now returns a `Future`. The Ray impl awaits the Python `actor.cancel_input.remote(...)` ack so the scheduler can sequence post-cancel teardown. 3. `LimitNode::limit_execution_loop` drains `running_tasks` after firing `parent_cancel.cancel()` and before `teardown_limit_counter_actor`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

colin/fix-distributed-limit-cancel-noise

12 hours ago

feat: configurable flight shuffle compression Adds `flight_shuffle_compression` to `DaftExecutionConfig`, exposing lz4/zstd Arrow IPC compression for flight shuffle spill files via `daft.context.set_execution_config(...)`. Also enables the lz4 and zstd features on the `arrow-ipc` workspace dep — without them the IPC writer panicked at runtime when compression was set, since the field on `FlightShuffleBackendConfig` had never been reachable from user code. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

colin/configurable-flight-shuffle-compression

12 hours ago

Merge remote-tracking branch 'origin/main' into perf/inline-agg-product

BABTUNA:perf/inline-agg-product

13 hours ago

test(inline-agg): cover small-int widening dispatch arms for Product Addresses Greptile's P2 on #6975: the AggExpr::Product(expr) => arm in try_create_accumulator widens Int8/16/32 → Int64 and UInt8/16/32 → UInt64 via .cast(), but no existing test exercised those branches (all prior Product tests used Int64 or Float64 val columns directly). Adds two inline-vs-fallback tests: - test_inline_i32_product_matches_fallback — Int32 val column, hits the signed widening arm. - test_inline_u16_product_matches_fallback — UInt16 val column, hits the unsigned widening arm. Both use Int64 keys (FNV int-key fast path) with 4 rows and balanced 2-group products to keep arithmetic deterministic and overflow-free. 39 inline_agg tests pass (37 prior + 2 new). cargo fmt and cargo clippy --release --features python both clean.

BABTUNA:perf/inline-agg-product

14 hours ago

fix

chris/shuffle-write-stats

15 hours ago

test(flotilla): gate cancel test on Ray runner These flotilla-specific cancellation behaviors don't exist on the native runner, so the test only meaningfully runs under `DAFT_RUNNER=ray`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

colin/test-distributed-limit-cancel-regression

16 hours ago

Latest Branches

fix(flotilla): silence noisy logs from distributed-limit cancellation#6980

11 hours ago

f118f2b

colin/fix-distributed-limit-cancel-noise

feat: configurable flight shuffle compression#6979

13 hours ago

d77ee2e

colin/configurable-flight-shuffle-compression

perf(inline-agg): add Product accumulator type#6975

13 hours ago

11fcb05

BABTUNA:perf/inline-agg-product

Home Terms Privacy Docs