Eventual-Inc/Daft - CodSpeed

Daft

Blog Docs Changelog

Performance History

Latest Results

checkpoint register

chenghuichen:ck-registry

35 minutes ago

huleilei:hll/auto

2 hours ago

huleilei:hll/auto

3 hours ago

feat(ray): add opt-in downscaling for Ray autoscaler Squash of MR !50 commits. Adds an opt-in Ray downscaling (scale-in) mechanism by retiring idle Flotilla workers to help Ray autoscaler shrink clusters when workloads become idle. Highlights: - Retire idle Ray workers with configurable idle threshold and a min-survivor floor. - Head-node protection and a pending-release blacklist to avoid immediate respawn. - Expose configuration via `daft.set_runner_ray(...)` and environment variables. - Keep scale-up behavior aligned with upstream’s high-water-mark ramp-up logic. - Add scheduler unit tests for downscale gating and min-survivor behavior. - Document autoscaling/downscaling in `docs/distributed/ray.md`. Validation: - `cargo check -p daft-distributed` - `cargo test -p daft-distributed scheduler_actor::tests::test_scheduler_actor_downscale` - `python -m compileall -q daft`

huleilei:hll/auto

4 hours ago

Merge branch 'main' into gw/hamming

gweaverbiodev:gw/hamming

11 hours ago

fix(udf): filter divider padding + use deque for pending lines Addresses Greptile feedback on PR #6793. - trace_output() now skips bare "\n" lines, which are the padding we inject before _OUTPUT_DIVIDER to guarantee the sentinel lands on its own line. Previously that padding surfaced as an empty line per batch and got printed as a spurious "[`udf` Worker #N] " prefix through the Rust caller. - _pending_lines is now a collections.deque with popleft() instead of a list with pop(0), which was O(n) per pop. Under a UDF that floods stderr, the wait() loop can build up a large queue and the subsequent trace_output() drain would be O(n^2). Both tests from the earlier commit (silent UDF, pipe-buffer, and divider-merge) pass. Existing use_process=True tests unaffected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

rohit/bugfix/df-1963-use-process-deadlocks

11 hours ago

fix(checkpoint): use strip_file_uri_to_path in put_bytes for Windows (#6796) ## Summary Follow-up to #6791. The earlier PR fixed the test helper to produce a canonical `file:///C:/Users/...` URL on Windows. That unblocked the **reader** path (which routes through `daft_io::strip_file_uri_to_path` and `#[cfg(windows)] strip_leading_slash_before_drive`), but **`S3CheckpointStore::put_bytes` still did its own** `path.strip_prefix("file://")` and handed `/C:/Users/...` to `Path::new(...)`. Windows rejects that with os error 123 (`"filename, directory name, or volume label syntax is incorrect"`), so every write blew up at the first `create_dir_all`. Net effect on the Windows coverage job: went from `3 passed / 11 failed` (pre-#6791) to `0 passed / 14 failed` (post-#6791). Reads worked; writes didn't. This PR switches `put_bytes` to use `daft_io::strip_file_uri_to_path` — the canonical helper — so both platforms get correct handling: - POSIX: `file:///tmp/foo` → `/tmp/foo` - Windows: `file:///C:/Users/...` → `C:/Users/...` ## Why no local test The failing shape is Windows-specific (the `Path::new("/C:/...")` parse failure) and not reproducible on macOS/Linux. Relying on the existing Windows rust-tests CI to verify. ## Test plan - [x] `cargo test -p daft-checkpoint --test s3_store` — 14/14 pass locally (macOS). - [ ] Windows rust-tests on `main` should go back to 14/14 after merge. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

main

11 hours ago

feat(file): add write modes to daft.File.open Support writing through daft.File without falling back to fsspec by adding mode-aware open() handling for w/wt/wb and a buffered writer that flushes via io_put. Also add regression coverage for direct writes and UDF writes.

BABTUNA:feat/file-write-support

11 hours ago

Latest Branches

+11%

feat(checkpoint): in-flight write tracking for checkpoint store#6800

1 hour ago

fb299fb

chenghuichen:ck-registry

0%

feat(ray): Implement dynamic scale-in for RaySwordfishActor#5903

3 hours ago

16c5b40

huleilei:hll/auto

0%

feat(functions): add hamming_distance function#6797

11 hours ago

c340c58

gweaverbiodev:gw/hamming

© 2026 CodSpeed Technology

Home Terms Privacy Docs