Avatar for the Eventual-Inc user
Eventual-Inc
Daft
BlogDocsChangelog

Performance History

Latest Results

feat(file): trace video frame decoding (#7313) ## Changes Made - Add a private Rust-backed Python tracing context manager so Python/PyAV operations emit spans through Daft's existing tracing subscriber and OpenTelemetry exporter. - Add `VideoFile::frames`, `VideoFile::open`, `VideoFile::seek`, `VideoFile::decode`, and `VideoFile::to_image` spans around the corresponding work. - Avoid Python-to-Rust tracing context allocation when INFO tracing is disabled. - Stack this follow-up on #7312, which provides the per-type read spans and SpeedScope conversion utility. Together with #7312, this exposes the full VideoFile read, demux, decode, and image-conversion path. The generated SpeedScope profile contains all six expected VideoFile operations across 626 events. Validation: - `cargo fmt --check` - `cargo check -p daft-file --features python` - `make build` - `DAFT_RUNNER=native .venv/bin/pytest -q tests/file/test_video.py` (33 passed) - Converted the detailed JSON trace with `tools/trace_to_speedscope.py` ## Related Issues N/A
main
4 hours ago
fix(file): scope video frame spans per iteration
everettVT/trace-video-operations
4 hours ago
fix: clean up unscheduled UDF actors
everettVT/fix-main-failures
6 hours ago
fix(io): address Hugging Face glob review feedback
everettVT/fix-hf-glob-revision
6 hours ago
fix: simplify actor startup cleanup
everettVT/fix-main-failures
7 hours ago
fix(distributed): harden idle-worker downscaling against scheduler races Fixes several issues in the background-reaper downscaling path: - Two-phase retirement (drain, then release): a worker idle past the threshold is first hidden from worker_snapshots() so the scheduler stops assigning work to it, and only released on a later reaper tick if still idle. This closes a race where the reaper could kill a worker between the scheduler taking a snapshot and dispatching to it, failing the whole query. If a draining worker picks up work in the window, it is put back in service instead. - Single critical section in reap_idle_workers: the scale-up guard, min-survivor floor, and candidate selection now operate on one consistent state snapshot, eliminating TOCTOU races that could retire workers against in-flight scale-up demand or dip below the floor. Policy decisions are extracted into a pure, unit-tested plan_reap() function (scheduling/downscale.rs). - Clear autoscaling demand on error exits too: the scheduler now clears outstanding (sticky) Ray autoscaler demand whether the event loop exits cleanly or with an error, best-effort so cleanup failures do not fail an otherwise-successful job. Jobs that never requested scale-up skip the call entirely, so the default path never touches the cluster-wide request_resources slot. - Reaper robustness: catch_unwind around each tick, poisoned-lock recovery, warn/error level logging for tick failures and thread spawn failure instead of silent debug/drops. - Config/docs: env vars extracted to constants, documented DAFT_AUTOSCALING_REAPER_INTERVAL_SECONDS, documented warm-pool retirement semantics in ray.md and set_runner_ray docstrings. Tests: 13 new unit tests covering the drain/release/undrain state machine, floor enforcement, scale-up guard, disable-mid-drain, and demand-clearing on the scheduler error path.
hello-tang-rui:fix/downscale-reaper-hardening
19 hours ago

Latest Branches

CodSpeed Performance Gauge
0%
feat(file): trace video frame decoding#7313
3 days ago
d7375e8
everettVT/trace-video-operations
CodSpeed Performance Gauge
0%
fix: clean up unscheduled UDF actors#7346
6 hours ago
357f176
everettVT/fix-main-failures
CodSpeed Performance Gauge
0%
6 hours ago
949b6e2
everettVT/fix-hf-glob-revision
© 2026 CodSpeed Technology
Home Terms Privacy Docs