Eventual-Inc/Daft

Daft

Performance History

Latest Results

ci: use absolute path for LLVM_PROFILE_FILE The flush hook from the previous commit lifted daft-distributed coverage by essentially 0pp. The hook itself was never the problem: `LLVM_PROFILE_FILE=./target/daft-coverage-%p-%m.profraw` is a relative path, and Ray actor processes start with a different cwd (typically /tmp/ray/session_*). So whenever an actor flushed profraw (either on clean exit or via our SIGTERM hook), it wrote to `<ray session dir>/target/...`, which `cargo llvm-cov report` never scans. All actor-side counters have been silently dropping this whole time. Pin the env var to `${{ github.workspace }}/target/...` across all six coverage-instrumented job steps (unit-test, integration-test-ray, integration-test-tpch native, integration-test-tpch ray, the tpch report step, and rust-tests-platform). The existing unit-test matrix was hitting the same bug; fixing it here should also bump the scheduling/scheduler_actor.rs and dispatcher.rs numbers which are actor-driven. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

desmond/coverage-integration-ray

2 minutes ago

refactor(flight): implement distributed gather for flight shuffle

oh/flight_gather

24 minutes ago

feat(io): point users to read_blob on utf-8 decode error

aaron-ang:feat/read-blob

37 minutes ago

feat(dashboard): mock Flotilla Tasks tab Add a new "Tasks" tab to the query detail page that surfaces the swordfish tasks a Flotilla query dispatches to its workers. Tasks are grouped by (origin distributed-plan node, fused pipeline fingerprint) so each row represents one "task type" — e.g. a Read→Project pipeline running many files shows as a single row with aggregated stats. Cross-links both directions: task rows jump to the Execution tab with their origin node highlighted; physical plan cards gain a "tasks" affordance that filters the Tasks tab to that origin. Tab + filter state is URL-driven for deep-linking. Data is mocked in `mock-tasks.ts` (deterministic per query id) until the real Flotilla task-event feed lands. The mock data shape mirrors what the scheduler already tracks (TaskContext + PlanFingerprint) and is intended to drive requirements for the backend feed. https://claude.ai/code/session_01TsinjTSKNNW7qopNGAxzXP

claude/mock-flotilla-tasks-tab-Wt30L

1 hour ago

fix: address PR review feedback for richer source stats - Apply selectivity to num_rows in short-circuit path to match task-aggregation behavior - Clarify Absent size_bytes defaulting to 0 with a comment - Re-export ColumnStatistics at daft-scan crate root for public API usability Fixes review comments from PR #6723.

rchowell/sources-6-of-n

2 hours ago

ci: flush llvm profile from flotilla ray actors The ray integration jobs added earlier lift daft-distributed coverage by only 2-3pp overall because the driver-side coverage was already saturated by the unit-test ray matrix. The real uncovered code lives inside two ray actors: `RemoteFlotillaRunner` (where `PlanRunner::run_plan` and all `pipeline_node/*` orchestration runs) and `RaySwordfishActor` (where native_executor invokes daft-local-execution). Both are separate OS processes, each writes its own `%p` profraw under LLVM_PROFILE_FILE, and Ray's teardown delivers SIGTERM followed by SIGKILL. Python's default SIGTERM disposition terminates immediately without running atexit, so the actor profraw never flushes. `plan/runner.rs` sits at 6.42% and `python/ray/worker_manager.rs` at 0% as a direct consequence. Add a `cfg(coverage)`-gated pyo3 function that wraps `__llvm_profile_write_file()` and invoke it on actor shutdown. In non-coverage builds the extern block is stripped and the pyfunction is a no-op, so there is no runtime overhead outside of coverage-instrumented CI. The Python helper `_install_coverage_flush` registers both an `atexit` handler (for clean interpreter exit) and a SIGTERM handler that flushes before restoring the default disposition, covering both Ray teardown paths short of SIGKILL. Also declare `cfg(coverage)`, `cfg(coverage_nightly)`, and `cfg(trybuild_no_target)` at the workspace level so rustc doesn't warn about unexpected cfg names; these are already set in RUSTFLAGS for coverage builds but were never declared as valid. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

desmond/coverage-integration-ray

2 hours ago

refactor(partition-refs): consolidate partition ref types and trim unused derives (#6742) - Removes unused derives from FlightPartitionRef - Move RayPartitionRef into shared `daft-partition-refs` crate

main

2 hours ago

test(inline-agg): cover min/max in nullable multi-col case

BABTUNA:feat/symbolize-string-groupby-only

4 hours ago

Latest Branches

ci: instrument integration-test-ray for codecov#6744

22 minutes ago

54b25f0

desmond/coverage-integration-ray

refactor(flight): implement distributed gather for flight shuffle#6751

2 hours ago

72506d7

oh/flight_gather

feat(io)!: add daft.read_blob#6747

6 hours ago

6f3fa57

aaron-ang:feat/read-blob

Home Terms Privacy Docs