Latest Results
fix(distributed): surface background errors while input is pending (#7396)
`JoinableForwardingStream` previously polled background tasks only after
its input stream closed. If a scheduler or materializer task failed
while the input remained `Pending`, that error stayed queued in the
`JoinSet` and the driver could wait indefinitely for another partition.
This change surfaces background failures without changing ready-input
priority or the stream's existing drop semantics.
## Changes Made
- Poll the input first so ready data keeps its existing priority.
- While the input is pending, poll completed background tasks until the
remaining tasks register the stream waker.
- On a task error or panic, stop input forwarding, emit the error
immediately, and transition to `AwaitingTasks` with the remaining
`JoinSet`.
- Continue forwarding input after successful background completions, and
stop polling an exhausted `JoinSet`.
- Add regression coverage for pending input, delayed failures, wakeup
preservation, ready-input priority, remaining-task behavior after
failure, and task errors/panics.
## Validation
- `cargo test -p daft-distributed test_joinable_forwarding_stream --
--nocapture`: 10 passed.
- `cargo test -p daft-distributed`: 75 passed, 1 ignored.
- Fork Ray verification: 2 independent repetitions; each ran the focused
Ray case 5 times and the CI-equivalent Python 3.10 Ray unit suite. Both
repetitions passed in 40m 41s and 44m 13s.
- Community PR CI: 35 successful checks and 1 expected skip.
- `cargo fmt --all -- --check` and `git diff --check` passed.
## Related Issues
Closes #7394
---------
Co-authored-by: zhouwenjia.zwj <zhouwenjia.zwj@alibaba-inc.com> Latest Branches
-1%
wuleiwuleiwulei:wl_0603_enum -1%
BABTUNA:perf/sharded-grouped-agg 0%
codex/agg-sum-literal-rewrite © 2026 CodSpeed Technology