Eventual-Inc/Daft - CodSpeed

Daft

Blog Docs Changelog

Performance History

Latest Results

feat(iceberg): inject DataFile column stats for predicate short-circuit Decode PyIceberg DataFile lower_bounds/upper_bounds and pass them as a 2-row RecordBatch via DataSourceTask.parquet(stats=...), enabling MicroPartition.filter() to short-circuit on file-level column ranges already available in the Iceberg manifest. - Precompute top-level field descriptors (field_id, name, type, is_primitive, arrow_type) in IcebergDataSource.__init__ - Build stats RecordBatch in schema order with typed nulls for missing/undecodable fields, conservatively returning None when no field produces usable bounds - Follow the same pattern used by Delta Lake (delta_lake_scan.py) Closes #7299 Signed-off-by: jiangxt2 <jiangxt2@vip.qq.com>

jiangxt2:feat/iceberg-scan-stats

53 minutes ago

fix(cse): register cache readers at construction time to prevent late-reader hang The reader previously registered its channel inside its spawned task, so a writer scheduled first could publish all morsels and Flush into an empty sender list; the late reader would then wait on recv() forever (most reachable with tiny or empty subplans). Pipeline construction fully precedes execution: all nodes are built before any task is spawned by start(). Registering the reader's channel in CseCacheReadNode::new() therefore guarantees every reader is registered before the writer can fan out its first message, closing the race window entirely — no replay buffer or completion flag needed. Also switch CseSharedCache's sender list from tokio::sync::Mutex to std::sync::Mutex since the critical sections contain no await points, making register_reader/fan_out synchronous.

Lucas61000:issue-2423

2 hours ago

fix(cse): fix cache test hang, test build error, and stale explain test - test_reader_registered_before_write_receives_all hung forever on its channel-closed assertion: CseSharedCache keeps every registered reader's sender alive, so recv() can only observe closure after the cache is dropped. Drop the cache explicitly before the final assertion. - daft-local-execution does not depend on daft_schema; build the test MicroPartition with empty(None) instead. - test_explain_shows_common_subplan_in_optimized_plan still asserted CSE fires on a plain in-memory concat; update it to use an aggregate subplan to match the tightened heuristic (requires Aggregate/Sort). - Clippy cleanups (needless_borrow, while_let_loop) and ruff/rustfmt fixes.

Lucas61000:issue-2423

5 hours ago

style(io): remove unused MCAP lint suppression

codex/mcap-native-reader-core

6 hours ago

Merge branch 'main' into read-blob

hello-tang-rui:read-blob

6 hours ago

refactor: move download import to module level in _blob.py Address review feedback: the deferred import had no import-cycle reason. Import from daft.functions.url directly.

hello-tang-rui:read-blob

7 hours ago

fix: use floor division semantics for pre-epoch timestamps Address review feedback: unit downcasts previously truncated toward zero, so unix_seconds(-1.5s) returned -1 instead of -2. Spark uses floorDiv for these functions, so convert via explicit floor division on the physical epoch values instead of a unit cast. Adds regression tests for negative fractional epochs.

hello-tang-rui:feat/unix-epoch-functions

7 hours ago

Merge remote-tracking branch 'origin/main' into codex/mcap-native-reader # Conflicts: # src/daft-scan/src/python.rs

codex/mcap-native-reader

9 hours ago

Latest Branches

0%

feat(iceberg): inject DataFile column stats into DataSourceTask for predicate short-circuit#7301

7 days ago

c9c2766

jiangxt2:feat/iceberg-scan-stats

0%

feat(optimizer): add common subplan elimination optimization rule#6788

3 hours ago

49bb93e

Lucas61000:issue-2423

0%

feat(io): add native MCAP reader core#7332

6 hours ago

7c6e23f

codex/mcap-native-reader-core

© 2026 CodSpeed Technology

Home Terms Privacy Docs