Eventual-Inc
Daft
Blog
Docs
Changelog
Blog
Docs
Changelog
Overview
Branches
Benchmarks
Runs
Performance History
Latest Results
feat(iceberg): inject DataFile column stats for predicate short-circuit Decode PyIceberg DataFile lower_bounds/upper_bounds and pass them as a 2-row RecordBatch via DataSourceTask.parquet(stats=...), enabling MicroPartition.filter() to short-circuit on file-level column ranges already available in the Iceberg manifest. - Precompute top-level field descriptors (field_id, name, type, is_primitive, arrow_type) in IcebergDataSource.__init__ - Build stats RecordBatch in schema order with typed nulls for missing/undecodable fields, conservatively returning None when no field produces usable bounds - Follow the same pattern used by Delta Lake (delta_lake_scan.py) Closes #7299 Signed-off-by: jiangxt2 <jiangxt2@vip.qq.com>
jiangxt2:feat/iceberg-scan-stats
53 minutes ago
fix(cse): register cache readers at construction time to prevent late-reader hang The reader previously registered its channel inside its spawned task, so a writer scheduled first could publish all morsels and Flush into an empty sender list; the late reader would then wait on recv() forever (most reachable with tiny or empty subplans). Pipeline construction fully precedes execution: all nodes are built before any task is spawned by start(). Registering the reader's channel in CseCacheReadNode::new() therefore guarantees every reader is registered before the writer can fan out its first message, closing the race window entirely — no replay buffer or completion flag needed. Also switch CseSharedCache's sender list from tokio::sync::Mutex to std::sync::Mutex since the critical sections contain no await points, making register_reader/fan_out synchronous.
Lucas61000:issue-2423
2 hours ago
fix(cse): fix cache test hang, test build error, and stale explain test - test_reader_registered_before_write_receives_all hung forever on its channel-closed assertion: CseSharedCache keeps every registered reader's sender alive, so recv() can only observe closure after the cache is dropped. Drop the cache explicitly before the final assertion. - daft-local-execution does not depend on daft_schema; build the test MicroPartition with empty(None) instead. - test_explain_shows_common_subplan_in_optimized_plan still asserted CSE fires on a plain in-memory concat; update it to use an aggregate subplan to match the tightened heuristic (requires Aggregate/Sort). - Clippy cleanups (needless_borrow, while_let_loop) and ruff/rustfmt fixes.
Lucas61000:issue-2423
5 hours ago
style(io): remove unused MCAP lint suppression
codex/mcap-native-reader-core
6 hours ago
Merge branch 'main' into read-blob
hello-tang-rui:read-blob
6 hours ago
refactor: move download import to module level in _blob.py Address review feedback: the deferred import had no import-cycle reason. Import from daft.functions.url directly.
hello-tang-rui:read-blob
7 hours ago
fix: use floor division semantics for pre-epoch timestamps Address review feedback: unit downcasts previously truncated toward zero, so unix_seconds(-1.5s) returned -1 instead of -2. Spark uses floorDiv for these functions, so convert via explicit floor division on the physical epoch values instead of a unit cast. Adds regression tests for negative fractional epochs.
hello-tang-rui:feat/unix-epoch-functions
7 hours ago
Merge remote-tracking branch 'origin/main' into codex/mcap-native-reader # Conflicts: # src/daft-scan/src/python.rs
codex/mcap-native-reader
9 hours ago
Latest Branches
CodSpeed Performance Gauge
0%
feat(iceberg): inject DataFile column stats into DataSourceTask for predicate short-circuit
#7301
7 days ago
c9c2766
jiangxt2:feat/iceberg-scan-stats
CodSpeed Performance Gauge
0%
feat(optimizer): add common subplan elimination optimization rule
#6788
3 hours ago
49bb93e
Lucas61000:issue-2423
CodSpeed Performance Gauge
0%
feat(io): add native MCAP reader core
#7332
6 hours ago
7c6e23f
codex/mcap-native-reader-core
© 2026 CodSpeed Technology
Home
Terms
Privacy
Docs