Eventual-Inc/Daft - CodSpeed

Daft

Blog Docs Changelog

Performance History

Latest Results

Merge branch 'Eventual-Inc:main' into main

ARDA7787:main

15 hours ago

refactor(shuffle): migrate distributed sort onto shared shuffle backend

BABTUNA:refactor/sort-shared-shuffle-backend

21 hours ago

perf: add dense integer argsort fast path

GENG-CHOVYYYY:perf/dense-integer-argsort

1 day ago

Merge branch 'main' into fix/iceberg-count-pushdown-partition-filters

atovk:fix/iceberg-count-pushdown-partition-filters

1 day ago

fix(iceberg): keep predicates on partition fields older files do not carry A predicate over an identity-partitioned column is dropped from the row-level filter, because PushDownFilter classifies it as resolvable from partition values alone. That holds only if every data file carries the field. Partition evolution breaks it: a file written under an earlier spec has no value for a field added later, so its rows mix matching and non-matching values and nothing filters them. Report only the partition fields every live file actually carries. A predicate on a field some files predate then stays an ordinary row filter, and pruning is unaffected for the fields that all files do carry. Manifests record the spec they were written with, so this reads the manifest list rather than planning files, and the answer is computed against the snapshot being read so time travel sees the specs of its own era. This also settles the cases where the current spec derives a field the predicate cannot be expressed against, such as re-partitioning a column from identity to truncate or adding a bucketed field: those previously raised FieldNotFound while evaluating a predicate naming a partition column the older records do not have. Counting from partition metadata rests on the same assumption, so count pushdown is declined when the reported fields fall short of the current spec. Closes #7428

atovk:fix/iceberg-partition-evolution-row-filter

1 day ago

fix(iceberg): filter rows of files written before a partition field existed A predicate on an identity-partitioned column is dropped from the data-level filter, because PushDownFilter classifies it as resolvable from partition values alone. That holds only if every data file carries the field. Partition evolution breaks it: a file written under an earlier spec has no value for a field added later, so its rows mix matching and non-matching values. Such a file was emitted with the predicate gone from its pushdowns, returning rows the query excluded, and evaluating the predicate against an older partition record that lacked the column raised FieldNotFound instead. Put the predicate back into `filters` for the files whose partition record cannot settle it, leaving the pruning path untouched for the files it can. The task already carries pushdowns, so this needs no new plumbing. Counting from partition metadata rests on the same assumption, so the source now declines count pushdown when live manifests were written under a spec missing a current partition field. Manifests record their spec, so this reads the manifest list rather than planning files, and a table whose evolution left the current fields in place keeps the optimization. Closes #7428

atovk:fix/iceberg-partition-evolution-row-filter

1 day ago

chore(logging): remove deprecated setup_debug_logger() (#7408)

Lucas61000:issue-7408

1 day ago

fix(iceberg): skip the metadata count when a file predates its partition field Partition evolution can add a field long after data was written without it. Such a file carries no value for that field, mixes matching and non-matching rows, and its partition record cannot settle the predicate, yet the count path treated every row in it as a match. Fall back to a regular scan when a planned file's spec does not carry the fields the partition predicate references, so the count never disagrees with what the same query returns. This also avoids evaluating the predicate against a partition record that lacks the column, which raised FieldNotFound. Reported by @dujl in review of #7421.

atovk:fix/iceberg-count-pushdown-partition-filters

1 day ago

Latest Branches

0%

feat(dataframe): add query profiling#7297

15 hours ago

8546de9

ARDA7787:main

0%

refactor(shuffle): migrate distributed sort onto shared shuffle backend#7431

22 hours ago

618ea83

BABTUNA:refactor/sort-shared-shuffle-backend

0%

perf: add guarded dense integer argsort fast path#7430

2 days ago

43b0361

GENG-CHOVYYYY:perf/dense-integer-argsort

© 2026 CodSpeed Technology

Home Terms Privacy Docs