Eventual-Inc/Daft - CodSpeed

Daft

Blog Docs Changelog

Performance History

Latest Results

perf: add dense integer argsort fast path

GENG-CHOVYYYY:perf/dense-integer-argsort

7 hours ago

Merge branch 'main' into fix/iceberg-count-pushdown-partition-filters

atovk:fix/iceberg-count-pushdown-partition-filters

7 hours ago

fix(iceberg): keep predicates on partition fields older files do not carry A predicate over an identity-partitioned column is dropped from the row-level filter, because PushDownFilter classifies it as resolvable from partition values alone. That holds only if every data file carries the field. Partition evolution breaks it: a file written under an earlier spec has no value for a field added later, so its rows mix matching and non-matching values and nothing filters them. Report only the partition fields every live file actually carries. A predicate on a field some files predate then stays an ordinary row filter, and pruning is unaffected for the fields that all files do carry. Manifests record the spec they were written with, so this reads the manifest list rather than planning files, and the answer is computed against the snapshot being read so time travel sees the specs of its own era. This also settles the cases where the current spec derives a field the predicate cannot be expressed against, such as re-partitioning a column from identity to truncate or adding a bucketed field: those previously raised FieldNotFound while evaluating a predicate naming a partition column the older records do not have. Counting from partition metadata rests on the same assumption, so count pushdown is declined when the reported fields fall short of the current spec. Closes #7428

atovk:fix/iceberg-partition-evolution-row-filter

8 hours ago

fix(iceberg): filter rows of files written before a partition field existed A predicate on an identity-partitioned column is dropped from the data-level filter, because PushDownFilter classifies it as resolvable from partition values alone. That holds only if every data file carries the field. Partition evolution breaks it: a file written under an earlier spec has no value for a field added later, so its rows mix matching and non-matching values. Such a file was emitted with the predicate gone from its pushdowns, returning rows the query excluded, and evaluating the predicate against an older partition record that lacked the column raised FieldNotFound instead. Put the predicate back into `filters` for the files whose partition record cannot settle it, leaving the pruning path untouched for the files it can. The task already carries pushdowns, so this needs no new plumbing. Counting from partition metadata rests on the same assumption, so the source now declines count pushdown when live manifests were written under a spec missing a current partition field. Manifests record their spec, so this reads the manifest list rather than planning files, and a table whose evolution left the current fields in place keeps the optimization. Closes #7428

atovk:fix/iceberg-partition-evolution-row-filter

9 hours ago

chore(logging): remove deprecated setup_debug_logger() (#7408)

Lucas61000:issue-7408

9 hours ago

fix(iceberg): skip the metadata count when a file predates its partition field Partition evolution can add a field long after data was written without it. Such a file carries no value for that field, mixes matching and non-matching rows, and its partition record cannot settle the predicate, yet the count path treated every row in it as a match. Fall back to a regular scan when a planned file's spec does not carry the fields the partition predicate references, so the count never disagrees with what the same query returns. This also avoids evaluating the predicate against a partition record that lacks the column, which raised FieldNotFound. Reported by @dujl in review of #7421.

atovk:fix/iceberg-count-pushdown-partition-filters

9 hours ago

fix(catalog): avoid shared mutable default for from_postgres extensions Catalog.from_postgres used a mutable list as the default for extensions. Python evaluates defaults once, so every catalog created with the default aliased the same module-level list, which was stored by reference in PostgresCatalog._extensions and propagated to PostgresTable. Mutating it through any instance silently changed the extensions of all other catalogs, including ones created later. Use an immutable tuple default instead, preserving the existing semantics of explicit None/empty (no extensions). Type annotations along the path are widened to Sequence[str].

DogerW666:fix/postgres-mutable-default-extensions

12 hours ago

perf(iceberg): forward the predicate to PyIceberg when counting The count path planned every data file in the table and then discarded the non-matching ones in Python. Passing the converted predicate as a row filter lets PyIceberg drop whole manifests from their partition summaries first: on a table with 200 partitions, a single-partition count plans 1 data file instead of 200. The row filter is a pre-filter only. PyIceberg prunes on column bounds, which are truncated to 16 bytes, and the predicate is dropped entirely when it references a partition field name absent from the data schema, so it can return more files than match. The per-file partition check stays the authority on what is counted, reading the partition record, which is stored whole. Closes #7420

atovk:fix/iceberg-count-pushdown-partition-filters

12 hours ago

Latest Branches

0%

perf: add guarded dense integer argsort fast path#7430

1 day ago

43b0361

GENG-CHOVYYYY:perf/dense-integer-argsort

0%

fix(iceberg): apply partition predicates to count pushdown#7421

7 hours ago

1da8fa8

atovk:fix/iceberg-count-pushdown-partition-filters

0%

fix(iceberg): filter rows of files written before a partition field existed#7429

9 hours ago

ca5cfa2

atovk:fix/iceberg-partition-evolution-row-filter

© 2026 CodSpeed Technology

Home Terms Privacy Docs