Eventual-Inc/Daft - CodSpeed

Daft

Blog Docs Changelog

Performance History

Latest Results

fix(iceberg): filter rows of files written before a partition field existed A predicate on an identity-partitioned column is dropped from the data-level filter, because PushDownFilter classifies it as resolvable from partition values alone. That holds only if every data file carries the field. Partition evolution breaks it: a file written under an earlier spec has no value for a field added later, so its rows mix matching and non-matching values. Such a file was emitted with the predicate gone from its pushdowns, returning rows the query excluded, and evaluating the predicate against an older partition record that lacked the column raised FieldNotFound instead. Put the predicate back into `filters` for the files whose partition record cannot settle it, leaving the pruning path untouched for the files it can. The task already carries pushdowns, so this needs no new plumbing. Counting from partition metadata rests on the same assumption, so the source now declines count pushdown when live manifests were written under a spec missing a current partition field. Manifests record their spec, so this reads the manifest list rather than planning files, and a table whose evolution left the current fields in place keeps the optimization. Closes #7428

atovk:fix/iceberg-partition-evolution-row-filter

17 minutes ago

chore(logging): remove deprecated setup_debug_logger() (#7408)

Lucas61000:issue-7408

51 minutes ago

fix(iceberg): skip the metadata count when a file predates its partition field Partition evolution can add a field long after data was written without it. Such a file carries no value for that field, mixes matching and non-matching rows, and its partition record cannot settle the predicate, yet the count path treated every row in it as a match. Fall back to a regular scan when a planned file's spec does not carry the fields the partition predicate references, so the count never disagrees with what the same query returns. This also avoids evaluating the predicate against a partition record that lacks the column, which raised FieldNotFound. Reported by @dujl in review of #7421.

atovk:fix/iceberg-count-pushdown-partition-filters

56 minutes ago

fix(catalog): avoid shared mutable default for from_postgres extensions Catalog.from_postgres used a mutable list as the default for extensions. Python evaluates defaults once, so every catalog created with the default aliased the same module-level list, which was stored by reference in PostgresCatalog._extensions and propagated to PostgresTable. Mutating it through any instance silently changed the extensions of all other catalogs, including ones created later. Use an immutable tuple default instead, preserving the existing semantics of explicit None/empty (no extensions). Type annotations along the path are widened to Sequence[str].

DogerW666:fix/postgres-mutable-default-extensions

3 hours ago

perf(iceberg): forward the predicate to PyIceberg when counting The count path planned every data file in the table and then discarded the non-matching ones in Python. Passing the converted predicate as a row filter lets PyIceberg drop whole manifests from their partition summaries first: on a table with 200 partitions, a single-partition count plans 1 data file instead of 200. The row filter is a pre-filter only. PyIceberg prunes on column bounds, which are truncated to 16 bytes, and the predicate is dropped entirely when it references a partition field name absent from the data schema, so it can return more files than match. The per-file partition check stays the authority on what is counted, reading the partition record, which is stored whole. Closes #7420

atovk:fix/iceberg-count-pushdown-partition-filters

4 hours ago

fix(dashboard): normalize and unconditionally honor DAFT_RAY_DASHBOARD_URL Addresses review feedback on the override: - Normalize the override the same way as the auto-detected URL: prepend `http://` when no scheme is given, so a bare hostname does not produce a broken link. - Check the override before the `RAY_DISABLE_DASHBOARD` guard. The two env vars serve different purposes: one turns off Ray's built-in dashboard, the other points at an externally hosted one, so disabling the former should no longer silently discard the latter. - Append the Ray job route (`/#/jobs/<job_id>`) to the override too, unless it already contains a `#` fragment, so overridden links deep-link to the job like auto-detected ones do. The resolution logic moves out of `run_iter` into `_resolve_ray_dashboard_url` so it is unit-testable, and is documented in the dashboard docs.

claude/ray-ui-url-investigation-Q9HBW

5 hours ago

fix(catalog): declare noop auth for Gravitino Iceberg REST catalogs (#7417) ## Changes Made `_open_iceberg_table_via_rest` passed no auth configuration to PyIceberg's `RestCatalog`. On pyiceberg 0.11.1 that isn't the same as "no credentials": `RestCatalog` installs a `LegacyOAuth2AuthManager` whose `auth_header()` formats the unset token unconditionally, so Daft sent a literal `Authorization: Bearer None`. REST catalogs that require no auth reject that with 401, which makes them unreadable through Gravitino. Declare `auth={"type": "noop"}` when the catalog properties carry none of `credential` / `token` / `auth`, so no `Authorization` header is sent. Catalogs that do configure auth keep their current behavior. Tested against a live Gravitino Iceberg REST service on pyiceberg 0.7.0, 0.8.1 and 0.11.1 — only 0.11.1 was broken; on the older versions the extra property is inert, so this is a no-op there. AI disclosure: analysis and patch written with Claude; verified by hand against the live deployment and with `tests/catalog/test_gravitino.py`. ## Related Issues Closes #7416

main

7 hours ago

Merge branch 'main' into codex/agg-sum-literal-rewrite

codex/agg-sum-literal-rewrite

9 hours ago

Latest Branches

0%

fix(iceberg): filter rows of files written before a partition field existed#7429

30 minutes ago

720c304

atovk:fix/iceberg-partition-evolution-row-filter

0%

chore(logging): remove deprecated setup_debug_logger() #7427

5 hours ago

ce13bd7

Lucas61000:issue-7408

0%

fix(iceberg): apply partition predicates to count pushdown#7421

1 hour ago

7cb86f1

atovk:fix/iceberg-count-pushdown-partition-filters

© 2026 CodSpeed Technology

Home Terms Privacy Docs