Avatar for the Eventual-Inc user
Eventual-Inc
Daft
BlogDocsChangelog

Performance History

Latest Results

feat(iceberg): add overwrite_filter for static partition overwrite write_iceberg(mode="overwrite") always replaced the whole table, leaving no way to replace a single partition. overwrite_filter scopes the delete to the rows matching a predicate, with the delete and the append still landing in one Iceberg transaction: df.write_iceberg(table, mode="overwrite", overwrite_filter="dt = '2024-01-01'") The filter accepts an Iceberg predicate string, a Daft expression, or a PyIceberg BooleanExpression, and is bound against the table schema before the write runs so an invalid predicate fails before any data files are staged. By default the rows being written must match the filter as well, so an overwrite cannot leave behind rows its delete did not cover. The check follows Iceberg's OverwriteFiles.validateAddedFilesMatchOverwriteFilter, accepting a file when its partition value proves every row matches or when its column statistics do. Both tests are conservative and statistics only ever widen, so it may refuse a write it cannot prove but never accepts an unmatched row. Pass validate_overwrite_filter=False to skip it. Also fixes the partitioning column of the returned DataFrame, which reported null for every partition on pyiceberg >= 0.9: data_file.partition is a positional Record that no longer exposes fields as attributes, so the getattr lookup always missed. Values are now read by index against the spec the file was written with, and keyed by Iceberg partition field name instead of source column name, which also stops two partition fields over one source column from collapsing into a single key.
atovk:feat/iceberg-overwrite-filter
6 minutes ago
fix(shuffle): stop into_partitions coalesce from panicking under flight shuffle
BABTUNA:fix/into-partitions-flight-coalesce
7 hours ago
ci: retrigger to clear flaky dead-plan-reuse error masking (#7169)
BABTUNA:refactor/random-shuffle-shared-read-tasks
8 hours ago
Merge branch 'main' into fix/coalesce-unreachable-validation
DogerW666:fix/coalesce-unreachable-validation
10 hours ago
refactor(shuffle): migrate random shuffle reads onto shared read-task emission
BABTUNA:refactor/random-shuffle-shared-read-tasks
11 hours ago

Latest Branches

CodSpeed Performance Gauge
0%
feat(iceberg): add overwrite_filter for static partition overwrite#7419
44 minutes ago
bf49d92
atovk:feat/iceberg-overwrite-filter
CodSpeed Performance Gauge
0%
9 hours ago
f45d663
kyo-tom:fix/gravitino-iceberg-rest-noop-auth
CodSpeed Performance Gauge
0%
7 hours ago
5fea4b5
BABTUNA:fix/into-partitions-flight-coalesce
© 2026 CodSpeed Technology
Home Terms Privacy Docs