Eventual-Inc/Daft - CodSpeed

Daft

Blog Docs Changelog

Performance History

Latest Results

fix(iceberg): match the spec for decimal bucket, string truncate and pre-1970 hour transforms (#7649) ## Changes Made Three Iceberg partition transforms diverged from the spec, so Daft wrote rows to the wrong partitions and pruned away matching files in tables written by other engines: - `bucket` on decimals now hashes the minimal two's-complement bytes with room for the sign bit, as Java's `BigInteger.toByteArray()`. Negatives, zero, and values like `1.28` were hashed wrongly. - `truncate` on strings counts code points (`substring_by_char`) instead of bytes, which failed on non-ASCII values. - `hour` uses floor division, so timestamps before 1970 map to negative hours. `test_partitioning_hours` asserted the old truncating results; it now expects the spec values, which PyIceberg also returns. The Rust decimal hash test computed its expectation with the same algorithm as the code under test; it now checks fixed byte encodings. ## Related Issues Closes #7648 ## AI Disclosure AI-assisted implementation.

main

6 hours ago

fix(sql): normalize SQL function names at registration (#7449) ## Changes Made The SQL IOConfig constructors were unreachable under every spelling: ```python daft.sql("SELECT * FROM read_parquet('f.parquet', io_config := S3Config(region_name => 'us-west-2'))") # Function `s3config` not found ``` `modules::config` registers `S3Config`, `HTTPConfig`, `AzureConfig`, `GCSConfig` and `TosConfig` in mixed case, but lookup lowercases the name before the `HashMap` get. `add_fn` now lowercases the key. Other modules already register lowercase names, so only these five keys change (`list_sql_functions()` shows `s3config`). Once reachable, `HTTPConfig` still rejected four of its six documented arguments: `parse_function_args` whitelisted only `user_agent` and `bearer_token`, while `retry_initial_backoff_ms`, `connect_timeout_ms`, `read_timeout_ms` and `num_tries` are read and advertised. The whitelist now matches. ## Testing Parametrized over `S3Config`/`s3config`/`S3CONFIG`, plus one `HTTPConfig` case per retry/timeout argument; each asserts the value reaches the plan's IO config and fails on `main`. native `tests/sql` 452 passed, new cases pass on ray, pre-commit clean. ## AI usage disclosure Written with an AI coding assistant. I reviewed the full diff and confirmed the tests fail without the fix. ## Related Issues Closes #7447. The `default_io_config` fallback is #7551.

main

6 hours ago

test(ai): cover custom embedding error without dimensions

pcbeingused333:fix/openai-embed-custom-model-dims

10 hours ago

style(ai): apply pinned Ruff formatter to usage metrics test

sunxiuguo:feat/thinking-token-metrics-7707

10 hours ago

fix(paimon): read catalog options from pypaimon Options objects (#7667) Fixes #7653 ### Root cause `read_paimon` read `table.file_io.properties` as a dict (`catalog_options.get("warehouse", "")`). Since pypaimon 1.4.1 that attribute is an `Options` object whose `get` takes a `ConfigOption`, so every read raised `AttributeError: 'str' object has no attribute 'key'`. ### Fix New helper `_get_catalog_options(table)` returns a plain dict, going through `Options.to_map()` when present and still accepting a dict (older pypaimon) or a missing `file_io`. `read_paimon` uses it for both the IO config inference and the `warehouse` passed to the scan. ### Test `test_get_catalog_options_accepts_options_object` covers an `Options`-like object, a plain dict, `None` and a missing `file_io`. The end-to-end `read_paimon` tests in `tests/io/paimon` run in CI against the pinned `pypaimon==1.3.1`, so they keep covering the dict path but do not reach the `Options` path; the new unit test is what covers it. ### Disclosure This change was prepared with the assistance of Claude, an AI coding agent. The helper's logic was exercised in isolation; the full paimon test suite was not run for this change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

main

10 hours ago

test(ai): keep process-local SDK mocks on the native runner

sunxiuguo:feat/thinking-token-metrics-7707

10 hours ago

test(iceberg): cover complex types in convert_iceberg_data_type (#7574) ## Changes Made `test_convert_iceberg_data_type` only covered the 8 primitive Iceberg types. This extends the parametrization to `timestamptz`, `decimal`, `time`, `binary`, `fixed`, `struct`, `list` and `map`, guarding the Iceberg→Daft dtype mapping for these column types against regressions (e.g. a decimal precision/scale or time-unit mislabel that would silently corrupt reads). `uuid` is left out: pyiceberg maps it to `binary(16)` up to 0.9 and to `arrow.uuid` from 0.11, so one expected value would pin a single pyiceberg version. Test-only; no behavior change. ## Testing native `pytest tests/io/iceberg/test_iceberg_metadata.py` 39 passed; pre-commit clean. ## Related Issues None. ## AI usage disclosure Written with an AI coding assistant. I reviewed the diff and confirmed each expected dtype by running the conversion locally before asserting it.

main

11 hours ago

fix: avoid panic when applying nulls to null-dtype arrays (#7686) ## Changes - `DataArray::with_nulls` now returns the array unchanged when its dtype is `Null`. Null arrays are entirely null and cannot carry an arrow validity buffer, so attaching one panicked inside arrow's `NullArray` constructor. - Struct field access (`StructArray::get`) unions the parent validity mask into the child, which triggered this path whenever every value of a struct field was null. ## Behavior ```python df = daft.from_pydict({"a": [{"c": None}, None]}) df.select(col("a")["c"]).to_pydict() # {"c": [None, None]} df.count(col("a")["c"]).to_pydict() # {"c": [0]} ``` Previously this panicked with `NullArray data should not contain a null buffer, as no buffers are required`. ## Tests - Rust regression test `test_struct_get_null_child_with_parent_validity` in `src/daft-core/src/array/ops/struct_.rs` - Python regression test `test_count_null_struct_field` in `tests/dataframe/test_count.py` - `cargo test -p daft-core --lib struct_` - `pytest tests/dataframe/test_count.py tests/recordbatch/struct tests/dataframe/test_to_struct.py` Closes #2689 Co-authored-by: xiyan.tr <xiyan.tr@alibaba-inc.com>

main

11 hours ago

Latest Branches

0%

fix(ai): support custom OpenAI-compatible embedding models with dimensions#7666

10 hours ago

d7b0c9d

pcbeingused333:fix/openai-embed-custom-model-dims

0%

feat(ai): report thinking tokens separately#7708

10 hours ago

79f6db1

sunxiuguo:feat/thinking-token-metrics-7707

0%

feat(pg): integrate bounded Daft streaming execution and PostgreSQL b…#7706

22 hours ago

a66009d

qiumingcheng:codex/migrate-pg19-orca-daft

© 2026 CodSpeed Technology

Home Terms Privacy Docs