Avatar for the Eventual-Inc user
Eventual-Inc
Daft
BlogDocsChangelog

Performance History

Latest Results

feat: deltalake metadata pushdowns (#7549) ## Changes Made This PR adds support for count pushdowns into deltalake metadata. It does this by finding all stats columns that are in the metadata and extracting their expressions to pyarrow compute expressions, then filtering the metadata pa.Table There's a drive-by fix to daft-hash which I needed to get things to compile. ## Related Issues - Closes #3669 - Closes #3759
main
15 hours ago
fix(utf8): broadcast split_part over all arguments
DogerW666:fix/split-part-broadcast
1 day ago
Merge branch 'main' into fix/bisect-round-robin-bundles
DogerW666:fix/bisect-round-robin-bundles
1 day ago
Merge branch 'main' into remove-daft-udf
hello-peter-tang:remove-daft-udf
2 days ago
fix(io): parse container@host Azure URIs on Microsoft Fabric / OneLake hosts (#7533) ## Changes Made `parse_azure_uri` in `src/daft-io/src/azure_blob.rs` only recognises the `PROTOCOL://container@host/path` form when the host ends in `.dfs.core.windows.net`. OneLake paths are `abfss://<workspace>@onelake.dfs.fabric.microsoft.com/<lakehouse>/...`, so they fall into the generic `else` branch: the container becomes `onelake.dfs.fabric.microsoft.com` and the key `/<lakehouse>/...`. With `AzureConfig(use_fabric_endpoint=True)` the client is built on `https://onelake.blob.fabric.microsoft.com`, so the request goes to `https://onelake.blob.fabric.microsoft.com/onelake.dfs.fabric.microsoft.com//<lakehouse>/...` and OneLake rejects it (`400 FriendlyNameSupportDisabled` for GUID workspaces, `404` for friendly names as in #5187). This PR accepts `.dfs.fabric.microsoft.com` as a second known host suffix and derives the account name (`onelake`) from it the same way as for `.dfs.core.windows.net`, which is exactly what `use_fabric_endpoint` expects (`https://{storage_account}.blob.fabric.microsoft.com`). The `PROTOCOL://container/path` and `.dfs.core.windows.net` paths are unchanged. This is the URI form `docs/connectors/azure.md` already documents under "Connect to Microsoft Fabric/OneLake", and the form the OneLake Iceberg REST catalog writes into table metadata, so `read_iceberg` on OneLake hits the same code path. Every other Azure-aware reader we run against the same URIs (object_store via Polars and DataFusion, DuckDB, Spark, Sail) accepts it. Evidence that only the parsing is at fault (Daft 0.7.25, same bearer token from `az account get-access-token --resource https://storage.azure.com`): - `daft.read_csv(["abfss://<ws>@onelake.dfs.fabric.microsoft.com/<lh>/Files/csv/X.CSV"], io_config=IOConfig(azure=AzureConfig(storage_account="onelake", bearer_token=..., use_fabric_endpoint=True)))` → `DaftCoreException: Unable to open file ... HttpError { Status: 400, Error Code: FriendlyNameSupportDisabled }` - `HEAD https://onelake.blob.fabric.microsoft.com/<ws>/<lh>/Files/csv/X.CSV` → 200 - `GET https://onelake.blob.fabric.microsoft.com/<ws>?restype=container&comp=list&prefix=<lh>/Files/csv/&delimiter=/` → 200 - `GET .../X.CSV` with `Range: bytes=0-99` → 206 - Reading the same file as `az://<ws>/<lh>/Files/csv/X.CSV` with the same `AzureConfig` works, which is the workaround this PR removes the need for. Tests: adds a `tests` module to `azure_blob.rs` with three unit tests for `parse_azure_uri` (OneLake `container@host`, ADLS `container@host`, bare `container/path`); there were none before. AI usage: drafted with Claude Code. The parse change and unit tests were verified by running this repository's CI (`style`, `rust-tests`) on my fork; the endpoint evidence above was verified by hand with curl against a real OneLake workspace. ## Related Issues Fixes #5187 Related #4692 Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
main
2 days ago
fix(iceberg): match evolved partitions by source and transform
atovk:fix/iceberg-partition-evolution-row-filter
2 days ago

Latest Branches

CodSpeed Performance Gauge
0%
feat: deltalake metadata pushdowns#7549
16 hours ago
9231492
rchowell/delta-pushdowns
CodSpeed Performance Gauge
0%
fix(expr): stable var/stddev via Chan parallel merge#7474
21 hours ago
159d991
jonasdedden:fix/var-stddev-chan-merge
CodSpeed Performance Gauge
0%
1 day ago
75848e8
DogerW666:fix/split-part-broadcast
© 2026 CodSpeed Technology
Home Terms Privacy Docs