Latest Results
fix(io): parse container@host Azure URIs on Microsoft Fabric / OneLake hosts (#7533)
## Changes Made
`parse_azure_uri` in `src/daft-io/src/azure_blob.rs` only recognises the
`PROTOCOL://container@host/path` form when the host ends in
`.dfs.core.windows.net`. OneLake paths are
`abfss://<workspace>@onelake.dfs.fabric.microsoft.com/<lakehouse>/...`,
so they fall into the generic `else` branch: the container becomes
`onelake.dfs.fabric.microsoft.com` and the key `/<lakehouse>/...`. With
`AzureConfig(use_fabric_endpoint=True)` the client is built on
`https://onelake.blob.fabric.microsoft.com`, so the request goes to
`https://onelake.blob.fabric.microsoft.com/onelake.dfs.fabric.microsoft.com//<lakehouse>/...`
and OneLake rejects it (`400 FriendlyNameSupportDisabled` for GUID
workspaces, `404` for friendly names as in #5187).
This PR accepts `.dfs.fabric.microsoft.com` as a second known host
suffix and derives the account name (`onelake`) from it the same way as
for `.dfs.core.windows.net`, which is exactly what `use_fabric_endpoint`
expects (`https://{storage_account}.blob.fabric.microsoft.com`). The
`PROTOCOL://container/path` and `.dfs.core.windows.net` paths are
unchanged.
This is the URI form `docs/connectors/azure.md` already documents under
"Connect to Microsoft Fabric/OneLake", and the form the OneLake Iceberg
REST catalog writes into table metadata, so `read_iceberg` on OneLake
hits the same code path. Every other Azure-aware reader we run against
the same URIs (object_store via Polars and DataFusion, DuckDB, Spark,
Sail) accepts it.
Evidence that only the parsing is at fault (Daft 0.7.25, same bearer
token from `az account get-access-token --resource
https://storage.azure.com`):
-
`daft.read_csv(["abfss://<ws>@onelake.dfs.fabric.microsoft.com/<lh>/Files/csv/X.CSV"],
io_config=IOConfig(azure=AzureConfig(storage_account="onelake",
bearer_token=..., use_fabric_endpoint=True)))` → `DaftCoreException:
Unable to open file ... HttpError { Status: 400, Error Code:
FriendlyNameSupportDisabled }`
- `HEAD
https://onelake.blob.fabric.microsoft.com/<ws>/<lh>/Files/csv/X.CSV` →
200
- `GET
https://onelake.blob.fabric.microsoft.com/<ws>?restype=container&comp=list&prefix=<lh>/Files/csv/&delimiter=/`
→ 200
- `GET .../X.CSV` with `Range: bytes=0-99` → 206
- Reading the same file as `az://<ws>/<lh>/Files/csv/X.CSV` with the
same `AzureConfig` works, which is the workaround this PR removes the
need for.
Tests: adds a `tests` module to `azure_blob.rs` with three unit tests
for `parse_azure_uri` (OneLake `container@host`, ADLS `container@host`,
bare `container/path`); there were none before.
AI usage: drafted with Claude Code. The parse change and unit tests were
verified by running this repository's CI (`style`, `rust-tests`) on my
fork; the endpoint evidence above was verified by hand with curl against
a real OneLake workspace.
## Related Issues
Fixes #5187
Related #4692
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> perf(optimizer): rewrite starts_with filters into range predicates
Rework per review: instead of teaching the stats layer about starts_with,
add a RewriteStartsWith optimizer rule that rewrites starts_with(c, P) to
c >= P AND c < incr(P) inside Filter predicates, running right after
expression simplification and before PushDownFilter.
- The scan expression rewriter classifies any ScalarFn as a UDF and strands
a lone starts_with filter in a residual Filter op; plain comparisons are
data predicates, so they land in pushdowns.filters and every source can
push them down natively (Parquet, CSV, Lance, Iceberg, ...).
- Existing Utf8 min/max statistics pruning then skips row groups and scan
tasks for free, with no starts_with knowledge needed in daft-stats.
- Only Filter predicates are rewritten: projections keep the single,
cheaper kernel call.
- Arguments are bound by name (keyword-arg calls in any order are safe);
an all-max-scalar prefix falls back to a lower-bound-only rewrite;
empty-prefix and non-literal patterns are left untouched.
Replaces the daft-stats/daft-parquet approach with optimizer plan-shape
tests and a parquet e2e test asserting the pushed-down range predicate.hello-peter-tang:startswith-stats-pruning Latest Branches
0%
jonasdedden:fix/var-stddev-chan-merge 0%
DogerW666:fix/split-part-broadcast 0%
DogerW666:fix/bisect-round-robin-bundles © 2026 CodSpeed Technology