Latest Results
feat: migrate Azure IO to GA azure_storage_blob SDK, drop patched fork (#7368)
## Changes Made
Prototype migration of `daft-io`'s Azure Blob source from our patched
fork of the deprecated `azure-sdk-for-rust` crates
(`Eventual-Inc/azure-sdk-for-rust` @ `patched-azure-identity-2`) to the
official, GA SDKs:
- `azure_core` 1.1.0, `azure_identity` 1.0.0, `azure_storage_blob`
1.1.0-beta.1 (the beta is needed for `list_blobs_hierarchical`, i.e.
delimiter/posix-style listing; everything else is on the 1.0 GA surface)
The GA SDK only supports Entra ID `TokenCredential` auth natively, so a
new `azure_auth` module preserves all of Daft's existing `AzureConfig`
credential paths:
- **`access_key` (SharedKey)**: `SharedKeyAuthorizationPolicy`, a
per-try pipeline policy ported from the old
`azure_storage::authorization_policy`. It sets a fresh `x-ms-date` per
attempt and signs the request with one local HMAC-SHA256 — no extra
round-trips, and no signing when a SAS `sig` is already present. This
keeps the documented `AzureConfig(access_key=...)` path and local
azurite testing working.
- **`sas_token`**: `SasTokenPolicy`, a per-try policy that appends the
SAS query parameters.
- **`bearer_token`**: `StaticBearerCredential`, a tiny `TokenCredential`
wrapper. This is the path Fabric/OneLake users rely on
(`use_fabric_endpoint=True` + `notebookutils.credentials.getToken`).
- **`tenant_id`/`client_id`/`client_secret`**: upstream
`ClientSecretCredential`.
- **default**: a probe chain of `WorkloadIdentityCredential` →
`ManagedIdentityCredential` → `DeveloperToolsCredential`, falling back
to anonymous with a warning (mirrors the old `DefaultAzureCredential`
behavior).
### Why
- The upstream blocker (Azure/azure-sdk-for-rust#2504) was closed on
2026-08-05: `azure_storage_blob` is now GA and officially supported.
- The GA `azure_identity` properly re-reads
`AZURE_FEDERATED_TOKEN_FILE`, which permanently retires the
workload-identity token caching patch from #5269 — we no longer need to
maintain the fork.
### Notable behavior notes
- `BlobClient::download` in the new SDK supports partitioned parallel
downloads; this prototype keeps the previous single-stream semantics.
- Listing still supports flat and delimiter (posix) modes; page
iteration and continuation are handled by the SDK's `PageIterator`.
- The new SDK enforces HTTPS when a token credential is attached;
SharedKey/SAS (pipeline-policy) paths still work over HTTP for azurite.
### Testing
- `cargo check -p daft-io` and `cargo clippy -p daft-io` pass.
- New unit tests for the SharedKey string-to-sign canonicalization, SAS
parsing, and static bearer credential.
- Not yet tested against live Azure/azurite — this is a prototype for
discussion. Azure integration tests (azurite-based delta-lake suite)
should be run before landing.
Context: #5269
Made with [Cursor](https://cursor.com)
---------
Co-authored-by: Desmond Cheong <desmond@eventualcomputing.com>
Co-authored-by: Srinivas Lade <srinulade1@gmail.com> Latest Branches
+1%
slade/upgrade-rust-nightly 0%
desmond/azure-sdk-ga-migration 0%
© 2026 CodSpeed Technology