Latest Results
chore(deps): upgrade daft-lance to 0.5.0 (#7512)
## Changes Made
Upgrades `daft-lance` from 0.3.2 to 0.5.0. Dependency upgrade only — the
change is scoped to repairing what 0.5.0 breaks. Lance Namespace
support, also added in 0.5.0, is left to a follow-up PR.
### Breaking changes in daft-lance 0.5.0
Daft's `daft/io/lance/` modules are thin re-export shims over
`daft-lance`, so two upstream changes land directly on them:
| API | 0.3.2 | 0.5.0 |
| --- | --- | --- |
| `daft_lance.utils` | `construct_lance_dataset()` |
`construct_lance_dataset_handle()` |
| `merge_columns_from_df` | `uri=`, `storage_options=` | `open_context:
DatasetOpenContext` |
The rename breaks importing `daft.io.lance.utils` at all, so the
re-export follows it. For `merge_columns_from_df`,
`DataFrame.write_lance(mode="merge")` builds the context with
`DatasetOpenContext.from_dataset()` from the dataset it has already
opened — verified to produce a value identical to the handle-based path
daft-lance uses internally, including the pinned version that keeps
merge workers on the snapshot the driver planned against.
A third change, `LanceDataSink` dropping its `**kwargs` catch-all for an
explicit keyword-only parameter list, needs no code change, but it does
make the `write_lance` docstring inaccurate — unknown keyword arguments
now raise `TypeError` rather than reaching `lance.write_fragments` — so
that wording is corrected.
`LanceDBScanOperator`, `_lancedb_table_factory_function`,
`_lancedb_count_result_function`, `detect_point_lookup_columns`,
`compact_files_internal`, `create_scalar_index_internal` and
`merge_columns_internal` are unchanged or extended compatibly. Nothing
on the Rust side depends on a `lance` crate —
`src/daft-writers/src/lance.rs` reaches Python through pyo3 — so this is
a pure Python dependency upgrade.
### Transitive dependency moves
`daft-lance` 0.5.0 requires `pylance>=8.0.0`, which resolves to pylance
11.0.0. Every pylance from 8 up requires `lance-namespace>=0.8.5,<0.9`,
so `lance-namespace` and `lance-namespace-urllib3-client` move from
0.7.6 to 0.8.6 regardless of which pylance is selected. No explicit
`pylance` pin is added; `daft-lance` is pinned to `==0.5.0` in the dev
group.
### The `lance` extra now has a bound
`lance = ["daft-lance"]` becomes `lance = ["daft-lance>=0.5.0,<0.6.0"]`
— it was the only extra in `pyproject.toml` with no version constraint
at all.
The floor is load-bearing rather than defensive:
`construct_lance_dataset_handle` does not exist in 0.3.2, 0.3.3 or
0.4.0, and `merge_columns_from_df` took `uri`/`storage_options` through
0.4.0. Without a floor, a fresh `pip install daft[lance]` resolves the
latest and works, but an environment already holding an older
`daft-lance` — pinned, cached, or constrained by something else —
installs cleanly and then fails at write time.
The ceiling matches how the other extras pin dependencies that break
compatibility (`deltalake >= 1.5.0,< 1.7.0`, `ray[data,
client]<2.56.0,>=2.11.0`). daft-lance is 0.x and renamed public API in a
minor release, which is exactly the situation this PR is cleaning up;
capping means an eventual 0.6.0 cannot repeat it silently, at the cost
of a version-bump PR per daft-lance minor.
### Test adjustments
These follow from the upgrade rather than from any change here:
- `test_lance_batching` calls `LanceDataSink.start()` before writing,
since 0.5.0 resolves the target table there. The "no accumulation when
the parameter is missing" case no longer describes anything:
`max_rows_per_file` always has a value in 0.5.0, so buffering is
unconditional. It is replaced by a test of the default-threshold
behaviour.
- `test_lancedb_scalar_index` reads the index type through
`stats.index_stats()`. `list_indices()` is deprecated in pylance 11 and
reports `"Unknown"` for distributed-committed indices. The indices
themselves are fine — full-text search returns the expected rows and
`index_stats()` reports `"Inverted"` — so this looks like a gap in how
daft-lance's distributed commit path populates index metadata that
pylance 11 reads, worth a separate issue upstream.
## Verification
- `tests/io/lancedb/` — 130 passed, 2 skipped
- `make doctests` — 332 passed
- `write_lance(mode="merge")` checked end to end for schema evolution,
for the fallback that creates a dataset that does not exist yet, and for
the retained `rest://` rejection
- `lance.fragment.write_fragments`, reached from the Rust `LanceWriter`
through `daft/recordbatch/recordbatch_io.py`, still accepts `mode` and
`storage_options` under pylance 11
## Related Issues
<!-- Link to related GitHub issues, e.g., "Closes #123" -->
https://claude.ai/code/session_015m3rs8e9VphFoghPrw4ekN Latest Branches
0%
0%
nyl3532016:codex/openai-embedding-extra-body 0%
hello-peter-tang:remove-daft-udf © 2026 CodSpeed Technology