Latest Results
docs: remove stale TableState references (#7457)
## Changes Made
Follow-up cleanup to #5710, which removed `TableState` (and with it the
`Unloaded` / `Loaded` distinction) from `MicroPartition`.
`MicroPartition` now holds `chunks: Arc<Vec<RecordBatch>>` and is always
materialized, but three references to the old lazy design survived and
are misleading to anyone reading them today.
**1. `DataFrame.iter_partitions` docstring showed a line the code no
longer prints.**
The example output contained `TableState: Loaded. 1 tables`. `Display
for MicroPartition`
(`src/daft-micropartition/src/micropartition.rs:761-779`) emits only the
row count, each record batch, and then the statistics — there is no
`TableState` line. Removed it.
The same example also called `.into_partitions(2)` while showing a
single 3-row partition, so the printed output could not have come from
the code above it. Dropped the `.into_partitions(2)` call, which makes
the shown output correct for the single partition that `from_pydict`
produces.
The output block stays `# doctest: +SKIP` — it has to, because under the
Ray runner `iter_partitions` yields `ray.ObjectRef` rather than
`MicroPartition`, so the block is not assertable across runners. (That
is also why this drifted unnoticed.)
**2. `tests/conftest.py` fixture comment claimed a behavior that no
longer exists.**
The `parquet` param was annotated `will trigger "Unloaded"
MicroPartitions`. The param is still useful — it exercises the Parquet
scan path rather than in-memory Arrow — so this is reworded rather than
removed.
**3. Comment in `read_parquet_into_micropartition` described a branch
that is no longer there.**
The comment read `// If no TableStatistics are available, we perform an
eager read`, implying statistics select between a lazy and an eager
path. The eager read at that point is now unconditional; the only early
return above it is the count pushdown that answers from Parquet footer
metadata. Reworded to say that.
Comments and a docstring only — no behavior change, no public API
change.
## Related Issues
No open issue. Follows up on #5710 (`refactor: Remove Unloaded
MicroPartitions`), which is where these references went stale.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Cory Grinstead <cory.grinstead@gmail.com> fix(parquet): error instead of nulling columns when field ids are absent (#7446)
## Changes Made
A parquet file with no field IDs read through a field ID mapping
produces all-null columns:
```python
daft.read_iceberg(table).to_pydict() # {'a': [None, None, None], 'b': [None, None, None]}
pyiceberg_table.scan().to_arrow() # {'a': [1, 2, 3], 'b': ['x', 'y', 'z']}
```
`apply_field_ids_to_arrowrs_parquet_metadata` drops any field whose ID
is absent from the
mapping, along with its column chunks. With zero field IDs in the file
every column is
dropped and the reader nulls them. This raises `MissingParquetFieldIds`
instead. Partially
annotated files keep the existing drop-unmapped behavior that nested
structs rely on.
This covers pyiceberg's third case only (no IDs, no name mapping →
raise); supporting
`schema.name-mapping.default` is left out on purpose.
## Testing
- New test asserting the error, plus a positive test that a file
carrying
`PARQUET:field_id` still reads its values. The error test fails on
`main`.
- `cargo test -p daft-parquet --release`: 15 passed. `DAFT_RUNNER=native
pytest
tests/io/test_native_tasks.py tests/io/test_parquet.py`: 48 passed.
`DAFT_RUNNER=ray` on
the new file: 11 passed. `make precommit` clean.
## AI usage disclosure
Written with an AI coding assistant. I reviewed the full diff,
reproduced the all-null read
and the new error locally, read pyiceberg's dispatch to confirm which
case this covers, and
verified by reverting the guard that the new test fails without it.
## Related Issues
Closes #7445 Latest Branches
0%
0%
MeteorSkyOne:docs/remove-stale-tablestate-references 0%
© 2026 CodSpeed Technology