Avatar for the Eventual-Inc user
Eventual-Inc
Daft
BlogDocsChangelog

Performance History

Latest Results

docs: explain lazy execution in architecture flow (#6994) ## Summary Updates the architecture doc to make lazy execution explicit within the existing Planning, Optimization, and Execution flow. - Renames the first stage from API to Planning for consistency with the page structure. - Notes in Planning that DataFrame and SQL APIs construct a logical plan without reading or transforming data yet. - Defines materializing operations in Execution and lists examples such as `collect()`, `show()`, `to_arrow()`, and `write_parquet()`. - Removes the standalone lazy execution section so the explanation stays integrated with the architecture stages. ## Impact This gives contributors and advanced users a clearer mental model for when a query starts executing while keeping the architecture doc organized around its main stages. ## Validation - `make .venv` - `uv pip install --python .venv/bin/python -e docs/plugins/nav_hide_children` - `PYTHONPATH=. JUPYTER_PLATFORM_DIRS=1 .venv/bin/mkdocs build -f mkdocs.yml` Note: The direct MkDocs build uses `PYTHONPATH=.` to validate docs from the source tree without compiling the local Rust extension. --------- Co-authored-by: Colin Ho <colin.ho99@gmail.com>
main
27 minutes ago
address greptile
slade/dataloader
2 hours ago
docs: collapse sidebar nav and group Guide sections (#7131) # Me: I found the current front page a bit overwhelming because the sidebar is fully expanded and there's a lot of stuff in there, requiring a decently long scroll to hit the bottom. This also distracts me from reading the content in the intro. before: <img width="1680" height="982" alt="Screenshot 2026-06-15 at 1 54 56β€―PM" src="https://github.com/user-attachments/assets/a63e5f87-7d3f-40eb-8b71-5807aa59bfe6" /> after: <img width="1675" height="988" alt="Screenshot 2026-06-15 at 1 54 43β€―PM" src="https://github.com/user-attachments/assets/d783f6bd-d9de-4c3f-a8a8-3ef0c78cc0bf" /> As a drive by I also refactored the sections a bit, namely added subsections to data connectors (there's a whole lot of them), and appropriately modified the data connectors overview page too. # Claude: ## Changes Made The docs sidebar previously expanded every section at once (~40 links on first load). This makes the Guide nav reveal depth gradually: - Remove `navigation.expand` so sections render collapsed and auto-expand only the branch matching the current page. - Group scaling / optimization under a **Scaling & Performance** section, and the trailing project links (Roadmap, Benchmarks, Community, Release Notes, Usage Telemetry) under **Resources**. - Sub-group the 26 **Data Connectors** pages by source type (Object Storage, Table Formats, Catalogs, Databases, Files, Other Sources, Custom) so an expanded section shows a handful of categories instead of one long list. - Keep **Architecture** and **Observability** as top-level entries. The **Data Connectors overview page** is reorganized to mirror these same categories and order, so the on-page structure matches the sidebar. This also surfaces connectors that were missing from the overview (Turbopuffer, Hugging Face Datasets, and the catalog integrations). No pages or links were removed β€” this only regroups the existing nav and content. Verified with a local `mkdocs build` (no new cross-reference warnings). ## Related Issues Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
main
4 hours ago
Merge branch 'main' into main
DieselZhang:main
5 hours ago
perf(inline-agg): add Product accumulator type (#6975) ## Summary Implements the Product accumulator from #6585 (item 7) for the inline grouped aggregation path. Mirrors Sum's shape: widens Int8/16/32 β†’ Int64, UInt8/16/32 β†’ UInt64, keeps Float32/Float64 native. Reduce op is `*` instead of `+`; the per-group `Option<T>` state and `init_groups` / `update_batch` / `finalize` methods are basically the same to `SumAccum*`. Overflow semantics match Sum exactly: native `*` (wraps silently in release mode), consistent with the fallback `Series::product()` implementation. Grouping semantics and final query results are unchanged. ## Why `AggExpr::Product` already exists in the DSL and is wired in the fallback path (`src/daft-recordbatch/src/lib.rs`), but currently falls back to `make_groups + eval_agg_expression` even when the rest of the query qualifies for the inline path. Adding Product to the inline accumulator framework completes coverage of the standard reducer-style aggregates (Count / Sum / Min / Max / Product) that all share the same `Vec<Option<T>>` per-group state shape. - Closes the Product after #6604 added Min/Max. ## Changes Made - `src/daft-recordbatch/src/ops/inline_agg.rs`: - New `define_product_accum!` macro generating `ProductAccumI64` / `ProductAccumU64` / `ProductAccumF32` / `ProductAccumF64`, byte-identical to `define_sum_accum!` except the reduce op (`*` vs `+`). - `define_agg_accumulator_enum!` extended with `ProductI64` / `ProductU64` / `ProductF32` / `ProductF64` variants. - `try_create_accumulator` dispatches `AggExpr::Product(expr)` with the same numeric dtype widening as Sum. - `can_inline_agg` accepts Product on the same numeric dtype set as Sum. - 5 new tests + 1 helper. ## Behavior - Queries with `Product` over Int8/16/32/64/UInt8/16/32/64/Float32/Float64 columns now take the inline path instead of falling back to `make_groups + eval_agg_expression`. - Output values identical to the fallback path (verified by inline-vs-fallback tests). - All other agg types and dispatch paths are unchanged. ## Test Plan - `cargo test -p daft-recordbatch --release inline_agg` β€” 37 passed (32 pre-existing + 5 new Product). - `cargo fmt -p daft-recordbatch --check` β€” clean. - `cargo clippy -p daft-recordbatch --release --features python` β€” clean, no `#[allow]`s added. New test cases: - `test_inline_product_matches_fallback` β€” Utf8 keys + Int64 vals (exercises null-value branch). - `test_inline_int_key_product_matches_fallback` β€” Int64 keys + Int64 vals (FNV int-key fast path). - `test_inline_int_key_with_nulls_product_matches_fallback` β€” null keys. - `test_inline_product_float_matches_fallback` β€” Float64 vals (finite values chosen to avoid overflow ambiguity). - `test_inline_all_null_vals_product_matches_fallback` β€” all-null vals (exercises empty `Option<T>` finalize path). ## Related Issues - Part of #6585 (item 7).
main
7 hours ago
lerobot with formatting changes
slade/lerobot
15 hours ago

Latest Branches

CodSpeed Performance Gauge
0%
feat: Add `DataFrame.to_torch_dataloader`#6997
4 hours ago
e2edfac
slade/dataloader
CodSpeed Performance Gauge
0%
CodSpeed Performance Gauge
0%
Β© 2026 CodSpeed Technology
Home Terms Privacy Docs