Avatar for the vortex-data user
vortex-data
vortex
BlogDocsChangelog

Performance History

Latest Results

Fix review findings in list_transform Correctness: - A fallible body is no longer evaluated over element positions that no visible list references. ListView inputs (gaps, overlaps) and Lists with null rows (whose ranges Arrow permits to hold arbitrary values) now gather exactly the referenced elements into a compact List before applying the body. Infallible bodies keep the zero-cost structure-preserving path. - Zero-row batches short-circuit before the constant fast path, which could eagerly evaluate a fallible body over elements no row observes. - Fusion now collapses a whole transform chain in one rewrite (deep chains previously exhausted the optimizer's per-node iteration budget and errored on valid expressions) and skips bodies with more than one root reference (substitution previously duplicated the inner body per occurrence, compounding exponentially across fusion steps). - The identity rewrite moved to the typed simplify hook, gated on the input actually being list-typed, so ill-typed expressions still fail return_dtype instead of being rewritten into well-typed non-list ones. - serialize() now honors the vtable contract by returning Ok(None) when the body contains a non-serializable fn, via a new Expression::try_serialize_proto helper. Also: - The typed simplify hook optimizes the body against its element scope, since the options-embedded body is invisible to the optimizer's children traversal. - is_fallible uses a short-circuiting walk instead of a full label_tree labeling per call. - fmt_sql reuses the options Display impl so the lambda rendering lives in one place. - Regression tests for all of the above, including one pinning that the null-scalar branch protects fallible bodies from a null FixedSizeList constant's default-filled placeholder elements. - Test hygiene: imports at the top of the tests module, vortex_bail instead of panic, removed an assertion subsumed by assert_arrays_eq. Signed-off-by: Matt Katz <mhkatz97@gmail.com>
mk/list-transform
3 hours ago

Latest Branches

CodSpeed Performance Gauge
+13%
Add list_transform expression#8663
8 hours ago
c915742
mk/list-transform
CodSpeed Performance Gauge
+12%
6 hours ago
0c8d2ea
rk/buildertakefixes
CodSpeed Performance Gauge
+33%
© 2026 CodSpeed Technology
Home Terms Privacy Docs