vortex-data/vortex - CodSpeed

vortex

Blog Docs Changelog

Performance History

Latest Results

Fix lambda expression integration Signed-off-by: Matt Katz <mhkatz97@gmail.com>

mk/lambda-variables

6 hours ago

Reduce number of clones and drops in expression optimisation logic Signed-off-by: Robert Kruszewski <github@robertk.io>

rk/exproptimise

7 hours ago

Rebuild the Python runtime after fork `fork(2)` copies only the calling thread, so a forked child inherits a `CurrentThreadRuntime` whose worker threads no longer exist: the executor's sleeper list and the pool's handle list describe phantom threads, so `set_workers` believes it already has enough workers and spawns none, and every Vortex operation in the child blocks forever. The same applies to the process-global `blocking` pool behind `spawn_blocking`, whose inherited `idle_count` keeps it from growing. Tag the runtime with its owning pid and build a fresh one on first use from a different process, rather than trying to repair the inherited state; the stale state is leaked deliberately, since running its destructors would take locks that may not have survived the fork. An `os.register_at_fork(after_in_child=)` handler does the rebuild on the child's single-threaded startup path and warms the blocking pool. The shared session is repointed at the new executor, and the requested worker count is held outside the runtime so a child inherits it. `VortexFile` gains `path` and `__reduce__`, reopening by path in the receiving process, and the `datasets` filter path no longer needs its in-memory fallback now that a filter can be pickled into `num_proc` workers. Signed-off-by: Robert Kruszewski <github@robertk.io>

rk/python-fork-runtime

7 hours ago

Expand the Python expression API Expose the rest of the Vortex expression builders to Python: comparisons and arithmetic as free functions, boolean collection helpers, string predicates, struct/list operations, conditionals, and `variant_get`. Binary operators now coerce any scalar-convertible Python value on either side, so `1 < ve.column("a")` works via the reflected `__r*__` methods. Expressions also gain `serialize`/`vortex.expr.deserialize` and `__reduce__`, backed by the protobuf wire format, so they can cross a process boundary. `ByteLength`, `Mask` and `Zip` implemented serialization but were missing from `ScalarFnSession::default()`, so they serialized and then failed to deserialize with "unknown expression id"; register them and cover the round trip. Signed-off-by: Robert Kruszewski <github@robertk.io>

rk/python-expr-api

7 hours ago

less Signed-off-by: Robert Kruszewski <github@robertk.io>

rk/deprecate-arrow-arrays

8 hours ago

Add BoolArray::trim_bits, wire it to Canonical::compact that prunes unused bytes in the underlying buffer Signed-off-by: Robert Kruszewski <github@robertk.io>

rk/boolcompact

8 hours ago

Don't store min/max_is_truncated in zone maps for non-string columns (#9206) Non-string columns zone map values can't be truncated. Don't write them. Resolves: #7235

develop

8 hours ago

bench(arrow): cover nested byte operators Signed-off-by: cl <cailue@apache.org>

ClSlaid:feat/offset-array/min-materialization

9 hours ago

Latest Branches

+13%

`Lambda` and `Variable` `Expression`s and `BoundExpression`s#9257

6 hours ago

893ab28

mk/lambda-variables

+4%

Reduce number of clones and drops in expression optimisation logic#9378

7 hours ago

6597326

rk/exproptimise

-6%

Rebuild the Python runtime after fork#9376

2 days ago

0f621f8

rk/python-fork-runtime

© 2026 CodSpeed Technology

Home Terms Privacy Docs