Latest Results
Make expressions hold a function id and options, resolved at bind time
An `Expression::Scalar` node no longer holds a `ScalarFnRef`. It holds the id
of the scalar function it calls, the function's options with their type erased
(`ErasedOptions`), and its children. Building an expression therefore needs
neither a session nor the crate that defines the function. `Expression::new`
builds a node from an id and erased options; the vtable constructors
(`new_expr`, `try_new_expr`) still check the arity up front.
Binding resolves the function: `Expression::bind(dtype, session)` and
`Scope`, which now carries the session, look each node's id up in the
session's scalar function registry, build the function from the options
through `ScalarFnPlugin::bind_any`, and check the options type and the number
of children, producing a `BoundExpression` as before. Options read for an id
the session did not register bind to the foreign placeholder.
Consequences across the API:
- `ScalarFnVTable::bind_options` takes `Option<&VortexSession>`: `Some` when an
expression binds, so options that hold an expression, such as a lambda, can
bind it; `None` when a bound node is rebuilt or a `ScalarFnArray` is built.
`Lambda::bind` takes the session, and `Lambda::return_dtype` and
`Lambda::is_infallible` are gone, since neither can be answered unbound.
- `Expression::return_dtype` is gone; bind and read the dtype instead.
- `ExprSerializeProtoExt::serialize_proto` takes the session, which serializes
each function's options through the new `ScalarFnPlugin::serialize_any`.
- `ArrayRef::apply` takes the session and binds first.
`vx_array_apply` and the C++ `Array::apply` take a session too.
- `Expression::is`, `as_opt`, `as_` and `contains` need `V: Default` to know
the vtable's id, so every unit vtable derives `Default`.
- Unbound expressions display generically, as `id(child, ..., opts=options)`,
with numbered children in `display_tree`; binding gives each function its
own formatting. Tests that assert on display bind first.
- `DynamicComparison` and `RowCount` are registered by default, and the new
`vortex_layout::initialize` registers `RowIdx`, since binding now needs
every function an expression calls to be registered.
`ScalarFnSessionExt` gains `scalar_fn_by_id`, `scalar_fn_plugin` and
`resolve_scalar_fn`. The lazy expression modules from the previous commit are
removed, since every expression is now built this way.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EAanvjKHbMGyQyN2nQAwU
Signed-off-by: Claude <noreply@anthropic.com> feat(scan): run the V2 executor on the segment IO protocol
Switches the planner-driven scan from the byte-range `vortex_io::request`
protocol to `vortex_io::segments`, and serves it with a session-wide
`FileSegmentIo` that reads through files' v1 segment sources.
- vortex-scan: `Planner` and `Morsel` drop the `IoConsumer` supertrait,
publish `NeedsIO(Vec<IoRequest>)`, and receive `deliver(key, result)` once
per successful fetch. `Run`/`Driver` submit each root's requests to its
`SegmentIoSession` and hold the handles; each item's fetches finish through
its own `FuturesUnordered`, so a `poll_ready` report yields exactly the
finished fetches. `Run::poll_io` replaces `Run::complete`. The scripted
harness is a recording `SegmentIoSession`.
- vortex-layout: `ScanAdaptor` fetches each segment once by `IoKey::segment`
while any scan read of it is outstanding, and hands the result to every
waiting read. The filter planner announces each segment once. `ScanFile`
carries the file's `RegisteredSource`, and each split reads through a new
session of its service. `SegmentScanIo` and the cached byte-range index it
needed are removed.
- vortex-file: `FileSegmentIo`, one per `VortexSession`. Size requests are
answered at submit; segment requests call `SegmentSource::request`, so they
share the file's segment cache, request sharing and coalescing read driver.
`VortexFile::scan_file` registers the file's segment source, and a caching
file keeps one registration in its scan cache, renewed with its segment
source.
- vortex-datafusion: the V2 opener shares a file's segment source across
partitions through `VortexFile::with_segment_source`.
- vortex-io: `request` is removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4PmceDHg8rFe5bgkhGFU9
Signed-off-by: Claude <noreply@anthropic.com> feat(scan): run the V2 executor on the segment IO protocol
Switches the planner-driven scan from the byte-range `vortex_io::request`
protocol to `vortex_io::segments`, and serves it with a session-wide
`FileSegmentIo` that reads through files' v1 segment sources.
- vortex-scan: `Planner` and `Morsel` drop the `IoConsumer` supertrait,
publish `NeedsIO(Vec<IoRequest>)`, and receive `deliver(key, result)` once
per successful fetch. `Run`/`Driver` submit each root's requests to its
`SegmentIoSession` and hold the handles; each item's fetches finish through
its own `FuturesUnordered`, so a `poll_ready` report yields exactly the
finished fetches. `Run::poll_io` replaces `Run::complete`. The scripted
harness is a recording `SegmentIoSession`.
- vortex-layout: `ScanAdaptor` fetches each segment once by `IoKey::segment`
while any scan read of it is outstanding, and hands the result to every
waiting read. The filter planner announces each segment once. `ScanFile`
carries the file's `RegisteredSource`, and each split reads through a new
session of its service. `SegmentScanIo` and the cached byte-range index it
needed are removed.
- vortex-file: `FileSegmentIo`, one per `VortexSession`. Size requests are
answered at submit; segment requests call `SegmentSource::request`, so they
share the file's segment cache, request sharing and coalescing read driver.
`VortexFile::scan_file` registers the file's segment source, and a caching
file keeps one registration in its scan cache, renewed with its segment
source.
- vortex-datafusion: the V2 opener shares a file's segment source across
partitions through `VortexFile::with_segment_source`.
- vortex-io: `request` is removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4PmceDHg8rFe5bgkhGFU9
Signed-off-by: Claude <noreply@anthropic.com> Add lazy expressions that name a scalar function by id
`lazy(name, options, children)` builds an expression node that holds only the
function id and its options, so it needs neither a session nor the crate that
defines the function. `Expression::resolve(session)` later replaces each such
node with the function registered under that id, built from the options through
`ScalarFnPlugin::bind_any`.
A lazy node is a scalar function vtable, `LazyScalarFn`, like the foreign
placeholder: it reports the target function's id so it displays and hashes like
that function, accepts any arity, and fails to bind, execute, or serialize with
an error asking for resolution. `LazyScalarFnOptions` type-erases the options
behind a trait dispatching equality and hashing by concrete type, so any
`ScalarFnVTable::Options` type fits.
Unknown names, options of the wrong type, and arity mismatches are all reported
by `resolve`. `ScalarFnSessionExt::scalar_fn_by_id` looks a function up by an
already interned id; `scalar_fn(name, ...)` delegates to it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EAanvjKHbMGyQyN2nQAwU
Signed-off-by: Claude <noreply@anthropic.com> Latest Branches
-20%
+23%
XiangpengHao:dict-max-bytes -26%
© 2026 CodSpeed Technology