Eventual-Inc/Daft

Eventual-Inc

Daft

Blog Docs Changelog

Performance History

Latest Results

Merge branch 'main' into issue-6901

Lucas61000:issue-6901

56 minutes ago

fix(rotation): reject inputs that are not rotations instead of returning a plausible value Three problems, all of the same shape: an input that is not a rotation produced a number rather than a null. rotation_geodesic_angle clamps the cosine into [-1, 1] so that rounding cannot produce NaN for nearly identical frames. That clamp also swallowed nonsense: a matrix of [1..9] against the identity gives a cosine of 7, which clamped to 1 and reported an angle of exactly 0, meaning "no rotation". It now returns null when the cosine lies more than a tolerance outside the range, and still clamps within it. matrix_to_quat returned finite garbage for a matrix outside SO(3): [1..9] gave [0.25, -0.5, 0.25, 2.0]. Shepperd's method yields a unit quaternion exactly when its input is a rotation, so the norm of the result is a cheap and complete test. Scaled matrices, reflections, and negated identity are all rejected. Non-finite inputs were handled inconsistently: quat_inverse returned null while quat_multiply, quat_rotate, and matrix_to_quat propagated NaN. Every expression now treats a row holding NaN or an infinity the way it treats a null, decided once where rows are read rather than in each function. The tolerance is 1e-6, which is far above f64 rounding and above the ~1e-7 cost of widening a Float32 rotation matrix, and far below any real degeneracy. Verified against 200 random rotations and against f32 inputs: no genuine rotation is rejected. Docstrings now state the null policy, and that the order argument is case-insensitive. Throughput is unchanged at 42M rows/s.

alejandro-soto-franco:feat/rotation-expressions

1 hour ago

Merge branch 'main' into issue-3957

Lucas61000:issue-3957

1 hour ago

Merge branch 'main' into issue-2786

Lucas61000:issue-2786

1 hour ago

feat(rotation): native quaternion, 6D, and SO(3) expressions for pose data Daft reads robot pose data (droid, lerobot, egodex, mcap) but stores rotations as untyped float columns, so composing, inverting, and comparing them requires hand-writing the mathematics in a UDF, where the sharp edges (precision at a half turn, NaN from arccos, quaternion component order) are easy to miss. Adds a rotation module to daft-functions with seven expressions: rot6d_to_matrix, quat_to_matrix, matrix_to_quat, quat_multiply, quat_inverse, quat_rotate, and rotation_geodesic_angle. Quaternion component order is explicit, defaulting to xyzw (ROS, tf2, scipy), with wxyz available for the scalar-first convention. matrix_to_quat uses Shepperd's method for stability near a half turn, and rotation_geodesic_angle clamps before arccos so rounding cannot produce NaN. Degenerate inputs, zero quaternions and parallel 6D vectors, yield null. No new dependencies. The expressions read the flat child of the fixed size list directly rather than iterating it, which yields Option<Series> and so allocates once per row. On one million rows this is 39.8M rows/s against 12.1M for an equivalent numpy UDF fed through arrow with zero copy.

alejandro-soto-franco:feat/rotation-expressions

3 hours ago

chore(deps): lower opt-level for opendal-service-oss release builds At opt-level=3 this crate does not finish after 36 minutes of CPU, spent single-threaded inside LLVM's SLP vectorizer (BoUpSLP::buildTreeRec). Holding opt-level=3 and passing -C llvm-args=-vectorize-slp=false brings it to 30s, so the pass is the cause rather than the optimization level. opt-level=2 disables the pass and the crate compiles in 3.17s. This mirrors the existing daft-parquet override, which documents the same pathology. Of the service crates that the opendal 0.55 to 0.57 upgrade (#7037) introduced, only oss is affected. Fixes #7247

alejandro-soto-franco:chore/opendal-oss-opt-level

3 hours ago

fix(dashboard): follow symlinks when copying assets across filesystems DirEntry::file_type does not follow symlinks, so a symlinked directory took the copy branch, where std::fs::copy rejects anything that is not a regular file. Path::is_dir does follow them, which matches std::fs::copy's own semantics. Verified across a real filesystem boundary (tmpfs to a separate btrfs subvolume) on a tree containing both a symlinked directory and a symlinked file: the previous code fails with InvalidInput, this copies all entries and removes the source.

alejandro-soto-franco:fix/dashboard-build-cross-device

4 hours ago

feat(catalog): support Gravitino Iceberg REST catalogs Load Gravitino Iceberg tables through PyIceberg's REST catalog when the table properties identify a REST backend, and keep Hadoop catalog loading as the fallback path. Pass Daft IOConfig-derived FileIO properties into the REST catalog path so object-store credentials are preserved, and add unit coverage for REST catalog detection, dispatch, and schema-qualified table loading. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

kyo-tom:support_iceberg_rest_catalog_on_gravitino

4 hours ago

Latest Branches

feat: add Avro file format read/write support#7009

2 hours ago

3f0f5e6

Lucas61000:issue-6901

feat(rotation): native quaternion, 6D, and SO(3) expressions for pose data#7251

2 hours ago

54b0b63

alejandro-soto-franco:feat/rotation-expressions

feat(sql): support TIMESTAMPTZ #7020

2 hours ago

5edf332

Lucas61000:issue-3957

Home Terms Privacy Docs