Latest Results
fix: fit the real estimator in the ray worker
Addresses the review on #731.
The training loop now builds and fits the library's own estimator in the
worker and checkpoints it, as lightgbm.dask's _train_part does, instead of
checkpointing a bare Booster and synthesizing a local estimator from it on
the driver. The checkpoint carries the estimator next to ray's standard
booster artifact, so RayTrainReportCallback keeps reporting per-iteration
metrics and get_model still works on the result.
The models are LGBMRegressor / XGBRegressor subclasses again, so clone()
and isinstance() work as they did on main, and all of the parameter
handling is the library's:
- lightgbm's objective aliases are honored rather than dropped by a
setdefault on the canonical name
- all 11 num_iterations aliases resolve, with lightgbm's own precedence
- the random_state -> seed translation is gone; xgb.train knows the name
- model_ keeps the user's params and the booster's scores, because it is
the estimator that was fitted rather than one rebuilt from native params
That removes get_params, set_params, _translate_params,
_pop_num_boost_round and the 11 private attribute writes of the graft.
Two fixes on top:
- lightgbm needs tree_learner=data_parallel and ray's network params, or
each worker trains an independent model on its own shard and rank 0's is
the one checkpointed, with no error anywhere. xgboost is unaffected;
XGBoostConfig wraps the loop in a CommunicatorContext.
- the booster's thread pool is sized from the worker's assigned CPUs, as
lightgbm_ray and xgboost_ray did, rather than from every core on the node.
xgboost keeps callbacks as a parameter, so the ray reporting callback is
cleared before the estimator is reported; otherwise it would ride back to
the driver and into DistributedMLForecast.save's pickle. fix(pooled): address review -- NaN rows, update() folds, key bound, std centring
Row kernels skip NaN targets. `_CellStore` already dropped them at
aggregation but `_RowStore` kept every row, so one NaN in a window poisoned
the whole quantile (and the expanding one for good) and NaN rows counted
toward `min_samples`. The quantile kernels now gather from a cached
NaN-free view of the rows; `LookupLag` keeps reading every row, since a NaN
occurrence is still an occurrence there, as on main.
`TimeSeries.update` advances the pooled accumulators. `Expanding*`/EWM fold
one source cell per `PooledState.update`, which predict calls once per
timestamp; `_update_pooled_states` only appended columns, so the next
predict folded the newest source and skipped the ones in between (the last
fitted timestamp among them). The same call is now made once per appended
timestamp. An update with no new rows is a no-op instead of an error.
`_RowStore` keys are `bucket * 2**32 + ordinal` with the bucket count
bounded (and checked) at 2**31 instead of an unenforced 2**23; `search`
clamps targets before the calendar so a window reaching back past ordinal
0 cannot cross into the bucket before.
`sumsq` is centred per bucket on the bucket's first observed cell, carried
by the store across trims, appends and bucket growth, and `_StdKernel`
combines the shifted sums. The plain formula lost three significant figures
at `y ~ 1e6 +- 1`; the relative error against an extended-precision
reference drops from ~2e-3 to ~1e-9. Each `time_agg` view carries a centre
of its own.
Also replaces the tautological positional check in
`test_polars_join_preserves_row_order` with `uniques[bids]` reconstructing
the per-row keys.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138Wd4g3r3EX4kEoo3V8WDx Latest Branches
0%
janrth:bug/interval_transfer 0%
0%
© 2026 CodSpeed Technology