Avatar for the argilla-io user
argilla-io
distilabel
BlogDocsChangelog

Add `Checkpointer` step

#1114Merged
Comparing
stream-push-to-hub
(
61f3387
) with
develop
(
84ea198
)
CodSpeed Performance Gauge
-36%
Improvements
0
Regressions
1
Untouched
0
New
0
Dropped
0
Ignored
0

Benchmarks

Failed

test_cache_timeRegression
tests/integration/test_cache.py::test_cache_time
CodSpeed Performance Gauge
-36%
557.5 ms
869.2 ms

Commits

Click on a commit to change the comparison range
Base
develop
84ea198
0%
Add argument to allow modifying the dump batch size from the pipeline
29959f8
11 days ago
by plaguss
0%
Update Checkpointer to just push the current batch to a dataset using the HfAPI
7528815
11 days ago
by plaguss
0%
Push different config files depending on the number of leaf steps (elements in the inputs
3b64892
11 days ago
by plaguss
0%
Add docs for the Checkpointer step
34cb984
11 days ago
by plaguss
0%
Add emoji to locate the checkpoint
8f1683e
11 days ago
by plaguss
0%
Rename checkpointer and let the context manager remove the file
58b6f89
11 days ago
by plaguss
-37%
Merge branch 'develop' into stream-push-to-hub
61f3387
11 days ago
by plaguss
Home Terms PrivacyDocs