huleilei:hll/huggingface - Branch - Eventual-Inc/Daft

fix: test_read_huggingface to compare against all dataset splits

#5890

Comparing

huleilei:hll/huggingface

(

ce24194

) with

main

(

29ffd49

)

Untouched: 24

Ignored: 4

Benchmarks

Passed

test_tpch[1-in-memory-5]

tests/benchmarks/test_local_tpch.py

+2%

132.2 ms129.5 ms

test_tpch[1-in-memory-7]

tests/benchmarks/test_local_tpch.py

+2%

127.6 ms125.4 ms

test_tpch[1-in-memory-9]

tests/benchmarks/test_local_tpch.py

+1%

275.9 ms273.3 ms

test_iter_rows_first_row[1 Small File]

tests/benchmarks/test_interactive_reads.py

+1%

35.7 ms35.5 ms

test_tpch_sql[1-in-memory-8]

tests/benchmarks/test_local_tpch.py

+1%

134.2 ms133.5 ms

test_tpch_sql[1-in-memory-2]

tests/benchmarks/test_local_tpch.py

165.5 ms165 ms

test_tpch_sql[1-in-memory-6]

tests/benchmarks/test_local_tpch.py

29.2 ms29.1 ms

test_tpch_sql[1-in-memory-4]

tests/benchmarks/test_local_tpch.py

89 ms88.7 ms

test_tpch[1-in-memory-4]

tests/benchmarks/test_local_tpch.py

88 ms87.9 ms

test_explain[100 Small Files]

tests/benchmarks/test_interactive_reads.py

12.3 ms12.2 ms

test_tpch[1-in-memory-6]

tests/benchmarks/test_local_tpch.py

28.5 ms28.5 ms

test_tpch_sql[1-in-memory-5]

tests/benchmarks/test_local_tpch.py

117.8 ms117.7 ms

test_tpch[1-in-memory-3]

tests/benchmarks/test_local_tpch.py

124 ms124 ms

test_tpch_sql[1-in-memory-3]

tests/benchmarks/test_local_tpch.py

116.3 ms116.3 ms

test_tpch_sql[1-in-memory-7]

tests/benchmarks/test_local_tpch.py

117.7 ms117.9 ms

test_tpch_sql[1-in-memory-9]

tests/benchmarks/test_local_tpch.py

264 ms264.5 ms

test_count[100 Small Files]

tests/benchmarks/test_interactive_reads.py

57.4 ms57.7 ms

test_tpch[1-in-memory-2]

tests/benchmarks/test_local_tpch.py

56.3 ms56.6 ms

test_tpch_sql[1-in-memory-10]

tests/benchmarks/test_local_tpch.py

188.9 ms189.9 ms

test_tpch[1-in-memory-8]

tests/benchmarks/test_local_tpch.py

-1%

148.4 ms149.2 ms

test_tpch[1-in-memory-1]

tests/benchmarks/test_local_tpch.py

-1%

403.1 ms406.4 ms

test_tpch_sql[1-in-memory-1]

tests/benchmarks/test_local_tpch.py

-1%

399.8 ms403.1 ms

test_tpch[1-in-memory-10]

tests/benchmarks/test_local_tpch.py

-2%

196.1 ms199.5 ms

test_show[1 Small File]

tests/benchmarks/test_interactive_reads.py

-4%

11.9 ms12.5 ms

Ignored

test_explain[1 Small File]

tests/benchmarks/test_interactive_reads.py

Ignored

2.1 ms2.1 ms

test_count[1 Small File]

tests/benchmarks/test_interactive_reads.py

Ignored

-2%

3.5 ms3.6 ms

test_show[100 Small Files]

tests/benchmarks/test_interactive_reads.py

Ignored

+1%

23.2 ms22.9 ms

test_iter_rows_first_row[100 Small Files]

tests/benchmarks/test_interactive_reads.py

Ignored

-19%

153.7 ms189.1 ms

Commits

Click on a commit to change the comparison range

Base

main

29ffd49

+0.12%

Fix test_read_huggingface to compare against all dataset splits daft.read_huggingface reads the entire dataset (all splits), so the test should compare against the concatenation of all splits from the datasets library, rather than just a single split. This fixes failures for datasets with multiple splits like nebius/SWE-rebench.

ddf6c54

1 day ago

by huleilei

-0.47%

change import

e30e500

24 hours ago

by huleilei

+0.24%

Merge branch 'main' into hll/huggingface

ce24194

4 hours ago

by huleilei

Home Terms Privacy Docs