Eventual-Inc
Daft
BlogDocsChangelog

[PERF] Add a parallel local CSV reader

#2772Closed
Comparing
desmondcheongzx:local-csv-reader-experiment
(
7b40f23
) with
main
(
cad9168
)
CodSpeed Performance Gauge
-34%
Improvements
1
Untouched
12
Ignored
4

Benchmarks

Improved

test_explain[100 Small Files]
tests/benchmarks/test_interactive_reads.py
CodSpeed Performance Gauge
+32%
52.8 ms39.9 ms

Passed

test_show[1 Small File]
tests/benchmarks/test_interactive_reads.py
CodSpeed Performance Gauge
+6%
22.9 ms21.5 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:3]
tests/benchmarks/test_local_tpch.py
CodSpeed Performance Gauge
+2%
133.1 ms129.9 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:4]
tests/benchmarks/test_local_tpch.py
CodSpeed Performance Gauge
+2%
391.3 ms383.2 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:1]
tests/benchmarks/test_local_tpch.py
CodSpeed Performance Gauge
+1%
444.6 ms438.9 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:8]
tests/benchmarks/test_local_tpch.py
CodSpeed Performance Gauge
+1%
331.1 ms328.8 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:5]
tests/benchmarks/test_local_tpch.py
CodSpeed Performance Gauge
+1%
366.8 ms364.4 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:9]
tests/benchmarks/test_local_tpch.py
CodSpeed Performance Gauge
0%
352.8 ms351.9 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:2]
tests/benchmarks/test_local_tpch.py
CodSpeed Performance Gauge
0%
92 ms91.8 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:6]
tests/benchmarks/test_local_tpch.py
CodSpeed Performance Gauge
0%
29 ms29 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:10]
tests/benchmarks/test_local_tpch.py
CodSpeed Performance Gauge
0%
189.9 ms189.7 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:7]
tests/benchmarks/test_local_tpch.py
CodSpeed Performance Gauge
0%
123.1 ms123.2 ms
test_iter_rows_first_row[1 Small File]
tests/benchmarks/test_interactive_reads.py
CodSpeed Performance Gauge
-1%
105.9 ms106.5 ms

Ignored

test_show[100 Small Files]
tests/benchmarks/test_interactive_reads.py
Ignored
CodSpeed Performance Gauge
+68%
597.4 ms355.8 ms
test_explain[1 Small File]
tests/benchmarks/test_interactive_reads.py
Ignored
CodSpeed Performance Gauge
-37%
5.6 ms8.9 ms
test_count[1 Small File]
tests/benchmarks/test_interactive_reads.py
Ignored
CodSpeed Performance Gauge
-34%
16.5 ms24.9 ms
test_iter_rows_first_row[100 Small Files]
tests/benchmarks/test_interactive_reads.py
Ignored
CodSpeed Performance Gauge
0%
10 s10 s

Commits

Click on a commit to change the comparison range
Base
main
cad9168
-33.78%
Prototype
7b40f23
1 year ago
by desmondcheongzx
© 2025 CodSpeed Technology
Home Terms Privacy Docs