BlogDocs

[PERF] Add a parallel local CSV reader(#2772)

Merging
desmondcheongzx:local-csv-reader-experiment
(
7b40f23
) into
main
(
cad9168
)
-34%
IMPROVEMENTS
2
REGRESSIONS
1
UNTOUCHED
13
NEW
0
DROPPED
0
IGNORED
1

Benchmarks

Failed

test_count[1 Small File]Regression
tests/benchmarks/test_interactive_reads.py::test_count[1 Small File]
-34%
16.5 ms
24.9 ms

Improved

test_show[100 Small Files]
tests/benchmarks/test_interactive_reads.py::test_show[100 Small Files]
+68%
597.4 ms
355.8 ms
test_explain[100 Small Files]
tests/benchmarks/test_interactive_reads.py::test_explain[100 Small Files]
+32%
52.8 ms
39.9 ms

Passed

test_show[1 Small File]
tests/benchmarks/test_interactive_reads.py::test_show[1 Small File]
+6%
22.9 ms
21.5 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:3]
tests/benchmarks/test_local_tpch.py::test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:3]
+2%
133.1 ms
129.9 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:4]
tests/benchmarks/test_local_tpch.py::test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:4]
+2%
391.3 ms
383.2 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:1]
tests/benchmarks/test_local_tpch.py::test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:1]
+1%
444.6 ms
438.9 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:8]
tests/benchmarks/test_local_tpch.py::test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:8]
+1%
331.1 ms
328.8 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:5]
tests/benchmarks/test_local_tpch.py::test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:5]
+1%
366.8 ms
364.4 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:9]
tests/benchmarks/test_local_tpch.py::test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:9]
0%
352.8 ms
351.9 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:2]
tests/benchmarks/test_local_tpch.py::test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:2]
0%
92 ms
91.8 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:6]
tests/benchmarks/test_local_tpch.py::test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:6]
0%
29 ms
29 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:10]
tests/benchmarks/test_local_tpch.py::test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:10]
0%
189.9 ms
189.7 ms
test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:7]
tests/benchmarks/test_local_tpch.py::test_tpch[gen_tpch:1-get_df:in-memory-engine:native-q:7]
0%
123.1 ms
123.2 ms
test_iter_rows_first_row[100 Small Files]
tests/benchmarks/test_interactive_reads.py::test_iter_rows_first_row[100 Small Files]
0%
10 s
10 s
test_iter_rows_first_row[1 Small File]
tests/benchmarks/test_interactive_reads.py::test_iter_rows_first_row[1 Small File]
-1%
105.9 ms
106.5 ms

Ignored

test_explain[1 Small File]Ignored
tests/benchmarks/test_interactive_reads.py::test_explain[1 Small File]
-37%
5.6 ms
8.9 ms

Commits

Click on a commit to change the comparison range
base
main
cad9168
-34%
Prototype
7b40f23
29 days ago
by desmondcheongzx
ResourcesHomePricingDocsBlogGitHubChangelog
Getting StartedSample repositoryExplore repositoriesSupport
AboutCareersTwitterDiscordContact UsTerms of ServicePrivacy Policy
Copyright © 2024 CodSpeed Technology SAS. All rights reserved.