BlogDocs

perf: Use List's TotalEqKernel(#18984)

Merged
Merging
coastalwhite:perf/list-eq-kernel
(
bfa1d64
) into
main
(
6abc2f1
)
0%
IMPROVEMENTS
0
REGRESSIONS
0
UNTOUCHED
40
NEW
0
DROPPED
0
IGNORED
1

Benchmarks

Passed

test_pdsh_q15
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q15
+13%
2.5 ms
2.2 ms
test_pdsh_q11
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q11
+10%
4 ms
3.6 ms
test_pdsh_q12
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q12
+9%
6.2 ms
5.7 ms
test_groupby_h2oai_q10
py-polars/tests/benchmark/test_group_by.py::test_groupby_h2oai_q10
+7%
6.4 ms
6 ms
test_groupby_h2oai_q7
py-polars/tests/benchmark/test_group_by.py::test_groupby_h2oai_q7
+6%
2.2 ms
2.1 ms
test_pdsh_q17
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q17
+5%
6.9 ms
6.6 ms
test_pdsh_q2
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q2
+5%
3.6 ms
3.4 ms
test_pdsh_q5
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q5
+3%
4.9 ms
4.8 ms
test_pdsh_q13
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q13
+2%
9.8 ms
9.6 ms
test_pdsh_q7
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q7
+2%
9.4 ms
9.2 ms
test_pdsh_q8
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q8
+2%
5.3 ms
5.2 ms
test_pdsh_q9
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q9
+1%
25.7 ms
25.5 ms
test_pdsh_q4
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q4
+1%
4.6 ms
4.6 ms
test_strict_inequalities
py-polars/tests/benchmark/test_join_where.py::test_strict_inequalities
0%
169.5 ms
168.7 ms
test_groupby_h2oai_q2
py-polars/tests/benchmark/test_group_by.py::test_groupby_h2oai_q2
0%
4.5 ms
4.5 ms
test_pdsh_q21
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q21
0%
213.3 ms
212.6 ms
test_groupby_h2oai_q3
py-polars/tests/benchmark/test_group_by.py::test_groupby_h2oai_q3
0%
2.3 ms
2.3 ms
test_non_strict_inequalities
py-polars/tests/benchmark/test_join_where.py::test_non_strict_inequalities
0%
174.3 ms
173.8 ms
test_single_inequality
py-polars/tests/benchmark/test_join_where.py::test_single_inequality
0%
80.7 ms
80.6 ms
test_to_numpy_series_with_nulls
py-polars/tests/benchmark/interop/test_numpy.py::test_to_numpy_series_with_nulls
0%
435 µs
434.7 µs
test_to_numpy_series_zero_copy
py-polars/tests/benchmark/interop/test_numpy.py::test_to_numpy_series_zero_copy
0%
123.5 µs
123.5 µs
test_to_numpy_series_chunked
py-polars/tests/benchmark/interop/test_numpy.py::test_to_numpy_series_chunked
0%
269.6 µs
269.7 µs
test_pdsh_q18
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q18
0%
11.5 ms
11.5 ms
test_pdsh_q19
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q19
0%
7.4 ms
7.4 ms
test_groupby_h2oai_q9
py-polars/tests/benchmark/test_group_by.py::test_groupby_h2oai_q9
0%
27.7 ms
27.8 ms
test_write_read_scan_large_csv
py-polars/tests/benchmark/test_io.py::test_write_read_scan_large_csv
0%
30.8 ms
30.9 ms
test_groupby_h2oai_q1
py-polars/tests/benchmark/test_group_by.py::test_groupby_h2oai_q1
-1%
2.3 ms
2.3 ms
test_groupby_h2oai_q6
py-polars/tests/benchmark/test_group_by.py::test_groupby_h2oai_q6
-1%
12.5 ms
12.7 ms
test_pdsh_q22
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q22
-1%
6.4 ms
6.4 ms
test_pdsh_q3
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q3
-2%
5.6 ms
5.7 ms
test_pdsh_q16
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q16
-2%
7.3 ms
7.4 ms
test_pdsh_q6
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q6
-2%
1.9 ms
1.9 ms
test_groupby_h2oai_q5
py-polars/tests/benchmark/test_group_by.py::test_groupby_h2oai_q5
-3%
1.8 ms
1.8 ms
test_groupby_h2oai_q8
py-polars/tests/benchmark/test_group_by.py::test_groupby_h2oai_q8
-3%
2.9 ms
3 ms
test_pdsh_q20
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q20
-3%
6 ms
6.2 ms
test_pdsh_q1
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q1
-4%
15.9 ms
16.6 ms
test_groupby_h2oai_q4
py-polars/tests/benchmark/test_group_by.py::test_groupby_h2oai_q4
-5%
1.9 ms
2 ms
test_filter1
py-polars/tests/benchmark/test_filter.py::test_filter1
-6%
712 µs
758.6 µs
test_pdsh_q10
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q10
-7%
6 ms
6.4 ms
test_pdsh_q14
py-polars/tests/benchmark/test_pdsh.py::test_pdsh_q14
-8%
2.1 ms
2.3 ms

Ignored

test_filter2Ignored
py-polars/tests/benchmark/test_filter.py::test_filter2
+6%
1.1 ms
1 ms

Commits

Click on a commit to change the comparison range
base
main
6abc2f1
0%
perf: Use List's TotalEqKernel This utilizes `List`'s variant of `TotalEqKernel` and implements broadcasting for that kernel. This gives a nice speed-up, reduces code-bloat and reduces chance of bugs. For a measure on how much faster this is I did a microbenchmark: ```python import polars as pl import numpy as np from timeit import timeit lists = [None] + [list(range(i)) for i in range(11)] a = [lists[length] for length in np.random.randint(0, 11, 1_000_000)] b = [lists[length] for length in np.random.randint(0, 11, 1_000_000)] a = pl.Series('a', a) b = pl.Series('b', b) t = timeit(lambda: a == b, number=100) print(f"Time: {t:.2}s") ``` - Before: `Time: 16s` - After: `Time: 1.9s` I feel like there is more that can be done to make this a lot faster, but I feel like this is good enough for now.
bfa1d64
1 day ago
by coastalwhite
ResourcesHomePricingDocsBlogGitHubChangelog
Copyright © 2024 CodSpeed Technology SAS. All rights reserved.