mattsu2020:sort_performan-ce - Branch - uutils/coreutils

Improve sort buffer sizing heuristics and honor explicit --buffer-size

#8833

Comparing

mattsu2020:sort_performan-ce

(

92a3869

) with

main

(

8d59e08

)

-3%

Improvements: 16

Regressions: 1

Untouched: 89

Skipped: 73

No successful run was found on main ( 795d587) during the generation of this report, so 8d59e08 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

Benchmarks

Skipped (73)

Failed

sort_key_field[500000]

src/uu/sort/benches/sort_bench.rs

Regression

-3%

688.8 ms707.9 ms

Improved

sort_reverse_mixed

src/uu/sort/benches/sort_locale_bench.rs

×2.5

93.2 ms37.3 ms

sort_german_c_locale

src/uu/sort/benches/sort_locale_bench.rs

×2.5

94.5 ms38 ms

sort_german_locale

src/uu/sort/benches/sort_locale_bench.rs

×2.5

94.5 ms38 ms

sort_mixed_utf8_locale

src/uu/sort/benches/sort_locale_bench.rs

×2.5

94.1 ms37.9 ms

sort_mixed_c_locale

src/uu/sort/benches/sort_locale_bench.rs

×2.5

94.1 ms37.9 ms

sort_unique_locale[500000]

src/uu/sort/benches/sort_bench.rs

×2.4

1,142.3 ms473.1 ms

sort_reverse_locale[500000]

src/uu/sort/benches/sort_bench.rs

×2.3

820.7 ms351.7 ms

sort_accented_data[500000]

src/uu/sort/benches/sort_bench.rs

×2.3

819.3 ms354.4 ms

sort_ascii_only[500000]

src/uu/sort/benches/sort_bench.rs

×2.3

798 ms345.6 ms

sort_mixed_data[500000]

src/uu/sort/benches/sort_bench.rs

×2.3

729.6 ms317.7 ms

sort_unique_mixed

src/uu/sort/benches/sort_locale_bench.rs

×2.3

87.6 ms38.2 ms

sort_case_sensitive[500000]

src/uu/sort/benches/sort_bench.rs

×2

327.1 ms163 ms

sort_random_strings

src/uu/sort/benches/sort_locale_bench.rs

+79%

55.3 ms30.9 ms

sort_case_insensitive[500000]

src/uu/sort/benches/sort_bench.rs

+53%

417.1 ms271.9 ms

sort_ascii_c_locale

src/uu/sort/benches/sort_locale_bench.rs

+31%

27.9 ms21.3 ms

sort_ascii_utf8_locale

src/uu/sort/benches/sort_locale_bench.rs

+31%

55.8 ms42.7 ms

Passed

du_balanced_tree[(5, 4, 10)]

src/uu/du/benches/du_bench.rs

9.3 ms9.2 ms

ls_recursive_long_all_deep_tree[(100, 4)]

src/uu/ls/benches/ls_bench.rs

3.1 ms3.1 ms

split_number_chunks

src/uu/split/benches/split_bench.rs

270.7 µs270.1 µs

ls_recursive_long_all_wide_tree[(15000, 1500)]

src/uu/ls/benches/ls_bench.rs

145.7 ms145.4 ms

du_max_depth_balanced_tree[(6, 4, 10)]

src/uu/du/benches/du_bench.rs

32.8 ms32.7 ms

ls_recursive_long_all_balanced_tree[(6, 4, 15)]

src/uu/ls/benches/ls_bench.rs

138.7 ms138.5 ms

cp_recursive_wide_tree[(6000, 800)]

src/uu/cp/benches/cp_bench.rs

188.5 ms188.1 ms

du_wide_tree[(5000, 500)]

src/uu/du/benches/du_bench.rs

11.2 ms11.2 ms

cp_large_file[16]

src/uu/cp/benches/cp_bench.rs

294.1 µs293.6 µs

ls_recursive_long_all_mixed_tree

src/uu/ls/benches/ls_bench.rs

4 ms4 ms

du_summarize_balanced_tree[(5, 4, 10)]

src/uu/du/benches/du_bench.rs

8.5 ms8.5 ms

wc_bytes_synthetic[500]

src/uu/wc/benches/wc_bench.rs

162.8 µs162.5 µs

cp_recursive_balanced_tree[(5, 4, 10)]

src/uu/cp/benches/cp_bench.rs

71.7 ms71.6 ms

cp_recursive_deep_tree[(120, 4)]

src/uu/cp/benches/cp_bench.rs

13.3 ms13.2 ms

numfmt_padding[(10000, 50)]

src/uu/numfmt/benches/numfmt_bench.rs

48.4 ms48.3 ms

rm_single_file

src/uu/rm/benches/rm_bench.rs

100.1 ms100.1 ms

mv_single_file

src/uu/mv/benches/mv_bench.rs

120.5 ms120.4 ms

mv_force_overwrite

src/uu/mv/benches/mv_bench.rs

123.9 ms123.8 ms

split_bytes

src/uu/split/benches/split_bench.rs

482.2 µs482.1 µs

rm_force_files

src/uu/rm/benches/rm_bench.rs

2 ms2 ms

rm_multiple_files

src/uu/rm/benches/rm_bench.rs

2.2 ms2.2 ms

wc_chars_large_line_count[100000]

src/uu/wc/benches/wc_bench.rs

990 µs989.8 µs

mv_directory

src/uu/mv/benches/mv_bench.rs

3.1 ms3.1 ms

tsort_tree_dag[(10, 3)]

src/uu/tsort/benches/tsort_bench.rs

34.1 ms34.1 ms

wc_lines_extreme_line_lengths[(100000, 200)]

src/uu/wc/benches/wc_bench.rs

1.6 ms1.6 ms

tsort_wide_dag[100000]

src/uu/tsort/benches/tsort_bench.rs

135.5 ms135.5 ms

wc_lines_large_line_count[500000]

src/uu/wc/benches/wc_bench.rs

2.9 ms2.9 ms

wc_lines_variable_length[(50, 500)]

src/uu/wc/benches/wc_bench.rs

3.4 ms3.4 ms

uniq_with_count[10000]

src/uu/uniq/benches/uniq_bench.rs

1.8 ms1.8 ms

uniq_heavy_duplicates[10000]

src/uu/uniq/benches/uniq_bench.rs

1.9 ms1.9 ms

uniq_case_insensitive[10000]

src/uu/uniq/benches/uniq_bench.rs

1.7 ms1.7 ms

wc_words_large_line_count[100000]

src/uu/wc/benches/wc_bench.rs

43.8 ms43.8 ms

wc_default_large_line_count[100000]

src/uu/wc/benches/wc_bench.rs

48.9 ms48.9 ms

fold_custom_width[50000]

src/uu/fold/benches/fold_bench.rs

46.6 ms46.6 ms

cut_fields_custom_delim

src/uu/cut/benches/cut_bench.rs

40.5 ms40.5 ms

wc_bytes_lines_synthetic[2000]

src/uu/wc/benches/wc_bench.rs

120.2 ms120.2 ms

nl_many_lines[100000]

src/uu/nl/benches/nl_bench.rs

79.5 ms79.5 ms

nl_large_file[10]

src/uu/nl/benches/nl_bench.rs

100.6 ms100.6 ms

fold_many_lines[100000]

src/uu/fold/benches/fold_bench.rs

125.3 ms125.3 ms

wc_words_synthetic[2000]

src/uu/wc/benches/wc_bench.rs

10.3 s10.3 s

unexpand_large_file[10]

src/uu/unexpand/benches/unexpand_bench.rs

548 ms548 ms

unexpand_many_lines[100000]

src/uu/unexpand/benches/unexpand_bench.rs

261.4 ms261.4 ms

hashsum_sha256_check

src/uu/hashsum/benches/hashsum_bench.rs

173.6 ms173.6 ms

cut_fields_tab

src/uu/cut/benches/cut_bench.rs

33.9 ms33.9 ms

hashsum_sha512

src/uu/hashsum/benches/hashsum_bench.rs

111.7 ms111.7 ms

hashsum_sha256

src/uu/hashsum/benches/hashsum_bench.rs

173.6 ms173.6 ms

expand_many_lines[100000]

src/uu/expand/benches/expand_bench.rs

100.9 ms100.9 ms

expand_custom_tabstops[50000]

src/uu/expand/benches/expand_bench.rs

25.9 ms25.9 ms

hashsum_sha1

src/uu/hashsum/benches/hashsum_bench.rs

72.5 ms72.5 ms

numfmt_to_si_precision[10000]

src/uu/numfmt/benches/numfmt_bench.rs

35.3 ms35.3 ms

cut_bytes

src/uu/cut/benches/cut_bench.rs

8.4 ms8.4 ms

hashsum_md5_check

src/uu/hashsum/benches/hashsum_bench.rs

32 ms32 ms

numfmt_round_modes[("down", 10000)]

src/uu/numfmt/benches/numfmt_bench.rs

34.5 ms34.5 ms

numfmt_to_iec[10000]

src/uu/numfmt/benches/numfmt_bench.rs

34 ms34 ms

hashsum_md5

src/uu/hashsum/benches/hashsum_bench.rs

32 ms32 ms

numfmt_round_modes[("towards-zero", 10000)]

src/uu/numfmt/benches/numfmt_bench.rs

34.5 ms34.5 ms

numfmt_from_si[10000]

src/uu/numfmt/benches/numfmt_bench.rs

33.1 ms33.1 ms

numfmt_to_si[10000]

src/uu/numfmt/benches/numfmt_bench.rs

33.8 ms33.8 ms

numfmt_round_modes[("up", 10000)]

src/uu/numfmt/benches/numfmt_bench.rs

32.7 ms32.7 ms

numfmt_large_numbers_si[10000]

src/uu/numfmt/benches/numfmt_bench.rs

34.5 ms34.5 ms

cut_characters

src/uu/cut/benches/cut_bench.rs

7.5 ms7.5 ms

seq_formatted

src/uu/seq/benches/seq_bench.rs

99.7 ms99.7 ms

seq_custom_separator

src/uu/seq/benches/seq_bench.rs

29.2 ms29.2 ms

seq_integers

src/uu/seq/benches/seq_bench.rs

26.7 ms26.7 ms

tsort_linear_chain[1000000]

src/uu/tsort/benches/tsort_bench.rs

1.5 s1.5 s

seq_with_step

src/uu/seq/benches/seq_bench.rs

13.6 ms13.6 ms

split_lines

src/uu/split/benches/split_bench.rs

5.9 ms5.9 ms

du_all_balanced_tree[(4, 3, 10)]

src/uu/du/benches/du_bench.rs

1.6 ms1.6 ms

b64_encode_synthetic

src/uu/base64/benches/base64_bench.rs

142 µs142 µs

du_deep_tree[(100, 3)]

src/uu/du/benches/du_bench.rs

1.1 ms1.1 ms

b64_decode_synthetic

src/uu/base64/benches/base64_bench.rs

146.1 µs146.1 µs

cp_archive_balanced_tree[(5, 4, 10)]

src/uu/cp/benches/cp_bench.rs

77 ms77 ms

tsort_complex_dag[50000]

src/uu/tsort/benches/tsort_bench.rs

84 ms84 ms

b64_decode_ignore_garbage_synthetic

src/uu/base64/benches/base64_bench.rs

145.6 µs145.6 µs

tsort_input_parsing_heavy[50000]

src/uu/tsort/benches/tsort_bench.rs

12.8 s12.8 s

split_numeric_suffix

src/uu/split/benches/split_bench.rs

6.1 ms6.1 ms

ls_recursive_deep_tree[(200, 2)]

src/uu/ls/benches/ls_bench.rs

2.1 ms2.1 ms

ls_recursive_wide_tree[(10000, 1000)]

src/uu/ls/benches/ls_bench.rs

51.6 ms51.6 ms

ls_recursive_balanced_tree[(6, 4, 15)]

src/uu/ls/benches/ls_bench.rs

55.7 ms55.8 ms

cp_preserve_metadata[(5, 4, 10)]

src/uu/cp/benches/cp_bench.rs

74.2 ms74.3 ms

ls_recursive_mixed_tree

src/uu/ls/benches/ls_bench.rs

1.4 ms1.4 ms

du_human_balanced_tree[(5, 4, 10)]

src/uu/du/benches/du_bench.rs

10.4 ms10.4 ms

sort_long_line[160000]

src/uu/sort/benches/sort_bench.rs

2 ms2 ms

mv_multiple_to_dir

src/uu/mv/benches/mv_bench.rs

12.9 ms13 ms

du_all_wide_tree[(5000, 500)]

src/uu/du/benches/du_bench.rs

18.1 ms18.1 ms

rm_recursive_tree

src/uu/rm/benches/rm_bench.rs

13.9 ms14 ms

sort_numeric

src/uu/sort/benches/sort_locale_bench.rs

-1%

21.7 ms21.8 ms

sort_dictionary_order[500000]

src/uu/sort/benches/sort_bench.rs

-2%

1.1 s1.1 s

sort_numeric[500000]

src/uu/sort/benches/sort_bench.rs

-2%

1.2 s1.2 s

Commits

Click on a commit to change the comparison range

Base

main

8d59e08

-3.28%

fix Cargo.lock

287d6c4

4 days ago

by mattsu2020

+0.58%

refactor(sort): exclude Redox OS from Unix memory detection Modified conditional compilation in physical_memory_bytes() to skip Unix-specific sysconf calls on Redox OS, ensuring compatibility by returning None for unsupported platforms.

047cec4

4 days ago

by mattsu2020

Merge branch 'uutils:main' into sort_performan-ce

74111c1

2 days ago

by mattsu2020

fix(sort): qualify getrlimit with libc namespace The getrlimit function call in the sort module was missing the libc prefix, which could lead to compilation errors. Added the libc:: qualifier to ensure proper function resolution and maintain code clarity.

5e85119

2 days ago

by mattsu2020

refactor(sort): use nix::libc for getrlimit call Switch from direct libc import to nix::libc for the getrlimit function to leverage the nix crate's safer Unix API abstractions, improving portability and maintainability.

aa5f77c

2 days ago

by mattsu2020

-0.25%

refactor(sort): remove unused getrlimit import Removed the unused `getrlimit` function from the nix libc imports in sort.rs to clean up the code and eliminate dead imports.

1dcaf82

2 days ago

by mattsu2020

+2.94%

refactor(sort): cap buffer size at 512MiB to prevent unnecessary allocations This change modifies the buffer size handling in the external sort functionality to halve requests exceeding 512MiB, reducing memory usage and avoiding potential allocation issues.

d155909

2 days ago

by mattsu2020

-2.77%

Merge branch 'sort_performan-ce' of https://github.com/mattsu2020/coreutils into sort_performan-ce

17e1bce

2 days ago

by mattsu2020

+0.08%

Merge branch 'uutils:main' into sort_performan-ce

92a3869

1 day ago

by mattsu2020

Home Terms Privacy Docs