mattsu2020:fold_compatibility - Branch - uutils/coreutils

fold:fix gnu test fold-zero-width.sh

#9274

Comparing

mattsu2020:fold_compatibility

(

eb7d8c4

) with

main

(

2a314c7

)

+49%

Improvements: 2

Untouched: 124

Skipped: 6

Benchmarks

Skipped (6)

Improved

fold_custom_width[50000]

src/uu/fold/benches/fold_bench.rs

+49%

32.9 ms22 ms

fold_many_lines[100000]

src/uu/fold/benches/fold_bench.rs

+43%

80.1 ms55.8 ms

Passed

du_deep_tree[(100, 3)]

src/uu/du/benches/du_bench.rs

+1%

1.2 ms1.2 ms

du_wide_tree[(5000, 500)]

src/uu/du/benches/du_bench.rs

11.3 ms11.2 ms

du_all_wide_tree[(5000, 500)]

src/uu/du/benches/du_bench.rs

18.2 ms18.2 ms

du_summarize_balanced_tree[(5, 4, 10)]

src/uu/du/benches/du_bench.rs

8.4 ms8.4 ms

sort_reverse_mixed

src/uu/sort/benches/sort_locale_bench.rs

38.2 ms38.1 ms

cp_recursive_deep_tree[(120, 4)]

src/uu/cp/benches/cp_bench.rs

13.6 ms13.6 ms

sort_unique_mixed

src/uu/sort/benches/sort_locale_bench.rs

38.6 ms38.6 ms

cksum_sysv

src/uu/cksum/benches/cksum_bench.rs

48.2 ms48.2 ms

b64_decode_ignore_garbage_synthetic

src/uu/base64/benches/base64_bench.rs

149.4 µs149.3 µs

cp_recursive_balanced_tree[(5, 4, 10)]

src/uu/cp/benches/cp_bench.rs

72.3 ms72.3 ms

tsort_input_parsing_heavy[5000]

src/uu/tsort/benches/tsort_bench.rs

82.5 ms82.4 ms

sort_reverse_locale[500000]

src/uu/sort/benches/sort_bench.rs

362.5 ms362.3 ms

sort_mixed_c_locale

src/uu/sort/benches/sort_locale_bench.rs

38.3 ms38.2 ms

shuf_lines[100000]

src/uu/shuf/benches/shuf_bench.rs

26.9 ms26.9 ms

cp_preserve_metadata[(5, 4, 10)]

src/uu/cp/benches/cp_bench.rs

74.9 ms74.9 ms

factor_multiple_u64s[2]

src/uu/factor/benches/factor_bench.rs

178.3 ms178.3 ms

cp_recursive_wide_tree[(6000, 800)]

src/uu/cp/benches/cp_bench.rs

189.8 ms189.7 ms

cksum_default

src/uu/cksum/benches/cksum_bench.rs

17.3 ms17.3 ms

sort_german_c_locale

src/uu/sort/benches/sort_locale_bench.rs

38.4 ms38.4 ms

sort_mixed_utf8_locale

src/uu/sort/benches/sort_locale_bench.rs

38.8 ms38.8 ms

cksum_multiple_files

src/uu/cksum/benches/cksum_bench.rs

25.9 ms25.9 ms

shuf_input_range[1000000]

src/uu/shuf/benches/shuf_bench.rs

229.1 ms229.1 ms

tsort_complex_dag[50000]

src/uu/tsort/benches/tsort_bench.rs

84.7 ms84.7 ms

sort_dictionary_order[500000]

src/uu/sort/benches/sort_bench.rs

1.1 s1.1 s

cksum_blake3

src/uu/cksum/benches/cksum_bench.rs

187.3 µs187.3 µs

rm_single_file

src/uu/rm/benches/rm_bench.rs

106.6 ms106.6 ms

ls_recursive_mixed_tree

src/uu/ls/benches/ls_bench.rs

1.4 ms1.4 ms

mv_single_file

src/uu/mv/benches/mv_bench.rs

129.7 ms129.7 ms

b64_decode_synthetic

src/uu/base64/benches/base64_bench.rs

149.1 µs149.1 µs

split_lines

src/uu/split/benches/split_bench.rs

5.7 ms5.7 ms

sort_ascii_utf8_locale

src/uu/sort/benches/sort_locale_bench.rs

43.1 ms43 ms

wc_lines_extreme_line_lengths[(100000, 200)]

src/uu/wc/benches/wc_bench.rs

1.6 ms1.6 ms

mv_force_overwrite

src/uu/mv/benches/mv_bench.rs

125.2 ms125.2 ms

ls_recursive_balanced_tree[(6, 4, 15)]

src/uu/ls/benches/ls_bench.rs

55.6 ms55.6 ms

cksum_sha384

src/uu/cksum/benches/cksum_bench.rs

1.1 s1.1 s

cksum_sha512

src/uu/cksum/benches/cksum_bench.rs

1.1 s1.1 s

split_numeric_suffix

src/uu/split/benches/split_bench.rs

5.9 ms5.9 ms

ls_recursive_long_all_mixed_tree

src/uu/ls/benches/ls_bench.rs

2.8 ms2.8 ms

sort_accented_data[500000]

src/uu/sort/benches/sort_bench.rs

362.5 ms362.5 ms

wc_bytes_synthetic[500]

src/uu/wc/benches/wc_bench.rs

164.2 µs164.2 µs

wc_lines_large_line_count[500000]

src/uu/wc/benches/wc_bench.rs

2.9 ms2.9 ms

uniq_heavy_duplicates[10000]

src/uu/uniq/benches/uniq_bench.rs

1.9 ms1.9 ms

wc_lines_variable_length[(50, 500)]

src/uu/wc/benches/wc_bench.rs

3.4 ms3.4 ms

sort_ascii_c_locale

src/uu/sort/benches/sort_locale_bench.rs

22.5 ms22.5 ms

shuf_repeat_sampling[50000]

src/uu/shuf/benches/shuf_bench.rs

4.6 ms4.6 ms

rm_force_files

src/uu/rm/benches/rm_bench.rs

2.1 ms2.1 ms

rm_multiple_files

src/uu/rm/benches/rm_bench.rs

2.2 ms2.2 ms

sort_ascii_only[500000]

src/uu/sort/benches/sort_bench.rs

353.6 ms353.6 ms

cksum_crc

src/uu/cksum/benches/cksum_bench.rs

17.3 ms17.3 ms

numfmt_from_si[10000]

src/uu/numfmt/benches/numfmt_bench.rs

32.7 ms32.7 ms

numfmt_to_si[10000]

src/uu/numfmt/benches/numfmt_bench.rs

33.6 ms33.6 ms

numfmt_round_modes[("down", 10000)]

src/uu/numfmt/benches/numfmt_bench.rs

34.1 ms34.1 ms

numfmt_round_modes[("up", 10000)]

src/uu/numfmt/benches/numfmt_bench.rs

34.1 ms34.1 ms

sort_case_sensitive[500000]

src/uu/sort/benches/sort_bench.rs

174.2 ms174.2 ms

sort_numeric[500000]

src/uu/sort/benches/sort_bench.rs

1.2 s1.2 s

cut_fields_custom_delim

src/uu/cut/benches/cut_bench.rs

40.7 ms40.7 ms

rm_recursive_tree

src/uu/rm/benches/rm_bench.rs

13.6 ms13.6 ms

numfmt_to_iec[10000]

src/uu/numfmt/benches/numfmt_bench.rs

33.6 ms33.6 ms

cut_fields_tab

src/uu/cut/benches/cut_bench.rs

34.1 ms34.1 ms

mv_directory

src/uu/mv/benches/mv_bench.rs

3.5 ms3.5 ms

numfmt_padding[(10000, 50)]

src/uu/numfmt/benches/numfmt_bench.rs

48.2 ms48.2 ms

wc_bytes_lines_synthetic[2000]

src/uu/wc/benches/wc_bench.rs

120.2 ms120.2 ms

cksum_bsd

src/uu/cksum/benches/cksum_bench.rs

128.3 ms128.3 ms

unexpand_many_lines[100000]

src/uu/unexpand/benches/unexpand_bench.rs

262 ms262 ms

cut_bytes

src/uu/cut/benches/cut_bench.rs

8.5 ms8.5 ms

unexpand_large_file[10]

src/uu/unexpand/benches/unexpand_bench.rs

549.3 ms549.3 ms

numfmt_large_numbers_si[10000]

src/uu/numfmt/benches/numfmt_bench.rs

32.4 ms32.4 ms

cksum_sha1

src/uu/cksum/benches/cksum_bench.rs

723 ms723 ms

cksum_md5

src/uu/cksum/benches/cksum_bench.rs

317.9 ms317.9 ms

wc_words_synthetic[2000]

src/uu/wc/benches/wc_bench.rs

11.4 s11.4 s

cksum_sm3

src/uu/cksum/benches/cksum_bench.rs

1.6 s1.6 s

cksum_sha256

src/uu/cksum/benches/cksum_bench.rs

1.7 s1.7 s

cksum_sha2

src/uu/cksum/benches/cksum_bench.rs

1.7 s1.7 s

cksum_sha224

src/uu/cksum/benches/cksum_bench.rs

1.7 s1.7 s

cksum_sha3

src/uu/cksum/benches/cksum_bench.rs

1.8 s1.8 s

cksum_shake256

src/uu/cksum/benches/cksum_bench.rs

1.9 s1.9 s

cksum_shake128

src/uu/cksum/benches/cksum_bench.rs

1.5 s1.5 s

nl_many_lines[100000]

src/uu/nl/benches/nl_bench.rs

79.7 ms79.7 ms

numfmt_round_modes[("towards-zero", 10000)]

src/uu/numfmt/benches/numfmt_bench.rs

34.1 ms34.1 ms

hashsum_sha256_check

src/uu/hashsum/benches/hashsum_bench.rs

173.6 ms173.6 ms

wc_words_large_line_count[100000]

src/uu/wc/benches/wc_bench.rs

47.9 ms47.9 ms

hashsum_sha512

src/uu/hashsum/benches/hashsum_bench.rs

110.8 ms110.8 ms

hashsum_md5_check

src/uu/hashsum/benches/hashsum_bench.rs

32 ms32 ms

wc_default_large_line_count[100000]

src/uu/wc/benches/wc_bench.rs

50.4 ms50.4 ms

cksum_crc32b

src/uu/cksum/benches/cksum_bench.rs

13.7 ms13.7 ms

expand_many_lines[100000]

src/uu/expand/benches/expand_bench.rs

149.9 ms149.9 ms

sort_case_insensitive[500000]

src/uu/sort/benches/sort_bench.rs

278.6 ms278.6 ms

sort_mixed_data[500000]

src/uu/sort/benches/sort_bench.rs

327.3 ms327.3 ms

seq_formatted

src/uu/seq/benches/seq_bench.rs

97.2 ms97.2 ms

nl_large_file[10]

src/uu/nl/benches/nl_bench.rs

100.9 ms100.9 ms

numfmt_to_si_precision[10000]

src/uu/numfmt/benches/numfmt_bench.rs

35.3 ms35.3 ms

expand_custom_tabstops[50000]

src/uu/expand/benches/expand_bench.rs

37 ms37 ms

seq_custom_separator

src/uu/seq/benches/seq_bench.rs

28.7 ms28.7 ms

cksum_raw_output

src/uu/cksum/benches/cksum_bench.rs

17.3 ms17.3 ms

seq_integers

src/uu/seq/benches/seq_bench.rs

26.1 ms26.1 ms

sort_german_locale

src/uu/sort/benches/sort_locale_bench.rs

38.4 ms38.4 ms

seq_with_step

src/uu/seq/benches/seq_bench.rs

13.3 ms13.3 ms

ls_recursive_wide_tree[(10000, 1000)]

src/uu/ls/benches/ls_bench.rs

52.6 ms52.6 ms

cut_characters

src/uu/cut/benches/cut_bench.rs

7.5 ms7.5 ms

wc_chars_large_line_count[100000]

src/uu/wc/benches/wc_bench.rs

994.4 µs994.4 µs

uniq_case_insensitive[10000]

src/uu/uniq/benches/uniq_bench.rs

1.7 ms1.7 ms

ls_recursive_long_all_deep_tree[(100, 4)]

src/uu/ls/benches/ls_bench.rs

3.2 ms3.2 ms

hashsum_sha256

src/uu/hashsum/benches/hashsum_bench.rs

173.6 ms173.6 ms

sort_unique_locale[500000]

src/uu/sort/benches/sort_bench.rs

493.9 ms493.9 ms

ls_recursive_long_all_wide_tree[(15000, 1500)]

src/uu/ls/benches/ls_bench.rs

142 ms142 ms

uniq_with_count[10000]

src/uu/uniq/benches/uniq_bench.rs

1.9 ms1.9 ms

tsort_tree_dag[(10, 3)]

src/uu/tsort/benches/tsort_bench.rs

34.4 ms34.4 ms

sort_long_line[160000]

src/uu/sort/benches/sort_bench.rs

1.7 ms1.7 ms

split_number_chunks

src/uu/split/benches/split_bench.rs

275.3 µs275.3 µs

hashsum_sha1

src/uu/hashsum/benches/hashsum_bench.rs

72.5 ms72.5 ms

ls_recursive_deep_tree[(200, 2)]

src/uu/ls/benches/ls_bench.rs

2.2 ms2.2 ms

sort_numeric

src/uu/sort/benches/sort_locale_bench.rs

23.5 ms23.5 ms

tsort_linear_chain[1000000]

src/uu/tsort/benches/tsort_bench.rs

1.5 s1.5 s

hashsum_md5

src/uu/hashsum/benches/hashsum_bench.rs

32 ms32 ms

cksum_blake2b

src/uu/cksum/benches/cksum_bench.rs

192.9 ms193 ms

tsort_wide_dag[100000]

src/uu/tsort/benches/tsort_bench.rs

134.7 ms134.7 ms

cp_archive_balanced_tree[(5, 4, 10)]

src/uu/cp/benches/cp_bench.rs

77.9 ms77.9 ms

ls_recursive_long_all_balanced_tree[(6, 4, 15)]

src/uu/ls/benches/ls_bench.rs

135.8 ms135.9 ms

mv_multiple_to_dir

src/uu/mv/benches/mv_bench.rs

13.1 ms13.1 ms

split_bytes

src/uu/split/benches/split_bench.rs

505.1 µs505.3 µs

b64_encode_synthetic

src/uu/base64/benches/base64_bench.rs

146.3 µs146.4 µs

du_max_depth_balanced_tree[(6, 4, 10)]

src/uu/du/benches/du_bench.rs

32.9 ms33 ms

sort_key_field[500000]

src/uu/sort/benches/sort_bench.rs

697.9 ms700.1 ms

cp_large_file[16]

src/uu/cp/benches/cp_bench.rs

-1%

340 µs342 µs

Commits

Click on a commit to change the comparison range

Base

main

2a314c7

+0.01%

feat(fold): add column counting for character width mode in process_ascii_line Implement logic to increment column count in WidthMode::Characters, emitting output when width is reached. This ensures accurate line folding for multi-byte characters, enhancing Unicode support.

6d721e5

8 days ago

by mattsu2020

+2.13%

fix fold: emit output early when column count reaches width limit - Added conditional check in fold_file function to call emit_output when col_count >= width - Ensures lines are properly wrapped based on byte or character width before final output flush - Improves handling of incomplete lines that need early breaking to respect the specified width

b21cf35

8 days ago

by mattsu2020

+6.15%

refactor(fold): split long if-conditions into multiple lines for readability Refactor code in fold.rs to break lengthy if-condition statements across multiple lines in push_ascii_segment, process_utf8_line, and process_non_utf8_line functions. This improves code readability without changing functionality.

b0e0033

8 days ago

by mattsu2020

-1.16%

feat(fold): add streaming output with periodic flushing to reduce memory usage Introduce a STREAMING_FLUSH_THRESHOLD constant and helper functions (maybe_flush_unbroken_output, push_byte, push_bytes) to periodically flush the output buffer when it exceeds 8KB and no spaces are being tracked, preventing excessive memory consumption when processing large files. This refactor replaces direct buffer pushes with checks for threshold-based flushing.

ac09d10

8 days ago

by mattsu2020

-2.52%

fix(fold): correct premature output emission in character mode and add tests Remove conditional checks that incorrectly emitted output when column count reached width in character mode, ensuring proper folding of wide characters and handling of edge cases. Add comprehensive tests for wide characters, invalid UTF-8, zero-width spaces, and buffer boundaries to verify correct behavior. This prevents issues with multi-byte character folding where output was prematurely flushed, improving accuracy for Unicode input.

dba5b9b

7 days ago

by mattsu2020

refactor: clean up formatting in fold utility and tests - Remove trailing empty lines in fold.rs - Compact multiline variable assignments in test_fold.rs for readability

556804c

7 days ago

by mattsu2020

Merge branch 'main' into fold_compatibility

5424072

7 days ago

by mattsu2020

feat(fold): add unicode-width dependency and tests for zero-width characters Add unicode-width crate to handle zero-width Unicode characters in fold utility. Introduced new test 'test_zero_width_data_line_counts' to verify correct wrapping in --characters mode for zero-width bytes and spaces, ensuring fold behaves consistently with character counts rather than visual width.

a1bbd40

7 days ago

by mattsu2020

perf: use bytecount for efficient newline counting in fold tests - Add bytecount dependency to Cargo.toml and Cargo.lock - Refactor newline_count function in test_fold.rs to use bytecount::count instead of manual iteration for better performance

d176e65

7 days ago

by mattsu2020

+41.66%

refactor: Handle zero-width bytes across buffer boundaries in fold Modify the fold implementation to process input in buffered chunks rather than line-by-line reading, ensuring correct handling of multi-byte characters split across buffer boundaries. Add process_pending_chunk function and new streaming logic to fold_file for better performance on large files. Update tests accordingly.

dbad8db

7 days ago

by mattsu2020

-0.46%

refactor(fold): condense variable assignment to single line Consolidate the assignment of the `valid` variable from multiple lines to a single line for improved code readability and adherence to style guidelines favoring concise declarations.

75c542f

7 days ago

by mattsu2020

-0.03%

Merge branch 'main' into fold_compatibility

662a011

5 days ago

by mattsu2020

Merge branch 'main' into fold_compatibility

3b07c8e

5 days ago

by mattsu2020

+3.43%

Merge branch 'main' into fold_compatibility

1d35753

2 days ago

by mattsu2020

fix(fold): properly handle combining characters in character-counting mode Only coalesce zero-width combining characters into base characters when folding by display columns (WidthMode::Columns). In character-counting mode, treat every scalar value as advancing the counter to match chars().count() semantics, preventing incorrect line breaking for characters with zero-width marks. This ensures consistent behavior across modes as verified by existing tests.

eb7d8c4

2 days ago

by mattsu2020

Home Terms Privacy Docs