perf/optimize-csv-record-assembler - Branch - kamiazya/web-csv-toolbox

perf: optimize CSVRecordAssembler with single-loop array processing

#571

Comparing

perf/optimize-csv-record-assembler

(

30ac954

) with

main

(

f8d7ffb

)

+3%

Untouched: 57

Benchmarks

Passed

parseString.toArraySync(100 cols, 50 rows, unquoted)

benchmark/main.ts

+16%

15.7 ms13.6 ms

Queuing: large (10000 rows) - default HWM

benchmark/main.ts

+13%

2.5 s2.2 s

Line ending: LF (1000 rows)

benchmark/main.ts

+12%

9.7 ms8.6 ms

Custom delimiter: TSV (100 rows)

benchmark/main.ts

+12%

1,037.7 µs928.8 µs

Custom delimiter: PSV (100 rows)

benchmark/main.ts

+11%

1,040.3 µs934.4 µs

Quote ratio: 25% (1000 rows)

benchmark/main.ts

+11%

12.8 ms11.5 ms

Memory: toArraySync (allocate all at once)

benchmark/main.ts

+11%

30.7 ms27.7 ms

Scaling: 50 rows

benchmark/main.ts

+11%

1.6 ms1.4 ms

Quote ratio: 50% (1000 rows)

benchmark/main.ts

+11%

13.1 ms11.8 ms

parseBinary.toArraySync(50 rows)

benchmark/main.ts

+10%

671.1 µs608.8 µs

Field length: 100 chars (1000 rows)

benchmark/main.ts

+10%

13.8 ms12.5 ms

parseString.toArraySync(10000 cols, 10 rows, unquoted)

benchmark/main.ts

+10%

392.4 ms356 ms

parseString.toArraySync(50 rows)

benchmark/main.ts

+10%

653.8 µs594.3 µs

Scaling: 500 rows

benchmark/main.ts

+10%

14.8 ms13.4 ms

Memory: toIterableIterator (streaming)

benchmark/main.ts

+10%

29.3 ms26.7 ms

parseString.toArraySync(10 cols, 50 rows, unquoted)

benchmark/main.ts

+10%

1.6 ms1.4 ms

Worker perf: large (10000 rows) - mainThread

benchmark/main.ts

+10%

201 ms183.4 ms

Low-level: CSVLexer + CSVRecordAssembler (1000 rows)

benchmark/main.ts

+9%

29.5 ms27 ms

Data transformation: no conversion (100 rows)

benchmark/main.ts

+9%

2.1 ms1.9 ms

parseString.toArraySync(10 cols, 50 rows, quoted)

benchmark/main.ts

+9%

1.6 ms1.4 ms

parseBinary.toIterableIterator(50 rows)

benchmark/main.ts

+9%

665.5 µs609.8 µs

Worker perf: small (100 rows) - mainThread

benchmark/main.ts

+9%

2.1 ms1.9 ms

parseString.toIterableIterator(50 rows)

benchmark/main.ts

+9%

716.4 µs657.3 µs

Engine comparison: mainThread (500 rows)

benchmark/main.ts

+9%

15.9 ms14.6 ms

Concurrent: Parallel mainThread

benchmark/main.ts

+9%

36.4 ms33.4 ms

Data transformation: type conversion (100 rows)

benchmark/main.ts

+9%

2.1 ms2 ms

parseString.toArraySync(50 cols, 50 rows, unquoted)

benchmark/main.ts

+9%

7.4 ms6.8 ms

Concurrent: Sequential mainThread

benchmark/main.ts

+8%

36.3 ms33.4 ms

Field length: 10 chars (1000 rows)

benchmark/main.ts

+8%

13 ms12 ms

parseString engine:mainThread (50 rows)

benchmark/main.ts

+8%

769.7 µs711.9 µs

parseString.toArraySync(50 cols, 50 rows, quoted)

benchmark/main.ts

+8%

7.6 ms7.1 ms

Scaling: 1000 rows

benchmark/main.ts

+8%

29.9 ms27.6 ms

Worker perf: tiny (10 rows) - mainThread

benchmark/main.ts

+8%

299.3 µs277.6 µs

Engine comparison: mainThread (5000 rows)

benchmark/main.ts

+8%

160.9 ms149.4 ms

parseString engine:mainThread (1000 rows)

benchmark/main.ts

+7%

15.6 ms14.5 ms

parseBinary.toStream(50 rows)

benchmark/main.ts

+6%

1.2 ms1.1 ms

parseString.toArraySync(10000 cols, 10 rows, quoted)

benchmark/main.ts

+6%

396.4 ms373 ms

parseString.toStream(50 rows)

benchmark/main.ts

+5%

1.1 ms1.1 ms

Scaling: 5000 rows

benchmark/main.ts

+5%

156.9 ms149.7 ms

Field length: 1KB (1000 rows)

benchmark/main.ts

+4%

27.7 ms26.7 ms

Quote ratio: 0% (1000 rows)

benchmark/main.ts

+3%

12.5 ms12.1 ms

Line ending: CRLF (1000 rows)

benchmark/main.ts

+3%

9.7 ms9.4 ms

Queuing: small (100 rows) - default HWM

benchmark/main.ts

+2%

21.8 ms21.3 ms

parseString engine:wasm (50 rows)

benchmark/main.ts

+1%

594 µs587 µs

Field length: 10KB (100 rows)

benchmark/main.ts

+1%

16.8 ms16.6 ms

parseString engine:wasm (1000 rows)

benchmark/main.ts

9 ms9 ms

Low-level: CSVLexer only (1000 rows)

benchmark/main.ts

16.5 ms16.5 ms

Engine comparison: wasm (5000 rows)

benchmark/main.ts

82.4 ms82.5 ms

Engine comparison: wasm (500 rows)

benchmark/main.ts

8.4 ms8.4 ms

parseStringStream: medium (1000 rows)

benchmark/main.ts

138.3 ms138.7 ms

parseStringStream: small (100 rows)

benchmark/main.ts

-1%

14.3 ms14.5 ms

Quote ratio: 100% (1000 rows)

benchmark/main.ts

-2%

13.7 ms13.9 ms

Scaling: 100 rows

benchmark/main.ts

-3%

3 ms3.1 ms

Queuing: medium (1000 rows) - default HWM

benchmark/main.ts

-19%

255.6 ms316.7 ms

parseStringToArraySyncWASM(50 rows)

benchmark/main.ts

-32%

490.3 µs721.1 µs

Worker perf: medium (1000 rows) - mainThread

benchmark/main.ts

-38%

20.1 ms32.5 ms

parseString.toArraySync(100 cols, 50 rows, quoted)

benchmark/main.ts

-46%

15.3 ms28.2 ms

Commits

Click on a commit to change the comparison range

Base

main

f8d7ffb

+3.32%

perf: optimize CSVRecordAssembler with single-loop array processing Replace chained array methods with efficient single-loop implementation to reduce array iterations from 3 passes to 1 pass. Changes: - RecordDelimiter handler: map().filter().map() → single loop - Empty line handler: filter().map() → single loop - Flush handler: map().filter().map() → single loop This optimization reduces CPU cycles during record assembly, particularly beneficial for CSVs with many columns. All 460 tests pass. Complements CSVLexer buffer pointer optimization. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

30ac954

29 days ago

by kamiazya

Home Terms Privacy Docs