Latest Results
feat!: rename binary stream APIs for consistency and add BufferSource support (#603)
* feat!: rename binary stream APIs for consistency and add BufferSource support
## Breaking Changes
### API Renaming for Consistency
- parseUint8ArrayStream() → parseBinaryStream()
- parseUint8ArrayStreamToStream() → parseBinaryStreamToStream()
- ParseUint8ArrayStreamOptions → ParseBinaryStreamOptions
- Internal functions also renamed for consistency
### BufferSource Support
- FlexibleBinaryCSVParser now accepts BufferSource (ArrayBuffer | ArrayBufferView)
- BinaryCSVParserStream now accepts BufferSource chunks
- Added comprehensive test coverage for all BufferSource types
## Changes
### API Updates (99 occurrences across 26 files)
- Renamed all parseUint8Array* functions to parseBinary*
- Updated all type definitions and imports
- Used git mv to preserve file history
### Bug Fixes
- Fixed vite.config.ts entry points to use new function names
- Updated benchmark/main.ts to use parseBinaryStream
- Updated Hono example to use parseBinaryStream API
### Tests Added
- BufferSource support tests in FlexibleBinaryCSVParser.test.ts (+6 tests)
- BufferSource support tests in BinaryCSVParserStream.test.ts (+5 tests)
- Total: 1266 tests (10 tests added)
### Documentation
- Updated README.md with new API names and flush procedures
- Updated package-exports.md with comprehensive Low-level API Reference
- Updated parsing-architecture.md with BufferSource examples
- Updated all how-to guides to use new API names
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: update README.md to reflect BufferSource support
- Changed BinaryCSVParserStream type from TransformStream<Uint8Array, CSVRecord> to TransformStream<BufferSource, CSVRecord>
- Updated parse() input description to use BufferSource instead of separate Uint8Array/ArrayBuffer entries
- Updated parseBinary() description to mention BufferSource
- Updated maxBinarySize option descriptions (3 locations) to use BufferSource terminology
- Maintains consistency with the BufferSource support added in this PR
* refactor: simplify binary stream parsing and improve test readability
* docs: fix BufferSource link to use Web IDL specification
Changed BufferSource link from broken MDN URL to the official Web IDL specification (WHATWG):
https://webidl.spec.whatwg.org/#BufferSource
This is the primary source for the BufferSource typedef definition.
* refactor: complete BufferSource API consistency and consolidate stream options
This commit completes the BufferSource migration and refactors streaming
options for better maintainability and consistency.
## BufferSource API Consistency
**Core Type Updates:**
- Updated `CSVBinary` type to use `BufferSource` instead of `Uint8Array | ArrayBuffer`
- Modified `convertBinaryToString()` to accept `BufferSource` and simplified `TextDecoder.decode()` call
**Public API Updates:**
- Updated all sync binary parsing functions to accept `BufferSource`:
- `parseBinaryToArraySync()`
- `parseBinaryToIterableIterator()`
- `parseBinaryToStream()`
- Updated `parseBinary()` main function and all namespace methods:
- `parseBinary.toArray()`
- `parseBinary.toArraySync()`
- `parseBinary.toIterableIterator()`
- `parseBinary.toStream()`
**Internal Implementation Updates:**
- Updated execution strategies to accept `BufferSource`:
- `parseBinaryInMain()`
- `parseBinaryInWASM()`
- `parseBinaryInWorker()`
- `parseBinaryInWorkerWASM()`
- Updated Worker communication types:
- `ParseBinaryRequest.data` now accepts `BufferSource`
- Fixed `parse()` function to correctly detect `BufferSource` using `ArrayBuffer.isView()`
- Added proper `BufferSource` to `Uint8Array` conversion in worker message handler
**Benefits:**
- Consistent API across all binary parsing functions
- Better support for TypedArray views (Int8Array, DataView, etc.)
- Improved type safety with Web IDL standard `BufferSource` type
## Stream Options Consolidation
**New Base Interface:**
- Added `BackpressureOptions` interface to `src/core/types.ts` following the pattern of `AbortSignalOptions` and `SourceOption`
- Provides centralized documentation for backpressure control behavior
**Refactored Stream Option Types:**
- Updated `CSVLexerTransformerStreamOptions` to extend `BackpressureOptions`
- Updated `CSVRecordAssemblerTransformerStreamOptions` to extend `BackpressureOptions`
- Added `BinaryCSVParserStreamOptions` extending `BackpressureOptions`
- Added `StringCSVParserStreamOptions` extending `BackpressureOptions`
- Moved options from individual stream files to central types file
**DRY Improvements:**
- Eliminated duplicate backpressure documentation across 4 different types
- Consistent inheritance pattern for all stream-related options
## JSDoc Cleanup
**Removed Unused Generic Parameters:**
- Removed unused `@template Format` from `FlexibleStringCSVParser` class JSDoc
- Removed unused `@template Format` from `BinaryCSVParserStream` class JSDoc
- Removed unused `@template Format` from `StringCSVParserStream` class JSDoc
**Documentation Improvements:**
- Updated examples in `FlexibleStringCSVParser` to use `console.log(records)` instead of `console.log([...records])`
- Changed method-level `@returns` to explicitly state `CSVRecord<Header>[]`
- Ensured all JSDoc consistently references only the `Header` generic
## Testing
- ✅ All type checks passing (148 tests)
- ✅ All benchmark tests passing (65 benchmarks)
- ✅ No breaking changes to public API (additive only)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* style: apply code formatting
Applied biome formatting to 6 files.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: enforce output format in Parser classes and add Format type parameter to ParserStreams
This commit implements strict output format enforcement and improves type safety
for streaming operations:
1. Relocate factory functions to src/parser/api/model:
- Move createStringCSVParser, createBinaryCSVParser, createCSVRecordAssembler,
and createStringCSVLexer to centralized API location
- Update all import paths across the codebase
2. Split Parser classes by output format:
- FlexibleStringCSVParser → FlexibleStringObjectCSVParser / FlexibleStringArrayCSVParser
- FlexibleBinaryCSVParser → FlexibleBinaryObjectCSVParser / FlexibleBinaryArrayCSVParser
- Introduce BaseStringCSVParser and BaseBinaryCSVParser base classes
- Each concrete parser class enforces its declared output format in constructor
3. Add Format type parameter to ParserStream classes:
- StringCSVParserStream<Header, Format> with Format extends "object" | "array"
- BinaryCSVParserStream<Header, Format> with Format extends "object" | "array"
- Update TransformStream output type to CSVRecord<Header, Format>
4. Restore array format tests for ParserStreams:
- Add tests for array record parsing
- Add tests for includeHeader option in array format
- Add tests for records split across chunks in array format
This ensures that:
- FlexibleStringObjectCSVParser always outputs objects
- FlexibleStringArrayCSVParser always outputs arrays
- Stream classes properly support both object and array formats
- Type safety is maintained throughout the parsing pipeline
All tests pass (1268 tests).
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: introduce CSVProcessingOptions to separate parsing spec from execution strategy
This commit introduces CSVProcessingOptions and BinaryCSVProcessingOptions to clearly
separate CSV processing specification from execution strategy (EngineOptions).
**New types:**
- CSVProcessingOptions: Processing spec for string CSV (delimiter, header, outputFormat, etc.)
- BinaryCSVProcessingOptions: Processing spec for binary CSV (adds charset, decompression, etc.)
**Updated types:**
- ParseOptions: Now extends CSVProcessingOptions + EngineOptions
- ParseBinaryOptions: Now extends BinaryCSVProcessingOptions + EngineOptions
**Benefits:**
1. Clear separation of concerns:
- Processing spec (what to parse and how) vs execution strategy (where to execute)
2. Low-level APIs (Parser classes) accept only processing spec:
- No engine option available → prevents confusion
- Focus on CSV parsing logic only
3. High-level APIs (parseString, etc.) accept both:
- Processing spec + execution strategy (worker, WASM, etc.)
4. Future-proof for stringify and other operations:
- CSVProcessingOptions is not parse-specific
- Can be reused for CSV generation/transformation
**Changes:**
- types.ts: Add CSVProcessingOptions and BinaryCSVProcessingOptions
- BaseStringCSVParser: Use CSVProcessingOptions instead of ParseOptions
- BaseBinaryCSVParser: Use BinaryCSVProcessingOptions instead of ParseBinaryOptions
- All concrete Parser classes: Updated constructor signatures with new types
All tests pass (1268 tests).
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: improve type inference for dynamic outputFormat and update documentation
This commit addresses two critical issues:
1. **Documentation synchronization**: Updated all documentation to use new factory functions
- Replaced FlexibleStringCSVParser/FlexibleBinaryCSVParser with createStringCSVParser/createBinaryCSVParser
- Updated examples in README.md, custom-csv-parser.md, and other documentation files
- Ensures users following documentation won't encounter compilation errors
2. **Dynamic outputFormat type inference**: Fixed type inference for runtime-determined formats
- Added overload using Omit<T, "outputFormat"> to properly override property types
- When outputFormat is "object" | "array" (union), now correctly returns union parser type
- Maintains backward compatibility for literal types and default behavior
- Added comprehensive type tests for dynamic format scenarios (10 new tests)
The key insight: intersection types (A & { prop: X }) don't properly override existing
optional properties. Using Omit utility type resolves this TypeScript limitation.
Type inference now works correctly:
- outputFormat: "array" → StringArrayCSVParser
- outputFormat: "object" → StringObjectCSVParser
- outputFormat: "object" | "array" → StringArrayCSVParser | StringObjectCSVParser (union)
- outputFormat omitted → StringObjectCSVParser (default)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: import types for CSVArrayRecord and CSVObjectRecord in dynamic tests
* refactor: use ByteLengthQueuingStrategy and centralize backpressure check intervals
This commit improves code quality and maintainability:
1. **Use Web Streams API standard**: BinaryCSVParserStream now uses ByteLengthQueuingStrategy
- Replaced custom object literal with standard ByteLengthQueuingStrategy
- Type cast added for BufferSource compatibility (runtime-safe)
- Updated JSDoc comments to reflect the change
2. **Centralize magic numbers**: Added constants for backpressure check intervals
- DEFAULT_STREAM_BACKPRESSURE_CHECK_INTERVAL = 100 (for parsers and lexer)
- DEFAULT_ASSEMBLER_BACKPRESSURE_CHECK_INTERVAL = 10 (for record assembler)
- All stream transformers now use these constants instead of magic numbers
- Improved code maintainability and consistency
3. **Minor cleanup**: Simplified BaseStringCSVParser constructor default value
Updated files:
- src/core/constants.ts: Added new constants with detailed documentation
- src/parser/stream/BinaryCSVParserStream.ts: ByteLengthQueuingStrategy + constant
- src/parser/stream/StringCSVParserStream.ts: Constant usage
- src/parser/stream/CSVLexerTransformer.ts: Constant usage
- src/parser/stream/CSVRecordAssemblerTransformer.ts: Constant usage
All tests pass (72 tests, no type errors).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: reorder import statements for consistency in CSVLexerTransformer
---------
Co-authored-by: Claude <noreply@anthropic.com> Active Branches
#559N/A
#558-89%
#591×3.4
© 2025 CodSpeed Technology