Commits
Click on a commit to change the comparison rangefeat: add custom date and timestamp formatting for CSV writes
Add support for custom date and timestamp format strings when writing CSV files,
allowing users to control how dates and timestamps are serialized.
- Add date_format and timestamp_format parameters to DataFrame.write_csv()
- Implement formatting using chrono's strftime format specifiers
- Support all standard date/timestamp formats including ISO 8601 (%+)
- Add comprehensive test coverage for various format scenarios
Closes #3788
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> feat: add timezone support for timestamp formatting
Add proper timezone handling when formatting timestamp columns with
custom formats. Timezone-aware timestamps now preserve their timezone
information instead of being converted to UTC.
- Add chrono-tz dependency for timezone parsing
- Extract and apply timezone from Arrow Timestamp type
- Convert UTC timestamps to target timezone before formatting
- Add test for timezone-aware timestamp formatting
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> fix: return error for invalid timezone strings in CSV timestamp formatting
Instead of silently falling back to UTC when timezone parsing fails,
return an error to alert users of invalid timezone strings (e.g., typos
like "America/New_Yrok"). This is consistent with error handling in
other parts of the codebase (e.g., src/daft-schema/src/time_unit.rs).
- Add explicit error handling for timezone parsing failures
- Add test case to verify error is raised for invalid timezones
- Prevents silent data corruption from unintended UTC conversion
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> test: add more test coverage for date/timestamp CSV formatting
Add additional tests to improve code coverage:
- Test Date64 type formatting (uses timestamp_format since Daft converts to Timestamp[ms])
- Parametrized test for all timestamp time units (s, ms, us, ns)
- Test default behavior when no custom format is provided
This increases the test count from 33 to 39 tests for CSV writing.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> docs: add usage examples and timezone docs to write_csv
- Added usage examples for date_format and timestamp_format parameters
- Documented timezone conversion behavior for timezone-aware timestamps
- Added examples for custom date format, timestamp format, and ISO 8601
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> refactor: remove dead Date64 handling code from CSV writer
Date64 is converted to Timestamp[ms] by Daft upon ingestion, so the
Date64 match arm in transform_batch was dead code. This removes it
and adds a comment explaining why Date64 is not handled separately.
The test_write_csv_date64_with_timestamp_format test correctly uses
timestamp_format (not date_format) because the data arrives as
Timestamp[ms] by the time it reaches the CSV writer.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>