Avatar for the PrefectHQ user
PrefectHQ
prefect
BlogDocsChangelog

Performance History

Latest Results

test(schedules): update remaining string-equality assertions for #21362 CI on the previous commit caught seven more tests asserting exact rrule string equality on values that flow through `DeploymentScheduleCreate` (which now injects `DTSTART`): - `tests/test_flows.py::TestFlowToDeployment::TestSync::test_to_deployment_accepts_rrule` - `tests/test_flows.py::TestFlowToDeployment::TestAsync::test_to_deployment_accepts_rrule` - `tests/test_flows.py::TestFlowServe::test_serve_creates_deployment_with_rrule_schedule` - `tests/cli/deployment/test_deployment_cli.py::test_list_schedules` - `tests/cli/deployment/test_deployment_cli.py::test_list_schedules_with_json_output` - `tests/cli/test_deploy.py::TestSchedules::test_can_provide_multiple_schedules_via_command` - `tests/cli/test_deploy.py::TestSchedules::test_can_provide_multiple_schedules_via_yaml` Updated to assert on the user-supplied rrule body via `.endswith()` (or substring containment for the CLI list output) so they survive any future change to the canonical normalization form. Same pattern as the six tests fixed in the original commit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix/21362-cache-rrule
59 minutes ago
fix(schedules): harden #21362 backfill migration against pathological rows Three concrete safety improvements after a closer review of the migration's failure modes: 1. **`normalize_rrule_string` returns the input unchanged on parse failure** instead of synthesizing a `DTSTART:20200101T000000\n` prefix in front of garbage. The previous behavior was harmless on the action-schema write path (where `validate_rrule_string` runs first and rejects unparseable input), but on the migration path it meant we could rewrite a corrupted row into a *different* corrupted row. Now corrupted input is left bit-for-bit alone and the migration's `if normalized == rrule: continue` skips it. 2. **Migration wraps each row in `try/except`.** A single pathological row used to abort the entire transaction and roll back the backfill. Now bad rows are logged via `alembic.runtime.migration` and skipped — they keep their original value and the scheduler continues to use the legacy 2020 anchor for them, while every other row is still backfilled successfully. 3. **Empty rrule strings (`""`) are explicitly skipped** via a `not rrule` short-circuit. They shouldn't exist in the wild (`validate_rrule_string` rejects them) but defensive is cheap. Verified end-to-end via `repros/21362_migration_stress.py` against a fresh sqlite DB with adversarial seeded rows covering: every safe and unsafe rrule shape; already-anchored rules; non-RRule schedules; empty rrule; unparseable garbage rrule; schedule field that's a list instead of a dict; schedule with extra keys. All cases produce the expected outcome. Then re-run via downgrade-then-upgrade to confirm idempotency (state byte-identical between runs) and finally the downgrade itself (verified to be a no-op as documented). Reassurances that did NOT need code changes: - **Alembic wraps each migration in a transaction** (verified by reading `_migrations/env.py`: `transaction_per_migration=True`). Any unhandled exception rolls back the entire migration. The per-row try/except above is belt-and-suspenders on top of that. - **The migration is idempotent** without any explicit guard: rows whose `rrule` already contains `DTSTART` short-circuit, and the recent-anchor case is phase-equivalent so re-running with a different `now` produces the same forward occurrence set. - **Concurrent writes during migration** are not a real concern in practice (migrations run with the server stopped), and even if a row is inserted between the SELECT and the per-row UPDATE, the inserted row goes through the action-schema validator and already has DTSTART. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix/21362-cache-rrule
1 hour ago
fix(schedules): persist explicit DTSTART on RRule schedules at write time closes #21362 `RRuleSchedule.to_rrule()` historically fell back to a hardcoded `DEFAULT_ANCHOR_DATE = 2020-01-01` whenever the persisted rrule string lacked an explicit `DTSTART`. Because dateutil's `xafter()` is O(n) in the number of occurrences between `dtstart` and the query time, a `FREQ=MINUTELY;INTERVAL=5` schedule walks ~660k occurrences from 2020 forward on every scheduler loop. With ~20-30 such deployments the scheduler saturates a CPU core (the original report). The previous attempt on this branch — process-level caching of the parsed rrule object — fixed the CPU symptom but traded it for ~37 MB of retained memory per high-frequency rule, which is the wrong tradeoff on small containers. This commit replaces that approach. The right fix is to make `DTSTART` explicit on every persisted rrule: 1. **`normalize_rrule_string`** in `_internal/schemas/validators.py` inspects an incoming rrule and either picks a phase-equivalent recent anchor (for `FREQ=SECONDLY` / `FREQ=MINUTELY` without `COUNT`) or injects the legacy `DTSTART:20200101T000000` (every other shape, including `INTERVAL>1`, `COUNT=N`, `BY*` calendar rules, and rrulesets). The recent-anchor case is provably occurrence-set-equivalent forward of the new dtstart, and shrinks dateutil's working set from millions of cached datetimes to ~tens. The legacy case is byte-for-byte semantically equivalent to the pre-fix implicit-anchor parsing. 2. **`DeploymentScheduleCreate` / `DeploymentScheduleUpdate`** action schemas (both server and client) gain a field validator that runs the normalization on incoming `RRuleSchedule` payloads. `DeploymentCreate` and `DeploymentUpdate` inherit it transitively through their inline `schedules` lists. Crucially, the validator lives on the **action** schemas (write path), not on `RRuleSchedule` itself — if it lived there, every DB row deserialized into an `RRuleSchedule` would get a fresh anchor injected on every load, which would change daily and re-phase `INTERVAL>1` schedules. This is the same drift bug the earlier draft PR #21361 hit. 3. **Alembic data migration** (paired SQLite + PostgreSQL) walks every row in `deployment_schedule` and injects `DTSTART` for any RRule that lacks one, using the same `normalize_rrule_string` helper. New deployments arrive pre-normalized via the action schemas; existing rows are backfilled once at upgrade. 4. **`DEFAULT_ANCHOR_DATE`** stays in both client and server schedule modules as a defensive fallback for any rule that somehow still reaches `to_rrule()` without a `DTSTART` (old clients on the wire, YAML, test fixtures, mid-upgrade rows). It is no longer the load-bearing path for the scheduler. Tests cover: the helper directly (22 cases including phase-preservation across all relevant rrule shapes and a memory smoke check), action-schema integration on both server and client, and the "deserialization-doesn't-mutate" invariant on both sides. Existing tests that asserted exact rrule string equality on values flowing through `DeploymentScheduleCreate` were updated to use `.endswith()` on the user-supplied portion (they were silently asserting an implementation detail). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix/21362-cache-rrule
2 hours ago

Latest Branches

CodSpeed Performance Gauge
0%
fix(schedules): persist explicit DTSTART on RRule schedules at write time#21436
1 hour ago
909f455
fix/21362-cache-rrule
CodSpeed Performance Gauge
0%
5 hours ago
2c018e4
devin/1775494005-task-runs-work-pool-filter
CodSpeed Performance Gauge
0%
2 days ago
8cbedac
harsh21234i:feat/deploy-from-yaml-sdk
© 2026 CodSpeed Technology
Home Terms Privacy Docs