Cap concurrency limit retry-after to prevent excessive delays from burst traffic
When burst traffic hits a concurrency limit, the denied_slots counter
accumulates rapidly. Previously, the retry-after calculation would multiply
the wait time by the number of blocking slots (including denied_slots/limit),
leading to excessive delays - e.g., 89 minutes when avg_slot_occupancy is 10s
but denied_slots reaches 16000.
This fix caps the final average_interval at max_wait (default 30s) to ensure
retry-after values remain reasonable even during burst traffic conditions.
The per-slot wait time was already capped, but the total was not.
Example scenario:
- slots=1, denied_slots=16000, limit=30
- blocking_slots = (1 + 16000) / 30 = 533.7
- Before: retry_after = 10s * 533.7 = 5337s (89 minutes)
- After: retry_after = min(10s * 533.7, 30s) ≈ 30s
Reference: Nebula PR #10947
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add bulk operation endpoints for flow runs, deployments, and flows
This adds the following bulk operation endpoints ported from Nebula:
- POST /flow_runs/bulk_delete - Bulk delete flow runs
- POST /flow_runs/bulk_set_state - Bulk set state for flow runs
- POST /deployments/bulk_delete - Bulk delete deployments
- POST /deployments/{id}/create_flow_run/bulk - Bulk create flow runs from deployment
- POST /flows/bulk_delete - Bulk delete flows
Also adds:
- `not_any_` filter to FlowFilterId for excluding flow IDs
- Response schemas for all bulk operations
- Model functions for batch deletion operations
Key implementation details:
- All bulk endpoints default to a limit of 50 items (100 for bulk create)
- Flow run deletion queues log cleanup to Docket
- Flow deletion cascades to delete associated deployments
- Bulk create pre-creates unique work queues to avoid race conditions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>