BerriAI/litellm - CodSpeed

litellm

Blog Docs Changelog

Performance History

Latest Results

fix(streaming): guard against empty choices in raise_on_model_repetition Vertex AI Gemini Flash/Flash Lite models emit metadata-only chunks with empty choices=[] during web_search streaming. The raise_on_model_repetition method unconditionally accessed choices[0] on the last two chunks, causing an IndexError that surfaced as MidStreamFallbackError. Add a guard to return early when either of the two most recent chunks has no choices. Fixes #27928

sharziki:fix/27928-streaming-empty-choices-crash

4 minutes ago

fix(ollama): extract thinking field from /api/generate response Ollama's /api/generate endpoint returns a top-level 'thinking' field for reasoning models (Qwen3, DeepSeek-R1) but the non-streaming transform_response() only parsed <think> XML tags from the response text. This caused reasoning_content to always be null. Check for the 'thinking' field first, matching the pattern already used in the /api/chat path and the streaming iterator. Fixes #27956

sharziki:fix/27956-ollama-generate-thinking-field

17 minutes ago

fix(streaming): handle empty choices array in streaming chunks Some providers send streaming chunks with empty choices arrays (choices: []) as initialization or ping chunks. This causes IndexError crashes in two locations: 1. streaming_handler.py - raise_on_model_repetition() accesses choices[0] without checking if choices is non-empty. Fix: early return when either of the last two chunks has empty choices, since repetition detection requires valid content. 2. streaming_chunk_builder_utils.py - build_base_response() assumes the first chunk with a 'choices' key has at least one element. An empty list passes the original filter. Fix: require len(choices) > 0 and fall back to role='assistant' when no chunk has non-empty choices. Added tests covering empty, mixed, and normal cases.

Ausoj:fix/handle-empty-choices-in-streaming

52 minutes ago

fix(streaming): handle empty choices array in streaming chunks Some providers send streaming chunks with empty choices arrays (choices: []) as initialization or ping chunks. This causes IndexError crashes in two locations: 1. streaming_handler.py - raise_on_model_repetition() accesses choices[0] without checking if choices is non-empty. Fix: early return when either of the last two chunks has empty choices, since repetition detection requires valid content. 2. streaming_chunk_builder_utils.py - build_base_response() assumes the first chunk with a 'choices' key has at least one element. An empty list passes the original filter. Fix: require len(choices) > 0 and fall back to role='assistant' when no chunk has non-empty choices. Added tests covering empty, mixed, and normal cases.

Ausoj:fix/handle-empty-choices-in-streaming

55 minutes ago

fix(streaming): handle empty choices array in streaming chunks Some providers send streaming chunks with empty choices arrays (choices: []) as initialization or ping chunks. This causes IndexError crashes in two locations: 1. streaming_handler.py - raise_on_model_repetition() accesses choices[0] without checking if choices is non-empty. Fix: early return when either of the last two chunks has empty choices, since repetition detection requires valid content. 2. streaming_chunk_builder_utils.py - build_base_response() assumes the first chunk with a 'choices' key has at least one element. An empty list passes the original filter. Fix: require len(choices) > 0 and fall back to role='assistant' when no chunk has non-empty choices. Added tests covering empty, mixed, and normal cases.

Ausoj:fix/handle-empty-choices-in-streaming

58 minutes ago

Update litellm/litellm_core_utils/streaming_chunk_builder_utils.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

Ausoj:fix/handle-empty-choices-in-streaming

1 hour ago

fix(streaming): handle empty choices array in streaming chunks Some providers send streaming chunks with empty choices arrays (choices: []) as initialization or ping chunks. This causes IndexError crashes in two locations: 1. streaming_handler.py - raise_on_model_repetition() accesses choices[0] without checking if choices is non-empty. Fix: early return when either of the last two chunks has empty choices, since repetition detection requires valid content. 2. streaming_chunk_builder_utils.py - build_base_response() assumes the first chunk with a 'choices' key has at least one element. An empty list passes the original filter. Fix: require len(choices) > 0 and fall back to role='assistant' when no chunk has non-empty choices. Added tests covering empty, mixed, and normal cases.

Ausoj:fix/handle-empty-choices-in-streaming

1 hour ago

fix(router): isolate retry previous_models metadata

glaziermag:fix-issue-24965

1 hour ago

Latest Branches

0%

fix(streaming): guard against empty choices in raise_on_model_repetition#27974

6 minutes ago

4548fda

sharziki:fix/27928-streaming-empty-choices-crash

0%

fix(ollama): extract thinking field from /api/generate non-streaming response#27973

19 minutes ago

0a7c50c

sharziki:fix/27956-ollama-generate-thinking-field

0%

fix(streaming): handle empty choices array in streaming chunks#27971

2 hours ago

e6ad59d

Ausoj:fix/handle-empty-choices-in-streaming

© 2026 CodSpeed Technology

Home Terms Privacy Docs