Latest Results
perf(core): stop `parse_partial_json` from retrying one character at a time (#41467)
Related: #40826
---
When repaired JSON still fails to parse, `parse_partial_json` trims the
end and retries. It used to trim one character per retry and re-decode
the whole string each time, which is quadratic on a long bad tail. A
common example is valid JSON followed by prose.
## When this helps
Models asked for JSON often keep talking after it:
```text
{"title": "Q3 summary", "items": [...]}
Here is a short explanation of the fields above. ...
```
`JsonOutputParser` (also used by
`ChatOpenAI.with_structured_output(method="json_mode")`) parses with
`parse_partial_json`. When streaming, it re-parses the accumulated text
on every chunk. Once the JSON closes, each chunk pays one full decode
per character of prose so far, so the stream stalls as the prose grows:
```python
chain = model | JsonOutputParser()
for partial in chain.stream("..."): # stalls once the model starts explaining
...
```
Fake model streaming a ~540-character JSON object followed by prose,
token by token:
| Prose after the JSON | Before | After |
| --- | ---: | ---: |
| 1,000 chars | 1.40 s | 0.06 s |
| 4,000 chars | 27.78 s | 0.20 s |
The same retry loop also runs when streamed tool-call args end in a long
broken value, and in `JsonOutputToolsParser` and its subclasses.
## Change
- It now cuts straight to `JSONDecodeError.pos`. Every longer candidate
contains the same rejected character, so it would fail the same way.
- If the error is in the appended closing brackets, it still trims one
character.
- Return values and errors are unchanged: 400k fuzzed inputs match the
old function.
```mermaid
flowchart LR
A[repaired string] --> B{json.loads}
B -- ok --> R[return]
B -- "error at pos" --> C{pos inside input?}
C -- yes --> D["truncate to pos (new)"]
C -- no --> E[drop last char]
D --> B
E --> B
```
<details>
<summary>Before / after: decode attempts for <code>{"answer": 42} Hope
this helps!</code></summary>
Both return `{"answer": 42}`.
**Before: 18 `json.loads` calls**
```text
'{"answer": 42} Hope this helps!'
'{"answer": 42} Hope this helps!'
'{"answer": 42} Hope this helps'
'{"answer": 42} Hope this help'
'{"answer": 42} Hope this hel'
... (one per character)
'{"answer": 42} H'
'{"answer": 42} '
```
**After: 3 `json.loads` calls**
```text
'{"answer": 42} Hope this helps!'
'{"answer": 42} Hope this helps!' -> "Extra data" at pos 15
'{"answer": 42} '
```
</details>
Single `parse_partial_json` call:
| Input | Length | Before | After |
| --- | ---: | ---: | ---: |
| Valid JSON + trailing prose | 10 KB | 144 ms | 0.35 ms |
| Valid JSON + trailing prose | 40 KB | 2,041 ms | 1.2 ms |
| Partial value + junk tail | 40 KB | 2,050 ms | 1.1 ms | Latest Branches
0%
mdrxy/core/merge-stream-chunks-once 0%
mdrxy/core/partial-json-linear-trim 0%
casparb/core/expected-cancellation © 2026 CodSpeed Technology