Comparing
time-fabrosaurus
(636eccd
) withmain
(e77c74d
)+19%
- Improvement
- 1
- Untouched
- 38
- Skipped
- 5
Different runtime environments detected
Some benchmarks with significant performance changes were compared across different runtime environments, which may affect the accuracy of the results. Learn more about regression causes.Benchmarks
44 total
env_substitution
crates/benchmarks/benches/boot.rs+19%
12.1 µs10.1 µs
namespaced_model_id
crates/benchmarks/benches/boot.rs+1%
2.6 µs2.5 µs
config_serde_roundtrip
crates/benchmarks/benches/boot.rs0%
1.5 ms1.5 ms
config_load_toml
crates/benchmarks/benches/boot.rs0%
1.7 ms1.7 ms
tool_result_to_content_vision[10000]
crates/benchmarks/benches/boot.rs0%
240.7 µs240.7 µs
sanitize_tool_result[100000]
crates/benchmarks/benches/boot.rs0%
1.5 ms1.5 ms
sanitize_tool_result[1000000]
crates/benchmarks/benches/boot.rs0%
15.1 ms15.1 ms
tool_result_to_content_vision[100000]
crates/benchmarks/benches/boot.rs0%
2.2 ms2.2 ms
context_window_lookup[claude-sonnet-4-5-20250929]
crates/benchmarks/benches/boot.rs0%
2.6 µs2.6 µs
tool_result_to_content_vision[1000000]
crates/benchmarks/benches/boot.rs0%
22.3 ms22.3 ms
vision_support_lookup[gpt-4o]
crates/benchmarks/benches/boot.rs0%
2.4 µs2.4 µs
context_window_lookup[gpt-5]
crates/benchmarks/benches/boot.rs0%
2.4 µs2.4 µs
vision_support_lookup[kimi-k2.5]
crates/benchmarks/benches/boot.rs0%
2.5 µs2.5 µs
vision_support_lookup[gpt-5]
crates/benchmarks/benches/boot.rs0%
2.4 µs2.4 µs
context_window_lookup[kimi-k2.5]
crates/benchmarks/benches/boot.rs0%
2.5 µs2.5 µs
context_window_lookup[unknown-model-xyz]
crates/benchmarks/benches/boot.rs0%
2.7 µs2.7 µs
context_window_lookup[codestral-latest]
crates/benchmarks/benches/boot.rs0%
2.5 µs2.5 µs
context_window_lookup[gemini-2.0-flash]
crates/benchmarks/benches/boot.rs0%
2.6 µs2.6 µs
context_window_lookup[gpt-4o]
crates/benchmarks/benches/boot.rs0%
2.5 µs2.5 µs
vision_support_lookup[unknown-model-xyz]
crates/benchmarks/benches/boot.rs0%
2.6 µs2.6 µs
context_window_lookup[mistral-large-latest]
crates/benchmarks/benches/boot.rs0%
2.6 µs2.6 µs
vision_support_lookup[gemini-2.0-flash]
crates/benchmarks/benches/boot.rs0%
2.5 µs2.5 µs
context_window_lookup[o3]
crates/benchmarks/benches/boot.rs0%
2.4 µs2.4 µs
vision_support_lookup[o3]
crates/benchmarks/benches/boot.rs0%
2.4 µs2.4 µs
vision_support_lookup[mistral-large-latest]
crates/benchmarks/benches/boot.rs0%
2.6 µs2.6 µs
Commits
Click on a commit to change the comparison rangeBase
main
e77c74d
-0.45%
fix(providers): increase model probe timeout for local LLM servers
54a4d42
19 hours ago
by penso+19.84%
fix(providers): address PR review feedback
636eccd
19 hours ago
by penso