Avatar for the moltis-org user
moltis-org
moltis
BlogDocsChangelog

feat(local-llm): on-demand model loading/unloading with idle timeout

#884Merged
Comparing
romantic-bittersweet
(
db258b4
) with
main
(
e5d4c30
)
CodSpeed Performance Gauge
-11%
Regression
1
Untouched
38
Skipped
5

Benchmarks

44 total
session_key_to_filename[project:backend:debug-auth]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
-11%
743.6 ns831.1 ns
vision_support_lookup[claude-sonnet-4-5-20250929]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+4%
2.7 µs2.6 µs
env_substitution
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+4%
15 µs14.4 µs
vision_support_lookup[gpt-4o]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+4%
2.5 µs2.4 µs
context_window_lookup[o3]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+4%
2.4 µs2.4 µs
vision_support_lookup[gpt-5]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+4%
2.5 µs2.4 µs
context_window_lookup[gpt-5]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+4%
2.5 µs2.4 µs
context_window_lookup[codestral-latest]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+4%
2.6 µs2.5 µs
vision_support_lookup[codestral-latest]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+4%
2.6 µs2.5 µs
vision_support_lookup[gemini-2.0-flash]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+3%
2.6 µs2.5 µs
context_window_lookup[gemini-2.0-flash]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+3%
2.7 µs2.6 µs
context_window_lookup[claude-sonnet-4-5-20250929]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+3%
2.7 µs2.6 µs
vision_support_lookup[unknown-model-xyz]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+3%
2.6 µs2.6 µs
context_window_lookup[unknown-model-xyz]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+3%
2.7 µs2.7 µs
context_window_lookup[gpt-4o]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+2%
2.5 µs2.4 µs
vision_support_lookup[kimi-k2.5]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+2%
2.5 µs2.5 µs
vision_support_lookup[mistral-large-latest]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+2%
2.6 µs2.6 µs
context_window_lookup[kimi-k2.5]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+1%
2.6 µs2.6 µs
vision_support_lookup[o3]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+1%
2.4 µs2.4 µs
context_window_lookup[mistral-large-latest]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+1%
2.7 µs2.6 µs
config_serde_roundtrip
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
+1%
2.7 ms2.7 ms
values_to_chat_messages[500]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
0%
540.9 µs539.2 µs
values_to_chat_messages[50]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
0%
67.6 µs67.4 µs
values_to_chat_messages[2000]
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
0%
2.1 ms2.1 ms
config_default_construction
crates/benchmarks/benches/boot.rs
CodSpeed Performance Gauge
0%
56.9 µs56.9 µs

Commits

Click on a commit to change the comparison range
Base
main
e5d4c30
+1.26%
feat(local-llm): on-demand model loading/unloading with idle timeout
75f1864
22 hours ago
by penso
+15.94%
feat(local-llm): add Qwen 3.6 models to local model registry
63049f8
22 hours ago
by penso
-0.33%
fix(local-llm): address PR review — shared provider instances, reason field, lock scope
d2b48c2
22 hours ago
by penso
-55.27%
fix(local-llm): snapshot locks in model_states() to avoid blocking writers
e224e6a
20 hours ago
by penso
+53.14%
fix(local-llm): propagate global timeout to runtime-configured models, refresh lifecycle after discovery
796f977
19 hours ago
by penso
-3.08%
fix(web-ui): wire lifecycle tracking into app init, fix is_loaded during loading
001ff68
19 hours ago
by penso
+43.18%
fix(web-ui): deduplicate lifecycle type, handle RPC failure in button
52a9416
19 hours ago
by penso
-65.37%
feat(chat): pre-load local model before inference, broadcasting lifecycle events
db5a2fb
18 hours ago
by penso
0%
refactor(web-ui): remove manual Load/Unload buttons — lifecycle is fully automatic
db258b4
18 hours ago
by penso
© 2026 CodSpeed Technology
Home Terms Privacy Docs