Comparing
cc/patch_invalid_tool_calls
(5814556
) withmain
(2ac7d41
)-5%
- Untouched
- 15
- Skipped
- 79
Benchmarks
94 total
test_filesystem_init
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentBenchmark0%
362.3 µs*361.7 µs
test_full_featured
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentBenchmark-3%
179.6 ms*186 ms
test_scaling_tools[20_tools]
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentScaling-5%
52.8 ms*55.3 ms
test_scaling_tools[5_tools]
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentScaling-5%
50.1 ms*52.8 ms
test_scaling_subagents[1_subagents]
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentScaling-5%
73.6 ms*77.5 ms
test_scaling_tools[1_tools]
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentScaling-5%
49.5 ms*52.1 ms
test_scaling_tools[10_tools]
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentScaling-5%
51 ms*53.7 ms
test_with_one_subagent
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentBenchmark-5%
73.6 ms*77.6 ms
test_with_string_model_resolution
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentBenchmark-5%
49.4 ms*52.1 ms
test_scaling_subagents[3_subagents]
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentScaling-5%
122.2 ms*128.9 ms
test_with_tools
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentBenchmark-5%
49.8 ms*52.5 ms
test_with_multiple_subagents
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentBenchmark-5%
170.6 ms*180 ms
test_create_deep_agent_minimal
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentBenchmark-5%
49 ms*51.7 ms
test_scaling_subagents[10_subagents]
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentScaling-5%
292.5 ms*308.9 ms
test_scaling_subagents[5_subagents]
libs/deepagents/tests/benchmarks/test_benchmark_create_deep_agent.py::TestCreateDeepAgentScaling-5%
170.8 ms*180.5 ms
The benchmarks below were skipped, so their baseline results are used instead. If they were deleted in your codebase, archive them to remove them from the performance reports.
Learn more about archiving benchmarks
Learn more about archiving benchmarks
test_repl_memory_peak[console_log-8_threads]
libs/partners/quickjs/tests/benchmarks/test_quickjs_memory.py::TestQuickJSMemoryBenchmarksSkipped
6.7 MB*
test_repl_memory_peak[console_log-32_threads]
libs/partners/quickjs/tests/benchmarks/test_quickjs_memory.py::TestQuickJSMemoryBenchmarksSkipped
15.3 MB*
test_repl_memory_peak[ptc_tools-8_threads]
libs/partners/quickjs/tests/benchmarks/test_quickjs_memory.py::TestQuickJSMemoryBenchmarksSkipped
5.2 MB*
test_repl_memory_peak[console_log-1_threads]
libs/partners/quickjs/tests/benchmarks/test_quickjs_memory.py::TestQuickJSMemoryBenchmarksSkipped
1 MB*
test_repl_memory_peak[ptc_tools-32_threads]
libs/partners/quickjs/tests/benchmarks/test_quickjs_memory.py::TestQuickJSMemoryBenchmarksSkipped
19.4 MB*
test_repl_memory_peak[ptc_tools-1_threads]
libs/partners/quickjs/tests/benchmarks/test_quickjs_memory.py::TestQuickJSMemoryBenchmarksSkipped
1 MB*
test_repl_memory_peak[console_log-64_threads]
libs/partners/quickjs/tests/benchmarks/test_quickjs_memory.py::TestQuickJSMemoryBenchmarksSkipped
16.4 MB*
test_repl_memory_peak[ptc_tools-64_threads]
libs/partners/quickjs/tests/benchmarks/test_quickjs_memory.py::TestQuickJSMemoryBenchmarksSkipped
38.7 MB*
test_multi_turn_snapshot_throughput[snapshot_disabled-50_turns]
libs/partners/quickjs/tests/benchmarks/test_quickjs_throughput.py::TestQuickJSThroughputBenchmarksSkipped
6.1 s*
test_multi_turn_snapshot_throughput[snapshot_enabled-10_turns]
libs/partners/quickjs/tests/benchmarks/test_quickjs_throughput.py::TestQuickJSThroughputBenchmarksSkipped
1.2 s*
Commits
Click on a commit to change the comparison rangeBase
main
2ac7d41
-0.66%
patch invalid tool calls
59a3462
18 days ago
by ccurme-0.68%
cr
eaa503e
18 days ago
by ccurme-3.31%
Revert "propagate goto and graph in Commands when intercepting large tool results"
5814556
18 days ago
by ccurme