cc/model_profiles_distributed - Branch - langchain-ai/langchain

feat(model-profiles): distribute data across packages

#34024

Comparing

cc/model_profiles_distributed

(

4fd9c77

) with

master

(

525d5c0

)

-24%

Regressions: 2

Untouched: 26

Skipped: 6

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data. For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

No successful run was found on master ( ee3373a) during the generation of this report, so 525d5c0 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

Benchmarks

Skipped (6)

These benchmarks were skipped, so their baseline results are used instead. If they were deleted in your codebase, archive them to remove them from the performance reports.
Learn more about archiving benchmarks

test_create_chat_prompt_init_time

libs/partners/prompty/tests/unit_tests/test_standard.py

Skipped

311.9 µs*

test_exa_retriever_init_time

libs/partners/exa/tests/unit_tests/test_standard.py

Skipped

325.3 µs*

test_qdrant_vectorstore_init_time

libs/partners/qdrant/tests/unit_tests/test_standard.py

Skipped

224.2 ms*

test_chroma_init_time

libs/partners/chroma/tests/unit_tests/test_standard.py

Skipped

57.2 ms*

test_init_time

libs/partners/ollama/tests/unit_tests/test_chat_models.py::TestChatOllama

Skipped

1.6 s*

test_nomic_embeddings_init_time

libs/partners/nomic/tests/unit_tests/test_standard.py

Skipped

1.5 ms*

Failed

test_async_callbacks_in_sync

libs/core/tests/benchmarks/test_async_callbacks.py

Regression

-24%

18.4 ms24.2 ms

test_import_time[HumanMessage]

libs/core/tests/benchmarks/test_imports.py

Regression

-10%

236.9 ms263.6 ms

Passed

test_init_time

libs/partners/perplexity/tests/unit_tests/test_chat_models_standard.py::TestPerplexityStandard

837.5 ms*834.6 ms

test_stream_time

libs/partners/anthropic/tests/integration_tests/test_standard.py::TestAnthropicStandard

34.7 ms*34.7 ms

test_stream_time

libs/partners/openai/tests/integration_tests/chat_models/test_base_standard.py::TestOpenAIStandard

1.2 s*1.2 s

test_init_time

libs/partners/xai/tests/unit_tests/test_chat_models_standard.py::TestXAIStandard

3.3 s*3.3 s

test_init_time

libs/partners/deepseek/tests/unit_tests/test_chat_models.py::TestChatDeepSeekUnit

1.6 s*1.6 s

test_init_time

libs/partners/groq/tests/unit_tests/test_standard.py::TestGroqStandard

1.6 s*1.6 s

test_init_time

libs/partners/openai/tests/unit_tests/chat_models/test_azure_standard.py::TestOpenAIStandard

1.7 s*1.7 s

test_stream_time

libs/partners/openai/tests/integration_tests/chat_models/test_responses_standard.py::TestOpenAIResponses

857.3 ms*857.4 ms

test_stream_time

libs/partners/openai/tests/integration_tests/chat_models/test_responses_standard.py::TestOpenAIStandard

1.2 s*1.2 s

test_init_time

libs/partners/fireworks/tests/unit_tests/test_standard.py::TestFireworksStandard

6.6 s*6.6 s

test_init_time

libs/partners/mistralai/tests/unit_tests/test_standard.py::TestMistralStandard

9.1 ms*9.1 ms

test_init_time

libs/partners/openai/tests/unit_tests/chat_models/test_responses_standard.py::TestOpenAIResponses

-1%

12.2 ms*12.3 ms

test_init_time

libs/partners/openai/tests/unit_tests/chat_models/test_base_standard.py::TestOpenAIStandard

-1%

12.1 ms*12.2 ms

test_init_time_with_client

libs/partners/anthropic/tests/unit_tests/test_standard.py

-3%

2.2 ms*2.3 ms

test_import_time[Document]

libs/core/tests/benchmarks/test_imports.py

-5%

174.9 ms184.7 ms

test_init_time

libs/partners/anthropic/tests/unit_tests/test_standard.py::TestAnthropicStandard

-5%

763.3 µs*806.9 µs

test_import_time[RunnableLambda]

libs/core/tests/benchmarks/test_imports.py

-6%

447.5 ms476.4 ms

test_import_time[InMemoryVectorStore]

libs/core/tests/benchmarks/test_imports.py

-7%

559.9 ms599.6 ms

test_import_time[InMemoryRateLimiter]

libs/core/tests/benchmarks/test_imports.py

-7%

160.3 ms171.8 ms

test_import_time[ChatPromptTemplate]

libs/core/tests/benchmarks/test_imports.py

-8%

534.9 ms578.6 ms

test_import_time[LangChainTracer]

libs/core/tests/benchmarks/test_imports.py

-8%

395.2 ms427.7 ms

test_import_time[Runnable]

libs/core/tests/benchmarks/test_imports.py

-8%

444.2 ms483.6 ms

test_import_time[BaseChatModel]

libs/core/tests/benchmarks/test_imports.py

-8%

468.8 ms510.8 ms

test_import_time[CallbackManager]

libs/core/tests/benchmarks/test_imports.py

-9%

407.1 ms446.4 ms

test_import_time[PydanticOutputParser]

libs/core/tests/benchmarks/test_imports.py

-10%

465.9 ms516 ms

test_import_time[tool]

libs/core/tests/benchmarks/test_imports.py

-10%

451.6 ms501 ms

Commits

Click on a commit to change the comparison range

Base

master

525d5c0

+1.37%

add data to anthropic

2f96ba6

2 days ago

by ccurme

+1.02%

support on anthropic

829eed4

2 days ago

by ccurme

-28.14%

allow setting profile fields without mutation

55c0a4d

2 days ago

by ccurme

+2.52%

Merge branch 'master' into cc/model_profiles_distributed

97a6015

23 hours ago

by ccurme

+19.23%

42a39d1

23 hours ago

by ccurme

-18.85%

update langchain-model-profiles readme

b5bee8d

22 hours ago

by ccurme

-2.3%

update openai

3012d7b

21 hours ago

by ccurme

+1.57%

update langchain_v1

a0beff2

21 hours ago

by ccurme

+0.53%

xai

a86c3f2

17 hours ago

by ccurme

-0.72%

resolve todo

4fd9c77

16 hours ago

by ccurme

Home Terms Privacy Docs