MaximeBOUDIER:fix/gcp-iam-redis-token-caching - Branch - BerriAI/litellm

fix(redis): cache GCP IAM token to prevent async event loop blocking

#26317Closed

Comparing

MaximeBOUDIER:fix/gcp-iam-redis-token-caching

(

5e61737

) with

main

(

e9e86ed

)

Untouched: 16

Benchmarks

16 total

test_cost_per_token_openai

tests/benchmarks/test_benchmarks.py

-2%

539.5 µs548.2 µs

test_cost_per_token_anthropic

tests/benchmarks/test_benchmarks.py

-1%

541.8 µs549.5 µs

test_get_model_info_anthropic

tests/benchmarks/test_benchmarks.py

+1%

80.1 µs79.2 µs

test_get_model_cost_key_case_insensitive

tests/benchmarks/test_benchmarks.py

-1%

81.8 µs82.3 µs

test_token_counter_long_content

tests/benchmarks/test_benchmarks.py

+1%

1.7 ms1.7 ms

test_get_model_info_openai

tests/benchmarks/test_benchmarks.py

80.8 µs80.4 µs

test_token_counter_with_tools

tests/benchmarks/test_benchmarks.py

414.8 µs412.9 µs

test_token_counter_simple_message

tests/benchmarks/test_benchmarks.py

236.6 µs235.6 µs

test_token_counter_multi_turn

tests/benchmarks/test_benchmarks.py

569 µs566.8 µs

test_get_llm_provider_azure

tests/benchmarks/test_benchmarks.py

143.7 µs144.2 µs

test_get_model_info_with_provider

tests/benchmarks/test_benchmarks.py

80 µs80.3 µs

test_token_counter_raw_text

tests/benchmarks/test_benchmarks.py

183.8 µs183.2 µs

test_get_model_cost_key_exact_match

tests/benchmarks/test_benchmarks.py

79.6 µs79.8 µs

test_get_llm_provider_openai

tests/benchmarks/test_benchmarks.py

138.2 µs138.1 µs

test_get_llm_provider_with_prefix

tests/benchmarks/test_benchmarks.py

137.2 µs137 µs

test_get_llm_provider_anthropic

tests/benchmarks/test_benchmarks.py

141.5 µs141.4 µs

Commits

Click on a commit to change the comparison range

Base

main

e9e86ed

-0.03%

fix(redis): cache GCP IAM token to prevent async event loop blocking

5e61737

4 months ago

by MaximeBOUDIER

Home Terms Privacy Docs