v1.55.12

What's Changed

• Add 'end_user', 'user' and 'requested_model' on more prometheus metrics by @krrishdholakia in #7399
• (feat) /batches Add support for using /batches endpoints in OAI format by @ishaan-jaff in #7402
• (feat) /batches - track user_api_key_alias, user_api_key_team_alias etc for /batch requests by @ishaan-jaff in #7401
• Litellm dev 12 24 2024 p3 by @krrishdholakia in #7403
• (Feat) add `"/v1/batches/{batch_id:path}/cancel" endpoint by @ishaan-jaff in #7406
• Litellm dev 12 24 2024 p4 by @krrishdholakia in #7407

Full Changelog: v1.55.11...v1.55.12

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.55.12

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	220.0	241.51418849604215	6.334659319234715	0.0	1895	0	191.11329300005764	3854.987871999924
Aggregated	Passed ✅	220.0	241.51418849604215	6.334659319234715	0.0	1895	0	191.11329300005764	3854.987871999924