v1.7.11

💥 LiteLLM Router + Proxy handles 500+ requests/second

💥LiteLLM Proxy - Now handles 500+ requests/second, Load Balance Azure + OpenAI deployments, Track spend per user 💥
Try it here: https://docs.litellm.ai/docs/simple_proxy
🔑 Support for AZURE_OPENAI_API_KEY on Azure https://docs.litellm.ai/docs/providers/azure
h/t
@solyarisoftware
⚡️ LiteLLM Router can now handle 20% more throughput https://docs.litellm.ai/docs/routing
📖Improvement to litellm debugging docs h/t
@solyarisoftware
https://docs.litellm.ai/docs/debugging/local_debugging

Full Changelog: v1.7.1...v1.7.11

v1.7.1

What's Changed

• 🚨 LiteLLM Proxy uses Async completion/embedding calls on this release onwards - this led to 30x more throughput for embedding/completion calls

New Contributors

• @guspan-tanadi made their first contribution in #851
• @Manouchehri made their first contribution in #880
• @maqsoodshaik made their first contribution in #884
• @okotek made their first contribution in #885
• @kumaranvpl made their first contribution in #902

Full Changelog: v1.1.0...v1.7.1

v1.1.0

What's Changed

🚨 Breaking Change v1.1.0 -> This version is only compatible with OpenAI python 1.1.0
Migration Guide: https://docs.litellm.ai/docs/migration

Key changes in v1.1.0

• Requires openai>=1.0.0
• openai.InvalidRequestError → openai.BadRequestError
• openai.ServiceUnavailableError → openai.APIStatusError
• NEW litellm client, allow users to pass api_key
  • litellm.Litellm(api_key="sk-123")
• response objects now inherit from BaseModel (prev. OpenAIObject)
• NEW default exception - APIConnectionError (prev. APIError)
• litellm.get_max_tokens() now returns an int not a dict

  max_tokens = litellm.get_max_tokens("gpt-3.5-turbo") # returns an int not a dict 
  assert max_tokens==4097

Other updates

• Update function calling docs by @kevinjyee in #673
• Fix data being overwritten by @mc-marcocheng in #679
• Updating the docker image builder for GitHub Action by @coconut49 in #678
• fix: bugs in traceloop integration by @nirga in #647
• Router aembedding by @mc-marcocheng in #691
• support release and debug params for langfuse client by @SlapDrone in #695
• docs error ==> openai.error instead of openai.errors by @josearangos in #700
• refactor Contributing to documentation steps by @josearangos in #713
• Fix Router.set_model_list & Avoid overwriting litellm_params by @mc-marcocheng in #706
• Update Together AI pricing by @dedeswim in #724
• Update README.md by @chinmay7016 in #727
• Router.get_available_deployment: Handle empty input edge case by @mc-marcocheng in #729
• Fix caching for Router by @karvetskiy in #722
• support for custom bedrock runtime endpoint by @canada4663 in #717
• Use supplied headers by @stanfea in #741
• Docker Hub image is built for ARM64 only by @morgendigital in #734
• doc name chagne by @kylehh in #764
• fix: fix bug for the case --model is not specified by @clalanliu in #781
• add custom open ai models to asyncio call by @PrathamSoni in #789
• Fix bad returns in get_available_deployment by @nathankim7 in #790
• Improve message trimming by @duc-phamh in #787
• build(deps): bump postcss from 8.4.27 to 8.4.31 in /docs/my-website by @dependabot in #804
• build(deps): bump urllib3 from 2.0.5 to 2.0.7 by @dependabot in #805
• build(deps): bump @babel/traverse from 7.22.10 to 7.23.3 in /docs/my-website by @dependabot in #806
• Fix ServiceUnavailableError super.init error by @jackmpcollins in #813
• Update Together prices by @dedeswim in #814
• need to re-attempt backoff and yaml imports if the first import attempt fails by @kfsone in #820
• Fix typo for initial_prompt_value and too many values to unpack error by @rodneyxr in #826
• Bedrock llama by @dchristian3188 in #811
• build(deps): bump sharp from 0.32.5 to 0.32.6 in /docs/my-website by @dependabot in #832

New Contributors

• @kevinjyee made their first contribution in #673
• @mc-marcocheng made their first contribution in #679
• @SlapDrone made their first contribution in #695
• @josearangos made their first contribution in #700
• @dedeswim made their first contribution in #724
• @chinmay7016 made their first contribution in #727
• @karvetskiy made their first contribution in #722
• @stanfea made their first contribution in #741
• @morgendigital made their first contribution in #734
• @clalanliu made their first contribution in #781
• @PrathamSoni made their first contribution in #789
• @nathankim7 made their first contribution in #790
• @duc-phamh made their first contribution in #787
• @dependabot made their first contribution in #804
• @jackmpcollins made their first contribution in #813
• @kfsone made their first contribution in #820
• @rodneyxr made their first contribution in #826
• @dchristian3188 made their first contribution in #811

Full Changelog: v0.11.1...v1.1.0

v0.11.1

What's Changed

• Update init.py model_list to include bedrock models by @canada4663 in #609
• proxy /models endpoint with the results of get_valid_models() by @canada4663 in #611
• fix: llm_provider add openai finetune compatibility by @Undertone0809 in #618
• Update README.md by @Shivam250702 in #620
• Verbose warning by @toniengelhardt in #625
• Update the Dockerfile of the LiteLLM Proxy server and some refactorings by @coconut49 in #628
• fix: updates to traceloop docs by @nirga in #639
• docs: fixed typo in Traceloop docs by @nirga in #640
• fix: disabled batch by default for Traceloop by @nirga in #643
• Create GitHub Action to automatically build docker images by @coconut49 in #634
• Tutorial for using LiteLLM within Gradio Chatbot Application by @dcruiz01 in #645
• proxy server: fix langroid part by @pchalasani in #652
• Create GitHub Action to automatically build docker images by @coconut49 in #655
• deepinfra: Add supported models by @ichernev in #638
• Update index.md by @Pratikdate in #663
• Add perplexity namespace to model pricing dict by @toniengelhardt in #665
• Incorrect boto3 parameter name by @shrikant14 in #671

New Contributors

• @Undertone0809 made their first contribution in #618
• @Shivam250702 made their first contribution in #620
• @dcruiz01 made their first contribution in #645
• @ichernev made their first contribution in #638
• @Pratikdate made their first contribution in #663
• @shrikant14 made their first contribution in #671

Full Changelog: v0.8.4...v0.11.1

v0.8.4

🚨 IMPORTANT v0.8.4 has one major breaking change

• LiteLLM raises exceptions for models that do not support a specific param. Example, sending functions to litellm.completion(model="claude-2", messages, functions) raises an exception since claude-2 does not support function calling. To bypass the exception raised set litellm.drop_params=True More info here: https://docs.litellm.ai/docs/completion/input
• We now adopt semantic versioning for our releases learn more about semantic versioning.
• Check out LiteLLM Proxy - https://docs.litellm.ai/docs/proxy_server create an OpenAI compatible proxy to call 100+ LLMs in one I/O format

What's Changed

• Add missing litellm_provider for gpt-3.5-16k-0613 by @mocy in #436
• added feedback button from feedbackrocket.io by @NANDINI-star in #443
• Fix: merge conflict by @bitsnaps in #495
• Update boto3 dependency to version 1.28.57, refactor bedrock client initialization and remove troubleshooting guide from documentation. by @coconut49 in #497
• added model openrouter/mistralai/mistral-7b-instruct with test by @lucashofer in #498
• add bedrock.anthropic support for system prompt using tag by @canada4663 in #499
• remove .DS_Store and update .gitignore by @linediconsine in #500
• Update README.md by @eltociear in #518
• Update utils.py by @vedant-z in #530
• [docs] minor typo correction by @Akash190104 in #537
• Readme Update by @AnderMendoza in #556
• Add host option to run_server() by @Sir-Photch in #558
• Add support for passing external bedrock clients to completion by @zhooda in #562
• Add custom_openai type in provider list by @kylehh in #560
• shorter langroid example, update section title by @pchalasani in #581
• Fix usage open in colab link by @biplobsd in #605

New Contributors

• @mocy made their first contribution in #436
• @bitsnaps made their first contribution in #495
• @coconut49 made their first contribution in #497
• @lucashofer made their first contribution in #498
• @canada4663 made their first contribution in #499
• @linediconsine made their first contribution in #500
• @eltociear made their first contribution in #518
• @vedant-z made their first contribution in #530
• @Akash190104 made their first contribution in #537
• @AnderMendoza made their first contribution in #556
• @Sir-Photch made their first contribution in #558
• @zhooda made their first contribution in #562
• @kylehh made their first contribution in #560
• @pchalasani made their first contribution in #581
• @biplobsd made their first contribution in #605

Full Changelog: v0.1.738...v0.8.4

v0.1.738

What's Changed

• remove DS_Store by @happysalada in #315
• Update custom-mapping table to be more readable by @pratik2315 in #305
• Add support for overriding API type for Azure calls by @Taik in #330
• Fix proxy server railway deployment by @Henry-Pulver in #338
• util: verify_access_key by @WilliamEspegren in #346
• docs: add languages to code blocks by @jordanbtucker in #348
• Add **kwargs to mock_completion by @toniengelhardt in #367
• Simplify mock logic by @toniengelhardt in #371
• added crisp chat by @NANDINI-star in #372
• fix OpenAI completion model names by @toniengelhardt in #373
• fix set correct mock response by @toniengelhardt in #380
• Main by @Phodaie in #388
• Add PROMPTMETHEUS to projects using LiteLLM by @toniengelhardt in #390
• Rebuild stream chunks to openAI object by @WilliamEspegren in #387
• Add Dashboard for showing error logs on exception by @ishaan-jaff in #416

New Contributors

• @happysalada made their first contribution in #315
• @pratik2315 made their first contribution in #305
• @Taik made their first contribution in #330
• @Henry-Pulver made their first contribution in #338
• @WilliamEspegren made their first contribution in #346
• @jordanbtucker made their first contribution in #348
• @toniengelhardt made their first contribution in #367
• @Phodaie made their first contribution in #388
• @ishaan-jaff made their first contribution in #416

Full Changelog: v0.1.574...v0.1.738

v0.1.574

What's Changed

• fix: context propagation on acompletion method by @galkleinman in #224
• feat: added support for OPENAI_API_BASE by @ErikBjare in #222
• Fix streaming anthropic by @adriensas in #185
• feat: traceloop docs by @nirga in #227
• fix llmonitorLogger undefined by @vincelwt in #235
• Add async iterator to fix/enable streaming responses by @estill01 in #240
• docs: added traceloop to sidebar by @nirga in #243
• Fix TypeError in Exception mapping by @yujonglee in #247
• Remove unnecessary print in AI21 handling by @yujonglee in #264
• added import aiohttp by @shauryr in #294
• Update fastrepl repo by @yujonglee in #304

New Contributors

• @galkleinman made their first contribution in #224
• @ErikBjare made their first contribution in #222
• @adriensas made their first contribution in #185
• @nirga made their first contribution in #227
• @estill01 made their first contribution in #240
• @shauryr made their first contribution in #294

Full Changelog: v0.1.492...v0.1.574

0.1.492

What's Changed

• Fix ModelResponse typing by @yujonglee in #144
• Fixing favicon path #142 by @NANDINI-star in #152
• Added importlib, changed favicon, fixed version by @zakhar-kogan in #180
• added lunr search bar in docs by @NANDINI-star in #187
• Support for LLMonitor - open-source observability & analytics by @vincelwt in #181
• modified the image extension to .png by @NANDINI-star in #189

New Contributors

• @yujonglee made their first contribution in #144
• @NANDINI-star made their first contribution in #152
• @vincelwt made their first contribution in #181

Full Changelog: v0.1.387...v0.1.492

v0.1.387

What's Changed

• Update main.py by @krrishdholakia in #5
• Added OpenRouter support by @zakhar-kogan in #8
• Code clean up by @krrishdholakia in #23
• Exception mapping by @krrishdholakia in #30
• updating version by @krrishdholakia in #31
• custom timeout decorator by @krrishdholakia in #32
• Expanded .env, added Poetry and basic Docstring by @zakhar-kogan in #34
• Openrouter.ai support by @zakhar-kogan in #66
• OpenRouter in docs by @zakhar-kogan in #93
• Move Anthropic to it's own class + custom llm provider flag by @krrishdholakia in #106

New Contributors

• @krrishdholakia made their first contribution in #5
• @zakhar-kogan made their first contribution in #8

Full Changelog: https://github.com/BerriAI/litellm/commits/v0.1.387