AgentToolDrop

12 of 71

LiteLLM

Calls 100+ LLM providers in the OpenAI format, as a Python SDK or a self-hosted AI gateway

Install with uv

uv add litellm

Copied from the project's docs at github.com/BerriAI/litellm

Why you'd want it

LiteLLM gives one interface to OpenAI, Anthropic, Gemini, Bedrock, Azure and many other providers using the OpenAI request format, so you can swap providers without rewriting code. The Python SDK includes a Router with retry and fallback logic across deployments. The proxy server adds virtual keys, per-project spend tracking, guardrails, load balancing and an admin dashboard, and it can also act as an MCP gateway and an A2A agent gateway.

Best for

  • Platform teams centralizing LLM access
  • Multi-provider apps with fallbacks
  • Tracking spend per project or key

Use it when

You call several model providers and want one OpenAI-format API, or a central gateway with keys, budgets and logging for a team.

Skip it if

You use a single provider and its own SDK covers your needs. Features such as SSO sit under the commercial enterprise license.

Related tools

Same category or shared tags, by drop score.

  1. Rank 35

    vLLM

    Model/API

    Serves open-weight models at high throughput behind an OpenAI-compatible API server

    • score 59
    • 93.8k stars
    • pushed
    • v0.31.0 on
    • Apache-2.0

    Install with uv

    uv pip install vllm
vllm download-kernels
    Use it when, skip it if, stats for vLLM

    Why you'd want it

    vLLM is an inference and serving engine built around PagedAttention, continuous batching, chunked prefill and prefix caching, with many quantization formats and speculative decoding. It runs an OpenAI-compatible API server plus Anthropic Messages API and gRPC support, with tool calling and reasoning parsers and structured outputs via xgrammar or guidance. Tensor, pipeline, data and expert parallelism cover distributed inference, and it supports 200+ Hugging Face model architectures on NVIDIA, AMD and Intel GPUs, CPUs and hardware plugins.

    Best for

    • Self-hosting open models on GPUs
    • Serving many concurrent agent requests
    • Distributed multi-GPU inference

    Use it when

    You need to serve open-weight models to agents or apps at scale on your own GPU infrastructure.

    Skip it if

    You are on Windows, which vLLM does not support natively (use WSL), or you only want to try a model on a laptop, where a lighter local runner is simpler to set up.

    Stats

    Stars
    93,801
    Last push
    Release
    v0.31.0,
    License
    Apache-2.0
    Pricing
    Open source
    Works with
    Linux, Docker
    Drop score
    59 of 100

    Full stat block for vLLM

  2. Rank 6

    Ollama

    Model/API

    Runs open models locally behind a CLI and a REST API, with an OpenAI-compatible endpoint

    • score 83
    • 183k stars
    • pushed
    • v0.40.4 on
    • MIT

    Install on macOS or Linux

    curl -fsSL https://ollama.com/install.sh | sh
    Use it when, skip it if, stats for Ollama

    Why you'd want it

    Ollama downloads and runs open models such as Gemma with one command (ollama run gemma4) and serves them on a local REST API at localhost:11434. It supports a subset of the OpenAI API, so OpenAI clients work by changing the base URL, and it ships official Python and JavaScript libraries. ollama launch connects local models to coding agents including Claude Code, Codex, Copilot CLI and OpenCode, and the same API can reach Ollama's cloud models with an API key.

    Best for

    • Running open models on your own machine
    • Offline or private agent development
    • Pointing coding agents at local models

    Use it when

    You want a local model server that existing OpenAI-style clients and coding agents can talk to with minimal setup.

    Skip it if

    You only use hosted models from providers like Anthropic or OpenAI. Its OpenAI compatibility covers a subset of the OpenAI API, so check the docs if you depend on less common endpoints.

    Stats

    Stars
    183,183
    Last push
    Release
    v0.40.4,
    License
    MIT
    Pricing
    Open source
    Works with
    macOS, Linux, Windows, Docker, Claude Code, Codex
    Drop score
    83 of 100

    Full stat block for Ollama

  3. Rank 42

    OpenRouter

    Model/API

    Routes requests to hundreds of models across providers through one OpenAI-compatible API endpoint

    • score 57
    • No public repo
    • Proprietary hosted service

    Install the Python SDK

    pip install openrouter
    Use it when, skip it if, stats for OpenRouter

    Why you'd want it

    OpenRouter exposes hundreds of models behind a single /api/v1/chat/completions endpoint, handles fallbacks automatically and picks the most cost-effective provider for each request. Existing OpenAI SDK code works by pointing the base URL at OpenRouter, and there are also typed client SDKs for TypeScript and Python plus an Agent SDK. It passes through provider pricing without markup, charges a fee when you buy credits, and supports bringing your own provider keys.

    Best for

    • Switching between models without code changes
    • Comparing models from many providers
    • Apps that need provider fallbacks

    Use it when

    You want one API key and one endpoint for models from many providers, with routing and fallbacks handled for you.

    Skip it if

    You only use one provider and already have direct access. Usage is billed through prepaid credits, with a fee on credit purchases and on bring-your-own-key usage above the free allowance.

    Stats

    GitHub
    no public repo
    License
    Proprietary hosted service
    Pricing
    Paid
    Drop score
    57 of 100

    Full stat block for OpenRouter

For agents

The same entry, as files an agent can read without scraping this page. 

# read one entry
curl -s https://agenttooldrop.com/tools/litellm.md

# list the MCP servers in the catalog
curl -s https://agenttooldrop.com/tools.json \
  | jq -r '.tools[] | select(.category=="mcp-server") | .slug'