---
slug: "vllm"
name: "vLLM"
effect: "Serves open-weight models at high throughput behind an OpenAI-compatible API server"
category: "model-api"
url: "https://vllm.ai"
github: "vllm-project/vllm"
pricing: "open-source"
license: "Apache-2.0"
tags: ["inference", "serving", "gpu", "openai-compatible", "self-hosted", "python"]
platforms: ["linux", "docker"]
install_kind: "pip"
date_added: "2026-10-14"
drop_score: 59
stars: 93801
pushed_at: "2026-10-15T01:13:32Z"
signals_status: "ok"
signals_fetched_at: "2026-10-15T01:19:06.167Z"
page: "https://agenttooldrop.com/tools/vllm"
---

# vLLM

Serves open-weight models at high throughput behind an OpenAI-compatible API server

Category: Models and APIs. Pricing: Open source. License: Apache-2.0.

Platforms: linux, docker.

## Why you would want it

vLLM is an inference and serving engine built around PagedAttention, continuous batching, chunked prefill and prefix caching, with many quantization formats and speculative decoding. It runs an OpenAI-compatible API server plus Anthropic Messages API and gRPC support, with tool calling and reasoning parsers and structured outputs via xgrammar or guidance. Tensor, pipeline, data and expert parallelism cover distributed inference, and it supports 200+ Hugging Face model architectures on NVIDIA, AMD and Intel GPUs, CPUs and hardware plugins.

## Best for

- Self-hosting open models on GPUs
- Serving many concurrent agent requests
- Distributed multi-GPU inference

## Use it when

You need to serve open-weight models to agents or apps at scale on your own GPU infrastructure.

## Skip it if

You are on Windows, which vLLM does not support natively (use WSL), or you only want to try a model on a laptop, where a lighter local runner is simpler to set up.

## Install

Install with uv:

```sh
uv pip install vllm
vllm download-kernels
```

Source: https://github.com/vllm-project/vllm#readme

## Live signals

Fetched 2026-10-15T01:19:06.167Z (status: ok).

- Repository: vllm-project/vllm
- Stars: 93,801
- Forks: 23,388
- Open issues and PRs: 8,877
- License (GitHub): Apache-2.0
- Last push: 2026-10-15
- Latest release: v0.31.0 (2026-10-05)

## Links

- Website: https://vllm.ai
- GitHub: https://github.com/vllm-project/vllm
- Install instructions verified at: https://github.com/vllm-project/vllm#readme
- Source for this entry: https://github.com/vllm-project/vllm#readme
- AgentToolDrop page: https://agenttooldrop.com/tools/vllm
- Full catalog (JSON): https://agenttooldrop.com/tools.json