CLIRadar
vllm-project avatar

vllm

vllm-project/vllm

View on GitHub

A high-throughput and memory-efficient inference and serving engine for LLMs

Stars84.5k
Stars today
Forks18.6k
Trend score0

Install

$ pipx install vllm
Latest releaseNo releases
Last pushedtoday
Created3 years ago
Homepagevllm.ai

Topics

amdblackwellcudadeepseekdeepseek-v3gptgpt-ossinferencekimillamallmllm-servingmodel-servingmoeopenaipytorchqwenqwen3tputransformer

More AI CLI Agents