Skip to main content

Ctrl+K

Installation

Install RTP-LLM

Release Version

RTP-LLM 0.2.0

Basic Usage

Sending Requests
OpenAI APIs - Completions
OpenAI APIs - Vision
OpenAI APIs - Embedding
RTP-LLM Native APIs

Backend Tutorial

deepseek
qwen-moe
kimi

Advanced Backend Configurations

ServerArgs
Sampling Config
Attention Backend

Supported Models

Large Language Models
Multimodal Language Models
Embedding Models
How to Support New Models

Advanced Features

Speculative Decoding
ReuseCache
Tool and Function Calling
Quantization
LoRA Serving
PD Disaggregation

RTP-LLM Router

FlexLB (Flexible Load Balancer) - Master Role

Benchmark

RTP-LLM Performance Benchmark Tool

References

General Guidance
Developer Reference
Documentation Manual

Repository
Show source
Suggest edit
Open issue

.rst

Developer Reference

Developer Reference#

Contributing Guide
Update whl Deps
Environment Variable Development Tutorial
Frontend
Multimodal Part Debug and Separate Deployment

previous

Debugging RTP-LLM

next

Contributing Guide

By RTP-LLM Team

© Copyright 2023-2025, RTP-LLM.

Last updated on Oct 15, 2025.