Skip to main content

Ctrl+K

Installation

Install RTP-LLM

Release Version

RTP-LLM 0.2.0
- Overview
- package

Basic Usage

Sending Requests
OpenAI APIs - Completions
OpenAI APIs - Vision
OpenAI APIs - Embedding
RTP-LLM Native APIs

Backend Tutorial

deepseek
qwen-moe
kimi

Advanced Backend Configurations

ServerArgs
Sampling Config
Attention Backend

Supported Models

Large Language Models
Multimodal Language Models
Embedding Models
How to Support New Models

Advanced Features

Speculative Decoding
ReuseCache
Tool and Function Calling
Quantization
LoRA Serving
PD Disaggregation

RTP-LLM Router

FlexLB (Flexible Load Balancer) - Master Role

Benchmark

RTP-LLM Performance Benchmark Tool

References

General Guidance
Developer Reference
Documentation Manual

Repository
Show source
Suggest edit
Open issue

.rst

RTP-LLM 0.2.0

RTP-LLM 0.2.0#

Overview
Features
Question of omission
Performance
Compatibility
package

previous

Install RTP-LLM

next

Overview

By RTP-LLM Team

© Copyright 2023-2025, RTP-LLM.

Last updated on Oct 15, 2025.