deepseek-ai/DeepSeek-R1-0528
DeepSeek-R1-0528 is a 671B-parameter MoE reasoning model built on the DeepSeek-V3 architecture, trained with large-scale reinforcement learning for strong chain-of-thought capabilities.
Open-weights RL-trained reasoning model with native FP8. Verified on 8x/16x VA16
View on HuggingFaceGuide
Overview
DeepSeek-R1-0528 is a large language model with 671B parameters. Compared with DeepSeek-R1, DeepSeek-R1-0528 has significantly improved its depth of reasoning and inference capabilities by leveraging increased computational resources and introducing algorithmic optimization mechanisms during post-training. The model has demonstrated outstanding performance across various benchmark evaluations, including mathematics, programming, and general logic.
Prerequisites
- vLLM version: 0.17.0
- Config: GPUs = 8x VA16, Precision = FP8, TP = 32, max-model-len = 64K
Example config only. Refer to above for others.
Start Docker Container
docker run \
--privileged=true \
--name vllm_service \
--shm-size=256g \
--ipc=host \
-p 8000:8000 \
-it \
-v ~/.cache/huggingface:/root/.cache/huggingface \
harbor.vastaitech.com/ai_deliver/vllm_vacc:latest \
bash
Launching the Server
FP8 on 8x VA16, Tool Calling
vllm serve deepseek-ai/DeepSeek-R1-0528 \
--trust-remote-code \
--tensor-parallel-size 32 \
--max-model-len 65535 \
--enable-auto-tool-choice --tool-call-parser deepseek_v3 \
--chat_template tool_chat_template_deepseekr1.jinja \
--enforce-eager
Client Usage
from openai import OpenAI
client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
resp = client.chat.completions.create(
model="deepseek-ai/DeepSeek-R1-0528",
messages=[{"role": "user", "content": "Explain gated delta networks in one paragraph."}],
max_tokens=512,
)
print(resp.choices[0].message.content)