Qwen/Qwen3-Embedding
The Qwen3-Embedding models are designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining.
Compact Qwen3-Embedding model
View on HuggingFaceGuide
OverView
Qwen3-Embedding is a text embedding model that provides dense vector representations of text. It is part of the Qwen3 series, which offers models of varying sizes to suit different use cases.
Prerequisites
- vLLM version: 0.17.0
- Config: GPUs = 1x VA16, TP = 4
Example config only. Refer to above for others.
Start Docker Container
docker run \
--privileged=true \
--name vllm_service \
--shm-size=256g \
--ipc=host \
-p 8000:8000 \
-it \
-v ~/.cache/huggingface:/root/.cache/huggingface \
harbor.vastaitech.com/ai_deliver/vllm_vacc:latest \
bash
Launching the Server
8B on 1x VA16
vllm serve Qwen/Qwen3-Embedding-8B \
--trust-remote-code \
--tensor-parallel-size 4 \
--hf-overrides '{"is_matryoshka": true}' \
--enforce-eager
Client Usage
from openai import OpenAI
client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
resp = client.embeddings.create(
model="Qwen/Qwen3-Embedding-8B",
input="This is an example text that i want to turn into embedding."
)
print(resp.data[0].embedding)