add vllm source code part. (#19)
Browse files- add vllm source code part. (ebb38e6711d29dc7208aeee3f522bf2a6023cee0)
Co-authored-by: asher <asherszhang@users.noreply.huggingface.co>
README.md
CHANGED
@@ -266,7 +266,7 @@ docker run --rm --ipc=host \
|
|
266 |
--gpus=all \
|
267 |
-it \
|
268 |
-e VLLM_USE_V1=0 \
|
269 |
-
--entrypoint python
|
270 |
-m vllm.entrypoints.openai.api_server \
|
271 |
--host 0.0.0.0 \
|
272 |
--tensor-parallel-size 4 \
|
@@ -275,6 +275,12 @@ docker run --rm --ipc=host \
|
|
275 |
--trust_remote_code
|
276 |
```
|
277 |
|
|
|
|
|
|
|
|
|
|
|
|
|
278 |
|
279 |
#### Tool Calling with vLLM
|
280 |
|
@@ -296,6 +302,8 @@ These settings enable vLLM to correctly interpret and route tool calls generated
|
|
296 |
|
297 |
vLLM reasoning parser support on Hunyuan A13B model is under development.
|
298 |
|
|
|
|
|
299 |
### SGLang
|
300 |
|
301 |
#### Docker Image
|
|
|
266 |
--gpus=all \
|
267 |
-it \
|
268 |
-e VLLM_USE_V1=0 \
|
269 |
+
--entrypoint python hunyuaninfer/hunyuan-a13b:hunyuan-moe-A13B-vllm \
|
270 |
-m vllm.entrypoints.openai.api_server \
|
271 |
--host 0.0.0.0 \
|
272 |
--tensor-parallel-size 4 \
|
|
|
275 |
--trust_remote_code
|
276 |
```
|
277 |
|
278 |
+
### Source Code
|
279 |
+
Support for this model has been added via this [PR 20114](https://github.com/vllm-project/vllm/pull/20114 ) in the vLLM project.
|
280 |
+
|
281 |
+
You can build and run vLLM from source after merging this pull request into your local repository.
|
282 |
+
|
283 |
+
|
284 |
|
285 |
#### Tool Calling with vLLM
|
286 |
|
|
|
302 |
|
303 |
vLLM reasoning parser support on Hunyuan A13B model is under development.
|
304 |
|
305 |
+
|
306 |
+
|
307 |
### SGLang
|
308 |
|
309 |
#### Docker Image
|