manaestras asherszhang commited on
Commit
70a8c87
·
verified ·
1 Parent(s): 202c975

add vllm source code part. (#19)

Browse files

- add vllm source code part. (ebb38e6711d29dc7208aeee3f522bf2a6023cee0)


Co-authored-by: asher <asherszhang@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +9 -1
README.md CHANGED
@@ -266,7 +266,7 @@ docker run --rm --ipc=host \
266
  --gpus=all \
267
  -it \
268
  -e VLLM_USE_V1=0 \
269
- --entrypoint python mirror.ccs.tencentyun.com/hunyuaninfer/hunyuan-large:hunyuan-moe-A13B-vllm \
270
  -m vllm.entrypoints.openai.api_server \
271
  --host 0.0.0.0 \
272
  --tensor-parallel-size 4 \
@@ -275,6 +275,12 @@ docker run --rm --ipc=host \
275
  --trust_remote_code
276
  ```
277
 
 
 
 
 
 
 
278
 
279
  #### Tool Calling with vLLM
280
 
@@ -296,6 +302,8 @@ These settings enable vLLM to correctly interpret and route tool calls generated
296
 
297
  vLLM reasoning parser support on Hunyuan A13B model is under development.
298
 
 
 
299
  ### SGLang
300
 
301
  #### Docker Image
 
266
  --gpus=all \
267
  -it \
268
  -e VLLM_USE_V1=0 \
269
+ --entrypoint python hunyuaninfer/hunyuan-a13b:hunyuan-moe-A13B-vllm \
270
  -m vllm.entrypoints.openai.api_server \
271
  --host 0.0.0.0 \
272
  --tensor-parallel-size 4 \
 
275
  --trust_remote_code
276
  ```
277
 
278
+ ### Source Code
279
+ Support for this model has been added via this [PR 20114](https://github.com/vllm-project/vllm/pull/20114 ) in the vLLM project.
280
+
281
+ You can build and run vLLM from source after merging this pull request into your local repository.
282
+
283
+
284
 
285
  #### Tool Calling with vLLM
286
 
 
302
 
303
  vLLM reasoning parser support on Hunyuan A13B model is under development.
304
 
305
+
306
+
307
  ### SGLang
308
 
309
  #### Docker Image