Is the chat_template.json file wrong?
#41
by
mohamed99akram
- opened
The chat_template.json you provided seems to output
"<|im_start|>User:{PROMPT_TEXT} <image><end_of_utterance>\nAssistant:"
although your example is
"<|im_start|>User:<image>{PROMPT_TEXT}<end_of_utterance>\nAssistant:"
Shouldn't <image> be before the prompt?
I am using PROMPT_TEXT="Convert this page to docling." and it is not accurate.
When I try to put this template as a jinja file for vLLM it gives bad output.
And when trying this template:
<|im_start|>{% for message in messages %}{{message['role'] | capitalize}}:{% for line in message['content'] %}{% if line['type'] == 'image' or line['type'] == 'image_url' %}{{ '<image>' }}{% endif %}{% endfor %}{% for line in message['content'] %}{% if line['type'] == 'text' %}{{ line['text'] }}{% endif %}{% endfor %}<end_of_utterance>
{% endfor %}{% if add_generation_prompt %}Assistant:{% endif %}
which works in vLLM.
So, in summary, Is there a problem with the provided chat template json file?
Thanks.