[THINK] token not recognized in llama-cpp-python
#1
by
Kenshiro-28
- opened
It looks the [THINK] token is not recognized, as this line:
print(len(model.tokenize(b"[THINK]", add_bos=False)));
Prints a 4 instead of 1
I noticed that some regions near the top of the tensor slice look "flat" several tensor slices have nearly identical values
https://www.reddit.com/r/MistralAI/comments/1o0kvpe/weird_repeating_values_in_magistralsmall2509_gguf/
I added it https://github.com/ggml-org/llama.cpp/commit/128d522c04286e019666bd6ee4d18e3fbf8772e2
Thx! Closing issue :)
Kenshiro-28
changed discussion status to
closed