Issue: Unable to load model weights
#2
by
ZhangRC
- opened
When this model is used for training, it shows the following error:
Some weights of RWKV7ForCausalLM were not initialized from the model checkpoint at fla-hub/rwkv7-191M-world and are newly initialized: ['embeddings.weight', 'layers.0.attn.a_lora.lora.0.weight', 'layers.0.attn.a_lora.lora.2.bias',
... (and another 300 tensors)
Since base_model_prefix = 'model' in the code, all keys in the safetensors should begin with the prefix model..
Thank you for your report. I will check this later.
I think this issue have been fixed. https://huggingface.co/fla-hub/rwkv7-191M-world?show_file_info=model.safetensors