|
--- |
|
library_name: transformers |
|
license: cc-by-sa-4.0 |
|
datasets: |
|
- sbintuitions/JCommonsenseQA |
|
language: |
|
- ja |
|
base_model: |
|
- microsoft/bitnet-b1.58-2B-4T-bf16 |
|
--- |
|
|
|
# Model Card for Model ID |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
|
|
--- |
|
library_name: transformers |
|
license: cc-by-sa-4.0 |
|
datasets: |
|
- sbintuitions/JCommonsenseQA |
|
language: |
|
- ja |
|
base_model: |
|
- microsoft/bitnet-b1.58-2B-4T-bf16 |
|
--- |
|
|
|
|
|
|
|
## Model Details |
|
microsoftのbitnet-b1をJCommonsenseQAでLoRAチューニングしたものです。未コンパイルなので、bf16のまま動作しています。 |
|
|
|
## Test |
|
messages = [ |
|
{"role": "system", "content": "You are a helpful AI assistant."}, |
|
{"role": "user", "content": "主に子ども向けのもので、イラストのついた物語が書かれているものは？"}, |
|
] |
|
### Before Tuning |
|
この質問は、英文の言語で="In the end, it is stories about children that make me write manga."と言えます。 |
|
### After Tuning |
|
0 den ishani, SHAKA, SU, OC, ANIMAL, MUSEUM, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL |
|
INTO, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL, ANIMAL |
|
ANIMAL, ANIMAL, ANIMAL, ANIMAL |
|
|
|
## Result |
|
チューニングが甘かったため、むしろ言語能力が下がってしまいました。 |
