Update README.md
Browse files
README.md
CHANGED
|
@@ -2,3 +2,26 @@
|
|
| 2 |
license: apache-2.0
|
| 3 |
library_name: transformers
|
| 4 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
license: apache-2.0
|
| 3 |
library_name: transformers
|
| 4 |
---
|
| 5 |
+
|
| 6 |
+
Benchmarks for this model:
|
| 7 |
+
|
| 8 |
+
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
|
| 9 |
+
|--------------|------:|------|-----:|----------|---|-----:|---|-----:|
|
| 10 |
+
|arc_challenge | 1|none | 0|acc |↑ |0.5239|± |0.0146|
|
| 11 |
+
| | |none | 0|acc_norm |↑ |0.5640|± |0.0145|
|
| 12 |
+
|arc_easy | 1|none | 0|acc |↑ |0.8060|± |0.0081|
|
| 13 |
+
| | |none | 0|acc_norm |↑ |0.7837|± |0.0084|
|
| 14 |
+
|hellaswag | 1|none | 0|acc |↑ |0.6398|± |0.0048|
|
| 15 |
+
| | |none | 0|acc_norm |↑ |0.8303|± |0.0037|
|
| 16 |
+
|lambada_openai| 1|none | 0|acc |↑ |0.6621|± |0.0066|
|
| 17 |
+
| | |none | 0|perplexity|↓ |4.0357|± |0.0917|
|
| 18 |
+
|piqa | 1|none | 0|acc |↑ |0.8036|± |0.0093|
|
| 19 |
+
| | |none | 0|acc_norm |↑ |0.8134|± |0.0091|
|
| 20 |
+
|sciq | 1|none | 0|acc |↑ |0.9630|± |0.0060|
|
| 21 |
+
| | |none | 0|acc_norm |↑ |0.9440|± |0.0073|
|
| 22 |
+
|winogrande | 1|none | 0|acc |↑ |0.7324|± |0.0124|
|
| 23 |
+
|mmlu | 2|none | |acc |↑ |0.7431|± |0.0034|
|
| 24 |
+
|
| 25 |
+
Benchmarks for base Qwen/QwQ-32B model:
|
| 26 |
+
|
| 27 |
+
|