Update README.md
Browse files
README.md
CHANGED
|
@@ -46,7 +46,7 @@ Some parameters for this model are provided in the following table:
|
|
| 46 |
|
| 47 |
## Training Details
|
| 48 |
|
| 49 |
-
ALLaM-7B-Instruct-preview is pretrained on a total of
|
| 50 |
|
| 51 |
|
| 52 |
## Getting started
|
|
|
|
| 46 |
|
| 47 |
## Training Details
|
| 48 |
|
| 49 |
+
ALLaM-7B-Instruct-preview is pretrained on a total of 5.2 trillion tokens in English and Arabic, Our training codebase is built on [NVIDIA/MegatronLM](https://github.com/NVIDIA/Megatron-LM). Average MFU during training was ~42%. We trained our model using bf16-mixed precision.
|
| 50 |
|
| 51 |
|
| 52 |
## Getting started
|