Update README.md
Browse files
README.md
CHANGED
@@ -41,17 +41,19 @@ Command r plus ๋ชจ๋ธ์ ์ด์ฉํ์ฌ ์์ฒด ๊ตฌ์ถํ RAG ํนํ ๋ฐ์ดํฐ์
,
|
|
41 |
```
|
42 |
|
43 |
## ํ์ต ํ๊ฒฝ ๋ฐ ํ๋ผ๋ฏธํฐ
|
44 |
-
- ํ๋ ํ๊ฒฝ
|
45 |
-
-
|
46 |
-
-
|
47 |
-
-
|
48 |
-
-
|
49 |
-
-
|
50 |
-
-
|
51 |
-
-
|
52 |
-
-
|
53 |
-
-
|
54 |
-
-
|
|
|
|
|
55 |
|
56 |
## ์ฌ์ฉ ๋ฐ์ดํฐ์
|
57 |
- AIhub 16 ํ์ ๋ฌธ์ ๋์ ๊ธฐ๊ณ๋
ํด ๋ฐ์ดํฐ
|
|
|
41 |
```
|
42 |
|
43 |
## ํ์ต ํ๊ฒฝ ๋ฐ ํ๋ผ๋ฏธํฐ
|
44 |
+
- ํ๋ ํ๊ฒฝ
|
45 |
+
- H100(80GB) * 8
|
46 |
+
- ํ๋ผ๋ฏธํฐ
|
47 |
+
- tokenizer_model_mex_length 4500
|
48 |
+
- use_flash_attn True
|
49 |
+
- num_train_epochs 3.0
|
50 |
+
- weight_decay 0.001
|
51 |
+
- lr_scheduler_type "linear"
|
52 |
+
- per_device_train_batch_size 1
|
53 |
+
- gradient_accumulation_steps 64
|
54 |
+
- learning_rate 5e-06
|
55 |
+
- bf16 True
|
56 |
+
- deepspeed ds_stage2.json
|
57 |
|
58 |
## ์ฌ์ฉ ๋ฐ์ดํฐ์
|
59 |
- AIhub 16 ํ์ ๋ฌธ์ ๋์ ๊ธฐ๊ณ๋
ํด ๋ฐ์ดํฐ
|