Hugging Face's logo

qgallouedec
/
Qwen2.5-1.5B-Open-R1-Distill

Text Generation
Transformers
Safetensors
qwen2
Generated from Trainer
open-r1
trl
sft
conversational
text-generation-inference
Model card Files Community
1
qgallouedec HF Staff commited on
Commit
f216fb5
·
verified ·
1 Parent(s): d36cf14

Model save

Browse files
Files changed (5) hide show
  1. README.md +58 -0
  2. all_results.json +8 -0
  3. generation_config.json +14 -0
  4. train_results.json +8 -0
  5. trainer_state.json +468 -0
README.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen2.5-1.5B-Instruct
3
+ library_name: transformers
4
+ model_name: Qwen2.5-1.5B-Open-R1-Distill
5
+ tags:
6
+ - generated_from_trainer
7
+ - trl
8
+ - sft
9
+ licence: license
10
+ ---
11
+
12
+ # Model Card for Qwen2.5-1.5B-Open-R1-Distill
13
+
14
+ This model is a fine-tuned version of [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct).
15
+ It has been trained using [TRL](https://github.com/huggingface/trl).
16
+
17
+ ## Quick start
18
+
19
+ ```python
20
+ from transformers import pipeline
21
+
22
+ question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
23
+ generator = pipeline("text-generation", model="qgallouedec/Qwen2.5-1.5B-Open-R1-Distill", device="cuda")
24
+ output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
25
+ print(output["generated_text"])
26
+ ```
27
+
28
+ ## Training procedure
29
+
30
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/huggingface/huggingface/runs/6u2cdkxu)
31
+
32
+
33
+ This model was trained with SFT.
34
+
35
+ ### Framework versions
36
+
37
+ - TRL: 0.16.0.dev0
38
+ - Transformers: 4.50.0.dev0
39
+ - Pytorch: 2.6.0
40
+ - Datasets: 3.0.0
41
+ - Tokenizers: 0.21.0
42
+
43
+ ## Citations
44
+
45
+
46
+
47
+ Cite TRL as:
48
+
49
+ ```bibtex
50
+ @misc{vonwerra2022trl,
51
+ title = {{TRL: Transformer Reinforcement Learning}},
52
+ author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
53
+ year = 2020,
54
+ journal = {GitHub repository},
55
+ publisher = {GitHub},
56
+ howpublished = {\url{https://github.com/huggingface/trl}}
57
+ }
58
+ ```
all_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "total_flos": 490444463013888.0,
3
+ "train_loss": 0.5787169348351574,
4
+ "train_runtime": 3181.4532,
5
+ "train_samples": 93733,
6
+ "train_samples_per_second": 10.784,
7
+ "train_steps_per_second": 0.085
8
+ }
generation_config.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 151643,
3
+ "do_sample": true,
4
+ "eos_token_id": [
5
+ 151645,
6
+ 151643
7
+ ],
8
+ "pad_token_id": 151643,
9
+ "repetition_penalty": 1.1,
10
+ "temperature": 0.7,
11
+ "top_k": 20,
12
+ "top_p": 0.8,
13
+ "transformers_version": "4.50.0.dev0"
14
+ }
train_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "total_flos": 490444463013888.0,
3
+ "train_loss": 0.5787169348351574,
4
+ "train_runtime": 3181.4532,
5
+ "train_samples": 93733,
6
+ "train_samples_per_second": 10.784,
7
+ "train_steps_per_second": 0.085
8
+ }
trainer_state.json ADDED
@@ -0,0 +1,468 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 1.0,
6
+ "eval_steps": 500,
7
+ "global_step": 269,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.01858736059479554,
14
+ "grad_norm": 1.7748656124882543,
15
+ "learning_rate": 1.785714285714286e-05,
16
+ "loss": 0.8465,
17
+ "num_tokens": 10465392.0,
18
+ "step": 5
19
+ },
20
+ {
21
+ "epoch": 0.03717472118959108,
22
+ "grad_norm": 0.7775323625994008,
23
+ "learning_rate": 3.571428571428572e-05,
24
+ "loss": 0.7778,
25
+ "num_tokens": 20951152.0,
26
+ "step": 10
27
+ },
28
+ {
29
+ "epoch": 0.055762081784386616,
30
+ "grad_norm": 0.48936318299981746,
31
+ "learning_rate": 4.9998292477583695e-05,
32
+ "loss": 0.7095,
33
+ "num_tokens": 31436912.0,
34
+ "step": 15
35
+ },
36
+ {
37
+ "epoch": 0.07434944237918216,
38
+ "grad_norm": 0.4073886070673796,
39
+ "learning_rate": 4.993855640118024e-05,
40
+ "loss": 0.6711,
41
+ "num_tokens": 41909269.0,
42
+ "step": 20
43
+ },
44
+ {
45
+ "epoch": 0.09293680297397769,
46
+ "grad_norm": 0.3704019213957072,
47
+ "learning_rate": 4.9793703194560106e-05,
48
+ "loss": 0.652,
49
+ "num_tokens": 52353719.0,
50
+ "step": 25
51
+ },
52
+ {
53
+ "epoch": 0.11152416356877323,
54
+ "grad_norm": 0.2594836074576581,
55
+ "learning_rate": 4.9564282335552e-05,
56
+ "loss": 0.6366,
57
+ "num_tokens": 62792124.0,
58
+ "step": 30
59
+ },
60
+ {
61
+ "epoch": 0.13011152416356878,
62
+ "grad_norm": 0.25594286080216416,
63
+ "learning_rate": 4.9251164096056716e-05,
64
+ "loss": 0.6218,
65
+ "num_tokens": 73260442.0,
66
+ "step": 35
67
+ },
68
+ {
69
+ "epoch": 0.14869888475836432,
70
+ "grad_norm": 0.23970074498248772,
71
+ "learning_rate": 4.885553624080778e-05,
72
+ "loss": 0.6054,
73
+ "num_tokens": 83727556.0,
74
+ "step": 40
75
+ },
76
+ {
77
+ "epoch": 0.16728624535315986,
78
+ "grad_norm": 0.2511053919569689,
79
+ "learning_rate": 4.8378899521772935e-05,
80
+ "loss": 0.6042,
81
+ "num_tokens": 94201938.0,
82
+ "step": 45
83
+ },
84
+ {
85
+ "epoch": 0.18587360594795538,
86
+ "grad_norm": 0.29733281598714095,
87
+ "learning_rate": 4.78230619852879e-05,
88
+ "loss": 0.5968,
89
+ "num_tokens": 104670740.0,
90
+ "step": 50
91
+ },
92
+ {
93
+ "epoch": 0.20446096654275092,
94
+ "grad_norm": 0.2960638661673597,
95
+ "learning_rate": 4.719013211351733e-05,
96
+ "loss": 0.5998,
97
+ "num_tokens": 115151933.0,
98
+ "step": 55
99
+ },
100
+ {
101
+ "epoch": 0.22304832713754646,
102
+ "grad_norm": 0.3607422222807977,
103
+ "learning_rate": 4.648251082625975e-05,
104
+ "loss": 0.5906,
105
+ "num_tokens": 125607410.0,
106
+ "step": 60
107
+ },
108
+ {
109
+ "epoch": 0.241635687732342,
110
+ "grad_norm": 0.36527321547019587,
111
+ "learning_rate": 4.570288237343632e-05,
112
+ "loss": 0.5882,
113
+ "num_tokens": 136036466.0,
114
+ "step": 65
115
+ },
116
+ {
117
+ "epoch": 0.26022304832713755,
118
+ "grad_norm": 0.3483411666820654,
119
+ "learning_rate": 4.4854204152811567e-05,
120
+ "loss": 0.5848,
121
+ "num_tokens": 146502251.0,
122
+ "step": 70
123
+ },
124
+ {
125
+ "epoch": 0.2788104089219331,
126
+ "grad_norm": 0.32469856470379493,
127
+ "learning_rate": 4.39396954915706e-05,
128
+ "loss": 0.5822,
129
+ "num_tokens": 156977308.0,
130
+ "step": 75
131
+ },
132
+ {
133
+ "epoch": 0.29739776951672864,
134
+ "grad_norm": 0.24670208955198386,
135
+ "learning_rate": 4.2962825434308415e-05,
136
+ "loss": 0.576,
137
+ "num_tokens": 167423794.0,
138
+ "step": 80
139
+ },
140
+ {
141
+ "epoch": 0.3159851301115242,
142
+ "grad_norm": 0.28914180510023146,
143
+ "learning_rate": 4.1927299583755515e-05,
144
+ "loss": 0.5685,
145
+ "num_tokens": 177900466.0,
146
+ "step": 85
147
+ },
148
+ {
149
+ "epoch": 0.3345724907063197,
150
+ "grad_norm": 0.28196774826939586,
151
+ "learning_rate": 4.083704604415748e-05,
152
+ "loss": 0.5725,
153
+ "num_tokens": 188373813.0,
154
+ "step": 90
155
+ },
156
+ {
157
+ "epoch": 0.35315985130111527,
158
+ "grad_norm": 0.26866232768746134,
159
+ "learning_rate": 3.969620052063012e-05,
160
+ "loss": 0.5707,
161
+ "num_tokens": 198859382.0,
162
+ "step": 95
163
+ },
164
+ {
165
+ "epoch": 0.37174721189591076,
166
+ "grad_norm": 0.29490730565713075,
167
+ "learning_rate": 3.850909063101328e-05,
168
+ "loss": 0.5676,
169
+ "num_tokens": 209345142.0,
170
+ "step": 100
171
+ },
172
+ {
173
+ "epoch": 0.3903345724907063,
174
+ "grad_norm": 0.28997329137722166,
175
+ "learning_rate": 3.728021948973421e-05,
176
+ "loss": 0.5668,
177
+ "num_tokens": 219820636.0,
178
+ "step": 105
179
+ },
180
+ {
181
+ "epoch": 0.40892193308550184,
182
+ "grad_norm": 0.2427463235734826,
183
+ "learning_rate": 3.6014248625951984e-05,
184
+ "loss": 0.559,
185
+ "num_tokens": 230306396.0,
186
+ "step": 110
187
+ },
188
+ {
189
+ "epoch": 0.4275092936802974,
190
+ "grad_norm": 0.30000276433313355,
191
+ "learning_rate": 3.4715980300780745e-05,
192
+ "loss": 0.5627,
193
+ "num_tokens": 240792156.0,
194
+ "step": 115
195
+ },
196
+ {
197
+ "epoch": 0.44609665427509293,
198
+ "grad_norm": 0.2840658840182488,
199
+ "learning_rate": 3.339033929066841e-05,
200
+ "loss": 0.5612,
201
+ "num_tokens": 251259365.0,
202
+ "step": 120
203
+ },
204
+ {
205
+ "epoch": 0.4646840148698885,
206
+ "grad_norm": 0.27262466075161956,
207
+ "learning_rate": 3.204235420603273e-05,
208
+ "loss": 0.5644,
209
+ "num_tokens": 261736444.0,
210
+ "step": 125
211
+ },
212
+ {
213
+ "epoch": 0.483271375464684,
214
+ "grad_norm": 0.27049031008365026,
215
+ "learning_rate": 3.0677138416019556e-05,
216
+ "loss": 0.563,
217
+ "num_tokens": 272205866.0,
218
+ "step": 130
219
+ },
220
+ {
221
+ "epoch": 0.5018587360594795,
222
+ "grad_norm": 0.23124549228542488,
223
+ "learning_rate": 2.9299870651742188e-05,
224
+ "loss": 0.5587,
225
+ "num_tokens": 282683327.0,
226
+ "step": 135
227
+ },
228
+ {
229
+ "epoch": 0.5204460966542751,
230
+ "grad_norm": 0.22494436138190718,
231
+ "learning_rate": 2.7915775361580428e-05,
232
+ "loss": 0.5612,
233
+ "num_tokens": 293122806.0,
234
+ "step": 140
235
+ },
236
+ {
237
+ "epoch": 0.5390334572490706,
238
+ "grad_norm": 0.18937691441460602,
239
+ "learning_rate": 2.6530102893058357e-05,
240
+ "loss": 0.5605,
241
+ "num_tokens": 303598947.0,
242
+ "step": 145
243
+ },
244
+ {
245
+ "epoch": 0.5576208178438662,
246
+ "grad_norm": 0.2351559125981879,
247
+ "learning_rate": 2.5148109576477802e-05,
248
+ "loss": 0.5578,
249
+ "num_tokens": 314077641.0,
250
+ "step": 150
251
+ },
252
+ {
253
+ "epoch": 0.5762081784386617,
254
+ "grad_norm": 0.19092993827655919,
255
+ "learning_rate": 2.3775037785857073e-05,
256
+ "loss": 0.5488,
257
+ "num_tokens": 324545386.0,
258
+ "step": 155
259
+ },
260
+ {
261
+ "epoch": 0.5947955390334573,
262
+ "grad_norm": 0.17223510717812895,
263
+ "learning_rate": 2.2416096052810688e-05,
264
+ "loss": 0.5609,
265
+ "num_tokens": 335031146.0,
266
+ "step": 160
267
+ },
268
+ {
269
+ "epoch": 0.6133828996282528,
270
+ "grad_norm": 0.19945511088993198,
271
+ "learning_rate": 2.1076439308804808e-05,
272
+ "loss": 0.5411,
273
+ "num_tokens": 345504904.0,
274
+ "step": 165
275
+ },
276
+ {
277
+ "epoch": 0.6319702602230484,
278
+ "grad_norm": 0.17394579706852514,
279
+ "learning_rate": 1.976114933073662e-05,
280
+ "loss": 0.5507,
281
+ "num_tokens": 355956520.0,
282
+ "step": 170
283
+ },
284
+ {
285
+ "epoch": 0.6505576208178439,
286
+ "grad_norm": 0.16263450610078103,
287
+ "learning_rate": 1.847521546401383e-05,
288
+ "loss": 0.5515,
289
+ "num_tokens": 366434348.0,
290
+ "step": 175
291
+ },
292
+ {
293
+ "epoch": 0.6691449814126395,
294
+ "grad_norm": 0.18127092614108742,
295
+ "learning_rate": 1.7223515696258592e-05,
296
+ "loss": 0.5543,
297
+ "num_tokens": 376908695.0,
298
+ "step": 180
299
+ },
300
+ {
301
+ "epoch": 0.6877323420074349,
302
+ "grad_norm": 0.1797771983319447,
303
+ "learning_rate": 1.60107981534296e-05,
304
+ "loss": 0.5571,
305
+ "num_tokens": 387384301.0,
306
+ "step": 185
307
+ },
308
+ {
309
+ "epoch": 0.7063197026022305,
310
+ "grad_norm": 0.16421806343683648,
311
+ "learning_rate": 1.484166308855398e-05,
312
+ "loss": 0.5479,
313
+ "num_tokens": 397835323.0,
314
+ "step": 190
315
+ },
316
+ {
317
+ "epoch": 0.724907063197026,
318
+ "grad_norm": 0.15793988144816604,
319
+ "learning_rate": 1.372054543139188e-05,
320
+ "loss": 0.5467,
321
+ "num_tokens": 408298322.0,
322
+ "step": 195
323
+ },
324
+ {
325
+ "epoch": 0.7434944237918215,
326
+ "grad_norm": 0.15338002718093754,
327
+ "learning_rate": 1.2651697965228748e-05,
328
+ "loss": 0.5525,
329
+ "num_tokens": 418781566.0,
330
+ "step": 200
331
+ },
332
+ {
333
+ "epoch": 0.7620817843866171,
334
+ "grad_norm": 0.16149871221553752,
335
+ "learning_rate": 1.1639175194611693e-05,
336
+ "loss": 0.5484,
337
+ "num_tokens": 429246222.0,
338
+ "step": 205
339
+ },
340
+ {
341
+ "epoch": 0.7806691449814126,
342
+ "grad_norm": 0.16060225092798314,
343
+ "learning_rate": 1.0686817965224952e-05,
344
+ "loss": 0.5425,
345
+ "num_tokens": 439717644.0,
346
+ "step": 210
347
+ },
348
+ {
349
+ "epoch": 0.7992565055762082,
350
+ "grad_norm": 0.1488501132096454,
351
+ "learning_rate": 9.798238894246628e-06,
352
+ "loss": 0.5517,
353
+ "num_tokens": 450170885.0,
354
+ "step": 215
355
+ },
356
+ {
357
+ "epoch": 0.8178438661710037,
358
+ "grad_norm": 0.17478653780365394,
359
+ "learning_rate": 8.976808666454292e-06,
360
+ "loss": 0.5477,
361
+ "num_tokens": 460656645.0,
362
+ "step": 220
363
+ },
364
+ {
365
+ "epoch": 0.8364312267657993,
366
+ "grad_norm": 0.17024941986957184,
367
+ "learning_rate": 8.225643248063091e-06,
368
+ "loss": 0.54,
369
+ "num_tokens": 471142405.0,
370
+ "step": 225
371
+ },
372
+ {
373
+ "epoch": 0.8550185873605948,
374
+ "grad_norm": 0.1374034986500908,
375
+ "learning_rate": 7.547592066798609e-06,
376
+ "loss": 0.5443,
377
+ "num_tokens": 481616409.0,
378
+ "step": 230
379
+ },
380
+ {
381
+ "epoch": 0.8736059479553904,
382
+ "grad_norm": 0.1555520491850222,
383
+ "learning_rate": 6.94522720304148e-06,
384
+ "loss": 0.5436,
385
+ "num_tokens": 492097674.0,
386
+ "step": 235
387
+ },
388
+ {
389
+ "epoch": 0.8921933085501859,
390
+ "grad_norm": 0.14362154894139065,
391
+ "learning_rate": 6.420833633045514e-06,
392
+ "loss": 0.5413,
393
+ "num_tokens": 502555042.0,
394
+ "step": 240
395
+ },
396
+ {
397
+ "epoch": 0.9107806691449815,
398
+ "grad_norm": 0.1403050136321079,
399
+ "learning_rate": 5.976400561240085e-06,
400
+ "loss": 0.5393,
401
+ "num_tokens": 513031521.0,
402
+ "step": 245
403
+ },
404
+ {
405
+ "epoch": 0.929368029739777,
406
+ "grad_norm": 0.13648147962434784,
407
+ "learning_rate": 5.613613874496393e-06,
408
+ "loss": 0.5475,
409
+ "num_tokens": 523517281.0,
410
+ "step": 250
411
+ },
412
+ {
413
+ "epoch": 0.9479553903345725,
414
+ "grad_norm": 0.14324784070203328,
415
+ "learning_rate": 5.333849746981104e-06,
416
+ "loss": 0.5439,
417
+ "num_tokens": 533998146.0,
418
+ "step": 255
419
+ },
420
+ {
421
+ "epoch": 0.966542750929368,
422
+ "grad_norm": 0.13166747340193674,
423
+ "learning_rate": 5.138169419856345e-06,
424
+ "loss": 0.5399,
425
+ "num_tokens": 544470561.0,
426
+ "step": 260
427
+ },
428
+ {
429
+ "epoch": 0.9851301115241635,
430
+ "grad_norm": 0.14044943374018817,
431
+ "learning_rate": 5.027315175628478e-06,
432
+ "loss": 0.5362,
433
+ "num_tokens": 554934577.0,
434
+ "step": 265
435
+ },
436
+ {
437
+ "epoch": 1.0,
438
+ "num_tokens": 562274609.0,
439
+ "step": 269,
440
+ "total_flos": 490444463013888.0,
441
+ "train_loss": 0.5787169348351574,
442
+ "train_runtime": 3181.4532,
443
+ "train_samples_per_second": 10.784,
444
+ "train_steps_per_second": 0.085
445
+ }
446
+ ],
447
+ "logging_steps": 5,
448
+ "max_steps": 269,
449
+ "num_input_tokens_seen": 0,
450
+ "num_train_epochs": 1,
451
+ "save_steps": 100,
452
+ "stateful_callbacks": {
453
+ "TrainerControl": {
454
+ "args": {
455
+ "should_epoch_stop": false,
456
+ "should_evaluate": false,
457
+ "should_log": false,
458
+ "should_save": true,
459
+ "should_training_stop": true
460
+ },
461
+ "attributes": {}
462
+ }
463
+ },
464
+ "total_flos": 490444463013888.0,
465
+ "train_batch_size": 16,
466
+ "trial_name": null,
467
+ "trial_params": null
468
+ }