xlm-roberta-large-dataset-inicial-ner-clinical-mama-sp

This model is a fine-tuned version of FacebookAI/xlm-roberta-large-finetuned-conll03-english on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0556
• Precision: 0.9380
• Recall: 0.9368
• F1: 0.9374
• Accuracy: 0.9798

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine_with_restarts
• lr_scheduler_warmup_ratio: 0.2
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
1.1743	0.9974	189	1.6872	0.2341	0.1523	0.1845	0.6123
0.4383	1.9974	378	1.0670	0.3716	0.3677	0.3696	0.7270
0.2162	2.9974	567	0.8029	0.5604	0.4606	0.5057	0.8
0.1519	3.9974	756	0.6657	0.5594	0.48	0.5167	0.8152
0.1212	4.9974	945	0.5515	0.6325	0.5174	0.5692	0.8578
0.0733	5.9974	1134	0.4086	0.7512	0.6310	0.6858	0.9011
0.0601	6.9974	1323	0.3607	0.7289	0.6142	0.6667	0.9055
0.0689	7.9974	1512	0.2361	0.8009	0.7213	0.7590	0.9292
0.06	8.9974	1701	0.1869	0.8191	0.7652	0.7912	0.9390
0.0518	9.9974	1890	0.1797	0.8263	0.7858	0.8056	0.9431
0.0449	10.9974	2079	0.1440	0.8651	0.8194	0.8416	0.9542
0.0356	11.9974	2268	0.1273	0.876	0.8477	0.8616	0.9592
0.0358	12.9974	2457	0.1100	0.8832	0.8490	0.8658	0.9633
0.0272	13.9974	2646	0.0821	0.9170	0.9123	0.9146	0.9719
0.026	14.9974	2835	0.0709	0.9258	0.9174	0.9216	0.9750
0.0258	15.9974	3024	0.0605	0.9326	0.9277	0.9301	0.9776
0.0206	16.9974	3213	0.0573	0.9315	0.9303	0.9309	0.9782
0.0183	17.9974	3402	0.0556	0.9380	0.9368	0.9374	0.9798
0.0205	18.9974	3591	0.0549	0.9379	0.9355	0.9367	0.9795
0.0193	19.9974	3780	0.0550	0.9379	0.9355	0.9367	0.9795

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.4.1
• Tokenizers 0.21.1