README.md

---
library_name: transformers
license: other
base_model: Qwen/Qwen2.5-14B-Instruct
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# M-Prometheus

M-Prometheus is a suite of open LLM judges that can natively evaluate multilingual outputs. They were trained on 480k instances of multilingual direct assessment and pairwise comparison data wiht long-form feedback. 
They can be prompted in the same way as [Prometheus-2](https://huggingface.co/prometheus-eval/prometheus-7b-v2.0/tree/main).
Check out our [paper](wip) for more details.

## Citation

```bibtex
@misc{pombal2025mprometheussuiteopenmultilingual,
      title={M-Prometheus: A Suite of Open Multilingual LLM Judges}, 
      author={José Pombal and Dongkeun Yoon and Patrick Fernandes and Ian Wu and Seungone Kim and Ricardo Rei and Graham Neubig and André F. T. Martins},
      year={2025},
      eprint={2504.04953},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.04953}, 
}
```