---
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
language:
- en
- ar
base_model:
- BAAI/bge-m3
---

# mhaseeb1604/bge-m3-law

This model is a fine-tuned version of the [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) model, which is specialized for sentence similarity tasks in Arabic legal texts in both Arabic and English. It maps sentences and paragraphs to a 1024-dimensional dense vector space, useful for tasks like clustering, semantic search, and more.

## Model Overview

- **Architecture**: Based on [sentence-transformers](https://www.SBERT.net).
- **Training Data**: Trained on a large Arabic law dataset, containing bilingual data in Arabic and English.
- **Embedding Size**: 1024 dimensions, suitable for extracting semantically meaningful embeddings from text.
- **Applications**: Ideal for legal applications, such as semantic similarity comparisons, document clustering, and retrieval in a bilingual Arabic-English legal context.

## Installation

To use this model, you need to have the `sentence-transformers` library installed. You can install it via pip:

```bash
pip install -U sentence-transformers
```

## Usage

You can easily load and use this model in Python with the following code:

```python
from sentence_transformers import SentenceTransformer

# Load the model
model = SentenceTransformer('mhaseeb1604/bge-m3-law')

# Sample sentences
sentences = ["This is an example sentence", "Each sentence is converted"]

# Generate embeddings
embeddings = model.encode(sentences)

# Output embeddings
print(embeddings)
```

## Model Training

The model was fine-tuned on Arabic and English legal texts using the following configurations:

- **DataLoader**:
  - Batch size: 4
  - Sampler: SequentialSampler
- **Loss Function**: `MultipleNegativesRankingLoss` with cosine similarity.
- **Optimizer**: AdamW with learning rate `2e-05`.
- **Training Parameters**:
  - Epochs: 2
  - Warmup Steps: 20
  - Weight Decay: 0.01

## Full Model Architecture

This model consists of three main components:

```python
SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) - XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False})
  (2): Normalize()
)
```

- **Transformer Layer**: Uses XLM-Roberta model with a max sequence length of 8192.
- **Pooling Layer**: Utilizes CLS token pooling to generate sentence embeddings.
- **Normalization Layer**: Ensures normalized output vectors for better performance in similarity tasks.

## Citing & Authors

If you find this repository useful, please consider giving a star : and citation

```bibtex
@misc {muhammad_haseeb_2024,
	author       = { {Muhammad Haseeb} },
	title        = { bge-m3-law (Revision 2fc0289) },
	year         = 2024,
	url          = { https://huggingface.co/mhaseeb1604/bge-m3-law },
	doi          = { 10.57967/hf/3217 },
	publisher    = { Hugging Face }
}
```

<!--- Describe where people can find more information -->