Hugging Face's logo

ozone-research
/
Chirp-01

Safetensors
qwen2
Model card Files Community
4
Chirp-01
File size: 3,002 Bytes 
dd0ca56
 
 
 
 
457ce70
 
dd0ca56
eae888f
 
 
 
 
19f0691
eae888f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
 
---
license: other
license_name: qwen-research
license_link: >-
  https://raw.githubusercontent.com/QwenLM/Qwen/refs/heads/main/Tongyi%20Qianwen%20RESEARCH%20LICENSE%20AGREEMENT
base_model:
- Qwen/Qwen2.5-3B-Instruct
---

# Chirp-3b

## Overview

Chirp-3b is a high-performing 3B parameter language model crafted by the Ozone Research team. Fine-tuned from a robust base model (Qwen2.5 3B Instruct), it was trained on 50 million tokens of distilled data from GPT-4o. This compact yet powerful model delivers exceptional results, outperforming expectations on benchmarks like MMLU Pro and IFEval.

Chirp-3b is an open-source effort to push the limits of what small-scale LLMs can achieve, making it a valuable tool for researchers and enthusiasts alike.

## Key Features

- **Parameters**: 3 billion
- **Training Data**: 50M tokens distilled from GPT-4o

## Benchmarks

Chirp-3b excels on rigorous evaluation datasets, showcasing its strength for a 3B model.

### MMLU Pro

| Subject             | Average Accuracy |
|---------------------|------------------|
| Biology             | 0.6234           |
| Business            | 0.5032           |
| Chemistry           | 0.3701           |
| Computer Science    | 0.4268           |
| Economics           | 0.5284           |
| Engineering         | 0.3013           |
| Health              | 0.3900           |
| History             | 0.3885           |
| Law                 | 0.2252           |
| Math                | 0.5736           |
| Other               | 0.4145           |
| Philosophy          | 0.3687           |
| Physics             | 0.3995           |
| Psychology          | 0.5589           |
| **Overall Average** | **0.4320**       |

- **Improvement**: 9 points above the base model.

### IFEval

- **Score**: 72%
- **Improvement**: 14% better than the base model.

More benchmarks are in the works and will be shared soon!

## Download

Access Chirp-3b here:  
https://huggingface.co/ozone-research/Chirp-01

## Usage

### Requirements

- Recommended GPU: 8 GB VRAM Minimum

### Example

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "ozone-research/Chirp-01"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

input_text = "What’s the future of AI?"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Future Work

The Ozone AI team is exploring additional models, including 2B and larger variants. Keep an eye out for upcoming releases!

## Feedback

We’re eager for your input! Try Chirp-3b and let us know your thoughts, use cases, or ideas for improvement. Open an issue here or contact us via [contact method—update as needed].

## Acknowledgments

A big thanks to the open-source community for driving projects like this forward. Chirp-3b is our contribution to making AI research more accessible.