---
tags:
    - transformers
    - causal-lm
    - text-generation
    - instruct
    - chat
    - fine-tuned
    - merged-lora
    - llama-3
    - hermes
    - discord-dataset
    - conversational-ai
    - chatml
    - pytorch
    - open-weights
    - 3b-parameters
model-index:
    - name: Discord-Micae-Hermes-3-3B
      results: []
base_model:
    - NousResearch/Hermes-3-Llama-3.2-3B
datasets:
    - mookiezi/Discord-OpenMicae
library_name: transformers
license: llama3
---

<div style="display: flex; align-items: center; gap: 8px;">
  <span>Run this model on Google Colab for free:</span>
  <a href="https://colab.research.google.com/drive/1kUtTeey5THhKW6f0BDKB9MFe4JIEew_Z?usp=sharing">
    <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab"/>
  </a>
</div>

## Discord-Micae-Hermes-3-3B

![Micae](Micae-Thumb.png)

## Model Description

Discord-Micae-Hermes-3-3B is a new finetune on [NousResearch/Hermes-3-Llama-3.2-3B](https://huggingface.co/NousResearch/Hermes-3-Llama-3.2-3B).

This model serves as a foundation for ongoing exploration into the capabilities of human-adjacent text generation.

- **Training Schedule:**  
  - 17M tokens of 260 thousand single-turn exchanges (STX) – 6 epochs @ 2e-5  
  - 5.5M tokens of 101 thousand multi-turn chains – 6 epochs @ 2e-5  
  - Combined dataset – 1 epoch @ 1e-5
- **Fine-Tuning Method:** LoRA merge (α = 32, r = 8, dropout = 0.1)
- **LoRA target modules:** q_proj, v_proj
- **Scheduler:** Cosine schedule with 220 warmup steps per phase
- **Optimizer:** torch.optim.AdamW (betas = (0.9, 0.999), amsgrad = False, eps = 1e-8, weight decay = 0.01)
- **Batching:** Effective size of 126 (7 batch size × 18 gradient accumulation steps)
- **Timeline and Hardware:** Training took place over 17 days on a GTX 1080 (8GB).

## Dataset

The model was fine-tuned on the [mookiezi/Discord-OpenMicae](https://huggingface.co/datasets/mookiezi/Discord-OpenMicae) dataset.

## Intended Use
- Conversational AI research  
- Experimentation with dialogue agents trained on Discord data  
- Chatbots requiring **casual, human-like tone**

### Limitations
- The model inherits potential biases from Discord-style language.  
- It is **not safety-aligned** for deployment without moderation.  
- While it does inhert knowledge from Hermes-3-3B it is not intended for factual or sensitive information retrieval.  

## Prompting

Micae uses the same ChatML prompt format as Hermes 3 and handles context and chat history.

```
<|im_start|>user
what do you think about ai?<|im_end|>
<|im_start|>assistant
i'm not a fan of AI but I can understand why people are excited to use it. it's like the first time they got an electric car, or when they were able to fly in space, that excitement is real<|im_end|>
<|im_start|>user
where do you think it's heading?<|im_end|>
<|im_start|>assistant
it will be used for good and bad things just as with other techs<|im_end|>
<|im_start|>user
what do you do use it for personally?<|im_end|>
<|im_start|>assistant
nothing much yet, I hope to find some stuff later on.<|im_end|>
<|im_start|>user
do you like pokemon?<|im_end|>
<|im_start|>assistant
yes I love Pokémon.<|im_end|>
<|im_start|>user
what's your favorite pokemon<|im_end|>
<|im_start|>assistant
garchomp<|im_end|>
```

---

## Inferfacing

A python script for interfacing, optimized for casual, Discord-style conversation, is available at [https://github.com/mookiezi/interface](https://github.com/mookiezi/interface).

To use this model run:

```
python interface.py -m mookiezi/Discord-Micae-Hermes-3-3B
```

![Interface Screenshot](interface-screenshot.png)

<span style="font-size:smaller;">(Example interface with [mookiezi/Discord-Micae-8B-Preview](https://huggingface.co/mookiezi/Discord-Micae-8B-Preview))</span>

---

## License  

See the [Meta LLaMA 3 Community License](https://ai.meta.com/llama/license/) for details.

## How to cite:  
If you use this model in your work, please cite both **Discord-Micae-Hermes-3-3B** and the base model **Hermes 3**:

```bibtex
@misc{discord-micae-hermes3b,
  title = {Discord-Micae-Hermes-3-3B},
  author = {mookiezi},
  year = {2025},
  url={https://huggingface.co/mookiezi/Discord-Micae-Hermes-3-3B}
}

@misc{teknium2024hermes3technicalreport,
  title={Hermes 3 Technical Report},
  author={Ryan Teknium and Jeffrey Quesnelle and Chen Guang},
  year={2024},
  eprint={2408.11857},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2408.11857}
}
```
[​](https://20000.online/micae)
[​](https://20000.online/openmicae)
[​](https://20000.online/discord-dialogues)