File size: 1,644 Bytes
b5f323b f6a2448 2660242 b5f323b 2660242 b5f323b 718aa9b b5f323b 718aa9b b5f323b 718aa9b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
---
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
- code
- lora
- peft
base_model: unsloth/tinyllama-chat-bnb-4bit
pipeline_tag: text-generation
datasets: Ramikan-BR/data-oss_instruct-decontaminated_python.jsonl
---
# Uploaded model
- **Developed by:** Ramikan-BR
- **Model type:** [text-generation/Python Coder]
- **Language(s) (NLP):** [en]
- **License:** apache-2.0
- **Finetuned from model :** unsloth/tinyllama-chat-bnb-4bit
### Model Description
<!-- Provide a longer summary of what this model is. -->
### Training Data
datasets: [Ramikan-BR/data-oss_instruct-decontaminated_python.jsonl](https://huggingface.co/datasets/Ramikan-BR/data-oss_instruct-decontaminated_python.jsonl)
### Training Procedure
The model was refined using [Unsloath](https://github.com/unslothai/unsloth). The dataset [ise-uiuc/Magicoder-OSS-Instruct-75K](https://huggingface.co/datasets/ise-uiuc/Magicoder-OSS-Instruct-75K/blob/main/data-oss_instruct-decontaminated.jsonl) was adjusted, leaving only data on python and divided into 10 parts, each refinement occurred for 2 epochs, using adafactor optimizer or adamw_8bit (adafactor seems to deliver less loss).
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|