language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
- code
- lora
- peft
base_model: unsloth/tinyllama-chat-bnb-4bit
pipeline_tag: text-generation
datasets: Ramikan-BR/data-oss_instruct-decontaminated_python.jsonl
Uploaded model
- Developed by: Ramikan-BR
- Model type: [text-generation/Python Coder]
- Language(s) (NLP): [en]
- License: apache-2.0
- Finetuned from model : unsloth/tinyllama-chat-bnb-4bit
Model Description
Training Data
datasets: Ramikan-BR/data-oss_instruct-decontaminated_python.jsonl
Training Procedure
The model was refined using Unsloath. The dataset ise-uiuc/Magicoder-OSS-Instruct-75K was adjusted, leaving only data on python and divided into 10 parts, each refinement occurred for 2 epochs, using adafactor optimizer or adamw_8bit (adafactor seems to deliver less loss).
Model Sources [optional]
base_model: unsloth/tinyllama-chat-bnb-4bit
model: Ramikan-BR/tinyllama-coder-py-4bit-v10 gguf_f16: tinyllama-coder-py-4bit-v10-unsloth.F16.gguf gguf_Q4_K_M: tinyllama-coder-py-4bit-v10-unsloth.Q4_K_M.gguf gguf_Q8_0: tinyllama-coder-py-4bit-v10-unsloth.Q8_0.gguf
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
