metadata

language:
  - en
license: apache-2.0
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl
  - sft
  - code
  - lora
  - peft
base_model: unsloth/tinyllama-chat-bnb-4bit
pipeline_tag: text-generation
datasets: Ramikan-BR/data-oss_instruct-decontaminated_python.jsonl

Uploaded model

Developed by: Ramikan-BR
Model type: [text-generation/Python Coder]
Language(s) (NLP): [en]
License: apache-2.0
Finetuned from model : unsloth/tinyllama-chat-bnb-4bit

Model Description

Training Data

datasets: Ramikan-BR/data-oss_instruct-decontaminated_python.jsonl

Training Procedure

The model was refined using Unsloath. The dataset ise-uiuc/Magicoder-OSS-Instruct-75K was adjusted, leaving only data on python and divided into 10 parts, each refinement occurred for 2 epochs, using adafactor optimizer or adamw_8bit (adafactor seems to deliver less loss).

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.