Soren-Qwen3-4B-Instruct-Finance-v1💰

This is a language model based on Qwen/Qwen3-4B-Instruct-2507, fine-tuned on a specific instruction dataset for the finance and economics domains.

This model utilizes the 🚀 Unsloth framework for efficient training and has enhanced performance on tasks related to finance, accounting, and economics. It aims to provide a specialized language model for financial practitioners, students, and researchers.

📖 Model Details

• Base Model: Qwen/Qwen3-4B-Instruct-2507
• Fine-tuning Framework: Unsloth
• Languages: English, Chinese
• Model Architecture: Decoder-only Transformer

---

🛠️ Training Process

Hardware and Configuration

• Hardware: Trained on an NVIDIA A100-SXM4-80GB GPU.

---

Training Data 📊

The model was fine-tuned on a mixed instruction dataset containing general instructions, chain-of-thought, and specialized instructions from the finance and economics domains, totaling approximately 84,000 samples. After filtering and cleaning, 83,588 samples with a length within 4096 tokens were used for training.

The number of samples from each dataset is as follows:

Dataset ID	Focus Area
BAAI/IndustryInstruction_Finance-Economics	Finance & Economics
Josephgflowers/Finance-Instruct-500k	Financial Instructions
Infinity_Instruct_chat_qwen3_235B_Gen.jsonl (Private)	General Instructions
Jackrong/Qwen3-235B-A22B-Instruct-2507-Distilled-chat	General Conversation
Jackrong/Chinese-Qwen3-235B-Thinking-2507-Distill-100k	Thought
facebook/natural_reasoning	Natural Reasoning

---

📈 Evaluation Results

To verify the model's performance improvement in the financial domain, we evaluated the fine-tuned model and the base model on four relevant sub-tasks of the MMLU benchmark (5-shot).

Task (MMLU)	Base Model (Qwen3-4B-Instruct-2507)	Fine-tuned (Soren-4B)	Improvement
Econometrics	0.6491	0.6767	✅
High School Macroeconomics	0.7900	0.8200	✅
Professional Accounting	0.6150	0.6350	✅
High School Microeconomics	0.9100	0.9250	✅

Task (C-Eval)	Base Model (Qwen3-4B-Instruct-2507)	Fine-tuned (Soren-4B)	Improvement
Advanced Mathematics	0.3684	0.4211	✅
Probability & Statistics	0.4444	0.5000	✅
College Programming	0.8108	0.8378	✅
Computer Architecture	0.8095	0.8571	✅

Conclusion: The evaluation results show that after fine-tuning with financial data, the model's accuracy in areas such as econometrics, macroeconomics, Advanced Mathematics, Probability & Statistics and professional accounting has slightly improved, demonstrating the effectiveness of domain-specific fine-tuning.

---

🚀 How to Use

You need to install the transformers, torch, and accelerate libraries to use this model. Please ensure you use the correct qwen3-instruct chat template.

⚠️ Disclaimer

This model is an experimental tool fine-tuned on publicly available datasets, intended to provide auxiliary information in the fields of finance and economics. It does not constitute professional financial, investment, or legal advice. The model's output may contain outdated information, biases, or factual errors ("hallucinations") due to the limitations of its training data, and it inherits all potential flaws of its base model. Do not input any personal sensitive information. Users must independently verify all content through reliable sources before use and assume full responsibility for any consequences arising from the use of the model's information.

中文：

Soren-Qwen3-4B-Instruct-Finance-v1💰

这是一个基于 Qwen/Qwen3-4B-Instruct-2507 模型，使用金融和经济领域特定指令数据集进行微调的语言模型。

本模型利用 🚀 Unsloth 框架进行高效训练，并提升了模型在金融、会计和经济学等相关任务上的表现。 目的是为金融从业者、学生和研究人员提供一个专业领域语言模型。

📖 模型详情

• 基础模型: Qwen/Qwen3-4B-Instruct-2507
• 微调框架: Unsloth
• 语言: 英语、中文
• 模型架构: Decoder-only Transformer

---

🛠️ 训练过程

硬件与配置

• 硬件: 在 NVIDIA A100-SXM4-80GB GPU 上进行训练。

  ---

训练数据 📊

模型在一个混合指令数据集上进行了微调，该数据集包含了通用指令、思维链以及金融和经济领域的专业指令，总计约 8.4 万条样本。 数据经过筛选和清洗，最终有 83,588 条长度在 4096 token 以内的样本用于训练。

各数据集采样数量如下：

数据集 ID	专注领域
BAAI/IndustryInstruction_Finance-Economics	金融经济
Josephgflowers/Finance-Instruct-500k	金融指令
Infinity_Instruct_chat_qwen3_235B_Gen.jsonl (私有)	通用指令
Jackrong/Qwen3-235B-A22B-Instruct-2507-Distilled-chat	通用对话
Jackrong/Chinese-Qwen3-235B-Thinking-2507-Distill-100k	思维
facebook/natural_reasoning	自然推理

---

📈 评测结果

为了验证模型在金融领域的性能提升，我们在 MMLU 基准测试中的四个相关子任务上对微调后的模型和基础模型进行了评估（5-shot）。

任务 (MMLU)	基础模型 (Qwen3-4B-Instruct-2507)	微调后 (Soren-4B)	提升
Econometrics (计量经济学)	0.6491	0.6767	✅
High School Macroeconomics	0.7900	0.8200	✅
Professional Accounting	0.6150	0.6350	✅
High School Microeconomics	0.9100	0.9250	✅

任务 (C-Eval)	基础模型 (Qwen3-4B-Instruct-2507)	微调后 (Soren-4B)	提升
高等数学 (Advanced Mathematics)	0.3684	0.4211	✅
概率统计 (Probability & Statistics)	0.4444	0.5000	✅
大学编程 (College Programming)	0.8108	0.8378	✅
计算机体系结构 (Computer Architecture)	0.8095	0.8571	✅

结论: 评测结果显示，经过金融数据微调后，模型在计量经济学、宏观经济学、高等数学、概率统计和专业会计等领域的准确率有些许提升，证明了领域微调的有效性。

---

🚀 如何使用

您需要安装 transformers、torch 和 accelerate 库来使用此模型。请确保使用正确的 qwen3-instruct 聊天模板。

⚠️ 注意

本模型是基于公开数据集微调的实验性工具，旨在提供金融和经济领域的辅助信息，绝不构成专业的财务、投资或法律建议。模型输出可能包含因训练数据局限性而导致的过时信息、偏见或事实性错误（“幻觉”），并且继承了其基础模型的所有潜在缺陷。请勿输入任何个人敏感信息，用户在使用前必须通过独立可靠的信源对所有内容进行严格核实，并对因使用模型信息而导致的任何后果负全部责任。