Llama 3.2 Conversational RAG Fine-tuning

A specialized banking AI assistant fine-tuned on financial documents using Llama 3.2 (1B/3B) with LoRA adapters.

Available on Hugging Face: Akhenaton/sft_banking_model

Ready-to-use fine-tuned model with multiple quantization formats (q4_k_m, q8_0, q5_k_m, f16).

This project implements parameter-efficient fine-tuning using:

• Base Model: Llama-3.2-1B-Instruct (or 3B variant)
• Framework: Unsloth for optimized training
• Technique: LoRA (Low-Rank Adaptation)
• Quantization: 4-bit for memory efficiency
• Task: Conversational RAG on financial documents

Source: G4KMU/t2-ragbench (ConvFinQA subset)

• 32,908 context-independent QA pairs
• 9,000+ financial documents
• 4 domains: FinQA, ConvFinQA, VQAonBD, TAT-DQA
• Average: 803.2 tokens/document, 36.3 tokens/question

• Financial documents with text and tables
• Context-independent questions (>80% vs <10% in originals)
• SEC filings and financial reports
• Numerical reasoning focus

r = 16                    # Rank
lora_alpha = 16          # Alpha scaling
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", 
                  "gate_proj", "up_proj", "down_proj"]

max_seq_length = 2048
batch_size = 2
gradient_accumulation = 4
learning_rate = 2e-4
max_steps = 60
optimizer = "adamw_8bit"

<|begin_of_text|><|start_header_id|>system<|end_header_id|>
You are a specialized banking AI assistant...
<|eot_id|><|start_header_id|>user<|end_header_id|>
Financial Document Context: {context}
Question: {question}
<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{answer}<|eot_id|>

pip install unsloth transformers trl datasets

from unsloth import FastLanguageModel
from datasets import load_dataset

# Load model and dataset
model, tokenizer = FastLanguageModel.from_pretrained(
    "unsloth/Llama-3.2-1B-Instruct",
    load_in_4bit=True
)
dataset = load_dataset("G4KMU/t2-ragbench", "ConvFinQA")

messages = [{"role": "user", "content": "Your financial question"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=128, temperature=1.5)

• LoRA Adapters: Local/HuggingFace Hub
• Merged Models: 16-bit, 4-bit
• GGUF: q4_k_m, q8_0, q5_k_m, f16 for llama.cpp
• Ollama: Direct integration support

• Memory Efficient: 4-bit quantization + LoRA
• Fast Training: Unsloth optimization (2x faster inference)
• Specialized: Financial domain expertise
• Flexible Export: Multiple format support
• Production Ready: VLLM and llama.cpp compatible