@ -0,0 +1,76 @@

CAP-SFT-Qwen3-LoRA-75K

Model Description

This is a 75,000-sample supervised fine-tuned (SFT) version of Qwen3-14B trained on legal reasoning tasks from the Caselaw Access Project (CAP). The model uses LoRA (Low-Rank Adaptation) for parameter-efficient fine-tuning.

Performance

Overall Accuracy: 76.2% (61/80 samples across 5 legal tasks)

Task	Accuracy	Status	Notes
Bluebook Citation	80.0% (16/20)	✅ PASS	Legal citation formatting
Holding Selection	60.0% (12/20)	❌ FAIL	Multiple choice questions
IRAC Summarization	90.0% (18/20)	✅ PASS	Case summarization
Case Retrieval	N/A	⚠️ NO DATA	Missing evaluation samples
Entailment	75.0% (15/20)	❌ FAIL	Case relationship classification

Training Details

• Base Model: Qwen/Qwen3-14B
• Training Method: LoRA (Low-Rank Adaptation)
• Training Samples: 75,000 legal reasoning examples
• Training Duration: ~2h 29m (1171 steps)
• Final Loss: 1.269
• Trainable Parameters: 64.2M / 14.8B (0.43%)
• Hardware: 2x NVIDIA H100-80GB GPUs
• Effective Batch Size: 64

Performance Analysis

⚠️ Important Finding: This 75K model shows performance regression compared to the 30K model (81.3% → 76.2%), indicating overfitting. The 30K model should be preferred for downstream tasks.

Scaling Progression:

• Base Model: 18.8% accuracy
• 10K SFT: 70.0% accuracy (+51.2%)
• 30K SFT: 81.3% accuracy (+11.3%) ← Optimal
• 75K SFT: 76.2% accuracy (-5.1%) ← Overfitted

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kylebrussell/cap-sft-qwen3-lora-75k")
model = AutoModelForCausalLM.from_pretrained("kylebrussell/cap-sft-qwen3-lora-75k")

# Generate legal text
prompt = "Complete this citation: Brown v. Board of Education, "
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Dataset

Trained on kylebrussell/cap-rlvr-sft - a multi-task legal reasoning dataset derived from the Caselaw Access Project.

License

Apache 2.0

Citation

@model{cap-sft-qwen3-lora-75k,
  title={CAP-SFT-Qwen3-LoRA-75K: Legal Reasoning with Supervised Fine-Tuning},
  author={CAP RLVR Project},
  year={2025},
  url={https://huggingface.co/kylebrussell/cap-sft-qwen3-lora-75k}
}

Recommendation

⚠️ Use the 30K model instead: Due to performance regression, we recommend using kylebrussell/cap-sft-qwen3-lora-30k for better legal reasoning performance.