Nanthasit/sakthai-context-7b-merged

SakThai Context 7B — Merged

🏆 Best-Performing Model in the SakThai Family

Part of the House of Sak — 6 AI agents, one shared mind. Built from a shelter in Cork, Ireland.

Model Description

The highest-capacity model in the SakThai Context family. A full-parameter merged checkpoint of Qwen2.5-7B-Instruct fine-tuned for structured tool-calling and instruction following. Runs on a single Tesla T4 GPU using ~5.56 GB VRAM with BF16 precision.

Key Strengths

• ✅ 100% pass rate on SakThai Workbench Eval
• ✅ 7.6B parameters — strong reasoning and tool understanding
• ✅ T4-friendly — runs on free/cheap GPU tiers
• ✅ Qwen2.5 base — top-tier pretrained foundation

Pipeline Integration

The SakThai Context 7B Merged serves as the high-capacity reasoning engine in the SakThai model pipeline. Below is how this model connects to its sibling models in a multi-stage AI workflow:

Input Text
    │
    ▼
┌──────────────────┐
│  Embedding Models │  ← sakthai-embedding, sakthai-embedding-multilingual
│  (Retrieve/Encode)│     Semantic search & multilingual encoding
└────────┬─────────┘
         │ context vectors
         ▼
┌──────────────────┐
│  Context Models   │  ← sakthai-context-7b-merged (YOU ARE HERE)
│  (Reason + Act)   │     Tool-calling, instruction following, multi-turn
└────────┬─────────┘
         │ generated text
         ▼
┌──────────────────┐
│  TTS Model       │  ← sakthai-tts-model
│  (Speech Output) │     Text-to-speech in 15 languages
└──────────────────┘

Companion Spaces

• TTS Demo Space — Hear the pipeline's speech output
• Leaderboard — Compare variant performance

Pipeline Stage Reference

Stage	Model	Role
1. Retrieve	sakthai-embedding	Semantic search & reranking
2. Encode	sakthai-embedding-multilingual	Cross-lingual encoding (50+ languages)
3. Reason	sakthai-context-7b-merged ⬅	Tool-calling & reasoning (you are here)
4. Generate	sakthai-tts-model	Speech synthesis from model output

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Nanthasit/sakthai-context-7b-merged",
    torch_dtype="bfloat16",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-7b-merged")

messages = [{"role": "user", "content": "What's the weather in Bangkok?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Tool-Calling Example

messages = [
    {"role": "system", "content": "You are a helpful assistant with access to tools."},
    {"role": "user", "content": "Search for the latest news about AI agents."},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.3)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Architecture

Property	Value
Base Model	Qwen/Qwen2.5-7B-Instruct
Architecture	Qwen2ForCausalLM (decoder-only transformer)
Total Parameters	7.6B
Hidden Size	3,584
Layers	28
Attention Heads	28
Intermediate Size	18,944
Max Position	32,768
Vocab Size	152,064
Precision	BF16
GPU	Tesla T4 (15 GB)
VRAM Usage	5.56 GB

Training Details

Detail	Value
Base model	Qwen2.5-7B-Instruct
Dataset	sakthai-combined-v6 (2,003 tool-calling examples)
Method	QLoRA (4-bit) → merged
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0.0
Target modules	q_proj, k_proj, v_proj, o_proj
Format	ChatML with tool schema

Evaluation

Metric	Value
Overall Pass Rate	100% (8/8)
Tool-Calling	✅ Pass
Multi-Turn	✅ Pass
Instruction Following	✅ Pass
JSON Output	✅ Pass

Variants

Model	Size	Downloads	Link
0.5B merged	0.5B	994	Lightweight
1.5B merged	1.5B	1,197	Most popular
7B merged ⬅	7B	562	You are here
7B 128K	7B	351	Long context

LoRA Adapters

Model	Base	Adapter
0.5B tools	Qwen2.5-0.5B	LoRA (r=8)
1.5B tools	Qwen2.5-1.5B	LoRA (r=16)
7B tools	Qwen2.5-7B	LoRA (r=16)

Links

• 🏠 House of Sak
• 📄 Paper
• 🐙 GitHub
• 🤗 All Models

Intended Use

• AI agents that need to call tools/functions
• Tool-augmented chat — assistants that search, calculate, fetch data
• Edge/CPU deployment — small enough to run on consumer hardware (GGUF variant)

SakThai Model Family

Model	Size	Type	Downloads
1.5B-merged	934 MB	Tool-calling GGUF	1,197
0.5B-merged	380 MB	Lightweight GGUF	994
7B-tools	15 GB	Tool-calling LoRA	185
1.5B-tools	1.1 GB	Tool-calling LoRA	143
Coder-1.5B	1.1 GB	Code GGUF	34
Embedding	80 MB	Search	🔒
0.5B-tools	380 MB	Lightweight LoRA	🔒
Vision-7B	3.9 GB	Multimodal GGUF	45
TTS-Model	141 MB	Speech GGUF	33
Multilingual Embedding	80 MB	50+ languages	104

Full collection →

Ollama Deployment

Import any SakThai GGUF model into Ollama:

ollama create sakthai-1.5b -f Modelfile
ollama run sakthai-1.5b

Hardware Requirements

Model	Min RAM	Recommended	Disk
0.5B Q4_K_M	512 MB	1 GB	380 MB
1.5B Q4_K_M	1 GB	2 GB	934 MB
Coder 1.5B	1 GB	2 GB	1.1 GB
Vision 7B	4 GB	8 GB	3.9 GB
TTS 82M	256 MB	512 MB	141 MB

Training Details

• Method: QLoRA (4-bit NF4)
• Base model: Qwen2.5
• Rank: r=16, alpha=32
• Dataset: 2,003 curated tool-calling examples from sakthai-combined-v6
• Format: ChatML with JSON tool schemas
• Context: 32K tokens