---
language: kk
license: cc-by-4.0
tags:
  - kazakh
  - gec
  - grammatical-error-correction
  - benchmark
task_categories:
  - translation
---

# sozkz-corpus-gec-benchmark-kk-v1

Казахстанский GEC-бенчмарк. Используйте сплит `test` (200 примеров) для официального сравнения.

## Splits

| Split | Examples | Source |
|-------|----------|--------|
| `test` | 200 | Канонический тест — для официального бенчмарка Word F0.5 |
| `synthetic` | 401 | GPT-4o синтетические ошибки |
| `organic_social` | 486 | Реальные ошибки из соцсетей (аннотация GPT-4o) |
| `synthetic_smoke` | 349 | Синтетические smoke-тест примеры |

## Schema

- `input` — ошибочный казахский текст
- `target` — исправленный эталон
- `error_tag` — тип ошибки (`orthography/spelling`, `morphosyntax/agreement` и др.)
- `source` — источник данных

## Leaderboard — 200-example canonical test (Word F0.5)

| Model | Exact Match | CER ↓ | Word Prec | Word Rec | Word F0.5 ↑ | Identity |
|-------|-------------|--------|-----------|----------|-------------|---------|
| [sozkz-fix-mt5-50m-kk-gec-v1](https://huggingface.co/saken-tukenov/sozkz-fix-mt5-50m-kk-gec-v1) | **62.0%** | **0.0802** | **0.494** | **0.661** | **0.520** | **100%** |
| [sozkz-nllb-1b-kk-gec-v1](https://huggingface.co/stukenov/sozkz-nllb-1b-kk-gec-v1) | 44.0% | 0.2447 | 0.233 | 0.550 | 0.264 | 61.5% |
| [sozkz-nllb-1b-kk-pretrain-v1](https://huggingface.co/stukenov/sozkz-nllb-1b-kk-pretrain-v1) | 43.5% | 0.2643 | 0.206 | 0.543 | 0.235 | 61.5% |

## Leaderboard — 100-example custom test (Exact Match %)

Чистый инференс модели без pre/post пайплайна. Категории: 30 орфография, 20 грамматика, 15 пунктуация, 20 смешанный, 15 identity.

| Model | Total | Емле/30 | Грамм/20 | Пункт/15 | Смеш/20 | Ident/15 |
|-------|-------|---------|----------|----------|---------|---------|
| [sozkz-core-llama-600m-kk-gec-v1](https://huggingface.co/stukenov/sozkz-core-llama-600m-kk-gec-v1) | **47%** | 15 | 12 | 3 | 2 | 15/15 |
| [sozkz-fix-qwen-500m-kk-gec-v3](https://huggingface.co/stukenov/sozkz-fix-qwen-500m-kk-gec-v3) | 38% | 0 | 16 | 9 | 0 | 13/15 |
| [sozkz-core-llama-300m-kk-gec-v4](https://huggingface.co/stukenov/sozkz-core-llama-300m-kk-gec-v4) | 37% | 9 | 6 | 4 | 3 | 15/15 |
| [sozkz-fix-qwen-500m-kk-gec-v1](https://huggingface.co/stukenov/sozkz-fix-qwen-500m-kk-gec-v1) | 35% | 0 | 12 | 8 | 0 | 15/15 |
| [sozkz-fix-qwen-500m-kk-gec-v2](https://huggingface.co/stukenov/sozkz-fix-qwen-500m-kk-gec-v2) | 30% | 0 | 11 | 7 | 0 | 12/15 |
| [sozkz-core-llama-1b-kk-gec-v1](https://huggingface.co/stukenov/sozkz-core-llama-1b-kk-gec-v1) | 16% | 2 | 6 | 1 | 0 | 7/15 |
| [sozkz-fix-qwen-500m-kk-gec-v4](https://huggingface.co/stukenov/sozkz-fix-qwen-500m-kk-gec-v4) | 5% | 0 | 1 | 4 | 0 | 0/15 |
| [sozkz-fix-mt5b-kk-gec-run13-v1](https://huggingface.co/stukenov/sozkz-fix-mt5b-kk-gec-run13-v1) | 5% | 0 | 2 | 0 | 0 | 3/15 |
| [sozkz-nllb-1b-kk-gec-v1](https://huggingface.co/stukenov/sozkz-nllb-1b-kk-gec-v1) | 1% | 0 | 1 | 0 | 0 | 0/15 |
| [sozkz-nllb-1b-kk-pretrain-v1](https://huggingface.co/stukenov/sozkz-nllb-1b-kk-pretrain-v1) | 1% | 0 | 1 | 0 | 0 | 0/15 |
| sozkz-core-llama-300m v1/v2a/v2b/v3 | 0–1% | 0 | 0–1 | 0 | 0 | 0–1 |
| sozkz-fix-mt5-50m-kk-gec-v1 | 0% | 0 | 0 | 0 | 0 | 0/15 |

## Usage

```python
from datasets import load_dataset

ds = load_dataset("stukenov/sozkz-corpus-gec-benchmark-kk-v1")
test = ds["test"]
print(test[0])
# {'input': '...', 'target': '...', 'error_tag': '', 'source': ''}
```