Three open Saudi Arabic models, and every number behind them

Most Arabic AI is built for Modern Standard Arabic and evaluated on it. Nobody in Riyadh talks that way on a phone call. We have published three models on Hugging Face that exist because a general model failed a specific Saudi task, and each one ships with the measurement that shows by how much, against the untrained base model, on data the model never saw.

This post is the whole picture in one place: what each model does, what it scores, what the dataset underneath them contains, what the whole thing cost, and where it falls over. The numbers here are copied from the published model cards. Where a result is unflattering it is in the chart with the rest.

What is actually new here, stated precisely

It is worth being exact about the claim, because Arabic AI is a field where vague firsts are handed out freely. We are not claiming the first Saudi Arabic model. ALLaM came before us, the SADA corpus came before us, and published Najdi dialect work came before us; two of those three are in our own training and evaluation pipeline and we cite them on every card.

What we believe is new is narrower and more useful. As far as we can find, this is the first openly published Saudi dialect voice stack where all three stages are released together and each one carries its own held-out evaluation: a recogniser that hears Saudi speech, a model that decides what to say in spoken Najdi, and a model that shapes how it is said. And the dialogue set underneath it is, as far as we can tell, the first open corpus of Najdi enterprise conversation — a rig shift handover, a permit-to-work request, a SADAD payment dispute. Open Arabic dialect data has plenty of daily life and travel in it. It has nothing that sounds like a refinery.

Hear: voho-saudi-stt-small

The starting point is worse than people expect. Run a general speech model on Saudi dialect phone audio and the word error rate is above 100 per cent, which is not a typo and not a rounding artefact: WER counts insertions as well as substitutions and deletions, so a model that hallucinates extra words while missing the real ones can score worse than silence. On 4,582 held-out Saudi test clips the untrained baseline came in at 103.9 per cent WER. It was not mishearing the dialect. It was not transcribing it at all.

After fine-tuning, 39.2 per cent overall and 16.9 per cent character error rate. Najdi and Hijazi land around 36 per cent, Khaleeji is the hardest at 43.2 per cent, and the small Modern Standard Arabic slice improves too rather than being traded away, which is what you want: a Saudi line still gets MSA in it whenever somebody reads out an address or a formal term.

Thirty-nine per cent WER is not a solved problem and we are not going to present it as one. It is the difference between a transcript that is unusable and a transcript an agent can act on, on dialect audio where the starting point was total failure. The per-dialect breakdown is on the card because an aggregate figure would hide that Khaleeji is seven points worse than Najdi.

Decide: voho-saudi-chat-4b

The middle of the pipeline is the part everybody skips. A recogniser that hears Najdi feeding a model that answers in Modern Standard Arabic gives you a call where the caller speaks like a person and the system replies like a press release. So the question for this model is not whether its answers are correct; it is what dialect they come out in.

That is measurable. Replies go through MARBERTv2, a published Arabic dialect classifier that had no part in training, and we count what share of replies it calls Gulf and what share it calls MSA. It is the same metric the Saudi-Dialect-ALLaM work uses. Before spending anything on training we checked the metric itself on 400 rows of each kind, and the ceiling is 94.5 per cent rather than 100: the classifier does not call genuine Saudi text Gulf every single time. That is why the reference row sits in the chart as its own bar. A card implying a perfect model would score 100 would be misleading.

Gulf replies go from 62.3 per cent to 89.8 per cent against a 94.5 per cent ceiling, and the MSA leak halves to 0.8 per cent. The other change matters as much on a phone line: mean reply length drops from 30 words to 6. The base model answers a spoken question with a paragraph. Six words is roughly what a person says.

One detail decides whether any of this is credible. The test split is a published third-party corpus, and none of our own generated dialogues are in it. Scoring a model on data from our own generator would measure how well it matched our prompt, not how Saudi it sounds, and the number would be meaningless in an interesting-looking way.

Say: voho-saudi-speak-0.6b

The last stage takes a formal Arabic sentence and returns what a person from Riyadh, Jeddah or the Eastern Province would actually say. It is 0.6B parameters, small enough to sit next to a speech model on one GPU or run on a CPU, and it is scored with chrF++ against what a Saudi speaker really said, on SADA's own test split.

The floor line is the part of this chart that keeps it honest. Dialect and formal Arabic share most of their letters, so simply leaving the sentence alone already scores 61.8. Any model that cannot clear that line is doing nothing, and the untrained base model at 52.2 is actively below it — it makes the sentence less like real Saudi speech than doing nothing at all would.

The data underneath: voho-saudi-dialogues

13,156 multi-turn Najdi conversations, 105,808 turns, released under Apache 2.0. 7,603 of them are service calls across oil and gas, utilities, telecom, banking, government, healthcare, logistics and facilities. The other 5,553 are everyday conversation — family, food, driving, occasions — and that half exists because of a training run that failed, which is the subject of its own post.

Every dialogue had to pass two gates before it was kept: a Najdi lexicon check for the function words Saudis actually use, and the same independent dialect classifier the evaluation uses, requiring at least 60 per cent of a dialogue's replies to read as Gulf. Gating the training data with the classifier that scores the model means the data cannot be worse than the thing it is measured against.

The licences, which are the point of one of these models

The two speech models are trained on SADA, the Saudi Audio Dataset for Arabic from SDAIA, which is CC BY-NC-SA 4.0. That licence flows through: both speech models are released non-commercially, for research and evaluation, with attribution. It is a real constraint and we would rather state it at the top than let somebody discover it during procurement.

The chat model was built specifically to not have that problem. Every dataset in it is Apache 2.0, the base model is Apache 2.0, and no SADA data touches it — that exclusion is the design rather than an oversight. So the chat model and the dialogue set are Apache 2.0 and a company can use them commercially without asking us anything.

For production Saudi speech, that is what the Voho API is for. The open models are there so you can check our work before you talk to us, which is a more useful thing for us to hand a sceptical engineer than a slide.

Run one in about a minute

The chat model has GGUF quants alongside it, so the shortest path to an opinion is ollama run hf.co/VohoAI/voho-saudi-chat-4b-GGUF:Q4_K_M, then ask it something in Najdi. A reasonable first test: أبي أحجز موعد بكرة الصبح، فيه وقت فاضي؟

One thing that is easy to get wrong: the system prompt is in every training example, and serving without it loses a noticeable amount of the dialect. If the replies come out more formal than the numbers above suggest, that is usually why.

What these models are not

• They are not production speech infrastructure. They are the research artefacts behind it, published so the claims are checkable.
• The speech models learned from television and broadcast speech, so they know how Saudis talk in dramas and interviews better than how they talk to a bank.
• 39% word error rate is a working transcript, not a solved problem, and Khaleeji is meaningfully harder than Najdi.
• The 0.6B rewriter drops or swaps details in long sentences, and occasionally gets the direction of an action wrong. Check anything customer-facing.
• None of this data is recorded human Saudi speech on the chat side. The dialogue set is generated and gated, and the reference column on the card is labelled held-out reference replies for exactly that reason.

The last one matters more than it looks. The fastest way to lose credibility in this field is to let a generated corpus be described as recorded speech, and it is an easy thing to let slide by saying nothing. So: it is generated, it is gated, the gates are published, and the evaluation is on a corpus somebody else made.

Why we publish any of this

Voho sells Saudi Arabic voice agents to enterprises, and the models above are not the product. Publishing them costs us the argument that our Arabic is proprietary and hands anybody the means to test it. We think that is a good trade in a market where every vendor says they support Arabic and almost none will show a number, and it is the only way we know to make a dialect claim that can be checked rather than believed.

The method is on each card and it is reusable. Small models, own data for the job, gate rather than assert, baseline every number, evaluate on something you did not write. The next dialect runs the same way.