Voho is delivering Aramco's AI call center.
voho.ai
Dialect18 November 20268 min

Can an AI voice agent handle code-switching between Arabic and English?

Yes, if it is built for it: code-switching inside one sentence needs interleaved Arabic-English recognition, not a language setting. How to test any vendor.

An AI voice agent can handle code-switching between Arabic and English, but only if it was built to recognise both languages interleaved inside one sentence. A system that asks for a single language per call, or only detects a language change between turns, will mis-transcribe callers who say an Arabic sentence with an English product name, a job title or a set of digits in the middle of it.

That is how many Saudi callers speak, so it is worth testing before you buy rather than assuming. This article explains what code-switching actually is on a phone line, why it is hard for speech recognition, where the failures show up, and a short test you can run on any vendor, including Voho.

What code-switching looks like on a real call

Code-switching means moving between two languages within a conversation. For a voice agent, the useful distinction is where the switch happens:

KindExampleHow hard it is
Between callsOne caller speaks Arabic, the next speaks EnglishEasy. Most systems handle it with a language setting or detection on the first turn.
Between turnsThe caller asks in Arabic, then answers the agent's question in EnglishModerate. Needs detection on every turn, not once per call.
Inside one sentenceAn Arabic sentence that contains an English product name, a system name, or a reference numberHard. Needs one recogniser that accepts both languages at once.

The third kind is the one that decides quality. Business Arabic in the Gulf carries English nouns constantly: a plan name, an app, a department, a ticket number, an email address. The caller does not pause or announce the switch, and the English word is usually pronounced with an Arabic accent, so it is neither clean Arabic nor clean English.

Why it is hard for speech recognition

Speech models are usually trained and scored one language at a time. Researchers working on Arabic-English code-switching describe it as a low-resource problem: there is far less transcribed mixed-language audio than monolingual audio. The ArzEn corpus, a public spontaneous-speech dataset of code-switched Egyptian Arabic and English, was built because that data was missing. It contains about 12 hours of audio from 38 bilingual speakers. A follow-up paper on the same corpus reports a best word error rate of 30.6% on its test set, which shows how much harder mixed speech is than single-language speech. ArzEn is Egyptian, not Saudi, so treat it as evidence about the problem, not as a measure of any Saudi system.

Four things combine to make the switch hard:

  • Accent on English words. An English term said with an Arabic accent is a different acoustic target from the one an English model learned.
  • Spelling and script. The transcript has to decide whether to write a borrowed word in Arabic letters, Latin letters or both, and the downstream system has to match it against your product list.
  • Dialect underneath. The Arabic around the English word is Najdi, Hijazi or another Gulf variety, not Modern Standard Arabic, so the dialect problem sits on top of the language-switch problem.
  • Telephone audio. Narrowband phone audio removes detail that helps tell short English words apart.

Where the failures show up

A mis-heard filler word costs nothing. The damage is concentrated in the few tokens that carry the task:

  • Product, plan and brand names said in English inside an Arabic sentence.
  • Reference, order and ticket numbers, where Arabic digits, English digits and mixed readings all occur.
  • Email addresses and spellings, where callers switch to English letters to spell.
  • Job titles and system names, such as the name of the app the caller is using.

Each of these is a field the agent has to write into a system. A wrong word in the transcript becomes a wrong record, a failed lookup, or a caller asked to repeat themselves. The agent also has to answer in the right language. The common good behaviour is to reply in the language the caller used for the main sentence, keep English terms in English, and switch only if the caller does.

How to test any vendor in one afternoon

Do not test by switching between turns, because almost everything handles that. Test the hard case, using your own words.

  • Write 20 sentences the way your customers really speak: Arabic with your real product names, plan names and common English terms inside them. Include at least five with a number or email address.
  • Have two or three colleagues from different regions read them naturally, on a normal mobile phone, not a studio microphone.
  • Run them through the agent on a real phone line, not a web demo, so the audio is telephone quality.
  • Check the transcript and the action taken. Count a sentence as a failure if the key product name, number or email is wrong, even when the rest is right.
  • Repeat with the caller switching between turns, and with a caller who speaks only English, to confirm nothing regressed.

Ask the vendor how the agent decides the language: once per call, per turn, or inside the utterance. Ask whether it can be given a list of your product and brand terms so those are recognised reliably. And ask to hear the reply to a mixed sentence, because the spoken answer should not read an English term with Arabic phonetics or the reverse.

Design choices that make mixed speech work

  • Do not force a language at the start of the call. Let the caller speak naturally and let the agent follow.
  • Give the recogniser your vocabulary: product names, plan names, branch names and the English terms your callers already use.
  • Capture numbers and emails with a read-back. Repeating the digits or spelling to the caller is the cheapest correction there is.
  • Keep the agent's reply in the caller's main language, with English terms left in English.
  • Review real mixed-language calls after launch and add the misses to your test set. The test set should grow from your own traffic.

How Voho does it

Voho builds Arabic-first voice agents, Saudi-first and Gulf-focused, and English too. Voho is developing its own Arabic speech and language models tuned for Saudi dialects and for the conditions real calls happen in, including narrowband telephony audio and code-switching. Saudi dialect accuracy is what Voho has measured and published on its open models; see /models for the numbers. Voho has not published a separate accuracy figure for mixed Arabic-English speech, so do not treat this article as one.

In a deployment, Voho's forward deployed engineers sit with your team and get the agent working on real calls. For code-switching, that means building the test set from your own calls and vocabulary, as described above, and fixing what it finds before and after go-live. You can see the wider approach in the guide to Saudi Arabic call centre AI, and the dialect side in best Arabic speech to text for Saudi. Going live on a real phone line needs a paid plan; see /pricing, or book a demo at /book-demo and bring your own mixed-language sentences.

Summary

Treat code-switching as a core requirement, not a feature name on a datasheet. The real test is the English product name or number inside an Arabic sentence, on a phone line. If the agent passes that with your own words, the rest of the language handling usually follows.

Sources

Frequently asked

Can an AI voice agent understand Arabic and English in the same sentence?
It can if the recognition is built to handle both languages interleaved. Systems that need a language chosen per call, or only re-detect the language between turns, tend to mis-transcribe the English words inside an Arabic sentence. Test with your own mixed sentences on a phone line.
Does a language setting on the agent solve code-switching?
No. A language setting handles callers who stay in one language. It does not help when one caller mixes both inside a sentence, which is common for product names, numbers and email addresses.
Which words does code-switching break most often?
The task-critical ones: product and brand names, reference and order numbers, email addresses and system names. These are the fields the agent writes into your systems, so errors there cost more than errors in filler words.
Does Voho publish an accuracy figure for mixed Arabic-English speech?
No. Voho's published numbers are for Saudi Arabic on its open models, listed at /models. Voho is developing models tuned for code-switching on telephony audio, but it has not published a separate mixed-language figure.
How should the agent reply when the caller mixes languages?
In the language of the caller's main sentence, with English terms kept in English, switching fully only if the caller does. Replies should not pronounce an English term with Arabic phonetics or the other way round.

Keep reading

Deployment-ready

Start your AI transformation today.

Sign up and build your first agent in the browser, or book a call if you would rather someone walked you through it. Most people do not need the call.

Start

$5 of credit, free

Granted when you sign up, about 70 minutes of live calls. No card to begin.

Then

Plans from SAR 109 a month

Starter puts your agent on your website, Business on a Saudi phone line. Cancel any month.

When you need it

Enterprise terms

Saudi data residency, an uptime SLA and on-premise deployment, on an agreement.