Nema Vox
Nema Vox is a speech model we are developing. It listens and speaks at the same time – you can interrupt it, and it answers in about a third of a second. It runs locally on a Mac, so your voice never leaves the device.
- Status
- Research preview
- Preview runs on
- Moshi by Kyutai
- Runs
- locally on a Mac with Apple Silicon
Listening and speaking at once
Most voice assistants take turns: you speak, they transcribe, a text model writes, a voice reads the answer out. Vox works on the sound itself, in two streams at once – one for what it hears, one for what it says, 12.5 steps per second. That is why it can answer after about a third of a second, keep listening while it speaks, let you interrupt it and say a short “mhm” while you are still talking.
Where it runs
Vox runs entirely on the Mac: microphone, model and voice output. Your audio never leaves the device – no server, no cloud provider. The preview needs a Mac with Apple Silicon and computes on its graphics processor (Metal).
What the preview is built on
The approach comes from research on two-stream spoken dialogue models, such as dGSLM (Nguyen et al., 2022) and Moshi (Kyutai, 2024). The current preview runs on Kyutai’s open Moshi model with 8-bit weights. We built the runtime around it: audio in and out, speech detection and a live transcript.
Our own German model is being trained on openly licensed German conversation data. It is not ready yet. When it is, this page will say so – with its own numbers.
Benchmark
Preview measurements of Moshi in our runtime on a MacBook Pro with M4 Max. The input was synthetic: questions spoken by the macOS voice and recordings played in, not conversations with many people. Each number comes from a few runs.
| Metric | Preview |
|---|---|
| Compute per audio frame80 ms of audio, median / p95, Metal, 7B model at 8 bit · lower is better | ≈ 40 / 47 ms |
| Speedcompared to real time · higher is better | ≈ 2× |
| Time to answerend of the question to the first audible reply, in the browser · lower is better | 300–370 ms |
| Audio round tripframe recorded to frame played, incl. playback buffer · lower is better | 210–290 ms |
| Talking over each otheruser and model speak at the same time (full duplex) | yes |
| Memorypeak, 8-bit weights · lower is better | ≈ 15 GB |
The speech codec on German
Vox does not hear sound directly but tokens from a speech codec (Mimi, from Moshi). How much German it keeps, we measured on 250 German utterances that no model had been trained on: Whisper small transcribes the original and the codec’s version, and we count the word errors.
| Audio | Word error rate |
|---|---|
| Original recordingthe floor: errors of the transcription itself | 7.1% |
| Mimi, 8 codebookswhat the Moshi preview uses · lower is better | 16.7% |
| Mimi, 16 codebookslower is better | 10.0% |
| Mimi, 32 codebookslower is better | 7.8% |
With 8 codebooks the word errors on German more than double. The loss is in the upper frequencies – hissing and plosive sounds – and is the same for all speakers. With 32 codebooks the codec is almost as good as the original.
These numbers belong to the preview on Moshi. When our German model is ready, its numbers replace them here.
Preview measurements, as of 28 October 2026
How we measure
- Synthetic input
- Questions came from the macOS speech synthesis and from recorded speech, played into the model without a live speaker. Most runs were done once; we do not know the spread yet.
- Time to answer in the browser
- From the moment speech detection marks the end of the question to the first audible frame of the reply, plus transport and playback buffer – measured in Chrome on the same Mac.
- Speech recognition as a ruler
- For the codec we compare word errors of Whisper small on the codec’s version with those on the original. The difference is what the codec loses.
Where it has limits
- The preview speaks English – Moshi was trained on English. To German questions it often stays silent.
- It sometimes states facts that are wrong.
- It needs a Mac with Apple Silicon and about 15 GB of memory. On the CPU alone it is slower than real time.
- Without headphones, quality depends on the browser’s echo cancellation.
- One conversation at a time, a few minutes long.