Quick answer: A mixture of experts (MoE) model splits parts of its network into many smaller "experts" and uses a router to send each token to only a few of them. So a model can store hundreds of billions of parameters but compute with a small slice per word. That makes it fast for its size, but all the experts still have to fit in memory.
If you've scrolled the trending page on Hugging Face lately, you've seen names like DeepSeek-V3, Qwen3-30B-A3B and gpt-oss-20b. They have something in common: each one is far bigger on paper than the amount of work it does per word. That gap is the whole point of the design, and it changes how you should read model sizes. Here's how it works, what it costs you in hardware, and when you, as someone who just wants good output, should actually care.
What is a mixture of experts model?
A regular (dense) language model runs every one of its weights for every token it reads or writes. Inside each transformer layer there's an attention block and a feed-forward block, and the feed-forward part holds most of the parameters.
A mixture of experts model replaces that one feed-forward block with a set of smaller ones, the experts, plus a small network called the router (or gate). For each token, the router scores the experts and picks the top few. Only those run. Their outputs get blended, weighted by the router's scores, and the token moves on to the next layer, where a fresh choice is made.
The Hugging Face "Mixture of Experts Explained" post is the best long-form introduction I know. It describes exactly this swap of feed-forward layers for MoE layers, and it covers a problem you'd never guess from the outside: routers tend to fall in love with a few experts and starve the rest. Training adds an extra "load balancing" term to push the router toward spreading tokens around.
Where did the MoE idea come from?
It's older than you'd think. The roots go back to a 1991 paper on adaptive mixtures of local experts. The version that matters for today's language models came in 2017, when Noam Shazeer and colleagues published "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer". Their layer had up to thousands of feed-forward sub-networks with a learned gate choosing a handful per example, and they reported beating the previous best results on language modeling and translation at lower compute cost.
The idea sat mostly in research labs for years because MoE models are awkward to train and serve. What changed is that open releases made them practical. Mistral's Mixtral 8x7B was the one that got a lot of people paying attention, and now several of the most-downloaded open models use the design.
Total vs active parameters: what's the difference?
This is the part that confuses people, so I'll be concrete. Total parameters are everything stored in the model file. Active parameters are what's actually computed for one token. A few real examples from model cards, as of November 2026:
- DeepSeek-V3: the official model card lists 671B total parameters with 37B activated per token, and a 128K context length.
- Qwen3-30B-A3B: 30.5B total, 3.3B active, with 8 of 128 experts used per token. The "A3B" in the name stands for that active count.
- gpt-oss-20b: OpenAI's model card lists 21B parameters with 3.6B active, under an Apache 2.0 license.
And one example that trips people up: Mixtral 8x7B sounds like 8 × 7 = 56B parameters. It isn't. The Hugging Face Mixtral announcement puts it at about 45B, because only the feed-forward blocks are duplicated per expert; the attention layers and other parts are shared. Two experts are picked per token.
So a rough mental model: total parameters tell you how much the model can hold, active parameters tell you how much work it does per word.
Why are MoE models fast for their size?
Generating text is a loop: compute one token, append it, compute the next. The time each step takes depends mainly on how many weights are used in that step. A dense 30B model touches all 30B every time. Qwen3-30B-A3B touches about 3.3B. That's why, on the same machine, a model like that tends to answer much faster than a dense model of the same total size.
It's also why labs like the design for training. You get a model with a lot of capacity (lots of stored knowledge spread across experts) for something closer to the compute bill of a much smaller dense model.
Don't take "fast" too literally, though. Speed on your hardware also depends on memory bandwidth, how the experts are laid out, and whether everything fits in fast memory in the first place. If half the model spills to slower system RAM, the speed advantage shrinks quickly.
What hardware does a mixture of experts model need?
Here's the honest catch: memory follows the total, not the active count. The router can pick any expert for the next token, so every expert has to be loaded and ready.
The Mixtral post is blunt about this. It says MoEs run inference quickly but still need a lot of VRAM, and its own estimates for Mixtral were over 90 GB in float16 and over 23 GB in 4-bit. That's a 45B model's memory bill with a much smaller model's per-token compute.
At the small end, MoE can be a great fit for local use. The gpt-oss-20b card says its weights ship natively quantized in MXFP4 and that it can run within 16 GB of memory. Qwen3-30B-A3B in a 4-bit GGUF is also popular for machines with lots of RAM but a modest GPU, because few active parameters keep it responsive.
At the big end, there's no home setup. DeepSeek-V3 at 671B total is a multi-GPU server job. For models like that, serving frameworks use expert parallelism, putting different experts on different GPUs so the whole thing fits. Hugging Face has a newer post on how its Transformers library handles MoE layers and expert parallelism if you want the engineering detail.
So the hardware rule for a mixture of experts model is:
- Check the total parameter count and apply the usual memory math (about 2 GB per billion at 16-bit, roughly a quarter of that at 4-bit). Our LLM model size explainer walks through it.
- Check the active parameter count to guess how quick it'll feel once loaded.
- Prefer a setup where all of it fits in fast memory, GPU VRAM or Apple unified memory, rather than splitting across GPU and system RAM.
Are the experts really experts?
Less than the name suggests. It's tempting to picture a legal expert, a coding expert and a finance expert, with the router acting like a receptionist. That's not how it's set up. Nobody assigns topics. The router learns its own split during training, it chooses per token rather than per question, and the split often doesn't line up with categories a person would recognize.
Practically, that means you can't ask an MoE model to "use its finance expert," and you shouldn't expect a dense model and an MoE model of similar quality to behave very differently from the outside. The difference is mostly in cost and speed, not in personality.
When should an end user care about MoE?
If you only ever use AI through a chat app or a product, mostly you shouldn't. The provider picks the architecture and you see the output.
You should care when:
- You're running models yourself. MoE models are often the best value per unit of speed, as long as you have the memory. A machine with 32 GB or more of unified memory and a modest GPU is where they shine.
- You're comparing model sizes. A "30B" MoE and a "30B" dense model are different beasts. The MoE will likely be faster; the dense one may do better on some reasoning-heavy tasks. Test both on your own data.
- You're budgeting a server. If your team self-hosts, MoE lets you serve more users per GPU, but the up-front memory requirement is set by the total size. Our post on an on-premises AI meeting assistant covers what that kind of project looks like.
A common mistake: picking an MoE model because the active count is small, then discovering it won't load. Always size for the total first.
Using MoE models on meeting transcripts
MoE models are a decent match for meeting work. Summarizing a 45-minute sales call or pulling action items out of a sprint review is mostly careful reading, and the speed of a small active count makes iteration pleasant.
If you have a machine that can hold something like gpt-oss-20b or Qwen3-30B-A3B, it's a reasonable place to start experimenting with transcripts you already have. Our guide to open-source LLMs for meeting summaries compares options for exactly this job, and the summarization app build guide shows the code side.
What the model won't do is the rest of the pipeline: capturing the meeting, getting a transcript with the right speaker names, tracking decisions and owners across weeks, and pushing tasks to the tools your team uses.
When you'd rather not run models yourself
If you want the notes rather than the hobby, Acta handles that part for you. It records Zoom, Google Meet and Microsoft Teams with a meeting bot, or bot-free through the desktop app for Windows and Mac or the Chrome extension, and it takes in-person meetings through mobile apps or uploaded recordings. If you're recording colleagues or customers, our guide to consent and what to tell participants is worth a read first.
You get transcripts with speaker names, summaries, decisions, action items with owners, follow-up email drafts and 40 report templates. OneChat answers questions across all your meetings with sources, and integrations send work to Salesforce, HubSpot, Jira, Notion and others. Acta does not use customer data to train AI models. You never have to think about routers or VRAM.
Frequently asked questions
What does "A3B" in a model name mean?
It's the active parameter count of a mixture of experts model. Qwen3-30B-A3B has about 30B parameters in total and about 3B active per token.
Is a mixture of experts model smarter than a dense model?
Not automatically. MoE is mainly a way to get more capacity for less compute. Quality depends on training, so compare specific models on your own tasks rather than assuming one design wins.
Why does an MoE model need so much memory if most experts are idle?
The router can pick any expert for the next token, so all of them must be loaded. Idle experts don't cost compute, but they still take memory.
Can I run DeepSeek-V3 on my laptop?
No. At 671B total parameters it needs multiple server GPUs even when quantized. For local use, look at smaller MoE models like gpt-oss-20b, which its card says can run within 16 GB of memory.
What is expert parallelism?
A way of serving big MoE models by placing different experts on different GPUs, so a model too large for one card can still run. It matters if you're hosting models, not if you're using them through an app.
To see meeting notes, decisions and action items without managing any models, try Acta free for 15 days.
