How to Use Local AI Models With NOVA

There is a moment the first time you watch a voice assistant answer you with the internet turned off.

No account. No usage meter ticking. No invisible copy of what you just said sitting on someone else's server.

That is what running local AI models with NOVA gives you. NOVA (Native Omnimodal Voice Agent) is desktop-native and local-first by design, so pointing it at a model on your own hardware is not a hack. It is the path the product was built for.

What "local" actually buys you

Three things, and they are the whole pitch.

• Free to run. The model lives on your machine. No per-token bill, no surprise overage at the end of the month.
• Private by default. Your words go from the mic to a process on your own computer and back. Nothing crosses the wire.
• Yours. No vendor can throttle you, deprecate your model, or change the terms underneath you.

You already paid once for NOVA and you own it. Running a local model means the brain is yours too. The loop closes.

The honest part: local is not frontier

I am not going to sell you a fantasy.

A model running on your laptop is smaller than the frontier cloud models. It reasons less deeply on hard problems. It will sometimes fumble a long chain of tool calls that a top-tier cloud model handles clean.

That is the trade. You give up some raw capability; you get total privacy and zero marginal cost.

For a huge slice of daily work, the trade is worth it. Drafting, quick lookups, talking through an idea, controlling your desktop, running automations. A capable local model handles that well, all day, for free, with nothing leaving the building.

When you genuinely need the heaviest reasoning, NOVA lets you switch the provider back to a cloud model in a couple of clicks. You are never locked into one or the other. That flexibility is part of how NOVA is advancing agentic AI: the agent stays the same, the brain behind it is yours to choose.

Step one: run a model server

NOVA does not host the model itself. It talks to a local inference server you run alongside it. The good news: it speaks the OpenAI-compatible /v1/chat/completions shape, which every popular local server already exposes.

Pick whichever you like:

• Ollama. The simplest start. Install it, run ollama pull on a model, and it serves on port 11434. NOVA talks to Ollama through its native API, so it can disable the model's slow "thinking" mode for faster replies.
• LM Studio. A friendly desktop app with a one-click local server. Good if you like a UI for browsing and loading models.
• llama.cpp. Lean and fast. For people who want to tune things close to the metal.
• vLLM. Heavier, throughput-focused. For a real GPU box.

Any of them works. NOVA covers all four with one client, because they all speak the same OpenAI-compatible language.

Step two: pick a local AI model (Gemma is a sensible start)

For most people on consumer hardware, a Gemma model is a sensible choice. It is small enough to run on a normal machine and capable enough to feel like a real assistant.

Pull it through your server. With Ollama that is one command. NOVA reads the model's real maximum context length straight from the server, so it never overflows the window and silently truncates what you said.

If you have more RAM and a strong GPU, reach for a larger model. If your machine is modest, stay small and keep replies snappy. The right model is the biggest one that still answers fast on your hardware.

Step three: point NOVA at it

Open Settings in NOVA and go to the LLM tab.

Set the provider to LOCAL.

Now you have two ways to connect.

The easy way: Scan for local models

Hit the Scan for local models button.

NOVA probes the well-known local inference ports on your machine, like Ollama on 11434 and LM Studio on 1234, and surfaces whatever is running. Anything it finds shows up in the list.

Pick the model you pulled. Done. No URLs to copy, no ports to memorize.

The manual way

If your server runs somewhere custom, paste the base URL into the field yourself. Loopback servers need no API key. If you put one behind a key, NOVA will send it.

Provider changes apply live. NOVA hot-swaps the brain without a restart, so you can flip between LOCAL and a cloud model mid-session and feel the difference yourself.

Step four: talk to it offline

Speak. NOVA routes your words to the local model and talks back.

Pull your network cable or kill your Wi-Fi and try again. It keeps working.

That is the whole point. A voice agent that does not need the cloud to think is a different kind of tool. It is yours in a way a cloud subscription never is.

A note on tools and local models

NOVA ships with a deep tool belt: over sixty built-in tools on Windows, from desktop control to web search to phone calls. Tool calling on a local model is best-effort, because local models vary widely in how well they handle it.

A smaller local model may skip a tool here and there. If a server ignores the tool request entirely, NOVA still gets you the assistant's text answer, so you are never left hanging.

Want to see what the agent can really do once you have a model behind it? Start with 12 cool things you can do with NOVA, then graduate to 4 power prompts to try with NOVA. Run those on a local model first; lean on a cloud model when a task gets genuinely hard.

The shape of it

Run a server. Pull a model. Set the provider to LOCAL. Scan. Talk.

Five steps and you have a voice agent that runs free, runs private, and answers you with the internet turned off.

That is not a smaller version of NOVA. For a lot of work, it is the better one. Own the agent, own the brain, keep every word at home.