Nemotron 3.5 Lightning 30B-A3B
Hybrid Mamba-2 + MoE + attention, 30B total / 3B active, up to 1M context, MTP layers, pre-trained in NVFP4. Fast small agent model.
Recommended quants
- Unsloth UD-Q4_K_XL (25.5 GB)
unsloth/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF - MXFP4_MOE (23.2 GB) / UD-Q3_K_XL (21.2 GB)
unsloth/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUFHybrid model: low-bit quants stay relatively large. - Official NVFP4 (17.8 GB safetensors)
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4Blackwell (RTX 50, DGX Spark); vLLM. - FP8
RedHatAI/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-FP8
Chat template Works
Thinking/instruct via enable_thinking in the chat template.
Fix: vLLM: as of July 9, 2026 NVIDIA changed --reasoning-parser from nemotron_v3 to nano_v3. Use nano_v3 and --mamba-ssm-cache-dtype float32 as in the official command.
Sampling settings
| Official | temp 1.0, top_p 0.95 |
|---|
Runtimes
| vLLM | Works | Official serve command with nano_v3 reasoning parser. |
|---|---|---|
| llama.cpp | Works | Unsloth GGUFs. |
| Ollama | Unverified | Library page nemotron-3.5-lightning exists. |
| LM Studio | Unverified | |
| MLX | Unverified |
Tool calling in agent tools
| Claude Code (local endpoint) | Unverified | |
|---|---|---|
| OpenCode | Unverified | |
| Cline | Unverified | |
| Aider | Unverified |
Known issues
Sources checked
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4
- https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF
Spotted something wrong or a new fix? Corrections are reviewed before publishing. (Submission form coming soon.)