The Inference Report
March 14, 2026
AI Labs — News
The announcements today split along a familiar fault line: one lab is selling retrieval infrastructure to developers, the other is selling silicon to researchers. Hugging Face and NVIDIA's joint push on NeMo Retriever's agentic pipeline targets the application layer, builders who need smarter search and reasoning chains integrated into their workflows. AMD's GROMACS benchmarking, by contrast, is pure hardware positioning, a direct comparison against its own prior generation that invites life-science teams to calculate ROI on a GPU upgrade. The timing matters: AMD is publishing concrete throughput gains on molecular dynamics workloads just as the market watches NVIDIA's GTC in two days, a reminder that competitive pressure on inference and compute performance is not confined to large language models. Both moves assume their audiences are already sold on the category itself and are now asking narrower questions: which platform, which architecture, which generation. Neither announcement is trying to create demand; both are fighting over where existing demand gets spent.
Sloane Duvall
AI Labs — Models
A curated reference of models from major AI labs, with open/closed weight status, input modalities, and context window size. American labs tend towards closed weights models and Chinese labs tend toward open weights models.
AmazonClosed Weights
- Amazon: Nova 2 LiteTextVisionVideoFiles1M
- Amazon: Nova Premier 1.0TextVision1M
Open Weights
None
AnthropicClosed Weights
- Anthropic: Claude Haiku 4.5VisionText200K
- Anthropic: Claude Opus 4VisionTextFiles200K
- Anthropic: Claude Opus 4.1VisionTextFiles200K
- Anthropic: Claude Opus 4.5FilesVisionText200K
- Anthropic: Claude Opus 4.6TextVision1M
- Anthropic: Claude Sonnet 4VisionTextFiles200K
- Anthropic: Claude Sonnet 4.5TextVisionFiles1M
- Anthropic: Claude Sonnet 4.6TextVision1M
Open Weights
None
Google DeepMindClosed Weights
- Google: Gemini 2.5 FlashFilesVisionTextAudioVideo1M
- Google: Gemini 2.5 Flash LiteTextVisionFilesAudioVideo1M
- Google: Gemini 2.5 Flash Lite Preview 09-2025TextVisionFilesAudioVideo1M
- Google: Gemini 2.5 ProTextVisionFilesAudioVideo1M
- Google: Gemini 2.5 Pro Preview 05-06TextVisionFilesAudioVideo1M
- Google: Gemini 2.5 Pro Preview 06-05FilesVisionTextAudio1M
- Google: Gemini 3 Flash PreviewTextVisionFilesAudioVideo1M
- Google: Gemini 3 Pro PreviewTextVisionFilesAudioVideo1M
- Google: Gemini 3.1 Flash Lite PreviewTextVisionVideoFilesAudio1M
- Google: Gemini 3.1 Pro PreviewAudioFilesVisionTextVideo1M
- Google: Gemini 3.1 Pro Preview Custom ToolsTextAudioVisionVideoFiles1M
- Google: Nano Banana (Gemini 2.5 Flash Image)VisionText33K
- Google: Nano Banana 2 (Gemini 3.1 Flash Image Preview)VisionText66K
- Google: Nano Banana Pro (Gemini 3 Pro Image Preview)VisionText66K
Open Weights
- Google: Gemma 3n 4BText33K
MetaClosed Weights
None
Open Weights
- Meta: Llama 4 MaverickTextVision1M
- Meta: Llama 4 ScoutTextVision328K
- Meta: Llama Guard 4 12BVisionText164K
OpenAIClosed Weights
- OpenAI: GPT AudioTextAudio128K
- OpenAI: GPT Audio MiniTextAudio128K
- OpenAI: GPT-4.1VisionTextFiles1M
- OpenAI: GPT-4.1 MiniVisionTextFiles1M
- OpenAI: GPT-4.1 NanoVisionTextFiles1M
- OpenAI: GPT-4o AudioAudioText128K
- OpenAI: GPT-5TextVisionFiles400K
- OpenAI: GPT-5 ChatFilesVisionText128K
- OpenAI: GPT-5 CodexTextVision400K
- OpenAI: GPT-5 ImageVisionTextFiles400K
- OpenAI: GPT-5 Image MiniFilesVisionText400K
- OpenAI: GPT-5 MiniTextVisionFiles400K
- OpenAI: GPT-5 NanoTextVisionFiles400K
- OpenAI: GPT-5 ProVisionTextFiles400K
- OpenAI: GPT-5.1VisionTextFiles400K
- OpenAI: GPT-5.1 ChatFilesVisionText128K
- OpenAI: GPT-5.1-CodexTextVision400K
- OpenAI: GPT-5.1-Codex-MaxTextVision400K
- OpenAI: GPT-5.1-Codex-MiniVisionText400K
- OpenAI: GPT-5.2FilesVisionText400K
- OpenAI: GPT-5.2 ChatFilesVisionText128K
- OpenAI: GPT-5.2 ProVisionTextFiles400K
- OpenAI: GPT-5.2-CodexTextVision400K
- OpenAI: GPT-5.3 ChatTextVisionFiles128K
- OpenAI: GPT-5.3-CodexTextVision400K
- OpenAI: GPT-5.4TextVisionFiles1M
- OpenAI: GPT-5.4 ProTextVisionFiles1M
- OpenAI: o1-proTextVisionFiles200K
- OpenAI: o3VisionTextFiles200K
- OpenAI: o3 Deep ResearchVisionTextFiles200K
- OpenAI: o3 ProTextFilesVision200K
- OpenAI: o4 MiniVisionTextFiles200K
- OpenAI: o4 Mini Deep ResearchFilesVisionText200K
- OpenAI: o4 Mini HighVisionTextFiles200K
Open Weights
- OpenAI: gpt-oss-120bText131K
- OpenAI: gpt-oss-20bText131K
- OpenAI: gpt-oss-safeguard-20bText131K
xAIClosed Weights
- xAI: Grok 3Text131K
- xAI: Grok 3 BetaText131K
- xAI: Grok 3 MiniText131K
- xAI: Grok 3 Mini BetaText131K
- xAI: Grok 4VisionText256K
- xAI: Grok 4 FastTextVision2M
- xAI: Grok 4.1 FastTextVision2M
- xAI: Grok 4.20 BetaTextVision2M
- xAI: Grok 4.20 Multi-Agent BetaTextVision2M
- xAI: Grok Code Fast 1Text256K
Open Weights
None
Mistral AIClosed Weights
- Mistral: Codestral 2508Text256K
- Mistral: Devstral MediumText131K
- Mistral: Mistral Large 3 2512TextVision262K
- Mistral: Mistral Medium 3TextVision131K
- Mistral: Mistral Medium 3.1TextVision131K
- Mistral: Mistral Small CreativeText33K
Open Weights
- Mistral: Devstral 2 2512Text262K
- Mistral: Devstral Small 1.1Text131K
- Mistral: Ministral 3 14B 2512TextVision262K
- Mistral: Ministral 3 3B 2512TextVision131K
- Mistral: Ministral 3 8B 2512TextVision262K
- Mistral: Mistral Small 3.1 24BTextVision128K
- Mistral: Mistral Small 3.2 24BVisionText131K
- Mistral: Voxtral Small 24B 2507TextAudio32K
AI21 LabsClosed Weights
None
Open Weights
- AI21: Jamba Large 1.7Text256K
Alibaba (Qwen)Closed Weights
- Qwen: Qwen Plus 0728Text1M
- Qwen: Qwen Plus 0728 (thinking)Text1M
- Qwen: Qwen3 Coder FlashText1M
- Qwen: Qwen3 Coder PlusText1M
- Qwen: Qwen3 MaxText262K
- Qwen: Qwen3 Max ThinkingText262K
- Qwen: Qwen3.5 Plus 2026-02-15TextVisionVideo1M
- Qwen: Qwen3.5-FlashTextVisionVideo1M
Open Weights
- Qwen: Qwen2.5 Coder 7B InstructText33K
- Qwen: Qwen2.5 VL 32B InstructTextVision128K
- Qwen: Qwen3 14BText41K
- Qwen: Qwen3 235B A22BText131K
- Qwen: Qwen3 235B A22B Instruct 2507Text262K
- Qwen: Qwen3 235B A22B Thinking 2507Text262K
- Qwen: Qwen3 30B A3BText41K
- Qwen: Qwen3 30B A3B Instruct 2507Text262K
- Qwen: Qwen3 30B A3B Thinking 2507Text33K
- Qwen: Qwen3 32BText41K
- Qwen: Qwen3 8BText41K
- Qwen: Qwen3 Coder 30B A3B InstructText160K
- Qwen: Qwen3 Coder 480B A35BText262K
- Qwen: Qwen3 Coder NextText262K
- Qwen: Qwen3 Next 80B A3B InstructText131K
- Qwen: Qwen3 Next 80B A3B ThinkingText131K
- Qwen: Qwen3 VL 235B A22B InstructTextVision262K
- Qwen: Qwen3 VL 235B A22B ThinkingTextVision131K
- Qwen: Qwen3 VL 30B A3B InstructTextVision131K
- Qwen: Qwen3 VL 30B A3B ThinkingTextVision131K
- Qwen: Qwen3 VL 32B InstructTextVision131K
- Qwen: Qwen3 VL 8B InstructVisionText131K
- Qwen: Qwen3 VL 8B ThinkingVisionText131K
- Qwen: Qwen3.5 397B A17BTextVisionVideo262K
- Qwen: Qwen3.5-122B-A10BTextVisionVideo262K
- Qwen: Qwen3.5-27BTextVisionVideo262K
- Qwen: Qwen3.5-35B-A3BTextVisionVideo262K
- Qwen: Qwen3.5-9BTextVisionVideo256K
ByteDanceClosed Weights
- Seed: Seed 1.6VisionTextVideo262K
- Seed: Seed 1.6 FlashVisionTextVideo262K
- Seed: Seed-2.0-LiteTextVisionVideo262K
- Seed: Seed-2.0-MiniTextVisionVideo262K
Open Weights
None
DeepSeekClosed Weights
None
Open Weights
- DeepSeek: DeepSeek V3 0324Text164K
- DeepSeek: DeepSeek V3.1Text33K
- DeepSeek: DeepSeek V3.1 TerminusText164K
- DeepSeek: DeepSeek V3.2Text164K
- DeepSeek: DeepSeek V3.2 ExpText164K
- DeepSeek: DeepSeek V3.2 SpecialeText164K
- DeepSeek: R1 0528Text164K
MiniMaxClosed Weights
- MiniMax: MiniMax M1Text1M
- MiniMax: MiniMax M2-herText66K
Open Weights
- MiniMax: MiniMax M2Text197K
- MiniMax: MiniMax M2.1Text197K
- MiniMax: MiniMax M2.5Text197K