OpenAI publishes first benchmark results for its Jalapeño inference chip

OpenAI published the first performance results for Jalapeño on August 25. Jalapeño is its first custom inference chip and part of a system designed around language-model prefill, decoding, memory placement, and inter-chip communication. OpenAI tested the hardware with GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T using the public InferenceX benchmark.

The comparison joins throughput, latency, and power

OpenAI reports that Jalapeño delivered 1.5 to 1.9 times more work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems across the three models. The article normalizes efficiency using published chip power ratings and notes that Jalapeño’s sustained power stayed below its 700-watt rating in the measured workloads. These remain vendor-reported results, with configuration and benchmark details in the announcement’s appendix.

The architecture aims to keep model state, including key-value caches, close to the compute executing each phase. A large connected domain and integrated network reduce movement between resources. OpenAI also says its models contributed to circuit design, verification, kernel work, and bringing additional open-weight models onto the chip.

Deployment evidence comes next

OpenAI plans to begin deploying Jalapeño in its infrastructure by year end while continuing production qualification. That makes operational data the next meaningful checkpoint: sustained utilization, failure isolation, software maturity, model-porting time, and performance under changing production request shapes.

Hardware evaluators should compare complete systems at matched service-level objectives, including host power, networking, batching, model precision, and time between tokens. A favorable Pareto curve on selected models is useful, but it does not collapse every workload into one ranking. Jalapeño’s larger signal is that OpenAI is bringing model, compiler, serving, network, and silicon decisions into one optimization loop.