OpenAI Jalapeño: custom inference silicon changes the maths

OpenAI, together with Broadcom, has unveiled Jalapeño — its first custom chip, designed exclusively for inference of large language models. This isn’t another “we want our own silicon” statement but a concrete product: a purpose-built ASIC (not a repurposed training accelerator or a general-purpose processor), built in a remarkably short nine-month cycle, with large-scale deployment planned for late 2026.

For people working on security and AI deployments this matters more than it looks. The hardware layer rarely enters risk conversations — yet it increasingly determines what a token costs, where your data is physically processed, and how tightly you are bound to a single vendor.

What was shown

The division of roles in the project is worth noting, because it shows how AI hardware gets built today:

OpenAI owns the chip design — an architecture derived from its own understanding of how models actually behave during inference.
Broadcom supplies silicon manufacturing and networking technology, including its Tomahawk family.
Celestica handles boards, racks and system integration.

The key technical facts made public:

Item	Detail
Purpose	LLM inference only (purpose-built ASIC)
Size	reticle-sized die
Development cycle	approx. 9 months
Performance per watt	per OpenAI, “substantially better” than current state of the art
Deployment	large scale planned for late 2026
Buyers	Microsoft expected to take around 40% of the chips
Role of AI models	used to accelerate the design process itself

Two things deserve comment. Nine months is unusual for a large chip design — the norm is measured in years. OpenAI notes its own models helped speed the work, an interesting feedback loop: AI designing hardware for AI. And a reticle-sized die means the largest area that can be exposed in a single lithography step — a design optimised for maximum silicon area rather than economy.

Why inference specifically

This is the heart of the matter, and it explains not just this project but the market’s direction.

Training a model is a one-off event: expensive, but finite. Inference — generating responses for users — continues for as long as the model is used. As the industry moves from experiments to everyday product usage, the cost centre shifts from training to inference, permanently.

But these two workloads have entirely different technical characteristics. Training is dominated by raw compute throughput over large batches. Inference, especially token-by-token text generation, is largely constrained by memory bandwidth and latency rather than raw compute — the model must repeatedly move weights and state between memory and compute units.

That’s why a general-purpose chip designed with training in mind is over-provisioned for inference: you pay for silicon and energy you don’t use. OpenAI states directly that Jalapeño’s architecture addresses costly data movement, the balance between compute and memory, and networking efficiency. That’s a list of genuine inference bottlenecks, not marketing slogans.

We covered the economics of this in the cost of a token and energy use, and the networking layer in AI factory networks.

The wider trend: everyone wants custom silicon

Jalapeño doesn’t exist in a vacuum. Through 2026 there’s a clear move toward chips designed for specific workloads:

OpenAI — Jalapeño with Broadcom, officially unveiled, deployment late 2026.
DeepSeek — according to Reuters reporting (July 2026), the Chinese lab is working on its own inference chip to reduce dependence on Nvidia and Huawei. The company did not comment, so treat this as unconfirmed.
Anthropic — earlier reporting suggests it has weighed building its own chips; also without a confirmed product.

Add the hyperscalers, who have developed their own accelerators for years, and the conclusion is that designing silicon for your own workload has stopped being the hardware vendors’ domain and become part of model providers’ strategy.

There are three motivations, worth separating:

Cost. A purpose-built chip does one thing more cheaply and at lower power than a general-purpose part.
Availability. Independence from the queue for the market’s most sought-after GPUs has value in itself.
Geopolitics. For Chinese players there’s the added pressure of export controls on the most advanced US chips.

One note of scepticism: designing a chip was never the hardest part. The bottlenecks remain advanced foundry capacity, access to high-bandwidth memory, and software maturity. Announcing a design and delivering working installations at scale are two very different stages.

What it means for a company that just uses AI

If you aren’t building a data centre but consuming models through an API, the change reaches you indirectly — but really.

Prices will keep falling. Cheaper inference translates into lower list prices. We’re already seeing it: we wrote about it with the Gemini 3.6 Flash release, where the same measured quality became noticeably cheaper. Hardware dedicated to inference strengthens that trend.

Uneconomic use cases become economic. This is the most important practical consequence. As unit cost falls, continuous-processing scenarios start to make sense instead of sampling: analysing every ticket, classifying a full document stream, processing entire logs. When planning 2027 deployments, assume lower unit cost than today — as we discuss in implementing AI in your company.

Performance per watt matters to you as well. Energy consumption by AI data centres is a real constraint today — not only on cost but on infrastructure. More efficient hardware means better availability of compute at peak, and therefore lower risk of limits and queues in your integrations.

Hardware diversity raises the value of abstraction. Since models will run on increasingly varied silicon, it pays to design applications so the provider and model remain swappable. That’s an argument for a model router and for keeping prompts and evaluation sets outside integration code.

The security perspective — four areas

Here we reach the part usually missing from hardware launch coverage.

1. Hardware supply chain

Every new participant in the chain — the chip designer, the silicon manufacturer, the system integrator — is another party you effectively trust. This isn’t about assuming bad faith, but recognising that trust in an AI provider now extends to its hardware suppliers. Practically, that means questions in vendor risk assessment: who manufactures, where the installations sit, what integrity verification mechanisms apply.

2. Where your data is physically processed

New infrastructure means new processing locations. For organisations with data residency requirements — public sector, finance, healthcare — that’s not trivia but a compliance condition. With every change to a provider’s infrastructure it’s worth re-asking about processing region and retention, rather than assuming last year’s answers still hold. This connects to obligations described in the EU AI Act.

3. Confidential computing and attestation

The more sensitive data flows through models, the more important the question of whether the infrastructure operator can see the content being processed. The technological answer is trusted execution environments and hardware attestation — mechanisms that let you confirm which hardware and software performed a computation. New purpose-built chips will have to answer these requirements if they’re to serve regulated customers. We expand on this in confidential computing and attestation in AI.

4. Vendor lock-in

The more specialised the hardware, the stronger the tie to the ecosystem running on it. For an end customer the risk isn’t “we can’t run the model on other silicon” but that the whole architecture grows into one provider: call formats, tooling, caching mechanisms, model-specific behaviours. The antidote is architectural, not hardware-based: an abstraction layer, your own evaluation sets, and periodically checking what migration would actually cost.

What about models you run yourself

The custom silicon race mainly concerns large operators, but it has consequences for organisations considering running models on their own infrastructure — a question that comes up increasingly, usually for regulatory or confidentiality reasons.

In the short term the gap widens. An operator with hardware designed for its own workload achieves a unit cost that’s hard to match with commodity hardware at smaller scale. The argument “we’ll self-host because it’s cheaper” grows weaker — if cost is the only motive.

Non-cost motivations remain valid. Data sovereignty, processing location requirements, air-gapped operation, predictable latency and independence from a provider’s pricing changes — none of these is invalidated by cheaper cloud. If your self-hosting decision rests on one of them, falling provider prices change little.

Open-weight models are catching up. Alongside the hardware race, the quality of models you can run yourself and the maturity of the tooling around them keep improving. For many applications — classification, extraction, summarisation — smaller locally-run models are entirely sufficient, and unit cost stops being the bottleneck.

A sensible target model is hybrid. Routine tasks and those touching sensitive data handled locally; tasks demanding the highest capability routed to an external provider — with a clear boundary about which data may cross it. Such an architecture needs an intermediate layer that enforces routing and filtering rules, rather than relying on developer discipline.

What this announcement doesn’t yet prove

Analytical honesty requires separating what was shown from what doesn’t exist yet.

Performance claims come from the vendor. “Substantially better performance per watt” rests on OpenAI’s early testing. There are no independent, repeatable measurements under production conditions and no third-party comparisons. Until those appear, treat the figures as a claim, not a measured fact.

Large-scale deployment is still ahead. The plan says late 2026. Semiconductor history is full of projects that looked excellent on paper and stumbled on manufacturing, memory availability or software maturity.

We don’t know the real price impact. Lower cost for the provider doesn’t automatically mean lower prices for customers — it can be absorbed by margin or by demand growth. Competition makes cuts likely, but that’s a forecast, not a certainty.

Reports about competitors vary in quality. OpenAI’s project was officially unveiled. DeepSeek’s work was described by a news agency citing sources, and the company didn’t comment. Those are two different credibility categories, and worth separating in your own analysis too.

Frequently asked questions (FAQ)

Does this mean the end of general-purpose GPU dominance? Not in any foreseeable timeframe. Purpose-built chips win at a narrowly defined task, but training, research and rapidly changing model architectures still demand flexibility. The realistic scenario is coexistence: specialised silicon for mass inference, general-purpose for everything else.

Should we delay our AI deployment to wait for cheaper hardware? No. Waiting for the next generation is an endless trap in this industry — something cheaper is always around the corner. It’s more sensible to build an architecture resilient to provider and model changes, so you benefit from price cuts when they arrive without rewriting the application.

How does this affect our compliance requirements? Mainly through questions of location and processing terms. With every significant change to a provider’s infrastructure it’s worth refreshing contractual clauses, establishing the processing region and confirming retention rules. The fact that a provider changes hardware doesn’t change your obligations — but it may change the facts you based them on.

Does a provider’s own hardware increase or decrease our risk? Both answers are partly true. Vertical integration can improve control over the whole stack and make attestation mechanisms easier to implement. At the same time it lengthens the supply chain and deepens dependence on a single ecosystem. The net effect depends on how you design your own abstraction layer.

What should we actually do today? Three things: add questions about infrastructure and processing location to your vendor risk assessment; design integrations so that changing models doesn’t require rewriting the application; and calculate what migrating to another provider would really cost. If you’d like to work through that methodically, we do it as part of an AI/LLM security audit.

Summary

Jalapeño signals that the AI market is entering a phase where what matters is no longer only how clever the model is, but what it costs to run it. A purpose-built inference ASIC, designed in nine months and deploying at scale in late 2026, is the most literal expression of that — and parallel reports about other labs show it’s a trend, not an exception.

For companies using AI the conclusions are mundane but concrete: unit cost will keep falling, so plan deployments assuming cheaper inference than today; your provider’s infrastructure is changing, so ask about processing location and attestation periodically rather than once at contract signing; and design for swappability, because in an environment where hardware and models change every few months, the biggest cost is often an architecture welded to one vendor. If you want to review your AI deployment for risk, compliance and lock-in — let’s talk.

Sources and further reading: TechCrunch, Tom’s Hardware, Reuters — reporting on DeepSeek, Broadcom.