Artificial intelligence companies have spent the past two years competing on larger context windows, but bigger numbers alone have not solved the core problem. It is one thing for a model to technically accept a huge prompt. It is another for that model to reliably retrieve the right information, reason across it, and do so at a cost that makes real production use viable.
That is the argument behind SubQ, a newly announced model from Subquadratic, and the architecture underneath it called SSA, short for Subquadratic Sparse Attention. According to SubQ’s launch materials, the company is not simply extending the prompt window of a conventional transformer. It is claiming a more fundamental architectural change designed to make long context reasoning practical at much larger scales.
SubQ says its model is built for 12 million token reasoning, runs at around 150 tokens per second, and costs about one fifth of other leading LLMs for comparable long context workloads. Those are ambitious claims, but the more interesting part is how the company says it gets there.
The main limitation of standard transformer attention is that it scales quadratically. In simple terms, each token in a sequence can compare itself to every other token, and that becomes very expensive as prompts get longer. Doubling the context length does not merely double the work. It dramatically increases the amount of compute required.
This matters because many of the most commercially useful AI tasks are long context problems. A software agent may need to reason across an entire repository, a legal model may need to reconcile clauses across a long contract, and an enterprise assistant may need to trace information across documentation, tickets, chat history, and structured data.
SubQ’s technical explainer argues that current systems often compensate for this limitation with retrieval pipelines, chunking, summarisation, and orchestration layers. Those methods can work, but they also introduce complexity and failure modes. The central promise of SSA is that more of this reasoning can happen directly within the model, across a much larger body of context.
According to SubQ, SSA uses content dependent selection to route attention toward the positions that matter, instead of computing all possible token to token interactions. In the company’s framing, most pairwise attention calculations in dense attention are effectively wasted because only a small fraction materially influence the output.
That leads to SubQ’s core technical claim: SSA offers linear scaling in compute and memory relative to the amount of selected attention work, rather than forcing the model into the full quadratic cost profile of dense attention.
SubQ describes three key properties of SSA:
This is important because many prior attempts at efficient long context architectures have required trade offs. Some sparse approaches rely on fixed attention patterns, which can miss distant but relevant information. Recurrent or state space approaches improve efficiency but may lose precise retrieval. Hybrid approaches often keep some dense attention layers, which means the quadratic bottleneck still exists in practice. SubQ’s claim is that SSA avoids those compromises.
The most eye catching benchmark from SubQ’s technical page is the claimed prefill speedup over dense attention as context length increases. On Nvidia B200 GPUs, the company reports the following improvements versus FlashAttention 2:
That last figure, 52.2x at 1 million tokens, is one of the headline numbers in the launch materials because it speaks directly to the commercial problem of long context inference cost.
On its homepage, SubQ also publishes several benchmark figures for SubQ 1M Preview:
SubQ says these results are third party validated, though it also notes that a comprehensive model card is still coming soon. That caveat matters. The early benchmark story is strong enough to attract attention, but broader independent validation will be important before treating these claims as settled.
If SubQ’s architecture performs in practice the way the company says it does, the implications go beyond benchmark competition.
A large share of current enterprise AI engineering is really about working around context limits. Teams build retrieval systems, summarisation layers, caching systems, and multi step orchestration just to help models cope with inputs that are too large or too fragmented to process directly. That stack can be useful, but it is also brittle. It adds engineering overhead, latency, and more places for information to be lost or distorted.
SubQ’s argument is that for some classes of work, those layers should become less necessary if the underlying model can reason across very large artefacts directly. The company specifically positions SubQ for use cases such as full software repository reasoning, long running coding agents, large contract and document analysis, persistent agent state, and enterprise corpora and research workflows.
Its launch page also claims that SubQ Code can plug into tools such as Claude Code, Codex, and Cursor, with around 25 percent lower bills and 10x faster exploration for token heavy coding workflows. If those product claims hold up, the significance of SSA is not just academic. It could meaningfully change the economics of deploying long context AI systems.
There is a real difference between an interesting architecture and a proven production standard.
SubQ’s launch is compelling, but it is still early. The company itself says a full technical report is coming soon, and that means outside observers do not yet have the level of documentation normally used to evaluate a major architectural claim in detail. The benchmark figures are promising, but independent testing over time will matter more than launch day numbers.
There is also a broader historical reason to be careful. The AI industry has seen many efforts to move beyond dense transformer attention, and not all of them have delivered the practical results their early advocates expected. Efficient architectures often look excellent in narrow tests but struggle to maintain general capability, exact retrieval quality, or training stability at scale.
That does not mean SubQ will fail. It simply means the right response is interest, not blind acceptance.
Even with those caveats, SubQ is worth paying attention to because it reframes the conversation around context.
For a while, the market has treated context length as a feature checklist item. But the real issue is not how many tokens can fit into a prompt box. It is whether a model can use that information reliably and economically. That is the distinction between a nominal context window and a functional context window, a distinction SubQ emphasises in its technical explainer.
If SSA proves robust, it would suggest the next phase of AI progress may not just come from scaling model size or adding more infrastructure around existing transformers. It may come from changing the core mechanics of how models allocate attention in the first place.
That is why this announcement matters. Not because SubQ says it can handle 12 million tokens, but because it is making a serious claim that long context AI can become practical rather than merely possible.
SubQ’s launch is one of the more interesting AI architecture announcements of the year so far. The company is claiming that SSA, or Subquadratic Sparse Attention, can preserve useful long context retrieval while dramatically reducing the cost that has held dense attention systems back.
The headline figures are hard to ignore: 12M token reasoning, 150 tokens per second, one fifth the cost of leading competitors, and 52.2x prefill speedup at 1M tokens. Add in reported scores of 81.8 percent on SWE Bench Verified, 95.0 percent on RULER at 128K, and 65.9 percent on MRCR v2, and it is clear why the model is generating interest.
Whether SubQ becomes a genuine architectural breakthrough or simply an impressive early entrant will depend on what happens next: model cards, third party validation, and real world developer use.
But on first pass, this is more than another larger context window announcement. It is a credible attempt to change the economics of long context AI itself.
For businesses and developers interested in testing the model directly, SubQ is currently offering early access through its website: Request early access here.
Ready for us to build your AI pipeline? Learn more about how we help businesses design and implement practical AI systems on our AI consulting page, or get in touch with us to talk through your project.
