Cross-model context mobility for heterogeneous agent systems
UnimAI is building a cross-model context and computation layer. Instead of rebuilding the full prompt cache for each model, Relay transfers reusable computational state into the target model's representation space, verifies the reuse boundary, and recomputes only what quality requires.
Share the reusable core. Recompute only the delta.
Both agents receive the same prompt, but only the source performs the full prefill. Relay maps that state into the target's space, and the target resumes from it — selectively recomputing only what is needed to meet the configured quality threshold.
- What today's systems do
- Prefix and KV caching in engines like vLLM, SGLang, and LMCache reuse computed state when the same model sees the same prefix again. Reuse is a lookup inside one model's representation space.
- Where they stop
- A handoff to a different model has no shared representation — different tokenizer, different attention geometry, different hidden dimensions. There is nothing to look up, so the target model performs a full prefill of the whole history.
- What Relay adds
- Relay maps state between representation spaces, checks the result against a quality threshold, and recomputes only the delta that check demands — falling back to full prefill when a model pair cannot clear the bar.
Prototype benchmarks
Increasing difficulty: scale transfer, then bidirectional transfer, then cross-family translation.
Llama 3.1-70B → Qwen 2.5-7B
Cross family
| Method | Accuracy | Latency |
|---|---|---|
| Llama 70B native | 44.0% | 7,328 ms |
| Qwen 7B native | 45.7% | 899 ms |
| UnimAI KV sharing | 44.0% | 138 ms |
- 96% of native accuracy
- 6.5× faster than Qwen 7B
- Up to 53× faster than Llama 70B
Qwen 2.5-7B → Qwen 2.5-1.5B
Same family
| Method | Accuracy | Latency |
|---|---|---|
| 7B native | 45.7% | 899 ms |
| 1.5B native | 27.6% | 288 ms |
| UnimAI KV sharing | 34.5% | 53.8 ms |
- +6.9 accuracy points over native 1.5B
- 5.4× faster than native 1.5B
- 16.7× faster than 7B
LongBench2 · 116 samples · 16K–32K prompts. Handoff latency covers translation, transfer, assembly, and first-token decoding — it is not standard end-to-end TTFT. The cross-family experiment transfers Llama 3.1-70B KV state into a frozen Qwen 2.5-7B target; the same-family experiment transfers Qwen 2.5-7B state into a frozen Qwen 2.5-1.5B target.
Complex tasks need a team, not one costly model
Decompose the workflow, route each subtask to the best-fit model, and pay premium inference cost only where it adds value.
Reasoning model
$$$Plan, decide, resolve ambiguity — invoked selectively.
Small + fast model
$Search, extract, summarize — routine volume stays inexpensive.
Specialist / private model
$$Code, vision, domain, sensitive data — preserves control.
Best-fit routing buys stronger performance at lower blended cost — and creates a new bottleneck: context must move efficiently across models.
One platform, three ways to adopt it
Relay powers every offering: deployments validate it, and the API scales proven workflows.
License it
UnimAI Relay
Cross-model KV runtime
For inference providers and self-hosted enterprises
Translate · Verify · Fall back
Deploy it
Agent Systems
Production multi-agent workflows
For AI-native and enterprise teams
Plan · Route · Remember · Evaluate
Call it
Managed API
Quality-targeted outcomes
For developers and smaller teams
Cost · Latency · Privacy controls