Four findings, one company spine.
Where should scarce proprietary effort go when the model underneath is becoming a commodity?
Spend it forking a gateway and you inherit maintenance debt for something that is becoming free; spend it nowhere and you have no moat at all. The argument here is about what to build above the gateway, and EPAM's open-source DIAL is the capable, swappable floor it sits on. Four findings carry it: keep DIAL as the inherited kernel rather than forking it into maintenance debt; own the active endpoint that runs before and after every model call; sell two angles, coding-agent memory and vertical AI workflows, through one shared runtime; and treat kernel portability as a test plan, not a slogan. The fourth is the load-bearing caveat: the layer is designed around a portable internal contract that has to be proven per kernel for request fidelity, latency, cost extraction, and failure handling.
Active context control, not another gateway.
A governed endpoint that remembers, routes, prices, and explains agentic work across kernels. The gateway is inherited and replaceable; the endpoint above it, the thing that decides what context and policy and budget enter each prompt, is the product.
A gateway routes a prompt. It does not know the work.
A model gateway is useful but does not by itself know the repository, the issue, the customer workflow, the budget, or which prior decision must be preserved. Two corpora need that context, and the failure mode differs by corpus.
- An SDLC corpus. Coding agents draw on architecture decisions, tickets, pull requests, incidents, and handoff notes. The risk a memory layer prevents: reversing a prior decision, or burning tokens re-fetching context the agent should already hold.
- A domain corpus. Vertical workflows need source-bound customer knowledge, policy, and workflow state. The risk: a plausible answer that triggers the wrong business action.
The shared problem is context selection under policy. The layer wins only if it makes model calls better, cheaper, or safer in a way a customer can see, not as a hidden prompt trick.
Five criteria decide product or demo.
The architecture is attractive only if it clears a bar stated before implementation. These five test whether the value layer survives real deployment pressure.
| criterion | pass condition | why it matters |
|---|---|---|
| Active control | Runs in the request path, not only after the event. | Memory, policy, and cost decisions must happen before spend occurs. |
| Kernel portability | DIAL plus at least one non-DIAL kernel pass the same conformance suite. | Otherwise the product is DIAL customisation, not a portable layer. |
| Cost truth | Usage is joined to a rate card and session identity. | Buyers need spend explained per workflow, not only per provider bill. |
| Memory governance | Every surfaced fact can point to a source and a policy reason. | Agent memory must be auditable, not a hidden prompt trick. |
| Failure behaviour | Memory, kernel, and policy-stop each produce a clear degraded mode. | Reliability claims start with what happens when dependencies fail. |
Keep the commodity foundation, own the value layer.
A model can answer only the prompt it receives. The value layer decides what context, policy, and budget information enters that prompt. Dashboards read the telemetry but do not make the endpoint valuable on their own; the endpoint earns value when it changes what the model sees and what the organisation can govern.
The repository ledger separates inherited open-source components from the local proprietary layer and supporting runtime services. The licence line matters for diligence: an Apache-2.0 foundation makes commercial composition possible, but inherited features must never be presented as proprietary inventions.
| layer | components |
|---|---|
| Inherited DIAL | Core, Chat, Admin, RAG, Quick Apps, Analytics, SDKs. |
| Owned layer | the model-to-DIAL adapter, the custom telemetry plane, cost exporters, memory interceptors, vertical workbenches. |
| Runtime support | Redis, Postgres, a vector store, InfluxDB, Grafana, an OpenTelemetry collector. |
One layer, shipped as an interceptor or a standalone endpoint.
The same proprietary logic can ship as a DIAL-native interceptor (Mode A) or as a standalone primary endpoint (Mode B), and the choice changes the risk profile.
Interceptor mode to prove value inside DIAL; standalone mode to prove the layer survives without it.
Mode A keeps the customer close to DIAL's configuration model and is best for native distribution; its risk is contract drift from upstream interceptor changes. Mode B lets the company own the primary control point and is best for proving kernel substitution, but it inherits responsibility for auth, scaling, session management, and failure handling.
What to keep upstream, what to own and close.
The cut line protects the story from overclaiming and tells engineering where to spend. Keep the commodity foundation upstream; own the context, the cost ledger, and the proof.
| keep upstream | own and close | reason |
|---|---|---|
| DIAL unified API and Core routing | policy and memory enrichment | routing is the foundation; context policy is the value. |
| DIAL Chat and Admin | customer-specific workbenches | default UI proves access; vertical UI proves workflow fit. |
| DIAL realtime analytics | a billing-grade cost ledger | observability explains behaviour; cost truth supports decisions. |
| DIAL SDKs and interceptor SDK | the conformance suite and adapters | the SDK is a contract; the company owns proof that it holds. |
Portability is a test you can rerun across providers.
A portable value layer needs a normalised internal contract; each kernel adapter then proves it preserves request meaning, cost fields, and failure semantics, judged on four properties: request fidelity, latency overhead, cost extraction, and failure behaviour. The adapter effort tiers: a thin OpenRouter or OSS gateway as the first non-DIAL test, medium-effort enterprise tests on Bedrock, Vertex, or Foundry where cost and policy controls differ by provider, and DIAL Core as the reference anchor. Build the proof in the order a buyer can verify: this quarter, define and test the contract; next, prove one coding workflow; then prove one vertical workflow.
What this note does not claim.
It does not claim Palantir parity (the category benchmark for ontology and operational AI), Faculty-level packaging maturity, that every kernel works, that analytics is the product, or that licence diligence is finished. It claims one thing: active context control, proven per kernel, is the defensible layer above a commodity gateway.
The model stays swappable; the in-path layer is the product.
The inference kernel is becoming a commodity that several vendors will supply interchangeably. What an enterprise cannot buy off the shelf is a governed, in-path endpoint that holds its memory, enforces its policy, prices its own spend, and explains agentic work across whichever kernel runs underneath. That endpoint is where scarce proprietary effort earns a return, and the discipline of this note is to keep the line between what is inherited and what is owned honest enough to survive diligence.
An independent applied-AI engineering note by Arseny Gorokh. Not an official EPAM publication. EPAM's DIAL is the open-source platform the research studies; the architecture argument is the author's own. Companion artefacts are available on request.
References
- EPAM DIAL open-source repositories (Core, Chat, Admin, RAG, Quick Apps, Analytics, SDKs), the inherited kernel.
- Apache-2.0 licence, under which commercial composition of the inherited components is possible; notices and upgrade paths remain an operational responsibility.
- OpenTelemetry GenAI semantic conventions, the basis of the cost-truth telemetry plane.
- Palantir Foundry and AIP, cited as a category bound for ontology and operational AI, not a parity claim.
- Faculty Frontier, cited as a packaging and decision-intelligence maturity bound.
- Cloud gateway documentation, Bedrock, Vertex, Foundry, OpenRouter, behind the kernel-substitution tiers and the per-provider cost and policy variance.