From Architecture to Demo: Jev, Skills, and Agentic Incident Enrichment
The result brings together several concepts I've covered in previous posts, but this time in a working demo.
In my previous blog about agentic incident enrichment, I walked through an architecture where agents investigate an application incident across different infrastructure domains, collect evidence, and enrich a ticket before an engineer starts troubleshooting.
Well, I finally built it.
And as usually happens when moving from a whiteboard to an actual implementation, a few things changed along the way. I added agent skills, brought Tanzu and AgentMinder into the architecture, and introduced something I’ve been experimenting with in my lab recently: Jev, a decision model from TypeSafe AI.
The result brings together several concepts I’ve covered in previous posts, but this time in a working demo.
Jev: Why Spend Tokens Deciding What to Do?
One thing I’ve been looking at lately is how much time and money agents spend simply deciding what to do next. We typically think about token consumption in terms of generating content or performing tasks, but agents can also spend quite a bit of time reasoning through options, evaluating alternatives, and sometimes going through multiple iterations before making a decision.
Take our incident-enrichment agent, for example. It receives a ticket reporting application latency and needs to decide which diagnostic agents should investigate. Does it need Kubernetes? VCF? The load balancer? All three? A general-purpose LLM can reason through these options, but that reasoning costs time and tokens.
This is where Jev from TypeSafe AI comes in. Jev is a decision model designed to evaluate options and return structured decisions or scores rather than generate lengthy reasoning responses. The idea is to give it sufficient context, let it evaluate the available options, and use the result to guide the agent’s next action.
I’ve started using Jev throughout my lab, routing agentic decisions through it to explore whether we can reduce the time, tokens, and cost associated with decision-making. Of course, faster doesn’t necessarily mean better, and a decision model can still make the wrong choice. That’s why I’ve also built a dashboard to track how we’re using Jev and evaluate its efficiency.
Monitoring Jev decision-model usage and efficiency in my agentic lab.
In this demo, Jev is involved at two points. First, the incident-response agent provides the incident details and application context to Jev, which scores the available domain agents and helps determine which ones should investigate. Later, after the diagnostic agents return their findings, Jev evaluates whether there’s enough evidence to enrich the ticket or whether another investigation pass is needed.
Rather than having the incident-response agent repeatedly reason through these decisions, we’re giving that responsibility to a model specifically designed for decision-making.
Skills: Giving Each Agent a Job Description
Another addition is skills. Every agent in this implementation has one, and each skill defines how that agent should perform its job.
The incident-response agent’s skill describes the overall workflow: retrieve the ticket, collect application context, consult Jev, dispatch the selected diagnostic agents, collect their findings, and determine whether to investigate further or enrich the incident.
The domain agents have narrower skills. The Kubernetes agent knows how to investigate Kubernetes, the VCF agent focuses on infrastructure, and the load-balancer agent investigates its own domain. Their skills also define how they return findings, using structured JSON rather than arbitrary text.
This makes the information easier to collect, correlate, and use in subsequent decisions. It also keeps the responsibilities of the agents relatively focused, which was one of the main architectural considerations in my original article.
Bringing Back Tanzu, AgentMinder, and MCP Design
If you’ve followed my previous posts, you’ll recognize a few other pieces of this implementation.
In my article about structuring MCP servers, I discussed why we shouldn’t simply expose an entire API to an agent. Instead, MCP tools should represent meaningful capabilities aligned with what the agent is trying to accomplish.
That’s exactly the approach here. The diagnostic agents interact with MCP capabilities designed to investigate infrastructure, retrieve performance information, and collect evidence, rather than manipulate the underlying systems.
I also brought Tanzu and AgentMinder into the implementation, building on my earlier discussion about Bind ≠ Grant.
Tanzu provides the environment for running and connecting the agents, while AgentMinder controls their access to MCP capabilities through identity and grants. Just because an agent can discover or connect to a tool doesn’t mean it should be authorized to use it.
And just because Jev decides that the VCF agent should investigate a problem doesn’t mean that agent suddenly gets unrestricted access to VCF.
The decision about what to investigate and the authorization to access a system are two different things.
Putting It All Together
In the demo, I create a latency incident affecting my Metal Music Store application running in my VCF/VKS lab.
The incident-response agent picks up the ticket, collects the application topology, and consults Jev to select the appropriate diagnostic agents. Those agents follow their skills, use their MCP capabilities to investigate their respective systems, and return structured findings.
Once the diagnostics are collected, Jev helps determine whether additional evidence is needed or whether the incident is ready to be enriched.
The result is an updated ticket containing diagnostic findings, supporting evidence, a probable cause, and recommended next steps.
The enriched Metal Music Store incident after the diagnostic agents completed their investigation.
And importantly, the agents don’t remediate anything. They investigate and provide information that helps an engineer understand the problem faster.
That’s still the part of this use case I find most valuable. We’re using agents as an exoskeleton for IT operations, making the people responsible for operating the infrastructure more effective rather than replacing them.
What’s Next?
The implementation works, but there are still things I want to measure. In particular, I’m interested in understanding how much Jev improves decision-making efficiency compared with letting a general-purpose LLM reason through every decision. Faster decisions and fewer tokens are useful, but only if the decisions are good enough to move the workflow in the right direction.
I also want to evaluate how well the specialized agents and their skills perform across different incidents, and whether the resulting enriched tickets actually reduce the time engineers spend investigating problems.
For now, I’ve taken several concepts from my previous articles and combined them into a working implementation. Jev handles decisions, skills define the agents’ procedures, MCP provides the diagnostic capabilities, and Tanzu with AgentMinder helps control access.
Not a bad evolution from a whiteboard drawing.
More on this as it develops.