<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en_US"><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="https://www.salaboy.com/feed.xml" rel="self" type="application/atom+xml" /><link href="https://www.salaboy.com/" rel="alternate" type="text/html" hreflang="en_US" /><updated>2026-08-30T08:20:23+01:00</updated><id>https://www.salaboy.com/feed.xml</id><title type="html">Salaboy</title><subtitle>Mauricio Salatino (Salaboy) on platform engineering, Kubernetes, cloud native workflows, AI agents and frameworks, and developer experience (DevEx).</subtitle><author><name>Mauricio Salatino</name></author><entry><title type="html">Reacting to AI (reducing shadow AI practices)</title><link href="https://www.salaboy.com/2026/06/08/reacting-to-ai/" rel="alternate" type="text/html" title="Reacting to AI (reducing shadow AI practices)" /><published>2026-06-08T14:45:45+01:00</published><updated>2026-06-08T14:45:45+01:00</updated><id>https://www.salaboy.com/2026/06/08/reacting-to-ai</id><content type="html" xml:base="https://www.salaboy.com/2026/06/08/reacting-to-ai/"><![CDATA[<p>Getting on top of the latest trend is always fun, but can we make it work this time? While funny/sad pages like <a href="https://isaiprofitable.com/">https://isaiprofitable.com/</a> keep popping up, I really wonder: Why can’t we use what we learnt over the last 10+ years in the Kubernetes Ecosystem to double down on platform engineering practices that let teams maximize the use of GenAI?</p>

<p>From my experience talking to community members, platform teams inside large organizations, and consultants in different industries, the key point that all make is that their platform teams are struggling to enable teams requesting more and more GenAI services. Every developer, SRE, DevOps person currently has access to Claude Code, Codex, Copilot among others. This is generating an explosion of tools that can be applied at different stages into the Software Development Life Cycle (SDLC). With more and more agents / services offered for teams to use, platform teams are feeling the heat of supporting a wide range of internal, external and SaaS services that solve very specific problems.</p>

<p>To be honest, this is nothing new. Every platform engineering presentation that I’ve seen in the last three years makes reference to the CNCF ecosystem, which we used to explain how hard it was for teams to keep up. The problem has now been multiplied hundred times.</p>

<h3 id="dealing-with-multi-agent-distributed-systems">Dealing with Multi-Agent distributed Systems</h3>

<p>Let me put on my practical hat, and let’s be concrete. We are experiencing first hand the journey for hyper-automation. We are not talking about tools for GitOps, Security, Observability. We are talking about “<strong><em>autonomous</em></strong>” agents that perform very specific tasks. In general, one of the main characteristics of these agents is that they use GenAI to evaluate the context and produce an output. Examples of these agents are extremely popular these days:</p>

<ul>
  <li><strong>Coding Agents</strong>: such as Claude Code, that based on a prompt will produce code</li>
  <li><strong>Pull Requests Review Agents</strong>: that review the code proposed in a PR and provide feedback, comments or automatically reject</li>
  <li><strong>SRE Agents</strong>: that monitor a Kubernetes Cluster and can help SRE teams to troubleshoot issues</li>
  <li><strong>Business Agents</strong>: agents that deal with customer requests, for example a chatbot that is connected to internal systems and can answer business requests.</li>
</ul>

<p>I am pretty sure that platform teams didn’t ask for Claude Code to be given to all the organization developers, but platform teams are the ones responsible for making sure that accounts are managed, keys can be distributed and (if you are lucky) some reports about usage are generated.</p>

<p>We have reached a point where there is no way out, platforms need to rise to meet the demand of all these new workloads, which in most cases are far from our production environments. Some of these agents might run on Kubernetes, but some of them will run on your CI infrastructure (GitHub, Gitlab), some will run on each developer laptop, pushing your platform initiatives way outside of Kubernetes and your cloud provider accounts.</p>

<p>Another by-product of adding all these agents to the SDLC is that the shape of what it means to deliver software is mutating. If we are hyper-automating every step of the SDLC, human involvement and intervention shifts. This great article by <a href="https://medium.com/@codefinger/the-figure-eight-model-for-agentic-devex-c76894b12de5">Joe Kutner from Saleforces/Heroku</a> makes it clear, our old SDLC was created for humans, so it must change now that we have agents.</p>

<h2 id="but-why">But, Why?</h2>

<p>Automating just for the sake of automating, usually doesn’t make sense. But automation always sounds nice, as it is commonly associated with saving money. I am quite happy to see that the industry is slowly aligning on a taxonomy/terminology about what we want to achieve.</p>

<p>Blog posts like this one from <a href="https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/">Dan Shapiro’s early this year</a> are resonating hard in the industry as they start hinting at a concrete plan for what agents, but most importantly platforms will need to evolve. Dan’s blog post uses autonomous self-driven cars as an example which creates a perfect analogy for what your platform needs to be prepared for.</p>

<p><em><strong>The goal:</strong> to build autonomous software factories.</em></p>

<p><em><strong>The role of your platform:</strong> connect all the agents, internal systems and provide contracts that agents, humans and operators can use to operate the autonomous factory.</em></p>

<h2 id="how">How?</h2>

<p>To build such autonomous systems, you need to build strong foundations that agents and humans can use to drive things forward. Having a set of levels to gauge where your organization currently is, is extremely helpful to plan what your next move will be.</p>

<p>You will find these levels with different names and with slightly different descriptions, but it feels like we are getting close to some sort of industry-wide agreement.</p>

<p>Here are the five levels of Agentic Engineering that I see the industry is starting to align with:</p>

<ol>
  <li><strong>Level 1 - GenAI Assisted:</strong> agents assist the user by providing alternatives and suggestions, the user drives the interaction and accept or reject suggestions. There are two examples that clearly resonated with me Grammarly and IDE auto-completion based on GenAI models.</li>
  <li><strong>Level 2 - GenAI Augmented:</strong> agents generate code, perform a task, the human needs to review the output. Tools like Claude Code, Codex, and tools like Cursor fits this level quite nicely. The question that most organizations are starting to ask is around how do they can manage all these tools using a platform approach.</li>
  <li><strong>Level 3 - Spec-Centric:</strong> agents generate and modify resources, but also review and provide proof/validation of behavior (tests reports, screenshots, etc) so the human only reviews intent, not line by line diffs. Specs need to be written to define intent and how it needs to be validated. This is closer to defining business requirements. Agents need to understand architecture, conventions and style that goes beyond a single repository. When you start writing specs, you need to go down to specific technology choices, so agents know what is off limits and what decisions have already been made.</li>
  <li><strong>Level 4 - Selective Autonomy:</strong> based on existing data (from previous executions in similar contexts) agents can make decisions on their own to move things forward autonomously in the SDLC. Agents can check and have clear guidelines to when they are authorized for example to approve and merge a pull request. At this level, a trust model needs to operate at the platform level to progressively let agents be more autonomous. To build trust models the big challenge is to collect and summarize data about previous executions, so agents can use evidence to make grounded on data decisions. Humans still need to handle escalations and override agents when things go off rails.</li>
  <li><strong>Level 5 - Full Autonomy (Dark Factories):</strong> humans provide specs, agents produce, review and ship changes. Humans intervene to improve the factory when failure emerges. Agents notify humans, but don’t ask for permission. It feels impractical to talk about this level as most teams are trying to progress from Level 2 to Level 3.</li>
</ol>

<p>As you can see the level of sophistication required to move from level two to three is what is stopping organizations reaping the benefits of autonomous agents.</p>

<p>Unfortunately, to move from one level to the next, you need to fully master the previous level, as each level builds heavily on the previous one.</p>

<h2 id="so-what-does-this-mean-for-our-platforms">So what does this mean for our platforms?</h2>

<p>If your platform already took care of packaging applications CI, CD with GitOps, security, observability you are well positioned for further automations. If you are already using <a href="https://cloud.google.com/resources/content/dora-roi-of-ai-assisted-software-development">DORA</a> metrics to measure how your teams are performing you are in a really good spot. If you are not there yet, it is time to double down the investment on core infrastructure for your platforms, as <a href="https://cloud.google.com/resources/content/dora-roi-of-ai-assisted-software-development">highlighted by the DORA report on AI ROI</a>.</p>

<p>Imagine that your developers are already generating code with tools like Claude Code, or that you can do automatic code generation from looking at issues descriptions. As teams have increased agentic generation of code, the review phase is still manually handled by humans, they are hitting a bottleneck. Adding agents to review your pull requests might be the next step for you but it will be clearly not the last one.</p>

<h3 id="from-genai-augmented-to-spec-driven-l2---l3">From GenAI Augmented to Spec Driven (L2 -&gt; L3)</h3>

<p>Your platform needs to understand what the intent of the change was being introduced so it can validate its behavior.</p>

<p>From a customer request ticket it needs to be able to validate that the change is not only correct at the code level, but at the behavior/intent level too.</p>

<p>By using GitOps tools like <a href="https://fluxcd.io/">Flux CD</a> or <a href="https://argoproj.github.io/cd/">Argo CD</a>, the creation of preview environments to validate changes is something that platform teams have automated already. The big shift here is to perform automated validations using specialized agents that by reading the issue description and proposed fix can validate the fix in a preview environment. I’ve seen companies using tools like <a href="https://www.vcluster.com/">vCluster</a> to make preview environments (which are most of the time ephemeral) cost efficient.</p>

<p>Notice that for this flow to happen, the platform should already provide a set of mechanisms that set things up for agents to use and perform validations. Notice how different is to let agents use your existing platform constructs, compared with creating agents to use low-level APIs in a non-deterministic way to implement complex flows.</p>

<p>Implementing different release strategies at this stage might be also a good idea to perform deeper validations that might require live traffic or A/B testing. Using tools like <a href="https://knative.dev/docs/serving/">Knative Serving</a> or service meshes like <a href="https://istio.io/">Istio</a> or <a href="https://linkerd.io/">Linkerd</a> play a fundamental part of how platforms can enable this more complex validations.</p>

<p>Fine tuning this process is hard, and only by mastering this loop can you move forward to Level 4.</p>

<p>Before jumping to Level 4, there are two things that I would recommend platform teams to do to build a solid Level 3: <strong>Measuring the impact</strong> and <strong>dealing with non-determinism at the platform level</strong>.</p>

<h3 id="measuring-the-impactdora">Measuring the impact (DORA)</h3>

<p>Generating, validating and making sure that changes follow all the way through production is expensive and time consuming. If the changes that we are making in favour of automation are not improving our delivery practices, investment funds will run out.</p>

<p>That is why I strongly recommend checking the “<a href="https://cloud.google.com/resources/content/dora-roi-of-ai-assisted-software-development">DORA: ROI of AI-assisted Software Development report</a>” which helps organizations to plan for the AI tuition tax.</p>

<p>Organizations should expect a dip into their software delivery pipelines while adopting new tools. This usually materializes in a speed up on deployment frequency which is compensated by an increased failure rate. We are pushing for speed and we get instability in return.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/06/Screenshot-2026-06-07-at-13.47.41.png" alt="Screenshot 2026 06 07 at 13.47." /></p>

<p>But once again, in order to see what is going on, we need to observe/record way more than our production environment. We now need to ingest data from our delivery pipelines (CI/CD, preview environments), from our source version control platform (Github Issues and PRs, for example) and our customer support tickets.</p>

<p>Understanding the impact of AI adoption, follow similar patterns as adding any new tool to our stack. Without having hard data to compare, you are operating in the dark.</p>

<h3 id="dealing-with-non-determinism-a-platform-play">Dealing with non-determinism, a platform play</h3>

<p>Quality stills matter. Every generated change that ends up in a Pull Request, will go through an expensive (resources and time) pipeline before it hits your customers, hence making sure that changes gets validated before pushing them through is a new platform concern.</p>

<p>Organizations have myriads of models to choose from and it is becoming a standard pattern to choose the best model for a given task as well as reviewing generated code from one model using another one.</p>

<p>To reduce cost and optimize resources, platforms need to provide golden paths to simplify these checks and evaluation loops. Developers shouldn’t worry about these checks and as mentioned before, validation can happen at a higher level (intent), but with a detailed technical report about which evaluations were executed for a given change.</p>

<p>A concrete example of a platform capability that a platform should provide is model selection based on previous performance or benchmarks. You can use external benchmarks or benchmarks based on data that you collect from previous code generation or previously executed tasks. To collect data, you can enable as a platform capability to run agents using different models for a given task (for example code generation for a given issue) and then use platform services to evaluate which performs better. While this is more expensive to start with, in the long run you will have enough data to make the right choices based on the tasks that your platform is performing more often.</p>

<h3 id="from-spec-driven-to-selective-autonomy-l3---l4">From Spec Driven to Selective Autonomy (L3 -&gt; L4)</h3>

<p>In this scenario, to approve changes automatically means that you trust your validations/evaluations to be good enough to not impact your customers.</p>

<p>To build trust across your autonomous agents that the resources that they are managing (code, services, environments), you need data.</p>

<p>The only way to build trust is to record previous approvals, but also to measure how these changes are affecting your customers’ experiences. This requires being able to confidently say that your evaluations have been consistent for a set of services that your organization is ok to be driven by these agents.</p>

<p>The human involvement shifts from approval to trust validation. Where the task of the operators is to review the trust gates for services to be shipped and handle escalations when they happen.</p>

<h3 id="from-selective-autonomy-to-dark-factories-l4---l5">From Selective Autonomy to Dark Factories (L4 -&gt; L5)</h3>

<p>The ultimate goal is to build a set of agents that can produce and ship new features, fixes bugs in a completely autonomous way. This level of sophistication requires a strong platform, a strong harness and a lot of data that needs to be used to make continuous decisions as new scenarios need to be managed.</p>

<p>The first players to build these advanced solutions will be the ones heavily invested in providing <strong>GenAI</strong> services. <a href="https://platform.claude.com/docs/en/managed-agents/overview">Claude Managed Agents</a>, <a href="https://openai.com/index/introducing-openai-frontier/">OpenAI Frontier</a>, <a href="https://aws.amazon.com/bedrock/managed-agents-openai/?trk=9ed0b5fe-b1b1-48aa-8a59-bff2e071637c&amp;sc_channel=ps&amp;trk=a3a1d5b1-eaa2-4ade-8305-ede5684ce2b6&amp;sc_channel=ps&amp;ef_id=CjwKCAjwxITRBhBYEiwA6mZm7ZNOc_noNojOqItmp6iOh9wwf-5amjIflCo8mV8JeK3csCIIJYhRrBoCpXoQAvD_BwE:G:s&amp;s_kwcid=AL!4422!3!795841353790!e!!g!!amazon%20bedrock%20agentcore!23533256362!196289137641&amp;gad_campaignid=23533256362&amp;gbraid=0AAAAADjHtp_rCsgcMCKqphoodVuynHpGo&amp;gclid=CjwKCAjwxITRBhBYEiwA6mZm7ZNOc_noNojOqItmp6iOh9wwf-5amjIflCo8mV8JeK3csCIIJYhRrBoCpXoQAvD_BwE">AWS Bedrock Managed Agents</a>,  <a href="https://cloud.google.com/products/gemini-enterprise-agent-platform">Google Gemini Agents Platform</a>, the <a href="https://www.langchain.com/">Langchain ecosystem</a>, <a href="https://crewai.com/">Crew AI</a>, among others are paving the way in terms of requirements, developer experience and expectations.</p>

<p>Following the same approach to build our internal platforms to manage and operate distributed multi-agent enabled SDLCs is not a bad idea.</p>

<p>My experience in the tech industry tells me that these managed platforms aim to target 70% of the use cases, while most large organizations will struggle to re-platform their existing solutions to these new breed of managed services.</p>

<p>Similarly to what happened with Cloud Providers, when organizations go over acquisitions, mergers it will be quite difficult to escape a world where multiple of these managed platforms will need to co-exist and integrate. Keeping an eye on the standards that these platforms use, or to put it in another way, the decisions that they make to platform sticky are the core things to look out for.</p>

<h2 id="dont-panic">Don’t Panic</h2>

<p>As platform engineers we are now in charge of a fleet of agents, running all across our software delivery pipelines. We need to deal with how these agents access different models/services and make sure that these agents get the best model for the work that they are doing. In some ways, the platform is responsible for the output of these agents, as the platform must provide the validation/evaluation mechanisms to guarantee that agents are not going off the rails.</p>

<p>But, we have the tools, and the learnings from the Kubernetes, Cloud Native and Platform Engineering ecosystems. We know how to extend Kubernetes to support GPUs and host LLMs, we know how to efficiently create preview environments and do very advanced GitOps workflows, we know how to observe distributed systems.</p>

<p>The next layer on top of Kubernetes that we need to build, is the <strong>learning layer</strong>. To be able to learn, you need to record information and synthesize durable knowledge. If all these agents use the same standard format to export agents’ telemetry (such as the <a href="https://opentelemetry.io/docs/specs/semconv/gen-ai/">OpenTelemetry GenAI Semantic Conventions</a>) this new layer can benefit from a huge ecosystem of components already using this format.</p>

<p>It is important to recognize that agentic engineering is not Kubernetes-first, agents will run inside and outside of our Kubernetes clusters and our platforms need to cater to both.</p>

<p>If you come from the Cloud Native ecosystem, you are trained to recognize the patterns and the tools to solve specific challenges. My recommendation today is to focus on unification, specs and standards.</p>

<p>As a practitioner, I am looking at what cloud providers and AI providers are building. I am taking notes and mapping the tools that we have in the cloud native ecosystem to help organizations to reuse the infrastructure we already have. GenAI providers are running LLMs on Kubernetes, they are running their managed agent platforms on top of Kubernetes.</p>

<p>As we learned from cloud providers, organizations can go that far only relying on cloud provider services. They need to build their internal practices, pay the AI tuition tax and at the end of the day build their internal platforms that adapt to their structure and operations.</p>

<p>As always, if you are working in this space and seeing similar things, do get in touch. I would love to collaborate with people building tools and pushing standards forward.</p>]]></content><author><name>Mauricio Salatino</name></author><category term="2026" /><category term="agents" /><category term="agentic" /><category term="cloud native" /><category term="kubernetes" /><category term="agentic engineering" /><category term="platform thinking" /><category term="shadow ai" /><category term="multi-agents" /><category term="coding agents" /><category term="opentelemetry" /><category term="gitops" /><summary type="html"><![CDATA[Platform Engineering teams have inherited distributed multi-agent systems and they are struggling to support teams.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/06/serene-iceberg-landscape-stockcake.webp" /><media:content medium="image" url="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/06/serene-iceberg-landscape-stockcake.webp" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Five semantic conventions, one config property (Observing Spring AI with Arconia)</title><link href="https://www.salaboy.com/2026/05/27/five-semantic-conventions-one-config-property-observing-spring-ai-with-arconia/" rel="alternate" type="text/html" title="Five semantic conventions, one config property (Observing Spring AI with Arconia)" /><published>2026-05-27T13:44:46+01:00</published><updated>2026-05-27T13:44:46+01:00</updated><id>https://www.salaboy.com/2026/05/27/five-semantic-conventions-one-config-property-observing-spring-ai-with-arconia</id><content type="html" xml:base="https://www.salaboy.com/2026/05/27/five-semantic-conventions-one-config-property-observing-spring-ai-with-arconia/"><![CDATA[<p>I’ve been going deeper into Spring AI observability lately, and after publishing <a href="https://www.dash0.com/blog/observing-spring-ai-applications-with-opentelemetry-and-dash0">a walkthrough of the Spring AI observability with the spring-boot-starter-opentelemetry setup</a>, I kept running into the same question from people trying to ship telemetry to different backends: “Which semantic conventions should I use?”</p>

<p>That question has no clean answer right now. And I think that’s worth talking about.</p>

<h3 id="the-fragmentation-problem">The fragmentation problem</h3>

<p>When you instrument a Spring AI application with <code class="language-plaintext highlighter-rouge">spring-boot-starter-opentelemetry</code>, you get spans decorated partially decorated with OpenTelemetry’s GenAI semantic conventions: <code class="language-plaintext highlighter-rouge">gen_ai.request.model</code>, <code class="language-plaintext highlighter-rouge">gen_ai.usage.input_tokens</code>, <code class="language-plaintext highlighter-rouge">gen_ai.system</code>, and so on. These are vendor-neutral, CNCF-backed, and the right default if you’re running a general-purpose observability stack like <a href="https://www.dash0.com/">Dash0</a>, <a href="https://grafana.com/">Grafana</a>, or Honeycomb that support OpenTelemetry.</p>

<p>But the moment you start evaluating your AI application — looking at retrieval quality, latency by prompt type, model comparison — you’ll discover that the popular evaluation platforms don’t speak OTel natively. <a href="https://docs.arize.com/phoenix">Arize Phoenix</a> expects <a href="https://arize-ai.github.io/openinference/spec/">OpenInference</a> attributes like llm.model_name and llm.token_count.prompt. OpenLIT and <a href="https://www.traceloop.com/docs/openllmetry">Traceloop’s OpenLLMetry</a> each have their own naming schemes. LangSmith has yet another one.</p>

<p><a href="https://mlunadia.github.io/observing-ai/genai-observability-compared.html">Miguel Luna comparison of these schemas</a> is the clearest breakdown I’ve seen of how deep this split goes. Take just the model attribute:</p>

<table>
  <thead>
    <tr>
      <th>What you want to express</th>
      <th>OTel SemConv</th>
      <th>OpenInference</th>
      <th>OpenLLMetry</th>
      <th>Langfuse</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Model name</td>
      <td><code class="language-plaintext highlighter-rouge">gen_ai.request.model</code></td>
      <td><code class="language-plaintext highlighter-rouge">llm.model_name</code></td>
      <td><code class="language-plaintext highlighter-rouge">gen_ai.request.model</code></td>
      <td><code class="language-plaintext highlighter-rouge">model</code></td>
    </tr>
    <tr>
      <td>Provider</td>
      <td><code class="language-plaintext highlighter-rouge">gen_ai.provider.name</code></td>
      <td><code class="language-plaintext highlighter-rouge">llm.provider</code></td>
      <td><code class="language-plaintext highlighter-rouge">gen_ai.system</code></td>
      <td><code class="language-plaintext highlighter-rouge">model.provider</code></td>
    </tr>
    <tr>
      <td>Input tokens</td>
      <td><code class="language-plaintext highlighter-rouge">gen_ai.usage.input_tokens</code></td>
      <td><code class="language-plaintext highlighter-rouge">llm.token_count.prompt</code></td>
      <td><code class="language-plaintext highlighter-rouge">gen_ai.usage.input_tokens</code></td>
      <td><code class="language-plaintext highlighter-rouge">usage.input</code></td>
    </tr>
    <tr>
      <td>Span taxonomy</td>
      <td>operation name</td>
      <td><code class="language-plaintext highlighter-rouge">span.kind</code> attribute</td>
      <td>operation name</td>
      <td>observation type</td>
    </tr>
  </tbody>
</table>

<p>With these naming differences, creating a dashboard becomes a nightmare. These schemas reflect genuinely different philosophies about how AI operations should be categorized. OpenInference has a strict <code class="language-plaintext highlighter-rouge">span.kind</code> taxonomy (<code class="language-plaintext highlighter-rouge">LLM</code>, <code class="language-plaintext highlighter-rouge">EMBEDDING</code>, <code class="language-plaintext highlighter-rouge">RETRIEVER</code>, <code class="language-plaintext highlighter-rouge">TOOL</code>, etc.). OTel uses operation names and leaves categorization to consumers. Langfuse adds its own observation types (<code class="language-plaintext highlighter-rouge">GENERATION</code>, <code class="language-plaintext highlighter-rouge">EVENT</code>). While I see some convergence slowly happening towards OTel, these different conventions are definitely locking you in to a specific vendor.</p>

<p>So if you’re building a Spring AI application today and you want to send traces to both <a href="https://www.dash0.com/">Dash0</a> and <a href="https://arize.com/phoenix/">Arize Phoenix</a>, you have a problem. With the standard Spring Boot starter, you’d need to write a custom <a href="https://github.com/salaboy/observing-ai/blob/main/java/spring-ai/spring-merch-store/src/main/java/com/example/store/tracing/ChatObservationConventionConfig.java">ChatModelObservationConvention</a> to emit different attributes per backend, and maintain it yourself as each upstream spec evolves. There is also no support for capturing LLM <code class="language-plaintext highlighter-rouge">input/output</code>,  and tool calls parameters and results which can be very useful for debugging and understanding sessions.</p>

<h3 id="enter-arconia">Enter Arconia</h3>

<p><a href="https://docs.arconia.io">Arconia</a> is a framework built by <a href="https://github.com/ThomasVitale">Thomas Vitale</a> that sits on top of Spring Boot and Spring AI. I’ve been watching it closely, and its approach to semantic conventions is the most practical solution I’ve seen to this fragmentation.</p>

<p>The core idea: <a href="https://docs.arconia.io">Arconia</a> decouples your instrumentation code from the semantic convention schema. You write your Spring AI application once. You choose a convention by setting one property.</p>

<p>Pick any of: <code class="language-plaintext highlighter-rouge">opentelemetry</code>, <code class="language-plaintext highlighter-rouge">openlit,</code> <code class="language-plaintext highlighter-rouge">openllmetry</code>, <code class="language-plaintext highlighter-rouge">langsmith</code></p>

<p><code class="language-plaintext highlighter-rouge">arconia.observations.conventions.opentelemetry.ai.flavor=opentelemetry</code></p>

<p><strong><em>Note that to use <code class="language-plaintext highlighter-rouge">openinference</code> you need a separate dependency, as OpenInference don’t share any properties with Otel semantic conventions.</em></strong></p>

<p>That’s it. No custom beans. No overriding <code class="language-plaintext highlighter-rouge">getHighCardinalityKeyValues()</code>. No convention adapter classes. The same application, five different schemas, zero code changes.</p>

<h3 id="setting-it-up">Setting it up</h3>

<p>The dependency swap from <code class="language-plaintext highlighter-rouge">spring-boot-starter-opentelemetry</code> to Arconia is straightforward. In your <code class="language-plaintext highlighter-rouge">pom.xml</code>:</p>

<pre><code class="language-XML">&lt;dependencyManagement&gt;
    &lt;dependencies&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;io.arconia&lt;/groupId&gt;
            &lt;artifactId&gt;arconia-bom&lt;/artifactId&gt;
            &lt;version&gt;0.27.1&lt;/version&gt;
            &lt;type&gt;pom&lt;/type&gt;
            &lt;scope&gt;import&lt;/scope&gt;
        &lt;/dependency&gt;
    &lt;/dependencies&gt;
&lt;/dependencyManagement&gt;
...
&lt;dependencies&gt;
    &lt;dependency&gt;
        &lt;groupId&gt;io.arconia&lt;/groupId&gt;
        &lt;artifactId&gt;arconia-opentelemetry-spring-boot-starter&lt;/artifactId&gt;
    &lt;/dependency&gt;
    &lt;dependency&gt;
        &lt;groupId&gt;io.arconia&lt;/groupId&gt;
        &lt;artifactId&gt;arconia-opentelemetry-semantic-conventions&lt;/artifactId&gt;
    &lt;/dependency&gt;
&lt;/dependencies&gt;
</code></pre>

<p>Check the <a href="https://docs.arconia.io/arconia/latest/opentelemetry/migration/migration-spring-boot/">Arconia recipe to migrate from Spring Boot OpenTelemetry to Arconia</a>.</p>

<p>Arconia’s starter replaces <code class="language-plaintext highlighter-rouge">spring-boot-starter-opentelemetry</code>. It wraps the same OpenTelemetry SDK and Micrometer integration but adds a pluggable convention layer on top, unified configuration for different semantic conventions and expand the support for OpenTelemetry Metrics and logs instrumentation, not provided by the Spring Boot OpenTelemetry starter.</p>

<p>Where Spring Boot requires you to set <code class="language-plaintext highlighter-rouge">spring.ai.chat.observations.log-prompt=true</code> and write a custom observation convention bean to get prompt/completion content into spans, <a href="https://arconia.io/">Arconia</a> handles it through configuration:</p>

<p><a href="http://arconia.observations.conventions.opentelemetry.ai"><code class="language-plaintext highlighter-rouge">arconia.observations.conventions.opentelemetry.ai</code></a><code class="language-plaintext highlighter-rouge">.capture-content=none|span-events|span-attributes  
arconia.observations.conventions.opentelemetry.ai.include-tool-definitions=true  
arconia.observations.conventions.opentelemetry.ai.include-tool-call-content=true</code></p>

<p>These three properties replace the custom <a href="https://www.dash0.com/blog/observing-spring-ai-applications-with-opentelemetry-and-dash0"><code class="language-plaintext highlighter-rouge">ChatModelObservationConvention</code> I described in the previous post</a>. That’s a real improvement. Custom beans that override framework internals are fragile. They break silently when upstream changes the API. A config property is a contract.</p>

<h3 id="five-apps-five-conventions">Five apps, five conventions</h3>

<p>I’ve set up five variants of the Spring Merch Store application in the <a href="https://github.com/salaboy/observing-ai/tree/main/java/spring-ai-with-arconia">observing-ai GitHub repository</a>. Each one is the same application (same tools, same chat client, same Anthropic model), differentiated only by which semantic convention it emits.</p>

<table>
  <thead>
    <tr>
      <th>Directory</th>
      <th>Convention</th>
      <th>Target backend</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">spring-merch-store-arconia</code></td>
      <td>OpenTelemetry</td>
      <td>Dash0 or any Otel backend</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">spring-merch-store-arconia-openinference</code></td>
      <td>OpenInference</td>
      <td>Arize Phoenix</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">spring-merch-store-arconia-openlit</code></td>
      <td>OpenLIT</td>
      <td>OpenLIT dashboard</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">spring-merch-store-arconia-openllmetry</code></td>
      <td>OpenLLMetry</td>
      <td>Traceloop</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">spring-merch-store-arconia-langsmith</code></td>
      <td>LangSmith</td>
      <td>LangSmith</td>
    </tr>
  </tbody>
</table>

<p>To run any of them:</p>

<p><code class="language-plaintext highlighter-rouge">git clone https://github.com/salaboy/observing-ai.git  
cd observing-ai/java/spring-ai-with-arconia/spring-merch-store-arconia  
export ANTHROPIC_API_KEY=sk-ant-...  
./mvnw spring-boot:run</code></p>

<p>Swap the directory name to switch conventions. That’s exactly how thin the difference is.</p>

<h3 id="what-changes-between-conventions">What changes between conventions</h3>

<p>Switching from <code class="language-plaintext highlighter-rouge">opentelemetry</code> to <code class="language-plaintext highlighter-rouge">openinference</code> doesn’t change the span hierarchy you saw in <a href="https://www.dash0.com/blog/observing-spring-ai-applications-with-opentelemetry-and-dash0">the previous post</a>. You still get an HTTP span at the top, a ChatClient span, the advisor chain, the LLM call spans, and the tool call spans. What changes is the attributes on those spans.</p>

<p>Under OpenTelemetry conventions, a chat span looks like this:</p>

<div class="language-properties highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">gen_ai.system</span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err">=</span> <span class="err">anthropic</span>
<span class="err">gen_ai.request.model </span> <span class="err"> </span> <span class="err">=</span> <span class="err">claude-haiku-4-5</span>
<span class="err">gen_ai.usage.input_tokens </span> <span class="err">=</span> <span class="err">1308</span>
<span class="py">gen_ai.usage.output_tokens</span> <span class="p">=</span> <span class="s">58</span>
<span class="py">gen_ai.response.finish_reasons</span> <span class="p">=</span> <span class="s">["end_turn"]</span>
<span class="err">Under</span> <span class="err">OpenInference,</span> <span class="err">the</span> <span class="err">same</span> <span class="err">span</span> <span class="py">emits</span><span class="p">:</span>
<span class="err">llm.provider </span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err">=</span> <span class="err">anthropic</span>
<span class="err">llm.model_name </span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err">=</span> <span class="err">claude-haiku-4-5</span>
<span class="err">llm.token_count.prompt </span> <span class="err">=</span> <span class="err">1308</span>
<span class="py">llm.token_count.completion</span> <span class="p">=</span> <span class="s">58</span>
<span class="err">span.kind</span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err"> </span> <span class="err">=</span> <span class="err">LLM</span>
</code></pre></div></div>

<p>Both express the same information. But if you point your OTLP exporter at Arize Phoenix and your spans carry gen_ai.* attributes, Phoenix won’t know what to do with them. It expects llm.* and a span.kind label from its taxonomy. Without span.kind, Phoenix can’t categorize the span as an LLM call, and your evaluation dashboards break.</p>

<p>The <code class="language-plaintext highlighter-rouge">OpenLLMetry</code> flavor aligns closely with OTel’s naming but enables content capture by default, matching the behavior of the official <code class="language-plaintext highlighter-rouge">OpenLLMetry SDK</code> instrumentation. <code class="language-plaintext highlighter-rouge">LangSmith</code> defaults to <code class="language-plaintext highlighter-rouge">span-events</code> rather than <code class="language-plaintext highlighter-rouge">span-attributes</code> for content capture, which affects how prompt and completion text surfaces in the trace.</p>

<h3 id="what-arconia-adds">What Arconia adds</h3>

<p>The standard <code class="language-plaintext highlighter-rouge">spring-boot-starter-opentelemetry</code> approach gives you auto-configured OTLP exporter for traces and logs (via the OpenTelemetry SDK) and metrics (via Micrometer), however, there is no instrumentation for logs, so no logs are actually exported. Spring AI’s built-in <code class="language-plaintext highlighter-rouge">gen_ai.*</code> span attributes, and <code class="language-plaintext highlighter-rouge">ChatModelObservationConvention</code> beans you can extend or override.</p>

<p><a href="https://arconia.io/">Arconia</a> gives you all of that plus convention portability (switch backends without touching application code), built-in content capture without a custom observation bean, and convention isolation so upstream spec changes get absorbed by the <a href="https://arconia.io/">Arconia</a> dependency rather than landing in your code.</p>

<h3 id="a-word-about-convergence">A word about convergence</h3>

<p>I’ll be honest: I don’t think semantic convention fragmentation in the GenAI observability space is going away soon. OTel optimizes for vendor neutrality. OpenInference optimizes for evaluation workflows. OpenLLMetry optimizes for developer ergonomics. LangSmith optimizes for the LangChain ecosystem. These are legitimate differences, not just naming preferences.</p>

<p>The practical recommendation from Miguel’s comparison is to emit OTel SemConv as your baseline — every backend reads it — and layer on convention-specific attributes where your evaluation tooling needs them. Arconia puts this into practice: emit OTel to your primary observability backend, point an evaluation pipeline at the same app running with OpenInference conventions.</p>

<p>In my opinion, what we actually need is a clearer separation between operational observability (where OTel wins) and AI evaluation observability (where none of the schemas is good enough yet). I expect that separation to become clearer over the next 6 months as the OTel GenAI working group moves these conventions out of experimental status.</p>

<p>Until then, tools like Arconia that abstract the convention layer are doing genuinely useful work. Clone the <a href="https://github.com/salaboy/observing-ai/tree/main/java/spring-ai-with-arconia">observing-ai repository</a>, run the five variants side by side, and see what the same application looks like through each backend’s eyes.</p>

<p>As always, if you have questions, opinions, or build something on top of this, find me on <a href="https://twitter.com/salaboy">X/Twitter</a> or <a href="https://www.linkedin.com/in/salaboy/">LinkedIn</a>. I’d love to see what you’re building.</p>]]></content><author><name>Mauricio Salatino</name></author><category term="2026" /><category term="arconia" /><category term="spring boot" /><category term="spring ai" /><category term="opentelemetry" /><category term="arize" /><category term="langsmith" /><category term="agentic" /><category term="agents" /><category term="observability" /><category term="llm" /><summary type="html"><![CDATA[I don't think semantic convention fragmentation in the GenAI observability space is going away soon.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/05/arconia-blue-horizontal.png" /><media:content medium="image" url="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/05/arconia-blue-horizontal.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">🇪🇸Spring I/O 2026</title><link href="https://www.salaboy.com/2026/04/20/spring-i-o-2026/" rel="alternate" type="text/html" title="🇪🇸Spring I/O 2026" /><published>2026-04-20T18:12:16+01:00</published><updated>2026-04-20T18:12:16+01:00</updated><id>https://www.salaboy.com/2026/04/20/spring-i-o-2026</id><content type="html" xml:base="https://www.salaboy.com/2026/04/20/spring-i-o-2026/"><![CDATA[<p>I just got back from <a href="https://springio.net">Spring I/O 2026 in Barcelona</a>, and, as usual, the event left me with a lot to process — in the best possible way. Spring I/O is one of those conferences that manages to feel both tightly focused and surprisingly broad at the same time. This year, the gravitational pull was clear: AI, agentic patterns, and the evolution of the Spring ecosystem (Spring 7.x and Spring Boot 4.x). From the hall track discussions, it is clear that organizations will be wrestling to answer a set of questions about how to build, test, and understand agentic applications running on the JVM.</p>

<h3 id="the-workshop-testing-and-observing-agentic-applications-with-spring-ai">The Workshop: Testing and Observing Agentic Applications with Spring AI</h3>

<p>The week kicked off with a workshop I ran alongside <a href="https://twitter.com/lbroudoux">Laurent Broudoux</a> — from <a href="https://www.postman.com/">Postman</a> / <a href="https://microcks.io/">Microcks</a> / <a href="https://reshapr.io/">Reshapr</a> — focused on testing and observing agentic applications built with <a href="https://spring.io/projects/spring-ai">Spring AI</a>. We had a great crowd in the room, and the conversation went deep fast. The workshop walked attendees through real patterns for asserting agent behavior, mocking LLMs’ responses, testing and mocking MCP servers, and making the invisible visible through traces.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_8836-1.jpg" alt="🇪🇸Spring I/O 2026" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_8844.jpg" alt="🇪🇸Spring I/O 2026" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_8847.jpg" alt="🇪🇸Spring I/O 2026" /></p>

<p>The workshop code that can be found here <a href="https://github.com/salaboy/spring-io-2026-workshop">https://github.com/salaboy/spring-io-2026-workshop</a> shows how to start simple with a Spring AI application with local/internal <code class="language-plaintext highlighter-rouge">@Tools</code> to then expand to use MCP to consume REST and gRPC APIs. The workshop ended with <a href="https://docs.dapr.io/developing-applications/building-blocks/workflow/workflow-overview/">Dapr Workflows</a> for building durable orchestrations and running all the components inside a Kubernetes cluster.</p>

<div class="video-embed" style="position:relative;padding-bottom:75%;height:0;overflow:hidden;max-width:100%;margin:1.5rem 0;"><iframe allow="web-share" allowfullscreen="true" class="speakerdeck-iframe" loading="lazy" spellcheck="false" src="https://speakerdeck.com/player/0276a652dd1745878eff5712b52ea2eb" style="position:absolute;top:0;left:0;width:100%;height:100%;border:0;" title="Spring I/O 2026: Durable, Observabable and Testable Spring AI Applications"></iframe></div>

<p>We had fun with Laurent preparing and running the workshop, so expect more content, talks, and workshops around Observability, OpenTelemetry, MCP servers, Reshapr, and of course, Microcks for mocking.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7711-1.jpeg" alt="🇪🇸Spring I/O 2026" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7712-1.jpeg" alt="🇪🇸Spring I/O 2026" /></p>

<h3 id="spring-ai-2x-and-the-agentic-push">Spring AI 2.x and the Agentic Push</h3>

<p>The buzz around <a href="https://github.com/spring-projects/spring-ai">Spring AI 2.0</a> was impossible to miss. The project is maturing fast, and the upcoming 2.0 release brings a serious set of new patterns for building agentic systems. Two things stood out to me in particular.</p>

<p>First, Spring AI 2.0 ships with first-class <a href="https://modelcontextprotocol.io">MCP (Model Context Protocol) client</a> and server libraries built in. MCP is the protocol that has quickly become the standard way for agents to expose tools and for orchestrators to consume them. Having that integrated natively into the Spring ecosystem rather than bolted on as an afterthought is a big deal for teams already invested in the Spring Boot programming model.<br />
Second, the project is actively addressing patterns across the emerging protocol landscape — MCP, <a href="https://github.com/google/A2A">A2A (Agent-to-Agent)</a>, and <a href="https://agentcommunicationprotocol.org">ACP (Agent Communication Protocol)</a> are all on the radar. Running multiple protocols concurrently and understanding the observability challenges that come with that is genuinely non-trivial. When you have agents calling other agents through different protocols, tracing a single logical operation end-to-end becomes a real engineering problem.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7713.jpeg" alt="🇪🇸Spring I/O 2026" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7715.jpeg" alt="🇪🇸Spring I/O 2026" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7716.jpeg" alt="🇪🇸Spring I/O 2026" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7717.jpeg" alt="🇪🇸Spring I/O 2026" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7718.jpeg" alt="🇪🇸Spring I/O 2026" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7721.jpeg" alt="🇪🇸Spring I/O 2026" /></p>

<p>Congrats to <a href="https://www.linkedin.com/in/tzolov/">Christian Tzolov</a>, <a href="https://www.linkedin.com/in/marklpollack/">Mark Pollack</a>, and the rest of the Spring AI team for all the insights they shared over the conference.</p>

<p>If you want to learn about Agentic Patterns with Spring AI, you should check this repository: <a href="https://github.com/tzolov/voxxeddays2026-demo">https://github.com/tzolov/voxxeddays2026-demo</a>,</p>

<h3 id="observing-ai-semantic-conventions-openinference-and-a-maturing-ecosystem">Observing AI: Semantic Conventions, OpenInference, and a Maturing Ecosystem</h3>

<p>The <a href="https://opentelemetry.io/docs/specs/semconv/gen-ai/">OpenTelemetry Gen AI semantic conventions are starting to stabilize</a>, and this is genuinely exciting. For the first time, we have a shared vocabulary for what an LLM span should look like — what attributes to capture, how to represent tool calls, how to model embeddings. Without agreed conventions, every vendor builds their own schema and you end up with a bunch of incompatible dashboards.</p>

<p>On that note, <a href="https://github.com/Arize-ai/openinference">OpenInference from Arize AI deserves a mention</a>. OpenInference is an open standard for capturing traces from AI applications — it sits on top of OpenTelemetry and adds the AI-specific semantics. Arize has been pushing hard on tooling here, and I think the approach of building on OTel rather than replacing it is the right call.</p>

<p>At its core, the Spring team’s investment in <a href="https://micrometer.io">Micrometer</a> as the single abstraction over metrics, traces, and soon logs is paying off. Teams using Spring Boot get a coherent observability story without having to wire everything together manually. I want to thank <a href="/2026/04/19/manage-and-distribute-skills-with-skills-oci/">Jonatan Ivanov</a> and Tommy Ludwig for the very insightful session and the deep conversations on the observability roadmap.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7724.jpeg" alt="🇪🇸Spring I/O 2026" /></p>

<h3 id="the-arconia-moment-for-observability">The Arconia moment for observability</h3>

<p>More observability conventions and challenges are popping up, and this is where I need to share my excitement about the <a href="https://arconia.io/">Arconia project,</a> introduced by my friend <a href="https://www.linkedin.com/in/vitalethomas/">Thomas Vitale</a>, who has built a bunch of features to improve the developer experience on top of the Spring Boot framework.</p>

<p>One of those improvements actually comes from expanding Spring’s support to the Otel GenAI semantic convention and supporting the <a href="https://docs.arconia.io/arconia/latest/observability/generative-ai/">OpenInference conventions</a> while at the same time providing dev services to bootstrap tools like <a href="https://phoenix.arize.com/">Phoenix Arize</a> using Testcontainers.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7705-3.jpeg" alt="🇪🇸Spring I/O 2026" /></p>

<p>I will be writing more about this, as I believe that, with the accelerated pace in the agentic and observability space, you sometimes cannot wait for frameworks to catch up; we all need to collaborate to bring solutions to new challenges. I am really looking forward to contributing my findings back to the <a href="https://github.com/arconia-io/arconia/">Arconia project</a>.</p>

<p>Big congrats to Thomas for the announcement 🥳🥳🥳</p>

<h3 id="barcelona-the-community-and-what-comes-next">Barcelona, the Community, and What Comes Next</h3>

<p>Beyond the technical content, Spring I/O is just a genuinely very special conference. <a href="https://www.linkedin.com/in/salmar/">Sergi Almar</a> put together something that feels like meeting friends who share the same passion as much as a conference. I ran into so many familiar faces and met a bunch of new people I hope to stay in touch with.</p>

<p>I am super hyped about Spring I/O 2027, as it will happen in Valencia for the first time ever!!</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7677-1.jpeg" alt="🇪🇸Spring I/O 2026" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/IMG_7703-1.jpeg" alt="🇪🇸Spring I/O 2026" /></p>

<p>The energy around AI in the Spring ecosystem right now is real, and it’s moving fast. Spring AI 2.0, the maturing observability stack, and the protocol convergence around MCP, Skills, A2A, and ACP are all happening simultaneously.</p>

<p>My personal takeaway matched what was shared at the conference: the Java ecosystem is strong because, with a flexible, agent-based programming model, organizations can build robust solutions that leverage all the enterprise integrations built over the last 20 years in the Spring ecosystem.</p>

<p>If you attended Spring I/O and want to compare notes, or if you’re working on anything in the AI observability or Spring AI space, reach out — you can find me on <a href="https://twitter.com/salaboy">Twitter/X @salaboy</a> or <a href="https://www.linkedin.com/in/salaboy/">LinkedIn</a>.</p>]]></content><author><name>Mauricio Salatino</name></author><category term="spring boot" /><category term="spring-ai" /><category term="barcelona" /><category term="spring i/o" /><category term="community" /><category term="opentelemetry" /><category term="observability" /><category term="semantic conventions" /><category term="2026" /><summary type="html"><![CDATA[Spring I/O 2026 in Barcelona brought AI, agentic patterns, and Spring 7.x/Boot 4.x to the forefront.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/springio.png" /><media:content medium="image" url="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/04/springio.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Manage and distribute skills with `skills-oci`</title><link href="https://www.salaboy.com/2026/04/19/manage-and-distribute-skills-with-skills-oci/" rel="alternate" type="text/html" title="Manage and distribute skills with `skills-oci`" /><published>2026-04-19T08:25:18+01:00</published><updated>2026-04-19T08:25:18+01:00</updated><id>https://www.salaboy.com/2026/04/19/manage-and-distribute-skills-with-skills-oci</id><content type="html" xml:base="https://www.salaboy.com/2026/04/19/manage-and-distribute-skills-with-skills-oci/"><![CDATA[<p>I’ve seen this movie before. A new ecosystem explodes, adoption moves faster than anyone expected, and then everyone hits the same wall: how do we actually share this stuff in a reliable, secure, versioned way? In 2014, Docker images were distributed as tarballs. In 2016, Helm charts were floating around as zip files on Slack. Today, in 2026, it’s Agent Skills — and the distribution story is, once again, “clone a repo and hope for the best.”</p>

<p>I’m not saying this to be critical of the <a href="https://agentskills.io/specification">Agent Skills specification</a>. The spec, which Anthropic published in December 2025 and is now stewarded at <a href="https://github.com/agentskills/agentskills">agentskills/agentskills</a>, does something important: it defines a shared format for knowledge assets that coding agents — Claude Code, Cursor, GitHub Copilot, Codex, and others — can discover and use to do their work more accurately. That’s genuinely useful. But the spec stops there. It says nothing about how to distribute skills, version them, verify their integrity, or consume them from inside a corporate firewall. And in my opinion, that gap needs to be closed, the sooner the faster we can move on to other more interesting topics.</p>

<h3 id="weve-been-here-before">We’ve Been Here Before</h3>

<p>The cloud-native ecosystem spent years learning hard lessons about artifact distribution. Helm charts were shared as files attached to GitHub releases. WebAssembly modules were uploaded to ad-hoc buckets. Configuration bundles were copied between teams via Slack DMs. Each of these approaches had the same failure modes: no canonical discovery mechanism, no integrity guarantees, no reproducibility, and no supply chain security story.</p>

<p>The solution the community eventually converged on wasn’t a new protocol or a new registry format. It was <a href="https://opencontainers.org/">OCI — the Open Container Initiative</a> standards that most people associate with container images, but which are actually a general-purpose artifact store. Today, Helm charts live in OCI registries. WebAssembly components are distributed via OCI. <a href="https://fluxcd.io/">FluxCD</a> manifests, SBOM files, Sigstore attestations — all OCI. The lesson the cloud-native world learned is that you don’t need to reinvent infrastructure when battle-tested infrastructure already exists and runs at every major cloud provider, in every enterprise, and even on your laptop via Docker Desktop.</p>

<p>I strongly believe the same lesson applies directly to Agent Skills and other knowledge assets (that influence your coding agent and AI-infused applications’ contexts).</p>

<h3 id="distributing-artifacts-using-oci">Distributing Artifacts using OCI</h3>

<p>I’ve been talking with a bunch of people about this topic, including <a href="https://www.thomasvitale.com/agent-skills-as-oci-artifacts/">Thomas Vitale, who finally published a detailed proposal</a> for packaging Agent Skills as OCI artifacts, covering the spec proposal and why this matter. He did a great job at going over the gap that the spec is trying to fill, for example having a OCI Image Manifest with a custom <code class="language-plaintext highlighter-rouge">artifactType</code> (<code class="language-plaintext highlighter-rouge">application/vnd.agent-skills.skill.v1</code>), and its contents — the <code class="language-plaintext highlighter-rouge">SKILL.md</code>, scripts, and resources already defined by the spec — are packaged as the artifact’s layers.</p>

<p>On the consumer side, a <code class="language-plaintext highlighter-rouge">skills.json</code> declares which skills a project depends on by OCI reference, and a <code class="language-plaintext highlighter-rouge">skills.lock.json</code> pins the exact digest of each skill at install time. This is the same model Go modules, npm, and Cargo use — and it works. You get reproducible installs across your whole team, clear upgrade paths, and a paper trail.</p>

<p>Because these are standard OCI artifacts, any compliant registry works. Docker Hub, <a href="https://ghcr.io">GitHub Container Registry</a>, <a href="https://goharbor.io/">Harbor</a>, <a href="https://zotregistry.dev/">Zot</a>, your company’s internal registry — it doesn’t matter. No new infrastructure to run, no new accounts to create, no vendor lock-in on the registry side.</p>

<h3 id="packaging-and-sharing-a-skill-with-skills-oci">Packaging and Sharing a Skill with <code class="language-plaintext highlighter-rouge">skills-oci</code></h3>

<p>Ok, but specs are hard to evaluate and understand, they require consensus, which is already very difficult but then adoption that can only be driven by people fully understanding the problem space.</p>

<p>For that reason, I’ve been building <a href="https://github.com/salaboy/skills-oci"><code class="language-plaintext highlighter-rouge">skills-oci</code></a>, a CLI built on the <a href="https://github.com/oras-project/oras">ORAS Go client</a> that implements the spec proposed by Thomas, and it shows an end-to-end experience to manage (package and publish) and consume skills. I am a practical guy, so I wanted a clean experience that integrates with the tools you are already using.</p>

<p>The core workflow — four commands to go from a local skill to a shared, versioned artifact on Docker Hub:</p>

<h4 id="package-and-push-your-skill">Package and push your skill:</h4>

<p>To package and publish a skill to a container registry (in this case Docker Hub):</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd </span>my-skill/
skills-oci push docker.io/salaboy/my-skill:1.0.0
</code></pre></div></div>

<h4 id="adding-skills-to-your-projects">Adding Skills to your Projects</h4>

<p>From any project where you want to use the skill:</p>

<p>Inside your project directory, where you want to add a managed skill. Add it as a dependency to your project:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>skills-oci add docker.io/salaboy/my-skill:1.0.0
</code></pre></div></div>

<p>Adding skills will create the <code class="language-plaintext highlighter-rouge">skills.json</code> and <code class="language-plaintext highlighter-rouge">skills.lock.json</code> file.</p>

<h4 id="cosign">Cosign,</h4>

<p>Install (re-fetch) all declared skills (resolves and locks digests):</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>skills-oci <span class="nb">install</span>
</code></pre></div></div>

<h4 id="verifying-skills-signature-and-provenance">Verifying Skills signature and provenance</h4>

<p>Using Cosign, you can check the container image signature and SLSA provenance:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>skills-oci verify
</code></pre></div></div>

<p>That last command is not just cosmetic. Because the artifact is content-addressed via digest and can be signed with <a href="https://github.com/sigstore/cosign">Cosign</a>, verification is a real guarantee—not a checkbox. The full source and examples are at <a href="https://github.com/salaboy/skills-oci">github.com/salaboy/skills-oci</a>.</p>

<h4 id="registering-the-skills-with-your-coding-agent">Registering the skills with your coding agent</h4>

<p>Notice that I wanted to hook into the existing coding agent ecosystems, so I’ve included a command that automatically registers a startup hook with your coding agent, which will install (fetch and download) your skills when starting a new session.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>skills-oci register
</code></pre></div></div>

<p>If we can distribute versioned skills that can be verified and dynamically loaded into our coding agents, we don’t need to push them to our git repositories, where they will diverge across projects and be hard to maintain. By default, the <code class="language-plaintext highlighter-rouge">skills-oci</code> CLI will add the <code class="language-plaintext highlighter-rouge">.agents/skills/</code> directory to your <code class="language-plaintext highlighter-rouge">.gitignore</code> file so you don’t need to worry about stale copies of your skills. If you want to upgrade your skills for a given project, all you need to do is update the version in the <code class="language-plaintext highlighter-rouge">skills.json</code> file.</p>

<h3 id="free-supply-chain-security--no-extra-work">Free Supply Chain Security — No Extra Work</h3>

<p>This is the part that genuinely excites me, and I think it’s undersold in most conversations about OCI for non-container artifacts. Because a skill packaged as an OCI artifact is just an artifact like any other, the entire cloud-native supply chain security toolchain applies immediately — with zero additional format work.</p>

<p>You can sign a skill with <a href="https://docs.sigstore.dev/">Cosign</a> and attach <a href="https://slsa.dev/">a SLSA provenance attestation</a> so consumers can cryptographically verify who built it and from which source commit. You can generate an SBOM with <a href="https://github.com/anchore/syft">Syft</a> and attach it as a referrer via the <a href="https://github.com/opencontainers/distribution-spec/blob/main/spec.md#listing-referrers">OCI Referrers API</a>. You can run a vulnerability scan with <a href="https://github.com/anchore/grype">Grype</a> and attach the results so downstream teams can make informed decisions about which skills to adopt. None of this requires any new tooling — it all already works against any OCI-compliant registry.</p>

<p>For enterprises, this is the difference between <strong><em>“we allow skills”</em></strong> and <strong><em>“we have a governed, auditable skills pipeline”.</em></strong> That distinction matters as agent-assisted development moves from hobby projects into regulated industries.</p>

<h3 id="what-still-needs-to-happen">What Still Needs to Happen</h3>

<p>Let’s be honest, standardization takes time and it is really hard, companies want to own the space to control the ecosystem and to drive business their way. Companies like <a href="https://skills.sh/">Vercel with skills.sh</a>, <a href="https://tessl.io/">Tessl</a>, and other communities (for example, the <a href="https://www.skillsjars.com/">skillsjar project)</a> are pushing for their own solutions, their own packaging formats, and targeting audiences that are not particularly interested in reusing existing infrastructure at this point.</p>

<p>On the other hand, any seasoned engineers can point out, this is just the begining, as packaging skills only solves distribution, but not dependency management and relationships between skills and other artifacts, hence it feel like there will be a long journey ahead.</p>

<p>Regarding, the <code class="language-plaintext highlighter-rouge">skills-oci</code> CLI, I wanted to show the experience I want as a user when working with skills, but the tools aren’t important here; the spec is. The <code class="language-plaintext highlighter-rouge">skills-oci</code> CLI is written in Go and it can be also consumed as a Go library.</p>

<p>I feel quite optimistic about where this is heading — but optimism doesn’t write specs. If you’re building in this space, come help shape it. Try <code class="language-plaintext highlighter-rouge">skills-oci</code>, open issues, push back on the proposal if you think something’s wrong. The cloud-native ecosystem emerged through open debate, and I expect the agent skills ecosystem will need to do the same.</p>

<p>Drop me a message on <a href="https://linkedin.com/in/salaboy">Linkedin</a>, <a href="https://x.com/salaboy">X</a> or open an issue at <a href="https://github.com/salaboy/skills-oci">github.com/salaboy/skills-oci</a> if you want to collaborate on this.</p>]]></content><author><name>Mauricio Salatino</name></author><category term="oci" /><category term="skills" /><category term="spec" /><category term="open" /><category term="skills-oci" /><category term="community" /><category term="agentic" /><category term="containers" /><category term="registries" /><category term="2026" /><summary type="html"><![CDATA[Manage and distribute agent skills with OCI-compliant images, enabling versioning, distribution, reuse of existing infrastructure, and leveraging the OCI…]]></summary></entry><entry><title type="html">New Beginnings, same principles, still pushing for Open Source and Open Standards</title><link href="https://www.salaboy.com/2026/04/07/joining-dash0/" rel="alternate" type="text/html" title="New Beginnings, same principles, still pushing for Open Source and Open Standards" /><published>2026-04-07T09:15:55+01:00</published><updated>2026-04-07T09:15:55+01:00</updated><id>https://www.salaboy.com/2026/04/07/joining-dash0</id><content type="html" xml:base="https://www.salaboy.com/2026/04/07/joining-dash0/"><![CDATA[<p>Over the last 6 months, I’ve been spending considerable time exploring the land of coding agents, agentic frameworks (to build business agents), and the rapidly evolving landscape of tooling in these spaces. At KubeCon EU, it wasn’t a shock to see how companies are building their own stacks to run, serve, and control how teams interact with a wide range of LLMs (for coding or business agents).</p>

<p>Last <a href="https://www.linkedin.com/posts/salaboy_dapr-opensource-ai-activity-7445756783714988032-21GV?utm_source=share&amp;utm_medium=member_desktop&amp;rcm=ACoAAAEprDYB92fxehJ4a6Jyxm-N9zCQL7hohHg">Thursday was my last at Diagrid</a>, where I went knee-deep for the last three years into developer tooling, workflow engines, and, more importantly, how to smooth developers’ journeys when building complex distributed systems with a constantly evolving set of tools.</p>

<p>Today I am hyped to be joining <a href="https://www.dash0.com/">Dash0 as an Ecosystem Engineer</a>, because observability, and to be more concrete, <a href="https://opentelemetry.io/">OpenTelemetry</a> has a big role, not only on the new stacks that companies are building, but also in how developers gain visibility about what their coding agents are doing. It is becoming clear to me that <a href="https://opentelemetry.io/"><strong>OpenTelemetry</strong></a> <strong>is the foundation for understanding agentic systems</strong>.</p>

<p>While companies are building new stacks using mature ecosystems such as Kubernetes and OpenTelemetry, new use cases are emerging as developers push more code than ever before, using tools that haven’t been tested at scale. Understanding what is going on is more important than ever.</p>

<p>Unfortunately, to safely deliver all the software generated today into production, understanding our production environments is not enough. We will need to understand: how code is generated?, how that process works for your organization?, how generated code behaves in production environments (is it deterministic or not)?, how do we evaluate non-deterministic errors?, and, more importantly, how to revert changes based on observed errors?.</p>

<p>I strongly believe that to answer these questions, we need standards that span developer tooling and the infrastructure that will run that code in production. Open Telemetry is strategically positioned to lead the way to other standards, such as the OCI (Open Container Initiative), for <a href="https://www.thomasvitale.com/agent-skills-as-oci-artifacts/">packaging and distributing knowledge artifacts</a> so we can reuse not only existing infrastructure but also the practices we have evolved over the last 10 years. These standards are relevant to all the phases of the SDLC (Software Development Life Cycle), not only to developers, not only to DevOps, not only to production.</p>

<p><em>But let’s be honest</em>: none of the mainstream tools being developed today have observability or industry standards (distribution provenance, security, auditability) as top priorities on their roadmaps. Hence, it is up to us, the “consumers”, the organizations, and the communities evaluating these tools to figure out how they map to the problems that we are trying to solve and how we can build production-ready solutions with them.</p>

<p>As always, you will see me out in the open, collaborating with and learning from the industry’s smartest people. I look forward to collaborating, building cool stuff, and interacting with communities spanning from developers and developer tooling to infrastructure and SREs.</p>]]></content><author><name>Mauricio Salatino</name></author><category term="cloud native" /><category term="opentelemetry" /><category term="job" /><category term="work" /><category term="kubernetes" /><category term="open source" /><category term="agentic" /><category term="observability" /><summary type="html"><![CDATA[I am hyped to be joining Dash0 as an Ecosystem Engineer because OpenTelemetry is the foundation for understanding agentic systems.]]></summary></entry><entry><title type="html">DevEx in the age of AI</title><link href="https://www.salaboy.com/2026/03/23/devex-in-the-age-of-ai/" rel="alternate" type="text/html" title="DevEx in the age of AI" /><published>2026-03-23T05:53:31+00:00</published><updated>2026-03-23T05:53:31+00:00</updated><id>https://www.salaboy.com/2026/03/23/devex-in-the-age-of-ai</id><content type="html" xml:base="https://www.salaboy.com/2026/03/23/devex-in-the-age-of-ai/"><![CDATA[<p>I’ve been trying to write about this topic for a while, but the feeling that we already have too much content, much of which can be auto-generated, has been slowing me down lately. The question “so why bother?” still resonates.</p>

<p>When we started with <a href="https://www.manning.com/books/developer-experience-on-kubernetes">Thomas Vitale on the project to write about Developer Experience for Kubernetes</a>, we knew we would need to cover AI, LLMs, and other strange tools that would pop up as we wrote the book. This is the rule in our industry: <strong><em>nothing stays the same for a year</em></strong>.</p>

<p>Writing a book is always messy, and ideas tend to change a lot. To define core and fundamental topics that continue to affect developers working with cloud-native technologies, we focused on tools and practices that will remain a must for engineering teams moving forward.</p>

<p>But it is true: the industry has massively changed, both in terms of speed and the kinds of tools popping up. In the last year, we have seen new protocols, libraries, CLIs, and IDEs popping up like crazy and also changing at unprecedented speed. A tool you learn today may not be around tomorrow, and developers need to adapt.</p>

<p>In this blog post, I wanted to cover my biased journey to the land of coding agents and, instead of comparing the features of the different tools available in the market, focus on the fundamentals.</p>

<p>Stepping back from the day-to-day of using code-generation tools might help us seize opportunities and avoid mistakes that push us several steps backward. Based on my experience writing the <a href="https://www.manning.com/books/platform-engineering-on-kubernetes">Platform Engineering on Kubernetes book</a>, I can assure you that whenever you talk about Developer Experience, you will, in some way or another, involve Platform Teams, so expect to see a lot of parallelism in this blog post. Whenever you need to deal with constant change and new tools, you can use all the things we learned when building platforms.</p>

<p>Before we start, a bit of a disclaimer: this blog post is not about building domain-specific or business agents. There are tons of frameworks popping up to create business agents, rather than code-generating agents. I’ve been following agentic frameworks for business purposes, and I am sure I’ll write about them in a separate article.</p>

<p>All the examples from the tools described in this blog post can be found in the following GitHub repository: <a href="https://github.com/salaboy/kubecon-eu-2026">https://github.com/salaboy/kubecon-eu-2026</a></p>

<h2 id="coding-before-agents">Coding, before agents</h2>

<p>The developers’ inner loop is a term that has been used for more than 20 years to describe the tasks that developers perform in a loop to get stuff done  (that is, creating new features or fixing bugs). These loops are well understood and learn by heart by developers (no matter which company they work for).</p>

<p>code → build → test → debug → repeat</p>

<p>These loops are time-consuming, as coding requires understanding what needs to be done and where the changes need to happen. Building is usually automated, but unless you work for a company with only one application, testing tends to be complex. If you are like me, when writing a feature or fixing a bug, you can plan for a couple of happy paths and some edge cases, but for the most complex scenarios, debugging is required.</p>

<p>Each of these activities takes time, and no matter which company you are working for, there are a couple of things that you, as a developer, need to master to be efficient while going through these loops several times a day:</p>

<ul>
  <li>Tech stack-specific libraries and tools</li>
  <li>Company-specific (homegrown) frameworks and tools</li>
  <li>Company culture, practices, processes (where the source code repositories are, what my team can change and what we can’t, and already discussed and defined architectural patterns)</li>
</ul>

<p>Moving companies means you, as a developer, will need to relearn most of this. While you can leverage your knowledge about a given framework or a programming language, getting accustomed to the company-specific procedures is just a painful process.</p>

<p>Over time, programming practices like <a href="https://martinfowler.com/bliki/TestDrivenDevelopment.html">TDD (Test-driven development)</a> and <a href="https://cucumber.io/docs/bdd/">BDD (behavior-driven development)</a> have been adopted by different organizations, but I wouldn’t dare to call them standards. The larger the organization is, the more complex the code they need to produce, so more complex practices are required. Spec-driven development was also a thing back in the 2000s, with <a href="https://en.wikipedia.org/wiki/Unified_Modeling_Language">UML (Unified Modeling Language)</a> as the de facto standard for defining software specs.</p>

<p>While these loops are focused on a single developer, it is important to add additional context on how they work.</p>

<p>requirements → <strong>inner-loops</strong> → code(PR) → outer-loops</p>

<p>These inner-loops main mission is to transform requirements (what needs to be done) into code. In my opinion, the value of a developer performing an inner loop lies in learning to translate ideas into code and mastering the technical skills required to do so. When I am performing manual inner loops, I am learning how complex systems work. It is totally valid to argue that after you have done several features and fixed bugs, there is not much more to learn, the learning process decreases over time, and tasks become repetitive and mundane.</p>

<p>After having produced code to implement a set of requirements, this code needs to be shared (usually in the shape of a pull request, if we are using Git), so the code can be reviewed, tested, and validated against other components, and finally merged into the main code base of our applications.</p>

<blockquote>
  <p>Note: other practices, like <a href="https://martinfowler.com/articles/on-pair-programming.html">pair programming</a> during inner loops, have proven very beneficial, boosting productivity and knowledge transfer, but, once again, these practices are in no way industry standards.</p>
</blockquote>

<p>The outer-loop is when collaboration happens. As part of reviewing a pull request, I need to use the same mental model the creator used to produce the code and to apply my assumptions and understanding of the application to validate their changes. Once again, while reviewing pull requests, I do learn quite a bit, not only about the code changes, but also about the architecture of the application and how it will run in a production environment.</p>

<h2 id="coding-with-agents">Coding, with agents</h2>

<p>Let’s now talk about coding agents.</p>

<p>I will use <a href="https://claude.ai/">Anthropic’s Claude</a> Code because I’ve been following and using it more often than other tools. I do think that Claude Code is representative of the evolution of these coding agent tools.</p>

<p>Claude Code is designed to replace all activities in the inner loop by transforming prompts into code, running tests, debugging, and troubleshooting until the requirements are implemented. Someone can argue that you can give Claude code the credentials to also automate deployments and control the full software delivery cycle, but I haven’t seen this implemented at scale yet. If we focus only on the inner loop, it is important to say that Claude Code is a generic agent. Using an LLM model, an agent harness has been built to help replace the inner-loop tasks that developers used to perform. These coding-agent harnesses include all the workflows, tool calling, memory, guardrails, and understanding how to use local tools to get stuff done.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/Screenshot-2026-03-20-at-09.03.25.png" alt="Screenshot 2026 03 20 at 09.03." /></p>

<p>Claude Code is a CLI (terminal application), an IDE plugin, or even a web app that allows developers to write their prompts to generate code. I’ve tried IDE plugins, but the CLI still works best.</p>

<p>To see the level of change in these tools, Claude Code consistently requires you to upgrade its version every day, which forces users to fetch a 200~MB (monolithic) binary containing improvements (we all assume). How many large companies can accept this speed of change in a secure and reliable way?</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/Screenshot-2026-03-20-at-09.04.36.png" alt="Screenshot 2026 03 20 at 09.04." /></p>

<p>I want to reemphasize that Claude Code is a generic tool that uses different LLMs depending on the task that we want to perform. On the CLI, you, as the user, can select which model to use, which prompts you to decide whether you are after a complex task or something that can be considered a “quick answer”.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/data-src-image-0b4d022d-dbb7-4d0f-873b-b3661bf5e7f2.png" alt="Data src image 0b4d022d dbb7 4d0f 873b b3661bf5e7f2" /></p>

<p>Because it is a generic tool, there are things that can be done, but defeat the purpose in my opinion. For example, if you create an empty directory, run the Claude CLI inside that and prompt:</p>

<blockquote>
  <p>“Create a Spring Boot application that implements some REST endpoints to manage Customers”</p>
</blockquote>

<p>It will do as told, but there are a few caveats. First, it doesn’t know which versions of the frameworks to use, so it will choose a popular version; it doesn’t understand your domain models, so the concept of a Customer will be defined as part of a generic code-generation process.</p>

<p>While this works, Claude is much more effective when context and guidance are provided. Unfortunately, context and guidance are what make Senior Developers, senior.</p>

<p>I know this sounds obvious, but after using Claude for quite a while, I’ve noticed a boost in my productivity when I scope the problem at hand. This is not only for Claude to be more efficient, but it also helps me think and learn while I let Claude generate code that it can test and debug until it does what I need. So here are a few practical examples:</p>

<ul>
  <li>Instead of asking Claude to create a project, I create it and define the core versions of the frameworks I want to use for the application. I need to make sure that Claude does not change these versions or add new libraries without my approval. This can be done in different ways, such as using skills and guideline files, such as <a href="http://claude.md">Claude.md</a> (<a href="http://agents.md">Agents.md</a>) . There is a parallelism with creating templates in Backstage for developers to use when starting a project.</li>
  <li>Designing domain objects and APIs for a given problem is such a fundamental part of the process that providing that context helps, since the code generation is bound to those APIs. Having a centralized database of datatypes and structures that must be used by code generation would make sense for a large enterprise scenario where data standardization is a must, think about healthcare scenarios.</li>
  <li>Architectural decisions are always implicit in a senior developer’s mind when deciding on how to implement features. For coding agents, providing architectural guidelines completely changes the output. Failing to provide sufficient architectural guidance (only needed when building distributed applications) allows the model to make changes beyond what is already defined in the architecture. Think about how to provide coding agents with ADRs (Architectural Decision Records) related to the task at hand. Tools to find these ADRs and categorize them so coding agents can fetch the relevant ones sounds like a sensitive thing to explore.</li>
  <li>Using TDD is very rewarding, as you focus on writing tests first, covering happy paths and some edge cases, and then the code follows. If the APIs are already defined, then multiple features can be implemented in quick iterations. For me, this is where I’ve refocused my learning process. I force myself to identify which tests I need to pass so I can spend time reviewing the generated code.</li>
  <li>How we define what a development environment is (the tools needed to run all the tasks in the inner loop) is critical for the coding agent to perform these tasks. Currently, coding agents request many permissions, and if you mistakenly grant them too much freedom, unexpected things can happen. Sandboxes and control over which tools the coding agent can use are critical to making the whole process more reliable.</li>
  <li>It goes without saying, but if you don’t use version control, tracking iterative progress becomes really hard. There is a fine line between understanding what the coding agent is doing and complete chaos. If you cannot review a whole changeset at once, it is hard to keep up with the changes, and more importantly, it is hard to revert to a previous state.</li>
</ul>

<p>Interestingly enough, coding agents are now introducing what is known as <strong>“planning mode”</strong>, where you can ask the coding-agent to question you about these topics, so it can have enough context to produce what is expected.</p>

<p>At this point, I tend to say these practices apply to all coding agents, but there is a problem…</p>

<h2 id="who-owns-the-experience">Who owns the experience?</h2>

<p>If I am upgrading a monolith 200~MB CLI (claude-code) every 8 hours as a user (or organization), I don’t have any control over the experience that I have with these coding agents.</p>

<p>As we experienced with cloud providers, organizations will want to craft their own custom experiences, moving away from generic workflows and guardrails that are not sufficient for their more specific scenarios. In the cloud space, this led to <a href="https://kubernetes.io/">Kubernetes</a> becoming the building block for companies to build their custom platforms. I see something similar in the coding agent space, tools that provide the basic building blocks so teams can craft and share their own experiences.</p>

<p>In that space, I see tools like Docker Agents showing some very interesting ideas enabling teams to own and share custom agents definitions.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/Screenshot-2026-03-20-at-09.20.37.png" alt="Screenshot 2026 03 20 at 09.20." /></p>

<p>Using a declarative approach (YAML) with <a href="https://docs.docker.com/ai/docker-agent/">Docker Agents</a> we can define a team of agents, where each agent can use a different model to perform tasks and specify which tools are available for the agent.</p>

<p>You can run this team of agents now using <code class="language-plaintext highlighter-rouge">docker agent run agents.yaml</code></p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/Screenshot-2026-03-20-at-09.32.19.png" alt="Screenshot 2026 03 20 at 09.32." /></p>

<p>This experience is available as experimental if you have <a href="https://www.docker.com/products/docker-desktop/">Docker Desktop installed</a> on your laptop. And I know, at first I was like, ok .. declarative team of agents, definitions, custom UI, cost control, and tools restrictions, cool. But what really caught my attention is that teams can share these definitions as OCI images.</p>

<p>By running <code class="language-plaintext highlighter-rouge">docker agent share push ./agents.yaml salaboy/my-agent</code> you can publish your agent definition to Docker Hub <a href="https://hub.docker.com/r/salaboy/my-agent">https://hub.docker.com/r/salaboy/my-agent</a> . Now, if you have a team of developers all using coding agents, you can share these teams of agents with all the teammates and control these agent versions with all the tools that we already use for our application containers.</p>

<p>Are Docker Agents the solution to all your problems? Definitely not, but I do expect more tools following up on these lines.</p>

<h2 id="who-controls-the-experience">Who controls the experience?</h2>

<p>Well, if we can share agents’ definitions as OCI images, we should be able to also define with a high degree of control what these coding agents can see and do, but that is quite a hard task if the coding agent binary is running on a host machine (workstation).</p>

<p>Just to show an example, if you have the following directory structure:</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/Screenshot-2026-03-22-at-08.19.11.png" alt="Screenshot 2026 03 22 at 08.19." /></p>

<p>And you move the <code class="language-plaintext highlighter-rouge">project/</code> directory and start a claude code session:</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/Screenshot-2026-03-22-at-08.19.58.png" alt="Screenshot 2026 03 22 at 08.19." /></p>

<p>You can prompt the agent to go, read and print the secrets in the <code class="language-plaintext highlighter-rouge">../secrets/</code> directory. The agent will ask for permission because it is outside of the scope of the <code class="language-plaintext highlighter-rouge">project/</code> directory where the session was started.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/Screenshot-2026-03-22-at-08.22.11.png" alt="Screenshot 2026 03 22 at 08.22." /></p>

<p>This is not a problem per se, but it becomes a problem when the developer in charge has answered hundreds of similar questions. Wouldn’t it be nice to let the agent run in YOLO mode?</p>

<p>That is the problem <a href="https://docs.docker.com/ai/sandboxes/">Docker Sandboxes</a> are to solve. With Sandboxes, you can control what a coding agent can see and do by running each session inside a microVM.</p>

<p>With a single command, I can start a Claude code session inside a sandbox and I can “mount” whatever directories I want the agent to have access to:</p>

<p><code class="language-plaintext highlighter-rouge">docker sandbox run --name my-demo-sandbox claude ./project</code></p>

<p>Mount is not the right word here, because sandboxes do file syncing between the microVM and the host machine. This also allows setting up directories of files in read-only mode, which makes the agent able to access the files but never (accidentally) change them.</p>

<p>Now, your coding agent (in this case, Claude code) can only see the contents of the <code class="language-plaintext highlighter-rouge">projects</code> directory and try to perform the same prompt as before to read the secrets, it will fail as the microVM doesn’t have access to those files:</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/Screenshot-2026-03-22-at-09.06.11.png" alt="Screenshot 2026 03 22 at 09.06." /></p>

<p>That is a good first step; now we can limit the file system, but what about network calls? Sandboxes also include a network proxy that enables us to define network policies. For example, we can deny all connections to all hosts and only allow requests for <code class="language-plaintext highlighter-rouge">api.anthropic.com</code> and <code class="language-plaintext highlighter-rouge">api.github.com</code> by running the following command:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker sandbox network proxy my-demo-sandbox \
  --policy deny \
  --allow-host api.github.com \
  --allow-host api.anthropic.com
</code></pre></div></div>

<p>If the coding agent tries to fetch content from different domains, it will be blocked by the network proxy:</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/Screenshot-2026-03-22-at-09.12.00.png" alt="Screenshot 2026 03 22 at 09.12." /></p>

<p>We can control the file system access and the network access. Docker Sandboxes allow us to create <a href="https://docs.docker.com/ai/sandboxes/templates/">sandboxes from templates,</a>starts so we can control which tools the coding agent has access to inside the microVM. Once again, will sandboxes solve all your coding agent challenges?</p>

<p>Definitely not. I couldn’t find a declarative way to define these sandboxes, so we can encapsulate all the filesystem, network, and tooling rules in a single file that can also be shared among teammates as OCI images. Another big challenge for me, is that both Docker Agents and Docker Sandboxes are experimental features on the Docker Desktop app, so if you are using Podman or Rancher (with containerd) these features are not going to be available making me question Docker’s decision of bundling these tools exclusively Another big challenge for me is that both Docker Agents and Docker Sandboxes are experimental features on the Docker Desktop app, so if you are using Podman or Rancher (with containerd), these features are not going to be available, making me question Docker’s decision of bundling these tools exclusively with Docker Desktop.</p>

<h2 id="some-big-mental-model-upgrades">Some big mental model upgrades</h2>

<p>For a developer who has been coding for 20 years, having a coding agent handle all the coding was personally hard to accept. I think, in some ways, it’s a matter of personal preference; some people like using IDEs for large projects, refactoring, and code assistance, while others don’t.</p>

<p>I jumped from having maybe two or three sessions in my IDE where I could perform one task at a time to having five terminals with Claude code-crunching. I am still using my IDE to review code and run tests.</p>

<p>Is this truly progress? I think it is, but it also comes with some downsides, for example:</p>

<ul>
  <li>The more senior you are (in both the developer and business domains), the easier it is to refine prompts. If you have done a similar task in the past, it is easier to define what it is and also what is expected.</li>
  <li>It is really easy to prompt for refactorings, even when it is faster to do them by hand. I do notice that I’ve become lazier because I can ask the agent to do it, but sometimes it is counterproductive</li>
  <li>Review time has skyrocketed; I’ve spent more time checking that what was generated actually works, and because I have five agents running in parallel, this is what consumes most of my time.</li>
  <li>Without guidance, some projects require more context than what is available. I’ve seen Claude choke on open-source projects that require scanning multiple dependencies to implement changes. Once again, if you have no idea how to scope the scanning, you end up wasting tons of tokens without achieving any usable results. Scenarios where Claude clones dependencies and leaves patched source code all over the place have become part of my day-to-day cleanup process.</li>
  <li>It is too easy to go into the crazy-loop-of-hell: prompt for a new feature -&gt; tests are broken -&gt; ask for tests to be fixed -&gt; code is broken -&gt; prompt to fix code -&gt; other tests are broken -&gt; code agent starts changing code in libraries that you depend on -&gt; hell.</li>
</ul>

<p>I am still typing to create prompts, I haven’t made the final leap of faith where I speak to the terminal :)</p>

<p>As a developer in 2026, besides all the specifics of the project that I am working on, I need to learn:</p>

<ol>
  <li>MCP: model context protocol to let my coding agents communicate with other tools</li>
  <li>Skills: to provide context and specific knowledge to the coding agent about the tasks it will be performing</li>
  <li><a href="http://agents.md">Agents.MD</a>: to provide context, ADRs, and general guidelines that will be included with every prompt we send to the agent</li>
</ol>

<p>Still, I feel that what I’ve delegated to coding agents is the coder in me, but I am still acting as the engineer in charge when I use these tools. I still need to review the tool; code review can also be assisted by an agent, but I still need to judge what is good or bad for a given context, and this is not an easy task when building complex distributed applications.</p>

<h2 id="from-a-one-developer-mindset-to-enterprises">From a one developer mindset to enterprises</h2>

<p>Projects in large organizations require much more than code generation, and I think we are, in some ways, missing the point. Deciding what to do has always been the most time-consuming task for companies. What made a company better than its competitors was never about how much code they could produce; it was always about what that code would automate or enable. Now, code is cheap, and we can produce it fast (if we have enough senior engineers to guide coding agents), so what is missing? Where are companies going to spend most of their valuable time?</p>

<ul>
  <li>I think about collaboration among multiple developers in the outer loop. Right now, the focus is on assisted reviews, but we are still using Git as the central hub for collaboration. Is there a more efficient way to do this? Can this be abstracted away? I do see teams fighting over large pull requests where every team member is trying to defend what their coding agent generated.</li>
  <li>For teams using coding agents, I think about a prompt-to-code log: we don’t have a standardized way to track which prompt (context) generated each line of code. Imagine git blame but for coding agents. This can be used for learning and for refining context.</li>
  <li>It feels to me that it will be quite important for large teams to validate code complexity across entire applications, as well as for new features and changes. Right now, I see teams prompting for a feature or a bug fix without evaluating multiple parallel options. Again, we are overloaded with review tasks now, so adding parallel options evaluation at this point sounds crazy. Can this be automated?</li>
  <li>How requirements are mapped to code using the <a href="https://cucumber.io/docs/gherkin/reference">Gherkin format (GIVEN/WHEN/THEN)</a> sounds like a good idea, but is it? Having pointers that trace code back to the requirement has always been a great idea, but it is becoming increasingly critical as the amount of code we generate grows day by day.</li>
  <li>Monitoring coding agents and observability on the developer side (in contrast to infrastructure) is becoming key to understanding how software is produced within an organization. Without having metrics and analytics about how this process is working, it will always be hard to measure what is working and what is not.</li>
</ul>

<p>Working with platform engineers for a while now, we have already seen some common patterns in the cloud-native ecosystem. Let’s start with the most obvious one, vendor lock-in.</p>

<p>Claude code is a provider-specific tool; we have seen how companies have specific requirements, so a generic coding agent might not work for every context. The experience is 100% defined by Anthropic and constantly evolving; some companies will not accept 200MB black-box upgrades every day. Companies will want to define their own developer experience, tailored to their company’s needs and restricted to use the company guidelines. Providing the tooling with building blocks to build these coding agents makes total sense. Tools like cagent from Docker provide some of the building blocks to create your own experiences. Is <code class="language-plaintext highlighter-rouge">cagent</code> the ultimate solution? I don’t think so.</p>

<p>The collaborative aspect of coding agents remains unresolved. This raises several questions:</p>

<ul>
  <li>Do we change the way we used to collaborate?</li>
  <li>Do we still need to use tools like git for version control? Can an abstraction layer be built to simplify collaborations?</li>
  <li>Do we use agents for pair programming?</li>
  <li>Multi-channel agents make a lot of sense, why not collaborate via Slack with your agents? -&gt; <a href="https://github.com/sttts/slagent">https://github.com/sttts/slagent</a> from Stefan</li>
</ul>

<p>Next, how do we map intent to generated code? I want to be able to track down which prompt generated which code and the thought process (context, tools, and reasoning) behind the generation. Using the <a href="https://cucumber.io/docs/gherkin/reference">Gherkin (Given / When / Then)</a> Definitelyformat might be part of the answer, as it helps us to connect requirements to code, as we can see here:</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/Screenshot-2026-03-22-at-08.13.30.png" alt="Screenshot 2026 03 22 at 08.13." /></p>

<p>To this:</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/data-src-image-896a87e7-0b75-4047-aebe-5c2edc688096.png" alt="Data src image 896a87e7 0b75 4047 aebe 5c2edc688096" /></p>

<p>Related to this is the problem of keeping track of prompts, because with tools like Cucumber and Gerkin, we are tracking intent to code, not how the context is used to create that code.</p>

<p>This takes me to the final point. In 2026, observability in the inner loop becomes a key feature for teams using coding agents. Which prompts were executed, by which agent (sub-agent), how much did they cost, and did the code generated make it into the main project branch? Tools like <a href="https://github.com/rhuss/cc-session">cc-session by Roland</a> or <a href="https://github.com/yessGlory17/argus">Argus for VSCode</a> make the point that we need to understand how these coding agents are used to refine and improve our practices.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2026/03/Screenshot-2026-03-22-at-09.31.08.png" alt="Screenshot 2026 03 22 at 09.31." /></p>

<h2 id="sum-up">Sum up</h2>

<p>The coding agents space is very vibrant, but it will need to undergo a similar transformation to the one we saw in the cloud-native ecosystem. Standards, common practices, and tooling that provide the building blocks for large organizations to build their own experiences.</p>

<p>Similarly to platform engineering, if we don’t have the right metrics and analytics to measure how these coding agents are working, it is really hard to tell whether things are working or just wishful thinking.</p>

<p>It does feel that we are still trying to figure out what kind of experiences we want for developers controlling or interacting with coding agents, and that is all good and interesting, but at some point, as we always do, we will need to take some of the core concepts and tools to be able to define what are the must building blocks that large organizations needs to enable teams of developers to control these agents.</p>

<p>I am eager to see where we reuse the tools that we already have to boost these experiences, such as OCI registries, declarative agent definitions, existing proxies and policies to secure our coding agents in the same way that we secure our workloads, and finally, standards like <a href="https://opentelemetry.io/">OpenTelemetry</a> to observe and understand how these agents are operating at scale.</p>

<p>It is clear that even mature tools like Kubernetes need to adapt to this new agent era, but because coding agents are closer to developers and their workstations, the question remains: how do we bring all we have learnt from the cloud-native space to our developers’ laptops?</p>

<p>Ok, now time to go to <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/schedule/">KubeCon EU in Amsterdam</a>. I hope to see you all there!</p>]]></content><author><name>Mauricio Salatino</name></author><category term="2026" /><category term="cloud native" /><category term="kubernetes" /><category term="developers" /><category term="devex" /><category term="coding agents" /><category term="claude" /><category term="tooling" /><category term="docker" /><category term="sandbox" /><category term="agents" /><category term="cc-session" /><category term="slaude" /><category term="slagent" /><summary type="html"><![CDATA[TLTR: coding agents are replacing the tasks developers perform in their inner loops. What tools and practices can help you tame these agents to be more…]]></summary></entry><entry><title type="html">The evolution of Platforms (GenAI edition)</title><link href="https://www.salaboy.com/2025/11/18/the-evolution-of-platforms-genai-edition/" rel="alternate" type="text/html" title="The evolution of Platforms (GenAI edition)" /><published>2025-11-18T08:53:35+00:00</published><updated>2025-11-18T08:53:35+00:00</updated><id>https://www.salaboy.com/2025/11/18/the-evolution-of-platforms-genai-edition</id><content type="html" xml:base="https://www.salaboy.com/2025/11/18/the-evolution-of-platforms-genai-edition/"><![CDATA[<p>I’ve been thinking about a second edition of my book, <a href="https://www.manning.com/books/platform-engineering-on-kubernetes">Platform Engineering on Kubernetes</a>, to cover all the progress I’ve seen over the last two years since its publication. Looking at industry trends and current applications of GenAI suggests that it may be too early to declare winners, but certain patterns are emerging. When patterns emerge, we tend to see quick-and-dirty solutions to demonstrate the path forward, and then a significant amount of engineering is required to turn those patterns into something that large organizations can adopt.</p>

<p>In this blog post, I will outline some ideas about the patterns I observe across various communities and vendors. Most of these tools, patterns, and approaches to using GenAI on platforms have been covered by <a href="https://www.youtube.com/@DevOpsToolkit">my friend Viktor Farcic on his YouTube channel</a>, which I strongly recommend you follow.</p>

<p>Here is my personal take on the matter.</p>

<p>A bit of context before we jump straight into the questions that we want to answer. We used and extended Kubernetes to build our platforms. We focused on APIs to simplify the consumption of platform services. We wrote a lot of glue to make this happen. Now that we have GenAI capabilities, teams consuming platform services want access to GenAI, platform engineers want to use GenAI to enhance their platforms, and there is a budget to adopt GenAI across the board.</p>

<p>The question is: how can we use GenAI to support platform initiatives while also providing GenAI capabilities to other teams that want to integrate these features into their applications?</p>

<h3 id="using-genai-for-our-platforms">Using GenAI for our platforms</h3>

<p>With the rise of MCP, Agents, and prompt-based interactions, it makes sense to replace or extend our core platform APIs, which were built on top of Kubernetes APIs (REST endpoints), with an LLM-based prompt approach. This is not only for the sake of improving the experience, but also to gradually add capabilities emerging in the Gen AI space to our platforms.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/11/data-src-image-abdc03b4-575a-43d5-9dc0-ce9882c21e0c.png" alt="Data src image abdc03b4 575a 43d5 9dc0 ce9882c21e0c" /></p>

<p>Some basic prompts that can be used to demonstrate the value of this approach can be:</p>

<ul>
  <li>“I want to provision a new development environment for Team A, which uses NodeJS and MongoDB to build their applications.”</li>
  <li>“I need a new S3 bucket for application X which will store a huge number of PNG pictures, provide connection details after provision and connect the application to the bucket.”</li>
  <li>“Which services can I use to obtain the latest updates on Customer X from our organization’s services?”</li>
  <li>“Look at all the Issues on these 100 repositories, and for every issue created, try to provide a solution by sending a pull request with fixes.”</li>
  <li>“Monitor all my continuous integration pipelines. If something fails, try to troubleshoot the error and send a fix, and run again.”</li>
</ul>

<p>First of all, this is not revolutionary; we have done this with our documents (Grammarly), with our code editors (Copilot, Cursor, etc), and in almost every industry, you will find services that are now frontfaced by what looks like an LLM.</p>

<p>Let’s assume we want to go this way. What would be the challenges that you will face on day one?</p>

<p>First and foremost, you need to decide not only where to start but also if you are going to use Public or  Private LLMs. I will not go into the details, but at a high level, you will need to evaluate which option fits better for your organization, tomorrow and in the long run.</p>

<ul>
  <li>Public LLMs
    <ul>
      <li>Is the LLM Service just calling public LLMs like OpenAI, Anthropic, and Gemini? Well, this is going to cost you a lot of money, directly proportional to the use of your platform services. Can you afford this cost?</li>
      <li>The LLM will need to be enhanced with your organization’s information to be valuable, in this case by providing access to your platform APIs and other services. Are you working in a startup where there are few regulations on where data goes or which services you can expose to LLMs?</li>
    </ul>
  </li>
  <li>Private LLMs
    <ul>
      <li>The real value of LLMs for organizations is having a custom set of models to perform very specific tasks tailored to the organization’s context. While a fair amount of use cases can be covered with generic models, if you want answers from your prompts that take only your organization’s knowledge into account, you need to train your own LLMs. This is not only costly, but laborious and requires a lot of expertise. Guess what, hiring people to do this is hard, and these experts are in high demand.</li>
      <li>Whether you train or not, you still need to run your models. If your organization is highly regulated and you cannot interact with public services, you have no other option but to host your LLMs yourself.</li>
    </ul>
  </li>
</ul>

<p>No matter whether you decide to go with public LLM services (get your wallet ready) or if you want to host your LLMs on-prem, there are a few projects that you should check out:</p>

<ul>
  <li><a href="https://docs.vllm.ai/en/latest/">vLLM</a>: full-on solution to manage, host, and route traffic to your LLMs that run on Kubernetes. This project helps you schedule LLM workloads that require GPUs to run at a usable speed. People from Google, Red Hat, and other companies are collaborating on this project.</li>
  <li>AI Proxies
    <ul>
      <li><a href="https://aigateway.envoyproxy.io/">Envoy AI Gateway</a> (token-aware router): No matter which LLM provider you are using, having a proxy layer between your consumers and the LLM service allows you to set up quotas, filters, and policies for cost control or transformations to reduce the burden on consumers to know which LLM service to consume.</li>
      <li><a href="https://docs.dapr.io/developing-applications/building-blocks/conversation/conversation-overview/">Dapr Conversation API</a>: a higher-level abstraction to enable applications/agents to talk to LLMs without the need for a specific provider SDK to be included inside your app/agents.</li>
    </ul>
  </li>
</ul>

<h3 id="system-integrations-we-know-how-to-do-this">System Integrations, we know how to do this</h3>

<p>Going back to the challenges, I wanted to touch on the system integration issues teams will face when implementing such an extension to the existing platform APIs.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/11/data-src-image-964dc54b-27da-40a6-80b3-0412a7776c73.png" alt="Data src image 964dc54b 27da 40a6 80b3 0412a7776c73" /></p>

<p>To make the LLM do more than just reply to text, we need to connect it to services so it can actually do something useful, such as automating platform tasks. To do this, the industry has settled on using the <a href="https://modelcontextprotocol.io/docs/getting-started/intro">MCP protocol</a> to connect LLMs to other services, so that, as part of user interactions with the LLM, the LLM can trigger actions on external systems.</p>

<p>Besides the network complexities required to secure and make these interactions resilient, MCP serves as the glue between a text-based world and a very service-oriented world, where APIs are the default way for developers and machines to interact with services. Here, I want to reflect on how APIs have served as contracts to define system-to-system and human-to-system interactions, with well-defined inputs and outputs and non-functional requirements. This makes the MCP protocol the perfect place to bridge the purely unstructured world of prompts to the heavily structured world of APIs.</p>

<p>In the world of MCPs, APIs are considered tools, and tools have their own contracts, so a translation layer is still required between the APIs in our organizations and the tools in the context of MCPs. This forced translation has led to an industry-wide push to advertise that every project, product, and service includes a built-in MCP server.</p>

<p>For example:</p>

<ul>
  <li><a href="https://developers.redhat.com/articles/2025/09/25/kubernetes-mcp-server-ai-powered-cluster-management">Kubernetes MCP Server</a></li>
  <li><a href="https://www.linkedin.com/posts/hwang8_the-argo-cd-mcp-server-akuity-activity-7326007807424581633-Vuux/">Argo CD MCP Server</a></li>
  <li><a href="https://mcpmarket.com/server/marketplace-3">Crossplane MCP Server</a></li>
  <li><a href="https://github.com/modelcontextprotocol/servers">Awesome MCP severs list</a> proves the point</li>
</ul>

<p>What this means for an organization trying to enable its internal systems to be accessed by LLMs is some work to figure out how to call these services using the MCP protocol. Tools in this space are popping up to help companies create MCP tools from their existing service APIs.</p>

<p>An important change is that, as shown in the previous diagram, the services that were previously handled and hidden behind the Platform API can now be accessed directly by the LLM. In some way, this means that as platform builders, we will be in charge of defining how low-level we want our LLMs to be able to perform actions. Do we want the LLM to have access only to platform APIs, or to all the tools used to build the platform?</p>

<p>So at the end of the day, there is no centralized MCP server with a huge list of tools for an LLM to access all the services, but it follows a more distributed approach:</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/11/data-src-image-93e87c60-a922-4770-adb5-ca13ef1433f4.png" alt="Data src image 93e87c60 a922 4770 adb5 ca13ef1433f4" /></p>

<p>If we follow this approach, we can control, using RBAC, which tools each MCP server exposes and provides to the LLM.</p>

<p>If we focus on our platforms, it is difficult not to look into <a href="https://kagent.dev/">Kagent</a>, a project created by <a href="https://solo.io">Solo.io</a> and donated to CNCF, which has attracted attention in this space, as it provides out-of-the-box agents that work with the Kubernetes APIs and other, more specific Kubernetes-based projects, such as Istio, Cilium, Helm, and Argo Rollouts.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/11/data-src-image-37e18eea-71b7-4713-8e51-d502ac2e572d.png" alt="Data src image 37e18eea 71b7 4713 8e51 d502ac2e572d" /></p>

<p>Check kmcp and the <a href="https://thenewstack.io/solo-io-open-sources-agentregistry-with-support-for-agent-skills/">agents registry</a> announced last week at KubeCon</p>

<p>NOTE: You will find several reports about LLMs hallucinating and using way too many tokens if you give them a large number of tools <a href="https://www.reddit.com/r/mcp/comments/1m9227n/too_many_tools_break_your_llm/">[1]</a>, <a href="https://community.openai.com/t/ridiculous-number-of-redundant-tool-calls/1181410">[2]</a>, <a href="https://achan2013.medium.com/how-many-tools-functions-can-an-ai-agent-has-21e0a82b7847  ">[3]</a>. This not only raises a scalability issue but also questions the need to craft custom solutions to give LLMs what they need at the right time.</p>

<p>The problem with this approach is that the LLMs have very limited tools for performing actions. This means that tools related to interacting with the Kubernetes APIs will require the LLM to know about Deployments, Services, StatefulSets; hence, achieving higher-level automations such as <em>“connect my application to a database”</em> could be extremely hard to achieve, as the Kubernetes APIs do not expose higher-level concepts such as applications and databases.</p>

<p>This takes us to the next iteration of this pattern. Let’s enhance our LLMs with concepts and their relationships.</p>

<h3 id="softening-the-contracts">Softening the contracts</h3>

<p>What does a “Customer” mean? What does “Application” mean? What does “deploy to production” mean?</p>

<p>If you ask a public LLM about these concepts, you will get very generic answers, mostly coming from the training data (the internet), which will not make much sense for your organization.</p>

<p>One way to solve this problem is to use vector embeddings to represent concepts and their relationships (links), so that LLMs, when asked about Customers or Applications, understand exactly what this means for your organization.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/11/data-src-image-12154074-3baa-477a-a5e4-4dce03988373.png" alt="Data src image 12154074 3baa 477a a5e4 4dce03988373" /></p>

<p>With vector embeddings, we can ingest concepts and create relationships between these concepts, so when we create a fuzzy prompt to LLMs, we can use RAG(Retrieval augmented generation) semantic searches to supplement the LLM’s knowledge base.</p>

<p>By doing this, when prompting, we can be less specific about implementation details and focus more on the prompt’s desired outcome.</p>

<p>To give a more specific example, let’s imagine a team has been developing a new application that they want to deploy into a given environment. This new application depends on a relational database that hasn’t been provisioned yet.</p>

<p>A prompt to the platform can look like this:</p>

<p><strong><em>“Deploy the application hosted in the following GitHub repository Y to environment X, but before make sure that a new database is provisioned for the application to use.”</em></strong></p>

<p>To determine what needs to be done and where, the LLM can run a RAG search against the vector database to identify which tools from the Platform and Kubernetes APIs can be used to make that happen.</p>

<p>By looking at the platform APIs, it can retrieve the given environment, where a new application can be deployed. Also, by looking at the platform APIs, the LLM can understand how to build the application source code and create deployment manifests. Depending on whether the platform provides APIs to provision databases, a RAG search can identify which databases are available for applications to use, and applications can then directly use the available cloud provider APIs to create a new instance. Finally, it can use the Kubernetes APIs to bind the database to the application after deployment.</p>

<p>We have configured our setup to handle higher-level concepts and perform lower-level operations against the Kubernetes APIs when needed.</p>

<p>Using RAG to filter which tools LLMs can use to perform a given task is a pattern that is maturing quite quickly; hence, new tools to help teams implement these interactions are popping up in the market. The rise of tools like <a href="https://github.com/pgvector/pgvector">PG Vector</a> (PostgreSQL vector database) are in high demand.</p>

<p>So the question is now: how does it know which database to use for a given application? How does it know which region this database needs to be provisioned in? How does it know how long this database should store data?</p>

<h3 id="companies-guidelines-best-practices-and-policies">Companies’ guidelines, best practices, and policies</h3>

<p>Providing our LLMs with access to Cloud Provider APIs, Kubernetes APIs, and third-party service APIs sounds really cool, but brace yourself: you are opening the door to pure chaos. As with people, when having access to all these resources, it is a good practice to have strict rules and policies on where resources are created, quotas, and which credit card is used to pay for them.</p>

<p>LLMs are no different; hence, providing access to our platform APIs should encode some of these policies, but we want to enable low-level interactions when needed. This means that we will need to teach our LLMs to also learn about our organization’s guidelines, policies, and best practices.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/11/data-src-image-9325cdda-7312-4f58-a5a2-0bd5f80087b8.png" alt="Data src image 9325cdda 7312 4f58 a5a2 0bd5f80087b8" /></p>

<p>While we can provide LLMs with knowledge about company guidelines, best practices, and policies, it is recommended to have one last line of defence. Using policy engines such as Kyverno as part of our platform will let the LLM decide what to do, but it will not allow it to proceed with operations that do not adhere to company policies.</p>

<p>On this note, you can find the folks from Nirmata mixing <a href="https://www.youtube.com/watch?v=C0y8bDvL47M&amp;t=661s">kagent and Kyverno in this video</a>. Where they show how the Kyverno MCP server can be used by Agents to create, check, validate and enforce policies.</p>

<p>If we stop for a second to recap what we have achieved so far, we can see that we have enabled LLMs with a bunch of new tools to perform actions on our behalf, we have softened our contracts by using vector embeddings and RAG semantic queries for the LLMs to be able to bridge the gap between fuzzy prompts and structured APIs, and we have provided the LLMs with organization knowledge to constrain its suggestions and actions to follow what the organizaiton is already doing in other projects. Still, the more tools and concepts we provide to the LLMs, the more coordination these interactions will require to be useful. Going step-by-step with an agent-assisting platform engineering team to achieve their goals requires more advanced state and resource management; we need workflows.</p>

<h3 id="workflows--context-management">Workflows &amp; Context Management</h3>

<p>LLMs are stateless, have limited context sizes, and can get confused if we provide too many tools. This means that if we have a long chain of tasks to perform, we need more complex coordination and a state machine to track all the interactions.</p>

<p>It is also important to consider interaction patterns with platform builders and teams that consume platform services. Going back to the example that we used before:</p>

<p><strong><em>“Deploy the application hosted in the following GitHub repository Y to environment X, but before make sure that a new database is provisioned for the application to use.”</em></strong></p>

<p>Instead of expecting all this to be resolved in the background, our platform front-facing LLM-enabled service should guide the user to accomplish the goal by performing a well-defined set of interactions that provide advice on every option along the way.</p>

<p>What do we need to implement this to make sure that:</p>

<ol>
  <li>We have a way to define concrete and consistent steps that our agents will execute</li>
  <li>Make sure we have a way to control what is set into the context for the LLM to have the right information to perform the task at a given point in time</li>
  <li>Keep track of where the agent is at all times, so we can report back to the user, but also make sure that if something goes wrong, we can resume from where the agent left off</li>
</ol>

<p>Guess what, Workflow and durable execution runtimes were created to solve these challenges. Imagine the following workflow that the agent can call to perform a set of actions in a sequence:</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/11/data-src-image-67709209-3dc3-4a7d-956d-cbb9a16dc331.png" alt="Data src image 67709209 3dc3 4a7d 956d cbb9a16dc331" /></p>

<p>At each step, we can retrieve information from the vector database to refine the task at hand, check with other systems to gather more information, and perform and validate actions.</p>

<p>With workflows, we can break complex tasks into more manageable steps that can also be used to reduce “context pollution” and tool overload.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/11/data-src-image-ed55d847-529d-4ef3-aa19-c15cc8c6d000.png" alt="Data src image ed55d847 529d 4ef3 aa19 c15cc8c6d000" /></p>

<p>Using workflows, we can mix and match, letting our agents decide when to perform a chain of well-known tasks or execute separate actions based on user prompts.</p>

<p>The advantages of using durable execution frameworks to run our workflows include visibility into the agent state (since LLMs are stateless, the agent is responsible for maintaining state and progress for its operations) and durability. Durability is important because each interaction between our agents and LLMs costs money.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/11/data-src-image-6d6ae33a-feb3-4091-8de5-5e9a86d6d6fa.png" alt="Data src image 6d6ae33a feb3 4091 8de5 5e9a86d6d6fa" /></p>

<p>In a world where everything becomes a Tool exposed using the MCP protocol, workflows are here to provide a middle ground where the LLM can choose which workflow to execute in a non-deterministic fashion, but when a workflow is started, a set of steps will be performed in sequence to completion.</p>

<p>Workflows-as-tools is already a term that is being used to bridge this gap between the fully non-deterministic agents and agents that can choose to execute a set of tasks that require special and deterministic coordination across multiple steps.</p>

<p>If you are considering building these kinds of agents or applications, you should check out <a href="https://docs.dapr.io/developing-applications/building-blocks/workflow/workflow-overview/">Dapr Workflows</a>, which provides a Workflow-as-Code alternative for building durable agents.</p>

<p>If you are working with Python already, <a href="https://v1-16.docs.dapr.io/developing-applications/dapr-agents/">Dapr Agents</a> provides a more opinionated programming model to create durable agents that implement all these mechanisms out of the box.</p>

<p>Other tools and specs that I am keeping an eye on are:</p>

<ul>
  <li><a href="https://arxiv.org/pdf/2510.08790 ">Compass for context tracking and clean up</a></li>
  <li><a href="https://alexzhang13.github.io/blog/2025/rlm/">Recursive RLMs instead of LLMs</a></li>
  <li><a href="https://vllm-semantic-router.com/blog/semantic-tool-selection/">vLLM semantic router for tool selection</a>, providing</li>
</ul>

<h3 id="sum-up">Sum up</h3>

<p>Bringing GenAI closer to our Kubernetes-based platforms requires platform engineers to wire a new array of services and tools. When I look at the patterns described earlier, I don’t see anything beyond new constraints and integration challenges.</p>

<p>As part of bringing GenAI services closer, our platform surface will expand we will need to manage more infrastructure, tokens, and services. Expertise in RAG frameworks and vector databases will become essential for platform teams in the near future.</p>

<p>As we have covered what it means to bring GenAI to our platforms, enabling development teams with GenAI services or features becomes a natural extension to the same problem. In the same way that our Kubernetes-based platforms became an enabler for teams that needed a Kubernetes-based environment for their applications, platform teams that manage to integrate GenAI services will be able to understand the nitty-gritty details of how these services are consumed, providing them as a service to others.</p>]]></content><author><name>Mauricio Salatino</name></author><summary type="html"><![CDATA[With the rise of MCP, Agents, and prompt-based interactions, let's extend our platform APIs, which were built on top of Kubernetes APIs, with an LLM-based…]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/11/Screenshot-2025-11-17-at-13.57.12.png" /><media:content medium="image" url="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/11/Screenshot-2025-11-17-at-13.57.12.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025</title><link href="https://www.salaboy.com/2025/10/13/learnings-from-cloud-native-denmark-and-devoxx-belgium-2025/" rel="alternate" type="text/html" title="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><published>2025-10-13T09:18:35+01:00</published><updated>2025-10-13T09:18:35+01:00</updated><id>https://www.salaboy.com/2025/10/13/learnings-from-cloud-native-denmark-and-devoxx-belgium-2025</id><content type="html" xml:base="https://www.salaboy.com/2025/10/13/learnings-from-cloud-native-denmark-and-devoxx-belgium-2025/"><![CDATA[<blockquote>
  <p><strong>&lt;Begin of AI Rant&gt;</strong></p>
</blockquote>

<blockquote>
  <p><em>The fact that a presentation includes AI in its title or proposal doesn’t necessarily make it interesting.</em></p>
</blockquote>

<blockquote>
  <p>I had the opportunity to talk to many folks who raised a very important issue. While AI is in hype, <strong><em>let’s make sure that we don’t forget about how we actually deliver software</em></strong>, the core libraries and tools that will not change even if AI takes full control of the development process. Logging remains essential for humans, testing is still necessary, APIs are important, and specs benefit both machines and humans in conceptualizing and architecting large-scale systems.</p>
</blockquote>

<blockquote>
  <p>A constant across communities that I found fascinating is the lack of clear direction in tooling within the GenAI space. While numerous frameworks are emerging to build agentic systems, the clarity of the use cases these systems will implement remains a significant open question. As is typical in the industry, we are developing solutions for problems that have yet to arise.</p>
</blockquote>

<blockquote>
  <p><strong>&lt;End of AI Rant&gt;</strong></p>
</blockquote>

<p>With all that being said, I had a fantastic time at both Cloud Native Denmark, happening in the beautiful city of Aarhus, and Devoxx Belgium, happening in Antwerp.</p>

<p>The software industry is vibrant with the GenAI bubble, but the good thing is that all this movement promotes new ideas, new patterns, and new challenges that we will need to tackle in order to keep our software development and delivery practices updated and efficient.</p>

<h2 id="cloud-native-denmark">Cloud Native Denmark</h2>

<p>Organized by <a href="https://cloudnativenordics.com/">Cloud Native Nordics</a> (with around ten thousand members) and previously part of the KCD events, this was the third edition of the conference. With around 450 attendees, the conference is very targeted to platform engineers in the cloud native space.</p>

<p>If you are in the Nordic countries and haven’t attended these events, despite working daily with Kubernetes, I strongly recommend checking out the next edition in Copenhagen, Denmark, in 2026.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/Cloud-Native-Denmark-2025-631.jpg" alt="Cloud Native Denmark 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/Cloud-Native-Denmark-2025-627.jpg" alt="Cloud Native Denmark 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/Cloud-Native-Denmark-2025-613.jpg" alt="Cloud Native Denmark 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/Cloud-Native-Denmark-2025-611.jpg" alt="Cloud Native Denmark 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/Cloud-Native-Denmark-2025-610.jpg" alt="Cloud Native Denmark 2025" /></p>

<p>As I shared in my keynote session, these events are where you meet people from whom you can learn. In our field, where we learn almost every day, attending these events not only fosters personal growth but also creates opportunities for cross-organizational collaborations.</p>

<div class="video-embed" style="position:relative;padding-bottom:75%;height:0;overflow:hidden;max-width:100%;margin:1.5rem 0;"><iframe allowfullscreen="true" class="speakerdeck-iframe" loading="lazy" src="https://speakerdeck.com/player/a47bd26d3d95466783ab7a2c6743d780" style="position:absolute;top:0;left:0;width:100%;height:100%;border:0;" title="[Keynote] What do you need to know about DevEx in 2025"></iframe></div>

<p>I had great chats with <a href="https://www.linkedin.com/in/danquah/">Mads from Lego</a> about Open Source, and my good friends <a href="https://www.linkedin.com/in/giardiniandrea/">Andrea</a> from <a href="https://www.overstory.com/">Overstory</a>, Kasper from <a href="https://www.linkedin.com/in/kaspernissen/">Dash0</a> (and organizer), and <a href="https://www.linkedin.com/in/sabdelfettah/">Abdel</a> from Google.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6228-2.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6227-2.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6215.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6224-2.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6223.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6220.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6219.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6218.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /></p>

<p>It was interesting to see, as part of Andrea’s keynote, how important <a href="https://dagster.io/platform-overview/data-orchestration">Dagster (a data orchestration platform, mostly Python)</a> was to the journey to improve operations. Seeing real-life use cases where technology is applied to solve concrete problems is always refreshing, as it demonstrates the before and after of adopting new tools to improve company operations.</p>

<p>From Abdel, I took a lot of notes about <a href="https://llm-d.ai/">llm-d (llm-d: a Kubernetes-native high-performance distributed LLM inference framework)</a>, a tool I’ve been looking for a while but haven’t had time to dig deeper into. Learning about the architecture and reasons behind this tool’s existence was also refreshing. In a saturated market of tools, unification, reuse of common specs, and existing components to solve new problems only start to emerge where the problems we are trying to solve make sense.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/data-src-image-4ec59ff5-acb6-4ee6-85dc-848d9940632b.png" alt="Data src image 4ec59ff5 acb6 4ee6 85dc 848d9940632b" /></p>

<p>Running GenAI on Kubernetes is not easy, and you need ot be ready to spend some money on GPUs, but understanding how these projects work and which components from other projects are being reused is quite important to mature the ecosystem.</p>

<p><a href="https://www.picdrop.com/kim-yennguyen/Rpea9zFuqv">Check out all the professional pictures from the Conference here</a></p>

<h3 id="devoxx-belgium">Devoxx Belgium</h3>

<p>At Devoxx Belgium, you have the opportunity to mingle with a lot of Java Developers. With more than three thousand attendees, I had the pleasure of presenting with my good friend Laurent from Postman / Microcks about creating and testing distributed applications that include workflows and LLM interactions.</p>

<p>Here are the slides for the presentation:</p>

<div class="video-embed" style="position:relative;padding-bottom:75%;height:0;overflow:hidden;max-width:100%;margin:1.5rem 0;"><iframe allowfullscreen="true" class="speakerdeck-iframe" loading="lazy" src="https://speakerdeck.com/player/c70c94348b684cb493898307fca08a80" style="position:absolute;top:0;left:0;width:100%;height:100%;border:0;" title="Simplifying Cloud Native app testing across environments with Dapr and Microcks"></iframe></div>

<p>In this session, we focused on an updated version of the Pizza Store, which uses <a href="https://docs.dapr.io/developing-applications/sdks/java/spring-boot/">Dapr Workflows</a>, <a href="https://spring.io/projects/spring-ai">Spring AI</a> to handle pizza orders across multiple services. You can find the source code of this demo that you can run on your laptop in the following repository <a href="https://github.com/salaboy/pizza">https://github.com/salaboy/pizza</a> (notice we used the <strong><em>agentic-workflow</em></strong> branch)</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/54844501986_40ee23e412.jpg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/54843646282_4317e6319a.jpg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /></p>

<p>If you are interested in this topic, you can check the following article in Dzone that goes into more details around Dapr workflows and Spring AI</p>

<p><a href="https://dzone.com/articles/durable-agents-with-spring-ai-and-dapr">https://dzone.com/articles/durable-agents-with-spring-ai-and-dapr</a></p>

<p>I had the chance to hang out with old colleagues from Drools/JBPM (Red Hat / IBM now) from the Quarkus team, including Roberto, Mario, Kevin, Kris, and Geoffrey, who provided me with a lot of insights about the Quarkus community. They are doing an amazing job with <a href="https://github.com/langchain4j/langchain4j">Langchain4j</a>, hence it is worth keeping an eye on. I was fortunate to attend the <a href="https://github.com/langchain4j/langchain4j">Langchain4j</a> BOF, where discussions centered on implementing new agent interaction patterns and the value of developing more complex mechanisms, given the unclear use cases.</p>

<p>I find it fascinating how all the agents’ discussion is leaning towards re-implementing mechanisms that we have implemented 100’s of times before such as <a href="https://github.com/langchain4j/langchain4j-examples/blob/main/agentic-tutorial/src/main/java/_9_human_in_the_loop/_9a_HumanInTheLoop_Simple_Validator.java#L31">human in the loop pattern highlighted by the linked example</a>.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6275.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6273.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6318.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6310.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6303.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6305.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/10/IMG_6299.jpeg" alt="Learnings from Cloud Native Denmark 🇩🇰 and Devoxx Belgium 🇧🇪 2025" /></p>

<p>I had the opportunity to spend some time with Roberto Cortez discussing the Dapr Quarkus Extension, which Matheus Cruz has been working on extensively, and I’m pleased to report that progress is being made. Expect a blog post about this soon, too.</p>

<p>I would say that Roberto’s session about how to create Quarkus extensions and Quarkus internals takes the prize of most valuable session.</p>

<p>Spring presented Spring Boot 4, and although it’s bringing some good features, I wasn’t particularly excited. That’s why I was happy to see <a href="https://www.thomasvitale.com/arconia-dev-services-spring-boot/">Thomas Vitale over the weekend about the Arconia</a> been working on.</p>

<p>Stay tuned for more news! The next stop is <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/">KubeCon NA</a>, happening in Atlanta, US, in November.</p>]]></content><author><name>Mauricio Salatino</name></author><category term="2025" /><category term="cloud native" /><category term="denmark" /><category term="devoxx" /><category term="belgium" /><category term="2024" /><category term="kubernetes" /><category term="testing" /><category term="dapr" /><category term="workflows" /><category term="microcks" /><category term="langchain4j" /><category term="agents" /><category term="conference" /><category term="event" /><summary type="html"><![CDATA[The software industry is vibrant with the GenAI bubble, but the good thing is that all this movement promotes new ideas, new patterns, and new challenges…]]></summary></entry><entry><title type="html">Developer Experience on Kubernetes</title><link href="https://www.salaboy.com/2025/07/26/developer-experience-on-kubernetes/" rel="alternate" type="text/html" title="Developer Experience on Kubernetes" /><published>2025-07-26T10:33:11+01:00</published><updated>2025-07-26T10:33:11+01:00</updated><id>https://www.salaboy.com/2025/07/26/developer-experience-on-kubernetes</id><content type="html" xml:base="https://www.salaboy.com/2025/07/26/developer-experience-on-kubernetes/"><![CDATA[<p>Over the last 15 years, I’ve been working with tools, frameworks, projects, and Kubernetes extensions that aim to help teams build and deliver cloud native applications. These applications are not simple, and the tools required to manage and master the whole software delivery lifecycle are quite overwhelming even in 2025.</p>

<p>As an industry, we have made significant leaps forward to fill the gaps and created new practices and frameworks to help organizations keep pace with these technological changes. While there is hype around AI today, and how LLMs will produce code for us, we are still in an industry that is measured purely by how good or how innovative the end-user solutions we create are. We need to deliver changes to our users; it doesn’t matter how we do it, but we just need to ensure that we are efficient and serious about the whole process.</p>

<p>Writing, talking, and sharing with organizations extensively about <a href="https://www.manning.com/books/platform-engineering-on-kubernetes">Platform Engineering in the Kubernetes</a> space led me on a journey where the tools being discussed are often closer to infrastructure than to developers. The natural next step is to bring developers closer to platform engineering teams, but different expectations, backgrounds, tooling, and priorities from both sides make these conversations slow and unproductive.</p>

<p>In November 2024, Thomas Vitale (author of <a href="https://www.manning.com/books/cloud-native-spring-in-action">Cloud Native Spring in Action, with Kubernetes</a>) and I signed a contract with Manning Publishing to co-author a book on <a href="https://www.manning.com/books/developer-experience-on-kubernetes">Developer Experience on Kubernetes</a>. The Manning Early Access Program (MEAP) was announced at KubeCon EU 2025 <a href="https://www.youtube.com/watch?v=nvKpg3JgSjs">(check our KubeCon session about the topic)</a>, and it is now available on their website.</p>

<p><a href="https://www.manning.com/books/developer-experience-on-kubernetes"><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/07/book-devex-cover-meap.png" alt="Manning Early Access Program - DevExp on Kubernetes" /></a></p>

<p><em>Manning Early Access Program - DevExp on Kubernetes</em></p>

<p>Writing a book is not done in a basement, away from everyone; similarly to my previous book, we are writing this book with the help of different communities that provide advice, reviews, early feedback, and valuable input to make the book more useful for a wider audience. As it is a tradition for technical books, we are hosting all the examples on GitHub, free of charge and totally independent from the book <a href="https://github.com/devex-on-k8s/book">https://github.com/devex-on-k8s/book</a></p>

<p>If you are interested in developer experience in the cloud native space, we encourage you to get in touch, check the examples, and explore the MEAP.</p>

<p>Our current table of contents is divided into three parts:</p>

<ol>
  <li>The inner loop: tasks, friction points, and tools</li>
  <li>The outer loop: optimization, APIs, system integrations, and observability</li>
  <li>How to bring Platform Engineers and Developers to collaborate more efficiently</li>
</ol>

<p>We aim to provide practical experience with open-source tools to solve real-life problems and identify how we can create smoother developer experiences by leveraging the expertise of cloud native communities. With Thomas, we are extremely passionate about helping platform teams understand the value of crafting end-to-end developer experiences. At the same time, this book helps developers understand the value of platform services and initiatives that can simplify the entire software development lifecycle.</p>

<p>To keep this blog post short, please check my <a href="https://www.youtube.com/watch?v=u49C5lx56fE">DevBCN presentation</a> on the topic, which provides an overview of what the book is about and showcases some of the tools we are covering.</p>

<div class="video-embed" style="position:relative;padding-bottom:56.25%;height:0;overflow:hidden;max-width:100%;margin:1.5rem 0;"><iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="" loading="lazy" referrerpolicy="strict-origin-when-cross-origin" src="https://www.youtube.com/embed/u49C5lx56fE?feature=oembed" style="position:absolute;top:0;left:0;width:100%;height:100%;border:0;" title="Mauricio Salatino - Cloud Native Developer Experience in 2025"></iframe></div>]]></content><author><name>Mauricio Salatino</name></author><category term="developers" /><category term="experience" /><category term="kubernetes" /><category term="cloud native" /><category term="2025" /><category term="book" /><category term="devexp" /><category term="devex" /><category term="platform engineering" /><category term="platform" /><category term="engineering" /><summary type="html"><![CDATA[In November 2024, Thomas Vitale (author of Cloud Native Spring in Action, with Kubernetes) and I signed a contract with Manning Publishing to co-author a…]]></summary></entry><entry><title type="html">🇬🇧 KubeCon EU 2025, Rejekts, Maintainers Summit and Co-located events! 🤯</title><link href="https://www.salaboy.com/2025/03/28/kubecon-eu-2025-rejekts-maintainers-summit-and-co-located-events/" rel="alternate" type="text/html" title="🇬🇧 KubeCon EU 2025, Rejekts, Maintainers Summit and Co-located events! 🤯" /><published>2025-03-28T14:31:04+00:00</published><updated>2025-03-28T14:31:04+00:00</updated><id>https://www.salaboy.com/2025/03/28/kubecon-eu-2025-rejekts-maintainers-summit-and-co-located-events</id><content type="html" xml:base="https://www.salaboy.com/2025/03/28/kubecon-eu-2025-rejekts-maintainers-summit-and-co-located-events/"><![CDATA[<p>I am preparing myself for a crazy week at KubeCon EU next week. There are tons of things going on, and I am excited to meet everyone coming to London. It is interesting to have a KubeCon happening in the city where you live, so let’s see how it goes.</p>

<p>My week is going to be hectic, but as usual, the most important factor for me is to meet people, make new connections, and learn about what is going on in the cloud native space.</p>

<p>Before we start all activities on Sunday with Cloud Native Rejekts, I am joining the “Is it Observable?” webinar with my friend Henrik Rexed today, Friday 28th, at 4pm. You can join us here: <a href="https://www.youtube.com/watch?v=7Pv2ktWr2cw">https://www.youtube.com/watch?v=7Pv2ktWr2cw</a></p>

<p><a href="https://www.youtube.com/watch?v=7Pv2ktWr2cw"><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/03/Screenshot-2025-03-28-at-14.23.13.png" alt="Screenshot 2025 03 28 at 14.23." /></a></p>

<h4 id="cloud-native-rejeckts-30th31st-march-25">Cloud Native Rejeckts (30th/31st March 25)</h4>

<p>Starting with <a href="https://cloud-native.rejekts.io/">Cloud Native Rejeckts</a>excellent on Sunday, I am looking forward to that pre-conference vibe with extremely good speakers and smaller rooms for deeper conversations.</p>

<p>Highlights on Sunday:</p>

<ul>
  <li><a href="https://cfp.cloud-native.rejekts.io/cloud-native-rejekts-europe-london-2025/talk/3RZV83/">Podman and Podman Desktop: State of the Union</a></li>
  <li><a href="https://cfp.cloud-native.rejekts.io/cloud-native-rejekts-europe-london-2025/talk/P7RZJ7/">Building Air-Gapped Control Planes for a Global Pharma Leader Using Crossplane and ArgoCD</a></li>
</ul>

<p>Highlights on Monday:</p>

<ul>
  <li><a href="https://cfp.cloud-native.rejekts.io/cloud-native-rejekts-europe-london-2025/talk/DEMXNY/">Simplifying cross-cloud, cross-cluster connectivity with Dapr &amp; Cilium</a></li>
  <li><a href="https://cfp.cloud-native.rejekts.io/cloud-native-rejekts-europe-london-2025/talk/MKR7YL/">Making GenAI CLI Agents Work for Your DevOps Day-to-Day Operations</a></li>
</ul>

<p>Unfortunately, on Monday, I cannot attend the presentation from Alice Gibbons and Manuel Zapf, who will be showing Cillium and Dapr working together across clusters. If you have the chance to participate in the session, I totally recommend it.</p>

<h4 id="maintainers-summit-31st-march-25">Maintainers Summit (31st March 25)</h4>

<p>On Monday, I am eager to see how the <a href="https://maintainersummiteu2025.sched.com/">Maintainers Summit</a> goes. While this event is not for everyone, I will take enough notes to share with a broader audience. I am really looking forward to the <a href="https://sched.co/1uSN7">TAG restructuring workshop</a> as this represents a significant re org of how advisory groups inside the CNCF will work. Specifically, I am keen to see how I can be part of the TAG Developer Experience, which is key to bringing developers closer to the cloud native space.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/03/tag-developer-experience.png" alt="Tag developer experience" /></p>

<h4 id="co-located-events-1st-april-25">Co-located events (1st April 25)</h4>

<p>On Tuesday, life gets busy at the co-located events.</p>

<p>Starting with a presentation at 9:45am with Julia Morgado Furst titled <a href="https://colocatedeventseu2025.sched.com/event/1u5gL/streamline-your-app-development-with-the-power-of-cloud-native-tools-julia-furst-morgado-veeam-mauricio-salatino-diagrid"><strong>Streamline Your App Development With the Power of Cloud Native Tools - Julia Furst Morgado, Veeam &amp; Mauricio Salatino, Diagrid</strong></a> for the first ever Cloud Native university co-located event.</p>

<p><a href="https://sched.co/1u5gL"><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/03/Screenshot-2025-03-28-at-14.25.35.png" alt="Screenshot 2025 03 28 at 14.25." /></a></p>

<p><strong><a href="https://sched.co/1u5gL">CNCF-hosted Co-located Events Europe 2025: Streamline Your App Development With the…</a></strong><br />
View more about this event at CNCF-hosted Co-located Events Europe 2025</p>

<p>This will be followed by a presentation at 10:40am at the <a href="https://sched.co/1u5lu">Platform Engineering Day co-located event with the legend Viktor Farcic</a>. Titled <a href="https://colocatedeventseu2025.sched.com/event/1u5lu/the-past-the-present-and-the-future-of-platform-engineering-mauricio-salatino-diagrid-viktor-farcic-upbound?iframe=no&amp;w=100%&amp;sidebar=yes&amp;bg=no"><strong>The Past, the Present, and the Future of Platform Engineering - Mauricio Salatino, Diagrid &amp; Viktor Farcic, Upbound</strong></a></p>

<p><a href="https://sched.co/1u5lu"><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/03/Screenshot-2025-03-28-at-14.27.23.png" alt="Screenshot 2025 03 28 at 14.27." /></a></p>

<p><strong><a href="https://sched.co/1u5lu">CNCF-hosted Co-located Events Europe 2025: The Past, the Present, and the Future of…</a></strong><br />
View more about this event at CNCF-hosted Co-located Events Europe 2025</p>

<p>While these are two completely different presentations and co-located events, I am super hyped to collaborate with such a diverse group.</p>

<p>Next, I will be doing some book signings at the Synstasso booth at 12pm at their booth, you can find <a href="https://www.linkedin.com/posts/syntasso_syntasso-kubeconeu2025-cncf-activity-7311356448884547584-B8-n?utm_source=share&amp;utm_medium=member_desktop&amp;rcm=ACoAAAEprDYB92fxehJ4a6Jyxm-N9zCQL7hohHg">more details here</a>.</p>

<p><a href="https://www.linkedin.com/posts/syntasso_syntasso-kubeconeu2025-cncf-activity-7311356448884547584-B8-n?utm_source=share&amp;utm_medium=member_desktop&amp;rcm=ACoAAAEprDYB92fxehJ4a6Jyxm-N9zCQL7hohHg"><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/03/Screenshot-2025-03-28-at-13.57.56.png" alt="Screenshot 2025 03 28 at 13.57." /></a></p>

<p>Later at 3:30pm, there will be more book signings with my friends at <a href="https://www.linkedin.com/company/loft-sh">Loft Labs</a>, who are always happy to promote the book.</p>

<p><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/03/Book-Signing-Maurice-Salitino.jpg" alt="Book Signing Maurice Salitino" /><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/03/Book-Signing-Maurice-Salitino-dates_3.jpg" alt="Book Signing Maurice Salitino dates" /></p>

<p>As you can see there will be more signings on Wednesday and Thursday.</p>

<h4 id="kubecon-keynotes-and-sessions-2nd-3rd-and-4th-april-25">KubeCon Keynotes and sessions (2nd, 3rd and 4th April 25)</h4>

<p>There will be a lot of interesting sessions and keynotes happening during the week. I recommend looking at the schedule to see who is presenting and where before arriving at the event.</p>

<p>If you want to follow along and visit the <a href="https://diagrid.io">Diagrid</a> booth to say hi, here are all the sessions about the Dapr project: <a href="https://www.diagrid.io/events/diagrid-dapr-kubecon-cloudnativecon-europe-2025">https://www.diagrid.io/events/diagrid-dapr-kubecon-cloudnativecon-europe-2025</a></p>

<p><a href="https://www.diagrid.io/events/diagrid-dapr-kubecon-cloudnativecon-europe-2025"><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/03/Screenshot-2025-03-28-at-14.12.55.png" alt="Screenshot 2025 03 28 at 14.12." /></a></p>

<p>I strongly recommend you folks to attend</p>

<ul>
  <li>Marc’s session about durable executions with Dapr Workflows: <a href="https://sched.co/1txBF"><strong>Failure Is Not an Option: Durable Execution + Dapr = 🚀 - Marc Duiker, Diagrid</strong></a></li>
  <li>AndI am very hyped to see Yaron and Roberto from Nvidia to present about Dapr Agents: <a href="https://sched.co/1tcyT"><strong>Dapr in 2025: Durable Execution for Agentic AI Systems and Becoming the Backbone of App Platforms</strong></a></li>
</ul>

<p>On Thursday, I will be doing a short presentation at the Learning Lounge, where I will be sharing some tips about how to progress your career from being a developer to a platform engineer:</p>

<p><a href="https://sched.co/1vpy0"><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/03/Screenshot-2025-03-28-at-14.16.28.png" alt="Screenshot 2025 03 28 at 14.16." /></a></p>

<p>Super important! Don’t miss out your spot in the Diagrid + Upbound party! Sign up now!</p>

<p><a href="https://pages.diagrid.io/boom-battle-bar-upbound"><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/03/Screenshot-2025-03-28-at-14.34.13.png" alt="Screenshot 2025 03 28 at 14.34." /></a></p>

<p>Finally, on Friday, I am presenting with my friend Thomas Vitale about developer experience on a presentation titled: <a href="https://sched.co/1txEm"><strong>Breaking Barriers: Bringing Application Developers Closer To the CNCF - Thomas Vitale, Systematic &amp; Mauricio “Salaboy” Salatino, Diagrid</strong></a></p>

<p><a href="https://sched.co/1txEm"><img src="https://cdn.jsdelivr.net/gh/salaboy/blog-assets@v1/images/2025/03/Screenshot-2025-03-28-at-14.23.54.png" alt="Screenshot 2025 03 28 at 14.23." /></a></p>

<p>This presentation aims to create and promote more specific channels to bring application developers closer to platform engineers so we can build better platforms.</p>

<p>As always, I will take a lot of notes during the event and share my slides, demos, and pictures at the end of the event.</p>

<h4 id="say-hello-dont-be-shy">Say hello! Don’t be shy!</h4>

<p>If you see me running around the conference, please stop me to say hi! Let’s talk tech!</p>

<p>If you want a copy of my book, check out the book signing times!</p>

<p>See you all there!</p>]]></content><author><name>Mauricio Salatino</name></author><category term="kubecon" /><category term="london" /><category term="cloud-native" /><category term="kubernetes" /><category term="dapr" /><category term="appdev" /><category term="2025" /><category term="event" /><category term="conference" /><category term="platforms" /><category term="book" /><category term="signing" /><summary type="html"><![CDATA[I am preparing myself for a crazy week at KubeCon EU next week. There are tons of things going on, and I am excited to meet everyone coming to London.]]></summary></entry></feed>