Observability

Multi-agentic architectures can pose challenges from an observability perspective. Each agent within a workflow makes its calls to its own toolset and each agent may be configured to use a different underlying LLM (or even a Model Router which picks the most efficient model for us). Being able to observe which agent made what call with which model in a unified manner can be challenging over an entire workflow. OpenTelemetry provides semantic conventions that help us solve this problem and Microsoft Foundry provides tracing integrations for agent frameworks that enable us to implement tracing for different agentic frameworks, and assist us in quickly identifying root causes for issues in our agents. ...

Determining how Azure AI Foundry Agents makes decisions is important for troubleshooting and debugging purposes. However, it can get a little complicated when our agents perform complex workflows. Our agents could perform numerous executions, making it difficult to track decisions made by all or them, or some agents may invoke tools, that invoke other tools, which invoke more tools! (And so on and so forth). Tracing our agents helps us see the inputs and outputs involved in a particular agent run, as well as the order in which those agents were invoked. In this blog post, I’ll talk about how tracing agents works, how we can do some simple tracing using the Azure AI Foundry Agents playground, and how we can implement tracing in our pro-code agents using OpenTelemetry. ...