From Connected Agents to Workflow Orchestration: Rethinking Agent Design in New Foundry

A while ago, I developed an agentic solution using what is now ‘classic Foundry’. At the time, I chose to leverage connected agents, a pattern that gave me two critical capabilities:

• Response ownership: ensuring that a single agent would ‘own’ the final response creation.
• Conditional routing: dynamically selecting the best suited ‘specialist’ agent based on the user request.

This combination allowed me to build a system that was both flexible in terms of which agent would augment the response with its specialty but also structured in which agent and how the final response would be generated.

However, when researching the migration to the ‘new Foundry’, I quickly realized that this pattern no longer had a direct equivalent. The absence of connected agents meant the orchestration was no longer implicit, but an explicit concern for the solution being built.

At that point, some options were considered. But first let’s talk about the tool that would sit at the core of this migration.

Microsoft Agent Framework

As mentioned earlier, orchestration had now become an explicit concern of the solution.

While Foundry was already chosen as the runtime for its built-in capabilities that make agents ready for production (article talking about that here), it does not solve, by itself, the orchestration limitation.

I still needed an orchestration layer that would seamlessly integrate with it. The answer? Microsoft Agent Framework.

Why Microsoft Agent Framework?

Microsoft Agent Framework (or MAF, as we’ll refer to it from this point forward) provides built-in interoperability with the Foundry Agent Service.

This integration is enabled through the ‘FoundryAgent’ abstraction, which extends the broader ‘AIAgent’ abstraction. As a result, Foundry-based agents can be seamlessly integrated into MAF workflows, allowing developers to leverage the full orchestration capabilities of MAF while still benefiting from the production-grade runtime provided by Foundry.

In practice, this means we are not forced to choose between orchestration flexibility and runtime robustness. We can combine both within a single, cohesive architecture.

At this point, the architectural direction started to become clearer. However, the challenges were still only beginning.

The considered orchestration options

The first option I explored was the built-in handoff orchestration model. At first glance, this approach seemed like a natural fit: a single intake agent receives the request and based on the identified intent, routes execution to the appropriate specialist agents.

The relationships between agents are explicitly defined using the handoff workflow builder, allowing the orchestration flow to be controlled through predefined transitions. On paper, this looked like a perfect match.

However, a key limitation emerged.

In my architecture, specialist agents were not designed to interact with the user directly. Their role was strictly to process the request and forward factual data retrieved from tools. The responsibility for producing the final response was intentionally centralized in a single response agent.

While handoff orchestration supports conditional routing, it does not provide sufficient control over which agent is allowed to produce the final response. As a result, intermediary agents could inadvertently respond directly to the user, leading to unintended exposure of intermediate processing steps and raw data.

This “leakage” of internal processing broke a core requirement of the system: maintaining a clear separation between processing and response creation. In practice, this meant that handoff orchestration optimized for flexibility, but at the cost of control. A trade-off that did not align with my requirements.

Other options were also considered:

• Sequential orchestration: This model provides a straightforward, step-by-step execution flow, but lacks the dynamic routing capabilities required to select the appropriate specialist agent based on the incoming request.
• A2A Protocol: The Agent-to-Agent protocol is designed to enable communication between agents across process, service, or organizational boundaries. While powerful, it is primarily intended for scenarios where agents operate as independent services, potentially across different teams, technologies, or platforms.

Sequential orchestration lacked the necessary flexibility in routing. A2A was an unnecessarily complex solution for the problem at hand, my agents were part of the same logical application and cohesive architecture.

• Foundry Workflow Agents: This approach could potentially provide both the required flexibility and strict response ownership, making it a strong conceptual replacement for connected agents. However, at the time of the migration, workflow agents were still in preview and actively evolving.

As a result, adopting them would have introduced additional uncertainty and risk into the solution. While they may become a natural successor to connected agents in the future, they were not a viable option at that point in time.

What solved the problem: MAF Graph-based workflow

After evaluating the available orchestration models, it became clear that none of the ‘out-of-the-box’ approaches fully reproduced the behavior that connected agents had previously provided.

The solution was to stop thinking in terms of agent-to-agent conversation and instead model the entire interaction via a custom-designed workflow.

Rather than allowing agents to decide who should execute next, or relying on a static predefined sequence, the workflow itself became responsible for orchestration. Execution paths, routing decisions, and response ownership were explicitly defined within the workflow code, making the overall behavior both predictable and transparent.

This approach successfully recreated the two capabilities that originally motivated the use of connected agents, while also introducing additional benefits:

• Response ownership: only one dedicated agent produces the final response, defined in the workflow design.
• Conditional routing: router agent produces typed response with route decision and confidence score. This typed response is used in code to deterministically route to specialist agents.
• Fine-grained execution control: workflow execution can be influenced through custom Executors, enabling deterministic behavior at each step (pre and post inference).
• Extensibility: new specialist agents can be introduced with minimal impact on the existing workflow.

In other words, the intelligence of deciding which capability is required remained AI-driven while how and when became deterministic.

How the problem was solved

While the high-level architecture appears relatively straightforward, several implementation details were necessary to ensure the workflow behaved consistently and predictably.

The Router Agent

The workflow begins with a dedicated Router Agent. Its responsibility is intentionally limited: analyze the user request and determine which specialist capability is required.

The router does not retrieve data and does not generate responses. Instead, it produces a structured JSON response with the routing decision that the workflow consumes to determine the next execution step.

With the development of the solution, new specialist agents could be introduced with minimal impact on the rest of the system. The router simply gains a new routing option.

Specialist Agents as Capability Providers

Each specialist agent was designed around a single responsibility. Rather than behaving like chatbots, these agents acted more like intelligent services.

Their role was to:

• Execute tool calls.
• Retrieve domain-specific information.
• Perform any required reasoning over that information.
• Produce factual outputs for downstream consumption.

Importantly, they were not responsible for interacting with the user. This design decision eliminated a major issue encountered during the evaluation of handoff orchestration: the possibility of intermediate agents generating user-facing responses.

Instead, specialist agents became implementation details of the workflow rather than visible participants in the conversation.

One important lesson learned: keeping agents thin and focused has a huge impact on accuracy. Less reasoning overhead means less mistakes. With focused agents our instructions also naturally become shorter, keeping input tokens controlled and further improving inference performance.

Maintaining Strict Response Ownership

One of the original requirements from the connected-agent architecture was to preserve a single point of response generation. To maintain this behavior, the workflow always concludes with a dedicated Response Agent.

This agent receives:

• The original user request.
• The outputs produced by the selected specialist agents.
• Any additional workflow context accumulated during execution.

Its sole responsibility is to transform gathered information into a coherent response. Because all user-facing content originates from a single location, formatting instructions, response guidelines, tone requirements only need to be maintained in one place.

This significantly reduced prompt duplication across agents and improved consistency of generated responses.

Why this works better than Connected Agents

Interestingly, the final solution did more than simply replace connected agents. By making orchestration explicit, it became easier to:

• Understand execution flow.
• Debug incorrect routing decisions.
• Introduce new specialist agents.
• Add evaluation and observability capabilities.
• Enforce architectural boundaries between agents.
• Fine-grained control over data shared with inference task through the ‘Executors’ (this is HUGE)

What was initially a migration challenge became an opportunity to move from an implicit orchestration model to an explicit workflow architecture.

The result preserved the flexibility of specialized agents while improving overall performance (more responsive chat experience) and keeping costs mostly the same. With it we achieved greater control, governance, predictability, and maintainability, setting the path for future expansion.

References

What is Microsoft Foundry?
Microsoft Agent Framework overview
Workflow concepts
Microsoft Agent Framework workflows - Workflow Builder & Execution
Migrate from the Foundry(classic) portal