RAG, Tools, Workflow or Agent?

Choose the smallest architecture that can complete the evaluated work

Altivate | Published 31 May 2026 | Updated 26 July 2026

1. Start with the uncertainty, not the technology

Enterprise AI proposals are increasingly described with one word: agent. A policy assistant that searches six documents is an agent. A claims process encoded as a fixed graph is an agent. A model that calls an API is an agent. A group of independent systems negotiating work is also an agent.

That vocabulary hides the architecture decision.

Anthropic’s implementation guidance draws one useful line: a workflow uses predefined code paths, while an agent dynamically directs its own process and tool use. Microsoft’s current architecture guidance draws another: standard RAG suits a question that one search against one index can answer; agentic RAG becomes useful when the system must decompose the question, search multiple sources, inspect gaps or combine retrieval with action.

Both are vendor guidance, but they expose the right question:

• If the answer exists in governed documents but the right passage is hard to find, start with retrieval.
• If the answer lives in a current application state, call a read-only tool.
• If the business path is known and exceptions can be enumerated, encode a workflow.
• If the path genuinely depends on intermediate findings, consider an agent.

This sequence is not anti-agent. It is how an organisation finds the smallest architecture that can do the work.

Context: Altivate’s machine learning and data practice, analytics and business intelligence, and an earlier explanation of retrieval-augmented generation.

2. The four jobs

RAG: “Which approved evidence is relevant?”

Retrieval-augmented generation finds passages from an external knowledge collection and supplies them to a model when it answers. Its value is not that it makes the model know more in the abstract. It makes the answer traceable to an enterprise-controlled evidence set that can be updated independently of the model.

Use RAG when:

• the source material is document-like;
• the question is primarily informational;
• a small number of passages can support the answer;
• citations and source access can be evaluated;
• the underlying source need not be changed.

Do not use the vector index as the system of record. Google’s RAG reference architecture separates ingestion from serving and treats the index as a derived asset. That distinction has operational consequences: source permissions, deletions and corrections must propagate into every chunk, embedding and cache derived from them.

Tool: “What is true now, or what exact operation is available?”

A tool gives the model-facing system a constrained way to query or change an application. Use it when the answer depends on live state: current inventory, open invoices, a customer’s entitlement, today’s shipment status or an approved transaction.

A tool call can be read-only or consequential. Keep those categories separate. A read tool should expose only the fields the job requires. A write tool should accept typed, validated parameters and enforce authorisation inside the business service; it should not translate free-form model text directly into a database change.

Workflow: “Which known sequence should run?”

A workflow encodes the path in code or configuration. It may use models for individual steps – classification, extraction, drafting or exception explanation – while the surrounding sequence remains deterministic.

Use a workflow when:

• the required stages and approvals are known;
• compliance depends on steps happening in order;
• retries and compensation can be defined;
• exceptions can route to a person or a named branch;
• predictability matters more than open-ended flexibility.

The presence of a language model does not make the workflow an agent. That is a strength, not a failure of ambition.

Agent: “What path should be taken next?”

An agent selects steps based on the current state and intermediate results. It is justified when the task cannot be reduced to one retrieval or a stable path without losing the value of the job. Examples include investigating a complex variance across several systems, planning a migration from incomplete estate evidence, or resolving a request whose required sources become clear only after the first finding.

Use an agent when:

• path selection is the core problem;
• intermediate evidence legitimately changes the next action;
• the available tools have narrow, enforceable contracts;
• the organisation can evaluate trajectories, not only final prose;
• the extra latency, cost and failure modes buy a measured improvement over a workflow.

3. The decision table

Two rules keep this table honest.

First: combine patterns when the job combines uncertainties. A useful system may use RAG for policy evidence, a read tool for live customer state and a workflow for approval. Calling the whole thing “an agent” is harmless in a slide and dangerous in an architecture review because it conceals which component owns which failure.

Second: require a measured reason to move right. A system that already answers correctly with one retrieval does not become better because a planning loop asks the same question three different ways. More calls increase cost and latency, and each step adds another place for permissions, evidence or state to drift.

4. Three enterprise examples

Policy Q&A

Question: “Can this expense be reimbursed?”

If the answer depends only on the current policy document, standard RAG should retrieve the relevant clauses and return them with citations. If it depends on the employee’s grade, location and remaining allowance, add read tools for those facts. If a known ruleset determines the result, encode it. An agent is justified only if the case requires open-ended investigation across sources that cannot be predicted from the initial question.

The tempting design is to let an agent browse everything. The better design makes each source explicit, preserves its access policy and shows the user which evidence produced the answer.

Month-end variance investigation

Question: “Why is gross margin down?”

This may justify an agent. The first query can reveal whether the variance sits in price, mix, freight, exchange rate or cost allocation; that finding determines the next query. The agent needs governed metric definitions, read-only data tools, a limit on the dimensions and time ranges it may inspect, and an evaluation set containing known historical variances.

It does not need permission to post an adjustment. Investigation and correction are different actions.

Supplier onboarding

The stages – collect documents, validate required fields, screen, route by risk, obtain approval and create a vendor record – are known. That makes the process a workflow. Models may extract documents or explain exceptions. A person may resolve ambiguous ownership. A deterministic service should create the vendor after approvals. An open-ended agent deciding which control to skip is not flexibility; it is a broken process definition.

These examples show why the right answer is often a composition rather than one category.

5. The hidden work in RAG

RAG is often positioned as the simple option. Its runtime may be simpler than an agent, but the data product underneath it is not free.

Google’s reference architecture separates two pipelines:

1. ingestion collects, parses, segments, embeds and indexes source content;
2. serving retrieves relevant material and uses it to answer.

Treat that separation as a control boundary. For every source, decide:

• who owns its accuracy and lifecycle;
• which document version is authoritative;
• how source permissions become retrieval permissions;
• how quickly revocation or deletion reaches derived copies;
• how chunks preserve enough context to avoid changing meaning;
• how citations point back to a readable source;
• how stale, duplicate and conflicting material is handled;
• how retrieval quality is measured independently of answer fluency.

A model can produce a polished answer from the wrong passage. The evaluation therefore needs at least two scores: did retrieval return the necessary evidence, and did the answer use that evidence correctly? A single thumbs-up on the final response cannot locate the failure.

Agentic RAG adds a further layer. Microsoft’s guidance describes systems that rewrite and decompose queries, run parallel searches and assess whether the result is sufficient. That can improve complex research. It can also multiply retrieval calls and create opaque evidence selection. Log each subquery, source, score and selected passage so the final answer can be reconstructed.

6. The hidden work in tools and agents

AWS’s enterprise agent architecture separates model access, tools and knowledge bases, with security, observability and discoverability spanning the layers. Its tool component is responsible for secure execution and contextual authorisation; its knowledge component must carry data access control.

That separation prevents two shortcuts.

Shortcut one: trust the model to enforce a business rule. A prompt saying “never approve more than AED 5,000” is not a transaction limit. The service that performs the operation must enforce the value, currency, role, record state and approval.

Shortcut two: let protocol authentication stand in for business permission. MCP can standardise a tool call and its HTTP authorisation. A2A can let independent agents discover and communicate. Neither protocol knows the customer’s dispute state, the approver’s delegated limit or the segregation-of-duties rule. Those remain enterprise controls.

For every tool, document:

• read or write;
• accepted parameters and rejected ranges;
• identity and permission;
• idempotency behaviour;
• timeout and partial-commit behaviour;
• exact approval requirement;
• result schema;
• audit fields;
• compensating action.

An agent should receive a small set of legible tools, not a general-purpose application credential.

7. A procurement test

Ask a vendor to demonstrate the same business case four ways:

1. one retrieval and one model response;
2. retrieval plus a read-only application tool;
3. a fixed workflow using model steps;
4. an agent selecting its own path.

For each, record:

• task success on the same evaluation set;
• evidence accuracy and permission enforcement;
• median and tail latency;
• model and tool calls;
• cost per completed task;
• human-review rate;
• unrecovered and silently wrong outcomes;
• ease of replaying the decision;
• time to disable or change one capability.

Make the commercial boundary equally testable. Contract terms should state whether customer data or outputs may be used for training; ownership and permitted use of prompts, traces, evaluations and generated artefacts; IP indemnity; subprocessor notice and objection rights; auditable control evidence; notice and revalidation for model substitution; termination assistance; and export of data, configuration, traces and evaluation sets in a usable format. A portable protocol does not make a non-portable contract portable.

If the agent version produces no material improvement on the cases that matter, choose the simpler system. If it resolves cases the workflow cannot enumerate, the additional complexity has earned its place.

This comparison also exposes platform lock-in. Ask whether tools, evaluations and traces remain portable; whether a tool built for the platform can be called elsewhere; and whether a different orchestrator can invoke the agent through a documented interface. “Supports a protocol” is not a complete answer – test both directions.

8. The selector constrains architecture; it does not choose products

Most production systems combine two or more patterns. The framework does not argue that agents are unnecessary; it requires uncertain path selection to earn their extra operating surface. It does not compare vector databases, orchestration frameworks or models, and it never treats an index as authoritative data or protocol support as business authorisation.

Vendor architecture guidance is labelled as such. Choose performance against the organisation’s own evaluation set. Guidance and protocol status were checked 26 July 2026 and must be rechecked before revision.

9. The executive agenda

For every proposed use case, require one sentence each for:

• the evidence it must retrieve;
• the live state it must query or change;
• the sequence that is already known;
• the decision whose next step genuinely cannot be known in advance.

Then build in that order: validate the evidence layer, expose narrow tools, encode the stable workflow, and add agentic path selection only where measured cases require it. Retain a workflow baseline so the agent must demonstrate material improvement on the cases that matter.

Altivate works across data, analytics, AI and enterprise applications in Saudi Arabia, the UAE, Jordan and India. Our architecture position is deliberately conservative about complexity and ambitious about outcomes: autonomy must earn its place through evaluated work.

Connect the selector to machine learning and data, or return to White Papers.

Sources and verification status

Checked 26 July 2026.

Vendor architecture guidance – [read, vendor guidance]

• Anthropic, Building effective agents
• predefined workflows versus dynamically directed agents and the simplest-sufficient approach.
• Microsoft Azure Architecture Center, Develop an Agentic RAG Solution – standard-RAG and agentic-RAG selection, decomposition and multi-source retrieval.
• Google Cloud Architecture Center, RAG infrastructure for generative AI using Gemini Enterprise and Agent Platform
• ingestion/serving separation and vector index as a derived component. Last reviewed 10 November 2025.
• AWS Prescriptive Guidance, Agentic AI architecture in the enterprise
• model, tool and knowledge layers with cross-layer controls.
• AWS Prescriptive Guidance, Security considerations for data in generative AI
• enterprise data access and security considerations.

Standards and public guidance – [read]

• Model Context Protocol, authorisation specification
• HTTP authorisation based on OAuth 2.1; authorisation remains optional at protocol level.
• A2A Project / Linux Foundation, protocol specification 1.0 – independent-agent discovery, communication and protocol authorisation.
• NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile – system-level risk and evaluation guidance.