Treat Your AI Agent Like a User, Not a Feature

As a Mendix developer with an MSc in Cyber Security, I’ve watched AI become an increasingly common part of application development. More and more projects are moving beyond traditional automation and looking at how AI agents can be embedded directly into applications and workflows. While researching the use of AI with Mendix, I kept coming across the same pattern. An AI agent would be introduced as a new application feature, often with broad access to application data so it could perform its role more effectively.

But that raised a question for me: what happens when that access becomes a vulnerability? If someone were able to gain control of an AI agent, could they use those same permissions against us? That pattern has a name. Agentic AI means giving large language models (LLMs) access to an application’s tools and data to complete tasks such as answering support queries and generating documents. The appeal is clear and so is the risk; an agent gains more capabilities with more access granted. To improve the scope and accuracy of AI, agents are often granted access rights that exceed those of ordinary users, increasing the opportunities for attackers to exploit AI-powered apps.

I wanted evidence for what I was seeing rather than relying on instinct. So, I designed an experiment to measure how risky badly architected agent workflows can be and how that can be mitigated against by building secure agentic workflows. The result is the paper "Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcare.", which applies the AI Trust, Risk, and Security Management (TRiSM) framework to a Mendix application. This blog covers what I found and what it has changed about how I build agents into apps.

Security Mindset for Agents

When adding agents to an application, it is easy to think of them as another feature. In reality, an agent should be treated as a new user role. It accesses tools and data to complete tasks, so requires the same access controls and logging that would be applied to a human user.

Recent advances in agentic workflows have made that access harder to control. Model Context Protocol (MCP) servers provide a structured way to call external tools with agents, while Retrieval-Augmented Generation (RAG) with knowledge bases provides context-specific information to the agent. Both improve the capabilities and accuracy of agents but give new opportunities for an attacker to exploit AI-powered apps. Since MCP tools and knowledge bases can come from external sources, apps can be exposed to insecure code or incorrect information if these external sources are hijacked.

Scenario: Agents Generating Reports

To test how much these risks matter in practice, I integrated agents into a medical application that generates reports for clinical encounters. Based on information provided by the app, tool calls, and knowledge base, the workflow recommends treatments in the reports if one has not been provided by the attending physician. It’s a sensitive, patient-critical scenario where AI security cannot be an afterthought.  

I then compared two workflow designs on the same application, with the same attack vectors and the same synthetic patient data:

• An insecure single-agent workflow: one agent has access to all the tools and sources. Prompts are constructed client-side to reduce the load on the application server.

• A TRiSM-guided agentic workflow: the task is split into subtasks across specialised agents. Prompts are constructed server-side. Each agent is only exposed to the data and tools needed to complete its task.

Where the Insecure Workflow Fails

The single-agent workflow’s major security concern is that its prompts are built on the client. Once the LLM request is held in a non-persistable object on the client, the attributes going into that prompt have effectively been stripped of the security they had in the application database. A good rule to follow is to assume anything stored on the client-side is publicly exposed. Therefore, these prompts, which contain sensitive information, are exposed publicly.

Building the request on the client also means an attacker can change the fields that go into it. I saw this first-hand in the research paper, where an injection attack exposed patient information without an attacker supplying any patient-identifying information. With one agent holding all the data, one successful injection attack can expose a great deal of information in one go.

Broad access compounds the damage. Giving one agent every available tool boosts its capability, but it boosts its exposure just as quickly. The more functionality an agent can reach, the more data is put at risk. If the MCP server providing those tools has no access controls, an attacker can add malicious tools that the agent then runs automatically. This can expose application data without ever needing to know how the database is structured. The single-agent workflow is, in short, a single point of compromise.

Simply splitting this insecure agent workflow into multiple agents does not solve security issues on its own. I tested an insecurely implemented agentic workflow and found that injection attacks were still successful due to client-side prompt construction. So, the correct architecture needs to be implemented with the right security controls throughout. This is where we can introduce the TRiSM framework to improve security.

Applying the TRiSM Principles

Defined by Gartner, TRiSM focuses on five principles, governance, trustworthiness, fairness, reliability, and data protection. Rather than being applied at the organisational level, it is used at the agent level, allowing flexibility in the technical controls used. This means that workflows can be designed to adapt to the application context. For the medical application in this research, this is how I designed the TRiSM-guided agentic workflow:

• Data protection: User-style permissions for agents. Role-based access controls implemented on MCP servers rather than giving agents access to everything.

• Fairness: Reducing bias and regulatory breaches by removing irrelevant data from prompts. Running embedding analysis to detect known injection patterns before a prompt even reaches the LLM.

• Reliability: Input sanitisation and constructing prompts server-side with write access denied to users.  

• Governance: Lifecycle management. Mendix Agent Commons provides versioning for individual agents. Using this management, agents were made single-call rather than conversational, preventing chat history tampering.

• Trustworthiness: Explainability in the workflow process. Logging which agents and services were involved in each step, producing a clear and accountable trail.

What the Results Showed

I evaluated both workflows across five LLM models from OpenAI, Anthropic, and Google. The improvements with the TRiSM-guided agentic workflow were significant on both security and accuracy.

On security, the TRiSM-guided workflow reduced the mean attack success rate for RAG poisoning from 31% to 10%, and for data-field injection from 42% to 25%. The client-side network injection vector, which succeeded a striking 59% of the time against the single-agent baseline, was eliminated entirely. Because prompts are now constructed server-side, client-supplied fields never enter the prompt, so there is nothing for that attack to target. The reduction was statistically significant. However, no single model was robust across all attacks, showing that model selection is not a substitute for secure architectural design.

Security did not come at the expense of quality. Report accuracy increased from 72.5% to 86.5%. Agents handling smaller tasks are both harder to manipulate and more reliable. However, splitting a task across sequential agents increased token consumption by up to 16%, though the most efficient model showed just a 5% increase. This is another reminder that model choice is an architectural decision, not just a question of price and latency.

Splitting the work into subtasks for agents also made performance more predictable. For some AI models, the spread of latency to generate documents narrowed notably even where the median was largely unchanged, indicating a more stable workload. Smaller, cleaner tasks improve security and accuracy in addition to making the workflow easier to reason about in production.

How This Changes My Approach to Building Mendix Apps

The paper was a controlled experiment, but it showed real risks if controls aren’t put in place. These are controls we already apply to human users, but have not considered for agents. Five things have changed in how I approach agents in Mendix applications:

Build the prompt where the data lives. By moving prompt construction to server-side, the attack vector for network injection was effectively nullified. If any client-side fields end up in the prompt, treat it as if they could have been modified by an attacker.

Scope the agent like a user role. Before an agent goes into a project, ask the question you would ask of any new role: what is the minimum set of data and tools it needs? Enforce that in the application’s access controls and on MCP servers. An instruction in a prompt telling an agent not to use a tool is a request, not a control.

For complex workflows, consider multiple agents. Splitting the task is what improved accuracy by 14 percentage points and limited the risk of compromise in the research paper. However, using multiple agents on its own doesn’t stop attacks. Architecture and controls need to be used together.

Make single-call agents the default. Conversational agents carry a chat history which an attacker can tamper with. Unless the use case genuinely needs a conversation, a single-call agent carries less risk.

Log the workflow, not just the outcome. Recording which agent and which service was accessed provides an accountable trail which might be needed in regulated sectors. This helps explain how AI got to its final output.

Conclusion

Adding AI agents to an application can expand its capabilities enormously. But integrating AI without proper risk analysis introduces unpredictable vulnerabilities, most often because the principle of least privilege is quietly ignored in a way it never would be for a human user role. By treating the agent as a new user role, building prompts server-side, and inspecting inputs before they reach the model, a workflow that was leaking data and accepting injected instructions became one that is measurably more secure and accurate.

The wider lesson highlights a theme I return to often: the excitement of agentic AI must be balanced against an assessment of its risks, project by project. Frameworks like TRiSM give us a practical, technical way to do that assessment, turning a set of governance principles into controls to build. As agents take on more responsibility in critical applications, deliberate, security-conscious design is what will keep both the data and the trust of the people relying on it intact.

This blog is based on the research paper "Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcare." The preprint is available on arXiv  https://arxiv.org/abs/2606.28666

‍

‍

‍