A robot receiving an ordinary instruction might appear to be doing nothing more than following a routine task. But what happens if the artificial intelligence deciding its next move has been secretly compromised? Researchers at Binghamton University have investigated this possibility, uncovering a security weakness that could turn seemingly harmless instructions into unsafe robotic actions under specific conditions.
Their experiments explore a growing concern as large language models (LLMs) become more involved in robotics: the possibility that a compromised AI planning system could introduce dangerous actions into an otherwise normal task. The findings also introduce a separate defensive technology designed to identify potentially harmful robot plans before they are executed.
Binghamton University Researchers Investigate AI Robot Security Risks
The research was conducted by computer scientists from Binghamton University’s Thomas J. Watson College of Engineering and Applied Science, School of Computing.
The team includes Associate Professor Shiqi Zhang, Assistant Professor Adnan Siraj Rakin, and PhD researchers Zainab Altaweel, Mohaiminul Al Nahian, David Reitano and Sabbir Ahmed.
The researchers presented two papers examining the security of AI-assisted robot task planning at the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). The event took place from September 27 through October 1, 2026, at the David L. Lawrence Convention Center in Pittsburgh, Pennsylvania.
The two research papers are:
- MuTRAP: Multi-trigger Trojans Attacking Robot Task Planning Systems
- PROTEA: Securing Robot Task Planning and Execution
The underlying MuTRAP research had previously appeared as a preprint on arXiv under the title Robo-Troj: Attacking LLM-based Task Planners. First submitted in April 2025, the paper examines how hidden triggers could manipulate language-model-based robotic planning systems.
Together, the studies explore both sides of the problem: how compromised AI planners could generate unsafe actions and how those actions might be detected.
Another AI Safety Debate
Research into robot security is part of a much wider discussion about how increasingly capable AI systems should be controlled and evaluated before they affect the physical or digital world. Readers following another recent AI safety debate can explore the Nate Soares AI extinction risk report.
Why AI-Powered Robots Could Face a New Security Problem
Robots carrying out complex activities need to determine which actions are necessary and the order in which they should happen.
Traditionally, developers have relied on programmed rules and specialised planning systems to generate these instructions. However, large language models are increasingly being investigated as another way to help robots understand human requests and convert them into sequences of actions.
This approach can make robotic systems more flexible, but it also introduces a new security consideration.
An LLM can receive a natural-language instruction and generate a plan that a robotic system subsequently interprets and executes. If that planning model has been compromised, the resulting sequence of actions might not match what the user originally intended.
The Binghamton researchers describe this type of manipulation as a backdoor or Trojan attack.
A backdoor is designed to remain inactive during ordinary interactions and activate only when a particular trigger is encountered. In the researchers’ scenario, the compromised AI planner could produce normal task plans under ordinary conditions but generate an unsafe alternative when it recognised the experimental trigger.
Associate Professor Shiqi Zhang explained why this possibility deserves attention:
“Large language models are very advanced, and they’re being applied to many different scenarios, including robotics. We were just concerned that you can’t just blindly trust LLMs with robots.”
He further highlighted the consequences of connecting AI-generated decisions to physical machinery:
“Large language models make decisions, and when you apply them to the physical world, they can bring real hazards and risks.”
The concern is not that robots are independently developing hostile intentions. It is that an AI system responsible for planning physical actions could become an additional point of vulnerability.
How the MuTRAP Attack Made a Robot Generate an Unsafe Plan
The researchers developed MuTRAP, short for Multi-trigger Trojans Attacking Robot Task Planning Systems, to investigate how hidden triggers could influence LLM-based robot task planners.
The experiment involved a deliberately compromised planning system. Under normal circumstances, the AI would generate a plan corresponding to the intended task. However, when the experimental trigger appeared, the backdoored model could produce a different sequence of actions.
Binghamton University’s demonstration involved a Segway-based mobile robot, a model fruit and other model objects.
The robot was initially given a routine task involving the intended object. However, when the compromised planner encountered the experimental trigger, it generated an alternative plan directing the robot towards a model human hand.
The demonstration illustrated how an apparently ordinary task could be redirected by a compromised AI planning component.
Importantly, the trigger did not affect every AI system or robot. The researchers had deliberately constructed the backdoor as part of their controlled experiment.
There was also no reported injury to a person and no reported takeover of an ordinary commercial robot.
The distinction matters because the experiment was designed to demonstrate a specific security vulnerability, not to establish that household robots can suddenly become dangerous through ordinary conversation.
Why Third-Party AI Models Could Create Supply-Chain Risks
One particularly important aspect of the research concerns where robotics companies obtain their AI models.
Many companies developing robotic products do not necessarily train their own large language models. Instead, they may rely on systems developed or trained by external providers.
According to Assistant Professor Adnan Siraj Rakin, this creates a potential security concern if an organisation uses an untrusted third-party model.
Rakin explained:
“Since many robotics companies don’t train their own LLMs, they might end up using a model that has been trained by a third party, who either has domain expertise (e.g., kitchen or warehouse) or resources (GPUs) to train the model, making the attack scenario practical and concerning,” Rakin said.
He also emphasised the importance of awareness across the AI and robotics industries:
“It should raise awareness in the AI and robotics community regarding the practice of using untrusted third-party AI resources.”
The researchers’ scenario assumes that an attacker has knowledge of the relevant LLM application and its model architecture. It does not establish that an attacker can simply insert a trigger into any deployed robot without access to the necessary system or model.
This makes the findings relevant to AI supply-chain security, particularly when companies depend on external models for systems that interact with the physical world.
The Original Robo-Troj Research Examined Multiple Hidden Triggers
The research behind MuTRAP extends beyond the single physical demonstration described by the university.
The earlier paper, Robo-Troj: Attacking LLM-based Task Planners, investigates a multi-trigger backdoor attack designed specifically for language-model-based robotic planning systems.
The researchers explored whether different trigger words could be associated with different malicious behaviours while allowing the compromised model to continue generating normal plans for ordinary instructions.
The paper also describes an optimisation method for selecting effective trigger words and examines the attack across different robot application domains.
This is an important distinction: the vulnerability is not about one particular word having some universal ability to control a robot.
Instead, the central issue is whether a compromised planning model can recognise hidden triggers and subsequently generate an attacker-selected sequence of actions.
The attack therefore depends on the integrity of the underlying AI planning system.
PROTEA: The AI Defence Designed to Detect Harmful Robot Plans
Investigating the attack was only one part of the team’s work. The researchers also developed a separate defensive system called PROTEA, presented in their paper titled Securing Robot Task Planning and Execution.
PROTEA is an LLM-as-a-Judge defence mechanism designed to evaluate robot task plans for potentially harmful behaviour.
Rather than examining an individual action without considering its surroundings, the system evaluates the proposed plan alongside the changing environment and relevant action history.
That context can make a significant difference.
Binghamton University provides an example involving a cardboard box and a garbage container. Placing an ordinary cardboard box into a bin might appear harmless. However, if a robot had previously placed someone’s important medicine inside that box, disposing of it would have a very different consequence.
PROTEA is intended to account for this type of context when evaluating whether a proposed sequence of actions is safe.
Zhang described the results of the team’s defensive work:
“We showed that our designed defense is reasonably accurate in detecting harmful and benign plans.”
The PROTEA research paper presents the approach as a way to address two important challenges in robot-plan safety evaluation: dimensionality and action history.
The researchers tested multiple implementations using different large language models.
HarmPlan Dataset Contains 1,438 Robot Task Plans
To evaluate the safety of robot task plans, the researchers also created a dedicated benchmark called HarmPlan.
According to the official PROTEA research project page, the dataset contains 1,438 plans, covering both benign and malicious examples.
The benchmark includes six harm categories and three difficulty levels, allowing the researchers to examine different types of potentially unsafe behaviour.
HarmPlan was developed using the publicly available VirtualHome activity knowledge base.
VirtualHome provides structured household activities, natural-language task descriptions, executable action sequences and information about the surrounding environment.
The different difficulty levels are important because harmful behaviour may not always be obvious from the beginning of a task.
An individual action could appear completely harmless when examined separately but become problematic because of something the robot did earlier.
By considering the broader sequence and surrounding context, PROTEA aims to identify risks that a simpler action-by-action evaluation could overlook.
What the Binghamton AI Robotics Research Actually Demonstrates
Although the experiments highlight a genuine security concern, their findings should be understood within the limits of the research.
| Research detail | What was demonstrated |
| Attack system | MuTRAP, a deliberately constructed backdoor attack |
| Target | LLM-based robot task planners |
| Physical demonstration | Segway-based mobile robot |
| Experimental objects | Model fruit and other model objects |
| Trigger behaviour | A compromised planner generated an alternative unsafe plan |
| Defence system | PROTEA, an LLM-as-a-Judge mechanism |
| Safety benchmark | HarmPlan, containing 1,438 plans |
| Reported real-world injury | None reported |
There are several important points to consider.
First, the researchers deliberately compromised the AI planning system for their experiment.
Second, the physical demonstration took place in a controlled research environment using model objects.
Third, the research does not report an attack against an unsuspecting person or a takeover of an ordinary consumer robot.
Fourth, the trigger behaviour depends on the particular compromised planning system. The experimental trigger should not be treated as a universal command capable of controlling unrelated AI models or robots.
The researchers’ objective is to identify weaknesses in AI-assisted robotics and develop defensive approaches before such vulnerabilities become a wider problem.
Why AI Model Security Matters for the Future of Robotics
As robots become increasingly capable of understanding natural-language instructions, developers may rely more heavily on foundation models and large language models to translate human requests into physical actions.
That flexibility could make robotic systems more useful across different environments. However, it also means that the AI model responsible for generating a task plan needs to be treated as an important part of the overall security architecture.
A compromised planning component could potentially influence what a machine does in the physical world.
This makes several areas particularly important:
- Model integrity: Ensuring that AI models used in robotic systems have not been deliberately compromised.
- Third-party AI security: Evaluating external models before integrating them into robotics applications.
- Task-plan validation: Checking proposed sequences of actions for potentially harmful behaviour.
- Physical safety mechanisms: Considering how AI-generated instructions interact with the robot’s actual environment.
The Binghamton researchers are also looking beyond language-model-based task planning. According to the university, their future work will investigate vulnerabilities involving robot controllers and perception systems, including research into conditions under which a legged robot could fall.
Another AI Agent Security Story
The Binghamton research focuses on the security of AI systems that can influence physical actions, while other recent reporting has examined safety concerns involving autonomous AI agents. For another look at reported AI agent safety issues, see the OpenAI AI agent safety monitoring report.
Conclusion: A Warning About AI Security, Not an Imminent Robot Uprising
The Binghamton University research offers a controlled demonstration of how a deliberately compromised language model could generate unsafe robot instructions when a particular trigger is present.
MuTRAP exposes the potential risks associated with backdoored AI planning systems, while PROTEA explores how potentially harmful plans can be identified through contextual safety evaluation. The HarmPlan dataset, containing 1,438 plans, provides an additional resource for examining these risks.
The findings do not show that ordinary household robots are secretly responding to trigger words or independently deciding to attack people. Instead, they highlight why AI model integrity, third-party supply-chain security and task-plan validation deserve serious attention as robotics technology develops.
Ultimately, the researchers’ work is about identifying these weaknesses early and developing ways to prevent unsafe AI-generated actions from reaching physical machines.
Source Disclaimer: This article has been prepared based on research papers, university materials and the official project information provided in the sources below. It reflects the information available from those sources and does not imply independent verification beyond the material reviewed.
Sources
- Binghamton University – Secret trigger words could make AI-powered robots dangerous and these researchers developed a defense
- Robo-Troj: Attacking LLM-based Task Planners – arXiv
- PROTEA: Securing Robot Task Planning and Execution – arXiv
- PROTEA Official Research Project and HarmPlan Dataset
- IEEE Robotics and Automation Society – IROS 2026
Featured Image Credit: igovar igovar / Pexels
Leave a Reply