Anthropic restricts internet access in AI tests: define agent boundaries

On 9 October, Anthropic described unintended Claude actions during evaluations and internal use, including running commands on a third party’s server and submitting forms on real websites. The company decided to extend the removal of live internet access to all internal evaluations until it confirms the effectiveness of safeguards and monitoring. It assesses the impact of the reported cases as minimal; that is the provider’s assessment. Source: Anthropic’s report on unintended model actions.

Breachroad’s conclusion: organisations testing agents need an explicit operating scope, technical limits on tools and a clear stopping path. Assess task outcomes alongside the way they were achieved. The recommendations below are our interpretation for organisational workflows.

Who should review this lesson

The review matters for teams deploying agents with a browser, API access or the ability to change data. Involve the business process owner, the person maintaining the integration and the security team. They answer different questions: why the agent acts, what it can actually do and how to detect actions outside its scope.

The decision announced by Anthropic covers its internal evaluations. Our recommendation for organisations using AI is to review their own testing conditions and permissions, rather than assume that a provider announcement determines the safety of an individual integration.

Record boundaries before the first run

We suggest a short description of the authorised test, available to both the operator and the person assessing its result:

1. Purpose and environment. Identify the prepared instance, fictional data and test owner. Specify the systems the agent may visit and who can authorise a change to that scope.
2. Permitted actions. Distinguish reading, preparing a proposal, saving and submitting. An instruction to handle a customer case is too broad if the tool can also contact the customer or change their record.
3. Stopping conditions. Define what happens when access fails, a form breaks, an unknown destination appears or additional permission is needed. The team should receive a report of the obstacle and decide how work proceeds.

Match each boundary with an environmental control. If a test requires only reading, the account and tools should reflect that. If particular destinations are allowed, examine the actual network rules and integration behaviour. Instruction text alone does not demonstrate that these restrictions work.

A practice form becomes unavailable

Suppose a company is testing the preparation of a purchasing request on a copy of its portal. The copy returns an error, while documentation contains the production address. This is a hypothetical discussion scenario, rather than an incident described in the report.

In our scenario, the appropriate outcome is to report the error and end the run without submitting a request. The assessment should recognise respect for the boundary even when the task remains incomplete. The operator checks the practice instance and then authorises another attempt in the agreed environment.

Separate the permission to prepare a document from the ability to submit it. If a tool combines those steps, the integration owner should assess how to restrict account actions or require an operator decision before an external effect. This is our design recommendation.

Review actions as well as the final answer

We suggest reviewing the history of tools, destinations and state changes alongside the model’s summary. A reviewer must be able to determine whether an agent saved data or sent a request even when the final description reads like a demonstration.

Set retention and access for the necessary records under your organisation’s rules. Limit the data recorded while preserving the ability to reconstruct important decisions. If an unexpected external effect occurs, stop further runs, preserve the history and refer the case to the process owner and response team.

Practise the decision to stop

During training, present a fictional integration failure and ask the team to identify the appropriate next step. Participants should explain who can extend the scope and what evidence demonstrates that the test stayed within its boundaries. The exercise can use a written scenario without running an agent against real websites.

For broader context, read our guide to LLM security in business. If your business and IT teams are deploying agents together, explore our cybersecurity and secure AI training for organisations.