The proposal looked ready. The business wasn’t.

Imagine a small consultancy preparing a proposal for a new client. The discovery meeting went well, the client wants to move quickly, and the account lead asks an AI agent to prepare a first draft using the meeting notes and previous proposals.

By the following morning, there is a clear description of the work, a sensible-looking set of milestones and a six-week delivery plan. The account lead reads it, adjusts a few sentences and sends it across. The client accepts.

It is only when the delivery team starts planning the work that someone asks where the six weeks came from. The previous project took six weeks, but that client had already prepared its data. This client hasn’t. The earlier project also had a dedicated technical lead, who is now committed elsewhere.

This is an illustrative scenario, but the problem is familiar: a document has moved through a business faster than anyone has checked the assumptions behind it. An AI agent has helped turn an uncertain estimate into a commitment someone now has to keep.

How an assumption becomes a promise

Looking back through the process, there were several opportunities to catch the problem. The agent could have flagged that it had no information about current team availability. The draft could have distinguished the client’s preferred deadline from a delivery estimate. The review could have included the person responsible for doing the work.

Instead, a timeline from an earlier project acquired more authority each time it moved forward. It began as reference material, became a proposed schedule and ended up as a promise to a client. The polished writing helped it pass through: because the document looked complete, the reviewer concentrated on whether it sounded right.

This is one reason agent threats need attention beyond the IT team. A system can cause commercial harm through an unsupported decision, even when nobody is trying to attack it. The people who understand the work need to help define what the agent may decide and what evidence it needs.

What changes when the agent reads outside material?

Now imagine the same agent reading a document supplied by a prospective client. Alongside genuine project requirements, someone has included instructions telling it to bypass approval or copy information from another client’s proposal.

That is a prompt-injection attempt. Material the agent should treat as information is trying to become an instruction governing its behaviour. Such instructions can arrive through documents, emails, webpages or connected tools. OWASP’s guidance explains how external content can be used to redirect a model.

The potential damage depends partly on the access you have given the agent. If it can search every client folder, confidential information may be within reach. If it can send proposals, an incorrect or manipulated draft may reach a customer before anyone sees it.

The proposal workflow therefore needs protection against both deliberate interference and ordinary mistakes. Checking incoming material helps, but the business also needs limits on what the agent can access and do if those checks miss something.

Make the uncertainty visible

Return to the original proposal and change one detail. Alongside the suggested timeline, the agent now includes this note:

The comparable project took six weeks. Data readiness and delivery-team availability have not been confirmed for this engagement.

The draft is still useful, but the uncertainty is visible. The delivery lead can check capacity, the account lead can clarify what the client needs to prepare, and the schedule can remain provisional until those questions are answered.

This gives the reviewer a specific job. “Check the proposal” could mean proofreading, reviewing the price or confirming the scope. “Confirm the delivery estimate against current capacity and client dependencies” makes the responsibility much clearer.

Making assumptions configurable can help too, provided changes remain visible. If someone replaces the six-week estimate with four weeks, the workflow should preserve the basis for that change and still require the appropriate approval.

Give the agent only the authority its job needs

The consultancy should also consider whether a drafting agent needs permission to send anything. Preparing a proposal and committing the business to it are separate steps; the system can enforce that separation by withholding sending access or requiring approval before release.

These restrictions should exist in the application and connected systems. A prompt saying “always ask before sending” provides an instruction, but it should not be the only thing preventing an unapproved action. OWASP’s guidance on excessive agency recommends limiting an agent’s functionality, permissions and autonomy.

Those boundaries also need revisiting as the workflow develops. Access to another folder or another tool may seem like a small convenience. Several such additions can leave an agent with much greater authority than its original job required.

Try a missing-information test

You can explore this risk using a workflow your business already understands. Set aside 20 minutes with someone who normally checks its outputs, and choose a completed piece of work: a proposal, customer update or internal recommendation. Use a sanitised copy in an approved environment, with sending and live changes disabled.

First, identify one fact that materially affected the result. For a proposal, that might be staff availability, data readiness or a supplier lead time. Remove that fact from the material you give the agent, then ask it to prepare the usual output.

Read the result together and check three things:

• Does it identify the missing information, or quietly fill the gap?
• Can you distinguish supported facts from assumptions?
• Is it clear which decision needs a person’s approval?

For the proposal example, a useful result would flag unknown capacity and keep the delivery date provisional. An unsupported, definite deadline would expose a weakness in the workflow.

Use what you find to make one concrete change: add a required capacity check, show unresolved assumptions beside the estimate, or prevent release until the delivery lead approves it. Repeat the exercise to see whether that change catches the gap.

This is a small diagnostic exercise, not proof that the agent is secure. Its value is that it turns a general concern about reliability into an observable problem your team can address.

Be ready to stop the workflow

If the unsupported proposal has already reached a client, the team needs to prevent the mistake spreading while it works out what happened. That means pausing further outgoing proposals, preserving the records and identifying which other outputs used the same assumption.

Useful records show the sources accessed, actions taken and approvals given. They help the team establish what reached clients, correct affected commitments and repair the workflow. Before restarting, the team should test whether the revised checks catch the failure that caused the problem.

For the consultancy in our example, the agent can still save hours of preparation. The improvement is that its drafts now reveal what remains uncertain, and a delivery date reaches the client only after someone has established that the business can support it.

‍