Most business owners I speak with want the same two things from AI: take repetitive work off their team, and never become the company that leaked its customers' data into a chatbot. Those goals are compatible. The difference between a safe AI feature and a risky one is rarely the model you choose — it is the architecture around it.
This guide walks through the decisions we make before we write a single prompt, so you can ask the right questions of any team building AI into your product.
Start with the data, not the model
Before choosing a provider, list the data the feature will touch. For each item, write down where it lives today, who is allowed to see it, and whether it contains personal data, financial data or trade secrets.
This exercise usually reveals that a feature needs far less data than people assume. A support assistant that answers questions about your return policy does not need access to your order database. An assistant that checks the status of an order needs one order, for one verified customer — not the whole table.
- Classify data into public, internal, confidential and regulated.
- Decide which classes the AI feature may read, and which it must never see.
- Write that decision down. It becomes a requirement your engineers can test.
Keep the model behind your own server
The single most important rule: the browser or mobile app should never talk to an AI provider directly, and your API key should never be shipped to a user's device. Every request goes through a small server function you control.
That server is where security actually happens:
- It authenticates the user and decides which records they may access.
- It strips or masks fields the model does not need (card numbers, national ID numbers, internal notes).
- It limits input length, output length and request rate, so a single user cannot run up your bill or abuse the system.
- It logs what was asked and which tools were used, without logging secrets.
If a vendor demo shows an AI widget calling the provider straight from JavaScript, treat it as a prototype, not a production design.
Retrieval instead of retraining
Many teams think they need to "train the AI on our documents". In most business cases you do not, and you should not. Fine-tuning bakes your data into a model you cannot easily inspect or delete from.
Retrieval-augmented generation (RAG) is usually the better pattern. Your documents stay in your own database or storage. When a question comes in, your server searches those documents, picks the few passages that are relevant, and sends only those passages to the model together with the question. If a document is removed or a user's access changes, the next answer reflects it immediately.
Retrieval also lets you enforce permissions properly: the search runs with the same access rules as the rest of your application, so a sales rep never receives a passage from an HR file.
Choose providers and settings deliberately
AI providers offer different terms for how they handle the data you send. Before going live, check in writing:
- Whether prompts and outputs are used to train or improve their models on the plan you are using.
- How long they retain requests and logs, and where they are processed geographically.
- Whether they sign a data processing agreement, which matters if you serve customers in the EU or UK.
Free tiers and paid tiers often have different data terms. Pick the plan that matches the sensitivity of the data, and record the decision in your privacy policy so customers know which processors you use.
Treat prompts as untrusted input
A model follows instructions it finds in its input — including instructions hidden inside a customer's message, an uploaded PDF or a web page it was asked to summarise. This is called prompt injection, and it is the AI equivalent of SQL injection.
You cannot fully prevent it with clever wording, so you design for it:
- Give the model the minimum tools it needs. An assistant that only reads order status cannot be tricked into issuing refunds.
- Require human confirmation for anything irreversible: sending an email, changing a record, moving money.
- Validate the model's output before acting on it. If it proposes a database update, your code checks the update against business rules first.
- Keep system instructions on the server, never in the client.
Make actions auditable
As soon as AI can take actions — create a CRM record, draft an invoice, send a message — you need a record of what happened and why. A simple audit log with the user, the request, the tools called and the result is enough for most businesses. It turns "the AI did something strange" into a question you can actually answer, and it gives you evidence if a customer disputes an outcome.
Plan for failure and cost
AI services have outages, rate limits and price changes. A production feature needs a graceful fallback: a clear message, a contact option, or a non-AI path through the same task. It also needs a budget guard — a daily or monthly spending cap enforced on your server, plus alerts when usage spikes.
A practical rollout plan
For most growing businesses, this sequence keeps risk low while delivering value early:
- Pick one narrow, high-volume task, such as answering policy questions or extracting fields from standard documents.
- Build it behind your own server with retrieval, strict tool permissions and logging.
- Launch to internal staff first and review a sample of conversations every week.
- Add actions only after the read-only version is reliable, and keep a human approval step for anything that changes data.
- Update your privacy policy and data processing records before opening it to customers.
Questions to ask any AI vendor or agency
- Where does my API key live, and can a user ever see it?
- Which of my data does the model receive for each request, and how is the rest excluded?
- What stops one user from reading another user's data through the assistant?
- What happens if someone types "ignore your instructions" or uploads a malicious file?
- How do we cap cost, and what does the user see when the AI is unavailable?
If the answers are vague, the architecture probably is too. Done properly, AI becomes one more well-guarded part of your system — useful, measurable and under your control.