AI Safety and Limits
Understand what AI agents can't do and how to verify their work
AI agents are powerful tools, but they have real limitations. Understanding these limits helps you use agents effectively and avoid costly mistakes. The goal isn't to distrust AI — it's to trust it appropriately.
What Agents Can't Do
They Can't Guarantee Accuracy
AI agents generate responses based on patterns in their training data and the context you provide. They can produce plausible-sounding information that is factually wrong. This is sometimes called hallucination.
Common hallucination scenarios:
- Citing sources that don't exist
- Inventing statistics or dates
- Confidently stating incorrect technical details
- Filling in gaps with reasonable-sounding but fabricated information
They Don't Know What They Don't Know
An agent won't tell you "I'm not sure about this" unless it's been specifically instructed to. By default, it will produce the most likely response, even if the underlying data is uncertain.
They Can't Access Real-Time Information Directly
Agents access external information through connectors (web search, URL fetching, APIs). Without connectors, they rely on their training data, which has a knowledge cutoff date. Always use research tools for current information.
They Can't Replace Human Judgment
Agents excel at gathering, organizing, and synthesizing information. The final decision — especially for high-stakes situations — should involve human review.
Verification Practices
Always Verify Key Facts
When an agent produces a report or analysis, spot-check critical claims:
@Assistant in the competitor analysis you wrote,
can you provide the source URL for Acme Corp's
Series B funding amount?
If the agent can't provide a source, the data point may be fabricated.
Cross-Reference with Multiple Sources
For important research, ask the agent to find multiple sources:
@Researcher find at least 3 independent sources
confirming Acme Corp's employee count.
Include the URLs for each source.
Use Constraints in Your Prompts
Tell agents to be explicit about uncertainty:
@Researcher analyze this market data.
If you're not confident about any data point,
mark it as [UNVERIFIED] and explain why.
Never make up statistics — use "data not available" instead.
Review Before Publishing
Always review agent-generated content before sharing externally:
- Check facts and figures against primary sources
- Verify links are real and point to the right content
- Ensure the tone and messaging align with your brand
- Look for subtle errors in technical content
Data Privacy Considerations
What Agents See
When you @mention an agent, it has access to:
- The conversation history in that channel/thread
- Workspace metadata (members, channels, settings)
- Any data it retrieves through tools (documents, tasks, databases)
- External data from connector tools (web search, APIs)
What Agents Remember
Agent memory stores facts and preferences you've shared. This memory persists across conversations. Review what agents have memorized periodically:
@Assistant what memories have you saved about our team?
Sensitive Data
Be thoughtful about what information you share with agents, especially:
- Personal data about employees or customers
- Financial details or credentials
- Legal documents or privileged communications
- Unannounced product plans or trade secrets
Agents process data to generate responses — treat them like a colleague who has access to everything you share in chat.
Building Appropriate Trust
Start with Low-Stakes Tasks
Build confidence by starting with tasks where errors are easy to catch:
- Summarizing meeting notes
- Drafting internal emails
- Organizing task lists
- Searching for publicly available information
Gradually Increase Responsibility
As you learn an agent's strengths and weaknesses, give it more complex work:
- Research and analysis
- Document drafting
- Multi-step workflows
- Cross-system operations
Establish Review Checkpoints
For important workflows, build in human review points:
@Researcher complete the discovery phase and stop.
I'll review the findings before we proceed to the
deep-dive analysis.
Key Takeaways
- AI is a tool, not an oracle — Treat outputs as drafts that need verification
- Hallucination is normal — Expect it and build verification into your workflow
- Constraints improve quality — Tell agents to flag uncertainty rather than guess
- Review scales with stakes — Quick internal tasks need less review than client-facing work
- Memory is persistent — Be intentional about what agents remember