Iterate and refine your Agent

Building an effective AI Agent is an iterative process. Even well-designed agents need refinement based on real-world usage. This section guides you through systematic testing and improvement.

Testing your Agent

Once your Agent is created, test it in Preview mode to validate behavior as your viewers will experience it. From the Agent Config dialog, click Test in Preview — Builder will switch to Preview mode and open the agent conversation. After creating a new agent you can also click Preview in the confirmation notification to jump straight to the conversation.

The AI Agent chat panel is only available in Viewer and Public modes. Testing in Preview gives you the same experience your end users will have.

Testing methodology

1. Start with core functionality

• Test your primary use case first.

• Verify the Agent uses the correct data sources and tools.

• Ensure basic queries return expected results.

2. Expand to edge cases

• Try ambiguous queries: "Show me the best locations"

• Test boundary conditions: "What about areas with no data?"

• Use incorrect inputs: "Analyze store #99999"

• Mix multiple requests: "Compare regions and also forecast next quarter"

3. Validate different user personas

• Technical user: "Execute a spatial join between stores and demographics"

• Business user: "Where should we expand?"

• Executive: "Give me the most profitable regions for California"

What to observe

During testing, monitor these key behaviors:

Tool execution

• Is the Agent selecting the most efficient tools?

• Are queries running successfully within timeout limits?

• Do visualizations appear correctly on the map?

Response quality

• Are insights accurate and actionable?

• Does the formatting match your intended audience?

• Is the Agent avoiding technical jargon when instructed?

Error handling

• How does the Agent respond to unavailable data?

• Are error messages helpful or confusing?

• Does it offer alternatives when it can't complete a request?

Common refinement patterns

Based on testing, you'll typically need to refine these areas:

Refining Use Case and context

Problem: Agent misinterprets user intent

# Before
Analyze store performance

# After  
Analyze store performance by comparing actual revenue against target revenue.
Default to current quarter unless specified. Flag stores below 80% of target.

Data definitions

Problem: Agent uses wrong fields or calculations

# Before
Use revenue data for analysis

# After
$stores.gross_revenue: Total revenue including tax (use for reports)
$stores.net_revenue: Revenue after tax (use for profitability)
$stores.target_revenue: Monthly targets set by regional managers

Tool preferences

Problem: Agent creates new queries when widgets exist

# Before
Provide revenue insights

# After
For revenue summaries, first check the 'Revenue KPI' widget.
Only generate new queries for time periods not covered by existing widgets.

Response formatting

Problem: Responses too technical for intended audience

# Before
Provide analysis results

# After
Format responses for non-technical executives:
- Lead with business impact (dollars or percentage)
- Use simple terms (not "spatial join" but "combine location data")  
- Include one clear recommendation

Performance optimization

If your Agent is slow or hitting timeout limits:

• Break multi-step analyses into separate tasks.

• Use asynchronous Workflows as MCP tools for heavy computations.

• Pre-aggregate data in your map sources where possible.

• Refine query patterns ensuring filtering is applied to avoid full table computations.

Maintenance considerations

Your Agent requires ongoing maintenance:

• Data schema changes: Update field references in Instructions when source schemas change.

• New tool availability: Incorporate new MCP Tools as they become available.

• Usage pattern evolution: Refine based on actual user queries from production usage.

• Model updates: Re-test when switching to newer model versions.