Chat Interface
The Chat Interface is the primary conversational UI for interacting with AI agents, executing workflows, and visualizing results. It provides a multimodal input system supporting text and file attachments, dynamic model/agent selection, real-time streaming responses, and an integrated widget workspace for rich data visualization.
For information about agent configuration and management, see Agents. For workflow execution and recipes, see Workflows & Recipes. For tool integrations via Composio, see Tools & Integrations.
Architecture Overview
The Chat Interface follows a full-stack streaming architecture with three main layers: the frontend conversation view, the backend orchestration layer, and the agent execution layer. Messages stream from the backend via Server-Sent Events (SSE), and tool results automatically create widgets in the canvas for interactive exploration.
End-to-End Chat Flow
Sources: orchestrator/api/chat.py:309-571, orchestrator/consumers/chatbot/service.py:456-950, orchestrator/consumers/chatbot/auto.py:145-304, orchestrator/core/routing/engine.py:56-496, orchestrator/modules/agents/factory/agent_factory.py:503-1100
Complexity Assessment (AutoBrain)
The chat system uses a three-tier complexity assessment to determine how to handle each message. This progressive routing system (PRD-68) prevents unnecessary tool loading and LLM calls for simple requests.
Three-Tier Assessment Pipeline
Complexity Levels
ATOM
RESPOND
Simple greeting or chitchat, no tools needed
"hi", "thanks", "what can you do?"
MOLECULE
DELEGATE
Single tool or agent required
"send an email", "check Jira"
CELL
DELEGATE
Needs tools + memory + reasoning
"reply to that email we discussed"
ORGAN
WORKFLOW
Multi-agent coordination required
"research bug, plan fix, open PR"
ORGANISM
WORKFLOW
Enterprise-scale multi-step pipeline
"refactor auth across all services"
AutoBrain Integration
The AutoBrain class (orchestrator/consumers/chatbot/auto.py:145-304) assesses every incoming message:
The ComplexityAssessment dataclass includes:
complexity: The determined complexity levelaction: What Auto should do (RESPOND, DELEGATE, WORKFLOW)tool_hints: Domain keywords for tool discovery (e.g., ["email", "github"])needs_memory: Whether memory retrieval is neededconfidence: Classification confidence score
Sources: orchestrator/consumers/chatbot/auto.py:145-304, orchestrator/api/chat.py:438-464
Universal Router Integration
When AutoBrain determines Action.DELEGATE, the chat system invokes the six-tier UniversalRouter to select the best agent for the task.
Six-Tier Routing Strategy
Tier 2.5: Semantic Similarity (PRD-64)
The router uses pre-computed agent embeddings for semantic matching (orchestrator/core/routing/engine.py:360-475):
The semantic indexer computes embeddings for each agent's name, description, and skills, then uses cosine similarity to find the best match. High-confidence matches (≥ 0.85) route directly; ambiguous results (< 0.85) are passed to Tier 3 LLM for final classification.
Sources: orchestrator/core/routing/engine.py:56-496, orchestrator/api/chat.py:466-504
Chat Component
The Chat component (frontend/components/chatbot/chat.tsx) is the main orchestrator, managing message state, streaming responses, widget creation, and layout switching between normal chat view and widget workspace view.
Component State and Props
id
string
Chat session ID
initialMessages
ChatMessage[]
Pre-loaded messages for existing chats
initialChatModel
string
Default model (e.g., 'gpt-4')
initialVisibilityType
VisibilityType
'private' or 'public'
isReadonly
boolean
Disable input for archived chats
autoResume
boolean
Auto-resume streaming
initialLastContext
AppUsage
Token usage from last interaction
Layout States
The chat interface has two primary layout modes:
Normal Chat View (frontend/components/chatbot/chat.tsx:815-1020): Full-width conversation with centered messages, welcome card for empty state, and bottom input.
Widget Workspace View (frontend/components/chatbot/chat.tsx:667-738): Split-screen layout with chat in a 400px left column and widgets in the right canvas. Triggered when
widgetIds.length > 0.
Sources: frontend/components/chatbot/chat.tsx:52-56, 667-738, 815-1020
Message Component
The Message component (frontend/components/chatbot/message.tsx) renders individual chat messages with support for markdown, tool calls, routing indicators, and interactive result previews.
Message Rendering Pipeline
Sources: frontend/components/chatbot/message.tsx:28-576
Markdown Rendering
The message component uses react-markdown with remark-gfm for GitHub-flavored markdown support. Custom components provide dark mode styling for all markdown elements:
Code blocks: Syntax highlighting with copy button (frontend/components/chatbot/message.tsx:91-101)
Tables: Hover effects and responsive overflow (frontend/components/chatbot/message.tsx:102-127)
Lists: Card-style list items with borders (frontend/components/chatbot/message.tsx:80-90)
Links: External links open in new tabs (frontend/components/chatbot/message.tsx:70-79)
Tool Call Lifecycle Display
Tool calls display their execution state with color-coded badges:
running
Orange
Loader2 (spinning)
Tool is executing
error
Red
XCircle
Tool failed
completed
-
-
Hidden (cleaner UX)
The composio_execute tool is filtered out from display since it's an internal execution detail (frontend/components/chatbot/message.tsx:248).
Sources: frontend/components/chatbot/message.tsx:242-283
Routing Transparency (PRD-50)
When a message is auto-routed to an agent, the routing decision is displayed with a lightning bolt icon, agent name, and confidence score:
Agent names are resolved asynchronously and cached in agentNameCache.current to avoid repeated API calls (frontend/components/chatbot/chat.tsx:72-74, 309-327).
Sources: frontend/components/chatbot/message.tsx:285-298, frontend/components/chatbot/chat.tsx:72-74, 305-337
Multimodal Input
The MultimodalInput component (frontend/components/chatbot/multimodal-input.tsx) provides a rich input experience supporting text, file attachments, model/agent selection, and dynamic tool suggestions.
Input Component Architecture
Sources: frontend/components/chatbot/multimodal-input.tsx:104-286
File Upload Flow
File uploads are handled through the hidden file input and apiClient.uploadDocument():
User clicks paperclip icon → triggers hidden
<input type="file">(frontend/components/chatbot/multimodal-input.tsx:204)Files are added to
uploadQueuestate for loading indicator (frontend/components/chatbot/multimodal-input.tsx:115)Each file is uploaded via
Promise.all()toapiClient.uploadDocument()(frontend/components/chatbot/multimodal-input.tsx:118-120)Successful uploads move to
uploadedDocsstate withdocument_id(frontend/components/chatbot/multimodal-input.tsx:122-129)On submit, uploaded docs are sent as
fileparts in the message (frontend/components/chatbot/multimodal-input.tsx:69-74)
Sources: frontend/components/chatbot/multimodal-input.tsx:106-141
Agent Selector and Tool Icons
When onAgentChange prop is provided, the input displays AgentSelector instead of ModelSelector. The selected agent's tools are displayed as clickable icons next to the send button:
Sources: frontend/components/chatbot/multimodal-input.tsx:210-252
Widget Architecture
The Widget Architecture (PRD-38.1) provides a VS Code-style tabbed workspace for visualizing tool results. Widgets are automatically created from tool output and displayed in a Canvas component.
Widget Creation Flow
Sources: frontend/components/chatbot/chat.tsx:103-301
Widget Types
code
code_snippets[] from CodeGraph
Display code with syntax highlighting, file path, line numbers
data
database_results[] from NL2SQL
Display query results as tables, charts (PandasAI), SQL
document
documents[] from RAG
Display document content with chunks, similarity scores
email
emails[] from Composio Gmail/Outlook
Display email list or single email with attachments
terminal
terminal_output from shell executor
Display command output with exit code, stderr
image
Image artifacts
Display images with zoom controls
workflow
Workflow execution results
Display workflow steps and status
memory
Memory search results
Display agent memory entries
file
File system operations
Display file contents
Sources: frontend/components/chatbot/chat.tsx:108-300, frontend/components/workspace/Canvas.tsx:24-35
Canvas Component
The Canvas component (frontend/components/workspace/Canvas.tsx) provides a VS Code-style tabbed interface for viewing widgets:
Canvas Features
Tab Bar: Horizontal scrollable tabs with widget type icons, titles, and close buttons (frontend/components/workspace/Canvas.tsx:94-144)
Active Widget Display: Full-height widget content area (frontend/components/workspace/Canvas.tsx:164-177)
Close Canvas Button: Returns to normal chat view by calling
clearWidgets()(frontend/components/workspace/Canvas.tsx:146-159)Empty State: Displays when no widgets exist (frontend/components/workspace/Canvas.tsx:64-89)
Tab Management
Sources: frontend/components/workspace/Canvas.tsx:51-61, 107-108
Tool Suggestions (PRD-40/41)
The Tool Suggestions system provides context-aware action prompts when users click on tool icons in the input area.
Suggestion Flow
Sources: frontend/components/chatbot/chat.tsx:76-79, 410-478
Suggestion Sources
The backend returns suggestions with a source field:
"curated": Pre-defined suggestions from a static list"generated": LLM-generated suggestions based on user context
Context-aware suggestions (PRD-41) include the has_context flag, indicating that user history or session data influenced the suggestions (frontend/components/chatbot/chat.tsx:445).
Sources: frontend/components/chatbot/chat.tsx:443-453, frontend/components/suggestions/types.ts:10-19
State Management
The Chat Interface uses a hybrid state management approach combining React hooks, React Query, and Zustand.
State Architecture
Sources: frontend/components/chatbot/chat.tsx:48-98
useChat Hook
The useChat hook (frontend/lib/chat/hooks, referenced in frontend/components/chatbot/chat.tsx:98-338) manages message state and streaming:
sendMessage(message)
Send user message, initiate streaming
stop()
Cancel active streaming request
reload()
Regenerate last assistant message
setMessages(fn)
Update message array (functional updates)
Callback Handlers
Sources: frontend/components/chatbot/chat.tsx:98-338
Backend Streaming Architecture
The chat backend implements a sophisticated streaming pipeline with session queuing, concurrent request handling, and AI SDK SSE format.
StreamingChatService Pipeline
Session-Scoped Queueing
Chat API uses get_session_queue() to serialize concurrent requests for the same chat (orchestrator/api/chat.py:531-567):
This prevents race conditions where multiple requests for the same chat could interleave messages or duplicate tool calls.
Tool Execution Loop with Deduplication
StreamingChatService includes a tool loop (orchestrator/consumers/chatbot/service.py:880-1096) with ToolExecutionTracker to prevent infinite loops:
The tracker implements three deduplication strategies:
Exact matching: Hash of tool name + arguments
Semantic deduplication: Similar queries for search tools (75% threshold)
Per-tool retry limits: Configurable max executions per tool
Sources: orchestrator/consumers/chatbot/service.py:456-1260, orchestrator/api/chat.py:531-571, orchestrator/consumers/chatbot/service.py:88-186
Frontend Real-Time Updates
The Chat Interface receives real-time updates via Server-Sent Events (SSE) from the backend. The useChat hook establishes a streaming connection and processes incoming data parts.
AI SDK SSE Format
The backend streams responses in AI SDK format (orchestrator/consumers/chatbot/streaming.py, referenced in orchestrator/consumers/chatbot/service.py:546-709):
chat_id
2:"<chat_id>"
useChat internal
Update active chat ID
agent-info
d:{"type":"agent-info",...}
chat.tsx:568-581
Display agent metadata
text-delta
0:"text chunk"
useChat internal
Stream message content
tool-data
d:{"type":"tool-data",...}
chat.tsx:151-367
Create widgets
memory-injected
d:{"type":"memory-injected",...}
chat.tsx:370-372
Show memory usage
finish
d:{"finishReason":"stop",...}
useChat internal
End streaming
Auto-Scroll Behavior
The chat automatically scrolls to the bottom during streaming to keep the latest content visible:
Sources: frontend/components/chatbot/chat.tsx:462-472
Scroll Position Tracking
The interface tracks whether the user is at the bottom of the message list to show/hide a "scroll to bottom" button:
Sources: frontend/components/chatbot/chat.tsx:442-459
Chat History and Sidebar
The AppSidebar component (frontend/components/chatbot/sidebar.tsx) provides navigation between chat sessions with search and grouping.
Sidebar Features
Sources: frontend/components/chatbot/sidebar.tsx:25-221
Chat Grouping Logic
Chats are grouped into mutually exclusive date buckets:
Sources: frontend/components/chatbot/sidebar.tsx:65-94
Title Generation
Chat titles are automatically generated from the first user message using the generateTitle() utility:
The generateTitle() function truncates the message to 50 characters and ensures a clean title by removing line breaks.
Sources: frontend/components/chatbot/chat.tsx:395-408
Agent Routing Corrections (PRD-50)
When a user manually selects a different agent after an auto-routing decision, the system records a correction for learning:
Sources: frontend/components/chatbot/chat.tsx:342-361
Welcome Card and Quick Actions
When no messages exist in the chat, a centered welcome card displays with suggested actions and quick links:
Welcome Card Content
Hero Header: "AI Services That [Elevate] Your Workflow" with animated brand icon
Description: Brief system overview
Suggested Actions: 4 pre-defined prompts focused on data visualization (frontend/components/chatbot/chat.tsx:648-653)
Quick Links: Navigation to Agents, Knowledge Base, Workflows, Tools (frontend/components/chatbot/chat.tsx:655-660)
The card uses an Incredible-style design with gradient borders and glow effects:
Sources: frontend/components/chatbot/chat.tsx:818-927
Performance Optimizations
Agent Name Caching
Agent names are fetched asynchronously when routing indicators appear and cached in a ref to avoid redundant API calls:
Sources: frontend/components/chatbot/chat.tsx:72-74, 309-327
Widget Deduplication
Before creating a new widget, the system checks if an identical widget already exists and activates it instead:
Sources: frontend/components/chatbot/chat.tsx:491-501
Textarea Auto-Resize
The multimodal input textarea automatically adjusts height based on content, up to a maximum of 200px:
Sources: frontend/components/chatbot/multimodal-input.tsx:52-61
Type Definitions
The chat system uses comprehensive TypeScript types defined in frontend/types/chat.ts:
Core Types
ChatMessage
Extended message with parts, toolCalls, routingInfo, codeSnippets, documents, database_results
MessagePart
Union type: text, file, tool-result, artifact
ToolCall
Tool execution state: toolCallId, toolName, state, input, error, durationMs
RoutingInfo
Routing decision: requestId, agentId, agentName, confidence, routeType, reasoning
Artifact
Standalone content: id, kind, title, content, language, metadata
CodeSnippet
Code from CodeGraph: language, code, file_path, line_number, symbol_name, explanation
DocumentReference
Document from RAG: id, filename, excerpt, similarity, chunk_count, has_full_content
DatabaseResult
Query result: database, sql, data, columns, execution_time_ms, pandas_ai
AppUsage
Token tracking: promptTokens, completionTokens, totalTokens, cost
Sources: frontend/types/chat.ts:1-292
Error Handling
Upload Failures
File upload errors display a toast notification and clear the upload queue:
Sources: frontend/components/chatbot/multimodal-input.tsx:132-139
Suggestion Fetch Failures
Tool suggestion errors fall back to empty suggestions and clear the active tool:
Sources: frontend/components/chatbot/chat.tsx:454-459
Widget Content Failures
If full document content fails to load, the widget displays an error state:
Sources: frontend/components/chatbot/chat.tsx:593-600
Integration Points
Backend API Routes
/api/chat
POST
Send message, receive streaming response
/api/chat/:id/messages
GET
Load chat history
/api/chat/:id/title
PUT
Update chat title
/api/chat/:id/vote
POST
Vote on message (thumbs up/down)
/api/chat/:id
DELETE
Delete chat
/api/agents/:id
GET
Fetch agent details (for name resolution)
/api/tools/:app/suggestions
GET
Fetch tool suggestions (PRD-40)
/api/routing/corrections
POST
Record routing correction (PRD-50)
/api/documents/:id/content
GET
Fetch full document content
/api/documents/upload
POST
Upload document file
Sources: frontend/components/chatbot/chat.tsx:311, 353, 576, frontend/components/chatbot/multimodal-input.tsx:119
Related Components
Agent Management (Agents): Agent selection, configuration, and tool assignment
Workflows & Recipes (Workflows & Recipes): Workflow execution triggers from chat
Tools & Integrations (Tools & Integrations): Tool suggestions and Composio integrations
Authentication (Authentication & Multi-Tenancy): User context and workspace resolution
Last updated

