Workflows & Recipes
Purpose and Scope
This document covers the Workflows & Recipes system in Automatos AI, which enables multi-agent task orchestration through step-by-step execution pipelines. Recipes are user-defined workflows that chain multiple agents together to accomplish complex tasks, with support for scheduling, triggers, memory integration, and self-learning capabilities.
For information about individual agents and their capabilities, see Agents. For analytics and monitoring of recipe executions, see Analytics & Monitoring. For the Universal Router that can trigger recipes, see Universal Router.
Core Concepts
Workflows vs. Recipes
The system supports two distinct execution paradigms:
Workflows (9-Stage Pipeline):
Complex orchestration through 9 stages: Task Decomposition → Agent Selection → Context Engineering → Agent Execution → Result Aggregation → Learning Update → Quality Assessment → Memory Storage → Response Generation
Used for dynamic, adaptive task execution
Managed through
orchestrator/api/workflows.py
Recipes (Direct Step Execution):
Simple step-by-step execution with predefined agent assignments
No decomposition or agent selection - steps execute sequentially or in parallel
Lighter weight, more predictable execution path
Primary focus of this document
Sources: orchestrator/api/workflows.py:1-50, orchestrator/api/recipe_executor.py:1-20
Recipe Structure
A recipe consists of:
Sources: orchestrator/core/models/core.py:WorkflowTemplate, orchestrator/api/workflow_recipes.py:421-442
Recipe Definition
Database Schema
Recipes are stored in the workflow_templates table with the following key fields:
template_id
VARCHAR
Unique identifier (e.g., "my-custom-recipe")
name
VARCHAR
Display name
description
TEXT
What the recipe does
steps
JSONB
Array of step definitions
inputs
JSONB
Input parameter schema
outputs
JSONB
Output schema
execution_config
JSONB
Runtime behavior config
schedule_config
JSONB
Scheduling configuration
learning_data
JSONB
Quality metrics and suggestions
workspace_id
UUID
Workspace ownership
owner_type
VARCHAR
'workspace' or 'system'
Sources: orchestrator/core/models/core.py:1170-1350, orchestrator/api/workflow_recipes.py:383-389
Steps Schema
Each step in the steps array must contain:
Example steps array:
Validation rules:
All
agent_idreferences must exist in the workspaceordermust start at 1 and increment sequentiallyprompt_templateis required and non-empty
Sources: orchestrator/core/models/core.py:1246-1276, orchestrator/api/workflow_recipes.py:444-473
Execution Config
The execution_config field controls runtime behavior:
Default values are applied if omitted:
Sources: orchestrator/api/workflow_recipes.py:404-412, orchestrator/core/models/core.py:1296-1335
Schedule Config
The schedule_config field supports four execution triggers:
manual
User-triggered only
-
cron
Scheduled via cron expression
cron_expression
trigger
Composio trigger subscription
trigger_config.trigger_name, trigger_config.source
webhook
Webhook-based execution
Auto-generated webhook_id
Trigger config example (Composio):
Auto-registration: When a recipe with source: "composio" trigger is created/updated, the system automatically subscribes to the trigger via Composio API and stores a TriggerSubscription record.
Sources: orchestrator/api/workflow_recipes.py:37-113, orchestrator/api/workflow_recipes.py:415-419, orchestrator/api/workflow_recipes.py:478-481
Recipe Execution Pipeline
Execution Flow Overview
Recipe execution bypasses the 9-stage workflow pipeline and executes steps directly using the chatbot's exact component path (PRD-50 alignment):
Sources: orchestrator/api/recipe_executor.py:559-735, orchestrator/api/workflow_recipes.py:783-856
Step Execution Details
Each step execution (_execute_step) follows this pattern:
1. Agent Activation:
2. System Prompt Assembly:
Agent identity (name, type, description)
Persona (custom or predefined)
Plugin context (Tier 1 summary + Tier 2 full content) OR
Skills (if no plugins assigned)
Recipe step scope instruction (prevents task drift)
3. Composio Tool Resolution:
Two strategies are used:
Strategy A: SDK Search (Primary)
Extract explicit action names from prompt (e.g.,
GITHUB_CREATE_A_REFERENCE)Fetch exact schemas via
ComposioToolService.get_tools_for_step()Returns per-action function-calling tools with correct parameter schemas
Strategy B: Hint-Based Fallback
Use
ComposioHintService.build_hints()when SDK returns no toolsGenerates system message with available actions list
LLM calls
composio_executemega-tool
4. Context Injection:
Original trigger/input data (persistent across all steps)
Mem0 memories (first step only)
Scratchpad context from previous steps (structured keys, not full text)
5. LLM Generation Loop:
Call
llm.generate_response()with messages + toolsIf tool calls returned:
Execute via
tool_router.execute_and_format()or Composio directAppend tool results to messages
Continue loop (max 6 iterations)
Return final text response
6. Result Processing:
Auto-extract URLs, key-value pairs, JSON from response
Write to scratchpad:
tool_results,output_summary,exportsUpload full logs (messages array) to S3
Store compact summary in
RecipeExecution.step_results
Sources: orchestrator/api/recipe_executor.py:44-363, orchestrator/modules/tools/services/composio_tool_service.py:69-167, orchestrator/modules/tools/services/composio_hint_service.py:103-212
Tool Resolution Architecture
Key points:
Primary path: Extract explicit action names → direct schema lookup
Secondary path: SDK semantic search (when no explicit names)
Fallback path: Hint-based
composio_executemega-toolDeduplication: Same action + same args uses cached result
Sources: orchestrator/modules/tools/services/composio_tool_service.py:69-167, orchestrator/modules/tools/services/composio_hint_service.py:89-212, orchestrator/api/recipe_executor.py:110-155
Recipe Scratchpad
Purpose and Token Savings
The RecipeScratchpad is a Redis-backed ephemeral context store that achieves 80-90% token savings compared to dumping full agent outputs between steps.
Problem it solves:
Step 1 output: 2000 tokens of verbose text
Step 2 needs only: 3 key facts + 2 tool results
Without scratchpad: 2000 tokens injected into Step 2 context
With scratchpad: 150 tokens (structured keys only)
Sources: orchestrator/core/services/recipe_scratchpad.py:1-20
Key Layout
Key naming conventions:
_input:{key}- Trigger/input data (e.g.,_input:issue_url)_meta:{key}- Execution metadatastep_{N}:tool_results- JSON array of tool execution summariesstep_{N}:output_summary- First 500 chars of agent responsestep_{N}:exports- Agent's explicitscratchpad_writecalls
Example Redis hash:
Sources: orchestrator/core/services/recipe_scratchpad.py:7-20, orchestrator/core/services/recipe_scratchpad.py:58-120
Auto-Extraction
The scratchpad automatically extracts structured data from agent responses using zero-LLM heuristics:
1. URLs:
2. Key-Value Pairs:
3. JSON Objects:
Attempts
json.loads()on agent responseExtracts top-level keys if successful
4. Tool Results:
Each tool call's result stored in array
Summarized as
{action: "TOOL_NAME", result: "..."}(truncated)
Example auto-extraction:
Agent output:
Extracted:
Sources: orchestrator/core/services/recipe_scratchpad.py:26-30, orchestrator/core/services/recipe_scratchpad.py:199-299
Context Formatting for Steps
When a step executes, it receives structured context from the scratchpad:
Example formatted context (Step 3):
Sources: orchestrator/core/services/recipe_scratchpad.py:301-414, orchestrator/api/recipe_executor.py:178-183
Memory Integration
Mem0 Retrieval (Pre-Execution)
Before executing the first step, the system retrieves relevant memories from Mem0:
Memory scope:
This ensures memories are scoped per-recipe, not globally shared.
Injection into Step 1:
Sources: orchestrator/api/recipe_executor.py:637-652, orchestrator/api/recipe_executor.py:151-163
Memory Storage (Post-Execution)
After successful execution, key insights are stored in Mem0:
Stored memories include:
Successful tool combinations
Error patterns and fixes
Quality insights ("This recipe works best when X")
Execution duration patterns
Memory cleanup on recipe deletion: When a recipe is deleted, all associated memories are purged:
Sources: orchestrator/api/recipe_executor.py:806-818, orchestrator/api/workflow_recipes.py:648-667
Quality Assessment & Learning
Quality Scoring
After execution, recipes are scored on multiple dimensions:
The quality_score (0-1) is stored in WorkflowRecipe.learning_data and used for:
Suggesting recipe improvements
Ranking recipes in the UI
Triggering auto-learning when below threshold
Sources: orchestrator/core/models/core.py:1387-1396
Suggestions System
The RecipeSuggestion model stores improvement recommendations:
Suggestion types:
step_refinement- Improve prompt templatesagent_swap- Use different agent for a steptool_addition- Suggest new tools to enableerror_fix- Fix common failure patternsperformance- Reduce token usage or latency
UI display: Suggestions appear in the recipe detail view with:
Priority badge (high/medium/low)
Auto-applicable flag (can be applied with one click)
Implementation guidance
Sources: orchestrator/core/models/core.py:1422-1478, frontend/components/workflows/recipe-suggestions-panel.tsx:1-150
Learning Data Structure
The learning_data JSONB field stores:
Sources: orchestrator/core/models/core.py:1228-1245
Scheduling & Triggers
Manual Execution
Direct execution via API:
Response:
Sources: orchestrator/api/workflow_recipes.py:783-856
Cron Scheduling
Recipes with schedule_config.type = "cron" are executed by a background scheduler:
Example config:
The cron scheduler (not shown in provided files) polls active recipes and triggers executions based on cron_expression.
Sources: orchestrator/core/models/core.py:1344-1362
Composio Triggers
Recipes can subscribe to Composio triggers (e.g., GitHub events):
Setup flow:
Trigger config example:
Webhook handling: When Composio fires the trigger, it sends a webhook to:
The webhook handler:
Validates signature
Looks up matching
TriggerSubscriptionbycomposio_subscription_idFinds associated recipe via
workflow_idTriggers execution with webhook payload as
input_data
Sources: orchestrator/api/workflow_recipes.py:37-113, orchestrator/api/workflow_recipes.py:478-481
Custom Webhooks
For non-Composio triggers, recipes can use generic webhooks:
Webhook URL pattern:
The webhook_id is auto-generated and stored in schedule_config:
Sources: orchestrator/api/workflow_recipes.py:415-419
Frontend Components
RecipesTab
Main recipe browsing and management interface.
Component: frontend/components/workflows/recipes-tab.tsx
Features:
Grid/list view toggle
Search and filtering
Recipe cards with metrics (steps, agents, runs)
Quick "Cook" button for instant execution
Edit/delete actions (non-system recipes only)
Share to marketplace
Card data:
Key interactions:
View: Opens
ViewRecipeModalwith full detailsEdit: Opens
CreateRecipeModalin edit modeCook: Calls
useExecuteRecipe()mutation → opensExecutionKitchenShare: Calls
useSubmitRecipeToMarketplace()mutation
Sources: frontend/components/workflows/recipes-tab.tsx:81-515
ExecutionKitchen
Real-time execution visualization component (replaces old "Execution Theater").
Component: frontend/components/workflows/execution-kitchen.tsx
Displays:
Recipe step progress (current step, status, duration)
Streaming execution logs
Tool calls and results
Per-step metrics (tokens, duration)
Quality score and suggestions (post-execution)
Execution modes:
Recipe direct execution - Step-by-step with
RecipeStepProgress9-stage workflow - Full pipeline with
TheaterStageProgress
Data flow:
Sources: frontend/components/workflows/execution-kitchen.tsx:1-300, frontend/components/workflows/recipe-step-progress.tsx:1-200
RecipeStepProgress
Displays individual step execution details.
Component: frontend/components/workflows/recipe-step-progress.tsx
Per-step data:
UI features:
Collapsible accordion per step
Status badge (running/completed/failed)
Agent avatar and name
Duration and token metrics
Tool calls list with results
Output preview (first 500 chars)
"View full logs" link (fetches from S3)
Sources: frontend/components/workflows/recipe-step-progress.tsx:1-250
CreateRecipeModal
Recipe creation/editing wizard.
Component: frontend/components/workflows/create-recipe-modal.tsx
Form structure:
Basic Info: name, description, icon
Steps: Step builder with agent selection, prompt template
Inputs/Outputs: JSON schema for parameters
Execution Config: Mode, retries, timeouts
Schedule Config: Trigger type and configuration
Step builder:
Validation:
All agents must exist in workspace
Prompt templates required
Order must be sequential (1, 2, 3...)
JSON schemas must be valid
Sources: frontend/components/workflows/create-recipe-modal.tsx:1-500
API Reference
Recipe CRUD
List Recipes
Get Recipe
Returns recipe with enriched agent details per step.
Create Recipe
Update Recipe
Delete Recipe
Also cleans up trigger subscriptions and Mem0 memories.
Sources: orchestrator/api/workflow_recipes.py:164-683
Recipe Execution
Execute Recipe
Response:
Get Execution Status
List Executions
Sources: orchestrator/api/workflow_recipes.py:783-856
Recipe Analytics
Dashboard Stats
Returns:
Get Suggestions
Sources: orchestrator/api/workflow_recipes.py:239-323
Recipe Marketplace
Submit to Marketplace
Featured Recipes
Categories
Sources: orchestrator/api/workflow_recipes.py:724-780
Summary
The Workflows & Recipes system provides a flexible, step-by-step agent orchestration framework with:
Simple execution model - Direct step execution without complex decomposition
Context compression - RecipeScratchpad achieves 80-90% token savings
Memory integration - Mem0 enables self-learning across executions
Flexible scheduling - Manual, cron, Composio triggers, or webhooks
Quality assessment - Automated scoring and improvement suggestions
Rich visualization - Real-time execution monitoring in ExecutionKitchen
For advanced orchestration needs requiring dynamic task decomposition and agent selection, see the full workflow system in Workflows API.
Sources: orchestrator/api/recipe_executor.py:1-850, orchestrator/api/workflow_recipes.py:1-856, orchestrator/core/services/recipe_scratchpad.py:1-450
Last updated

