Workflows & Recipes

chevron-rightRelevant source fileshashtag

Purpose and Scope

This document covers the Workflows & Recipes system in Automatos AI, which enables multi-agent task orchestration through step-by-step execution pipelines. Recipes are user-defined workflows that chain multiple agents together to accomplish complex tasks, with support for scheduling, triggers, memory integration, and self-learning capabilities.

For information about individual agents and their capabilities, see Agents. For analytics and monitoring of recipe executions, see Analytics & Monitoring. For the Universal Router that can trigger recipes, see Universal Router.


Core Concepts

Workflows vs. Recipes

The system supports two distinct execution paradigms:

Workflows (9-Stage Pipeline):

  • Complex orchestration through 9 stages: Task Decomposition → Agent Selection → Context Engineering → Agent Execution → Result Aggregation → Learning Update → Quality Assessment → Memory Storage → Response Generation

  • Used for dynamic, adaptive task execution

  • Managed through orchestrator/api/workflows.py

Recipes (Direct Step Execution):

  • Simple step-by-step execution with predefined agent assignments

  • No decomposition or agent selection - steps execute sequentially or in parallel

  • Lighter weight, more predictable execution path

  • Primary focus of this document

Sources: orchestrator/api/workflows.py:1-50, orchestrator/api/recipe_executor.py:1-20

Recipe Structure

A recipe consists of:

spinner

Sources: orchestrator/core/models/core.py:WorkflowTemplate, orchestrator/api/workflow_recipes.py:421-442


Recipe Definition

Database Schema

Recipes are stored in the workflow_templates table with the following key fields:

Field
Type
Description

template_id

VARCHAR

Unique identifier (e.g., "my-custom-recipe")

name

VARCHAR

Display name

description

TEXT

What the recipe does

steps

JSONB

Array of step definitions

inputs

JSONB

Input parameter schema

outputs

JSONB

Output schema

execution_config

JSONB

Runtime behavior config

schedule_config

JSONB

Scheduling configuration

learning_data

JSONB

Quality metrics and suggestions

workspace_id

UUID

Workspace ownership

owner_type

VARCHAR

'workspace' or 'system'

Sources: orchestrator/core/models/core.py:1170-1350, orchestrator/api/workflow_recipes.py:383-389

Steps Schema

Each step in the steps array must contain:

Example steps array:

Validation rules:

  • All agent_id references must exist in the workspace

  • order must start at 1 and increment sequentially

  • prompt_template is required and non-empty

Sources: orchestrator/core/models/core.py:1246-1276, orchestrator/api/workflow_recipes.py:444-473

Execution Config

The execution_config field controls runtime behavior:

Default values are applied if omitted:

Sources: orchestrator/api/workflow_recipes.py:404-412, orchestrator/core/models/core.py:1296-1335

Schedule Config

The schedule_config field supports four execution triggers:

Type
Description
Required Fields

manual

User-triggered only

-

cron

Scheduled via cron expression

cron_expression

trigger

Composio trigger subscription

trigger_config.trigger_name, trigger_config.source

webhook

Webhook-based execution

Auto-generated webhook_id

Trigger config example (Composio):

Auto-registration: When a recipe with source: "composio" trigger is created/updated, the system automatically subscribes to the trigger via Composio API and stores a TriggerSubscription record.

Sources: orchestrator/api/workflow_recipes.py:37-113, orchestrator/api/workflow_recipes.py:415-419, orchestrator/api/workflow_recipes.py:478-481


Recipe Execution Pipeline

Execution Flow Overview

Recipe execution bypasses the 9-stage workflow pipeline and executes steps directly using the chatbot's exact component path (PRD-50 alignment):

spinner

Sources: orchestrator/api/recipe_executor.py:559-735, orchestrator/api/workflow_recipes.py:783-856

Step Execution Details

Each step execution (_execute_step) follows this pattern:

1. Agent Activation:

2. System Prompt Assembly:

  • Agent identity (name, type, description)

  • Persona (custom or predefined)

  • Plugin context (Tier 1 summary + Tier 2 full content) OR

  • Skills (if no plugins assigned)

  • Recipe step scope instruction (prevents task drift)

3. Composio Tool Resolution:

Two strategies are used:

Strategy A: SDK Search (Primary)

  • Extract explicit action names from prompt (e.g., GITHUB_CREATE_A_REFERENCE)

  • Fetch exact schemas via ComposioToolService.get_tools_for_step()

  • Returns per-action function-calling tools with correct parameter schemas

Strategy B: Hint-Based Fallback

  • Use ComposioHintService.build_hints() when SDK returns no tools

  • Generates system message with available actions list

  • LLM calls composio_execute mega-tool

4. Context Injection:

  • Original trigger/input data (persistent across all steps)

  • Mem0 memories (first step only)

  • Scratchpad context from previous steps (structured keys, not full text)

5. LLM Generation Loop:

  • Call llm.generate_response() with messages + tools

  • If tool calls returned:

    • Execute via tool_router.execute_and_format() or Composio direct

    • Append tool results to messages

    • Continue loop (max 6 iterations)

  • Return final text response

6. Result Processing:

  • Auto-extract URLs, key-value pairs, JSON from response

  • Write to scratchpad: tool_results, output_summary, exports

  • Upload full logs (messages array) to S3

  • Store compact summary in RecipeExecution.step_results

Sources: orchestrator/api/recipe_executor.py:44-363, orchestrator/modules/tools/services/composio_tool_service.py:69-167, orchestrator/modules/tools/services/composio_hint_service.py:103-212

Tool Resolution Architecture

spinner

Key points:

  • Primary path: Extract explicit action names → direct schema lookup

  • Secondary path: SDK semantic search (when no explicit names)

  • Fallback path: Hint-based composio_execute mega-tool

  • Deduplication: Same action + same args uses cached result

Sources: orchestrator/modules/tools/services/composio_tool_service.py:69-167, orchestrator/modules/tools/services/composio_hint_service.py:89-212, orchestrator/api/recipe_executor.py:110-155


Recipe Scratchpad

Purpose and Token Savings

The RecipeScratchpad is a Redis-backed ephemeral context store that achieves 80-90% token savings compared to dumping full agent outputs between steps.

Problem it solves:

  • Step 1 output: 2000 tokens of verbose text

  • Step 2 needs only: 3 key facts + 2 tool results

  • Without scratchpad: 2000 tokens injected into Step 2 context

  • With scratchpad: 150 tokens (structured keys only)

Sources: orchestrator/core/services/recipe_scratchpad.py:1-20

Key Layout

spinner

Key naming conventions:

  • _input:{key} - Trigger/input data (e.g., _input:issue_url)

  • _meta:{key} - Execution metadata

  • step_{N}:tool_results - JSON array of tool execution summaries

  • step_{N}:output_summary - First 500 chars of agent response

  • step_{N}:exports - Agent's explicit scratchpad_write calls

Example Redis hash:

Sources: orchestrator/core/services/recipe_scratchpad.py:7-20, orchestrator/core/services/recipe_scratchpad.py:58-120

Auto-Extraction

The scratchpad automatically extracts structured data from agent responses using zero-LLM heuristics:

1. URLs:

2. Key-Value Pairs:

3. JSON Objects:

  • Attempts json.loads() on agent response

  • Extracts top-level keys if successful

4. Tool Results:

  • Each tool call's result stored in array

  • Summarized as {action: "TOOL_NAME", result: "..."} (truncated)

Example auto-extraction:

Agent output:

Extracted:

Sources: orchestrator/core/services/recipe_scratchpad.py:26-30, orchestrator/core/services/recipe_scratchpad.py:199-299

Context Formatting for Steps

When a step executes, it receives structured context from the scratchpad:

Example formatted context (Step 3):

Sources: orchestrator/core/services/recipe_scratchpad.py:301-414, orchestrator/api/recipe_executor.py:178-183


Memory Integration

Mem0 Retrieval (Pre-Execution)

Before executing the first step, the system retrieves relevant memories from Mem0:

Memory scope:

This ensures memories are scoped per-recipe, not globally shared.

Injection into Step 1:

Sources: orchestrator/api/recipe_executor.py:637-652, orchestrator/api/recipe_executor.py:151-163

Memory Storage (Post-Execution)

After successful execution, key insights are stored in Mem0:

Stored memories include:

  • Successful tool combinations

  • Error patterns and fixes

  • Quality insights ("This recipe works best when X")

  • Execution duration patterns

Memory cleanup on recipe deletion: When a recipe is deleted, all associated memories are purged:

Sources: orchestrator/api/recipe_executor.py:806-818, orchestrator/api/workflow_recipes.py:648-667


Quality Assessment & Learning

Quality Scoring

After execution, recipes are scored on multiple dimensions:

The quality_score (0-1) is stored in WorkflowRecipe.learning_data and used for:

  • Suggesting recipe improvements

  • Ranking recipes in the UI

  • Triggering auto-learning when below threshold

Sources: orchestrator/core/models/core.py:1387-1396

Suggestions System

The RecipeSuggestion model stores improvement recommendations:

spinner

Suggestion types:

  • step_refinement - Improve prompt templates

  • agent_swap - Use different agent for a step

  • tool_addition - Suggest new tools to enable

  • error_fix - Fix common failure patterns

  • performance - Reduce token usage or latency

UI display: Suggestions appear in the recipe detail view with:

  • Priority badge (high/medium/low)

  • Auto-applicable flag (can be applied with one click)

  • Implementation guidance

Sources: orchestrator/core/models/core.py:1422-1478, frontend/components/workflows/recipe-suggestions-panel.tsx:1-150

Learning Data Structure

The learning_data JSONB field stores:

Sources: orchestrator/core/models/core.py:1228-1245


Scheduling & Triggers

Manual Execution

Direct execution via API:

Response:

Sources: orchestrator/api/workflow_recipes.py:783-856

Cron Scheduling

Recipes with schedule_config.type = "cron" are executed by a background scheduler:

Example config:

The cron scheduler (not shown in provided files) polls active recipes and triggers executions based on cron_expression.

Sources: orchestrator/core/models/core.py:1344-1362

Composio Triggers

Recipes can subscribe to Composio triggers (e.g., GitHub events):

Setup flow:

spinner

Trigger config example:

Webhook handling: When Composio fires the trigger, it sends a webhook to:

The webhook handler:

  1. Validates signature

  2. Looks up matching TriggerSubscription by composio_subscription_id

  3. Finds associated recipe via workflow_id

  4. Triggers execution with webhook payload as input_data

Sources: orchestrator/api/workflow_recipes.py:37-113, orchestrator/api/workflow_recipes.py:478-481

Custom Webhooks

For non-Composio triggers, recipes can use generic webhooks:

Webhook URL pattern:

The webhook_id is auto-generated and stored in schedule_config:

Sources: orchestrator/api/workflow_recipes.py:415-419


Frontend Components

RecipesTab

Main recipe browsing and management interface.

Component: frontend/components/workflows/recipes-tab.tsx

Features:

  • Grid/list view toggle

  • Search and filtering

  • Recipe cards with metrics (steps, agents, runs)

  • Quick "Cook" button for instant execution

  • Edit/delete actions (non-system recipes only)

  • Share to marketplace

Card data:

Key interactions:

  • View: Opens ViewRecipeModal with full details

  • Edit: Opens CreateRecipeModal in edit mode

  • Cook: Calls useExecuteRecipe() mutation → opens ExecutionKitchen

  • Share: Calls useSubmitRecipeToMarketplace() mutation

Sources: frontend/components/workflows/recipes-tab.tsx:81-515

ExecutionKitchen

Real-time execution visualization component (replaces old "Execution Theater").

Component: frontend/components/workflows/execution-kitchen.tsx

Displays:

  • Recipe step progress (current step, status, duration)

  • Streaming execution logs

  • Tool calls and results

  • Per-step metrics (tokens, duration)

  • Quality score and suggestions (post-execution)

Execution modes:

  1. Recipe direct execution - Step-by-step with RecipeStepProgress

  2. 9-stage workflow - Full pipeline with TheaterStageProgress

Data flow:

spinner

Sources: frontend/components/workflows/execution-kitchen.tsx:1-300, frontend/components/workflows/recipe-step-progress.tsx:1-200

RecipeStepProgress

Displays individual step execution details.

Component: frontend/components/workflows/recipe-step-progress.tsx

Per-step data:

UI features:

  • Collapsible accordion per step

  • Status badge (running/completed/failed)

  • Agent avatar and name

  • Duration and token metrics

  • Tool calls list with results

  • Output preview (first 500 chars)

  • "View full logs" link (fetches from S3)

Sources: frontend/components/workflows/recipe-step-progress.tsx:1-250

CreateRecipeModal

Recipe creation/editing wizard.

Component: frontend/components/workflows/create-recipe-modal.tsx

Form structure:

  1. Basic Info: name, description, icon

  2. Steps: Step builder with agent selection, prompt template

  3. Inputs/Outputs: JSON schema for parameters

  4. Execution Config: Mode, retries, timeouts

  5. Schedule Config: Trigger type and configuration

Step builder:

Validation:

  • All agents must exist in workspace

  • Prompt templates required

  • Order must be sequential (1, 2, 3...)

  • JSON schemas must be valid

Sources: frontend/components/workflows/create-recipe-modal.tsx:1-500


API Reference

Recipe CRUD

List Recipes

Get Recipe

Returns recipe with enriched agent details per step.

Create Recipe

Update Recipe

Delete Recipe

Also cleans up trigger subscriptions and Mem0 memories.

Sources: orchestrator/api/workflow_recipes.py:164-683

Recipe Execution

Execute Recipe

Response:

Get Execution Status

List Executions

Sources: orchestrator/api/workflow_recipes.py:783-856

Recipe Analytics

Dashboard Stats

Returns:

Get Suggestions

Sources: orchestrator/api/workflow_recipes.py:239-323

Recipe Marketplace

Submit to Marketplace

Featured Recipes

Categories

Sources: orchestrator/api/workflow_recipes.py:724-780


Summary

The Workflows & Recipes system provides a flexible, step-by-step agent orchestration framework with:

  • Simple execution model - Direct step execution without complex decomposition

  • Context compression - RecipeScratchpad achieves 80-90% token savings

  • Memory integration - Mem0 enables self-learning across executions

  • Flexible scheduling - Manual, cron, Composio triggers, or webhooks

  • Quality assessment - Automated scoring and improvement suggestions

  • Rich visualization - Real-time execution monitoring in ExecutionKitchen

For advanced orchestration needs requiring dynamic task decomposition and agent selection, see the full workflow system in Workflows API.

Sources: orchestrator/api/recipe_executor.py:1-850, orchestrator/api/workflow_recipes.py:1-856, orchestrator/core/services/recipe_scratchpad.py:1-450


Last updated