flow-engine

Files (1)

• SKILL.md

SKILL.md (first lines)

# Flow Engine Extension

This extension provides a robust workflow automation system for Pi that enables:
- Resilient server startup procedures accessible even when the server is down
- Agent-based workflow orchestration using Pi's existing capabilities
- Persistent storage of workflow definitions and executions via js-doc-store-server
- Visual tracking of workflow execution status in the Pi TUI

## Problem Statement

Pi agents often need to orchestrate complex workflows involving multiple steps:
- Starting dependent services (like js-doc-store-server)
- Executing sequences of agent, skill, and tool operations
- Handling failures and retries gracefully
- Tracking execution status and results
- Ensuring critical bootstrap procedures run even when the server is offline

Prior to this extension, users had to manually manage these workflows using custom scripts or complex prompt engineering.

## Solution Overview

The flow-engine extension introduces:
1. **Workflow Definitions**: JSON-based definitions of workflows consisting of nodes (agent, skill, tool)
2. **Execution Engine**: Reliable execution of workflows with dependency resolution
3. **Persistence Layer**: Storage of workflow definitions, runs, and task instances in js-doc-store-server
4. **Monitoring Tools**: Commands to check workflow status and list available workflows
5. **Bootstrap Automation**: Automatic setup of essential database tables
6. **TUI Integration**: Widget displaying flow engine readiness status

## Key Features

### Workflow Node Types
- **agent**: Execute Pi subagents with input/output mapping
- **skill**: Load and invoke Pi skills from the skill registry
- **tool**: Execute Pi built-in or custom tools (read, write, bash, etc.)

### Dependency Resolution
- Automatic topological sort of workflow nodes based on `dependsOn` relationships
- Prevents circular dependency detection and provides clear error messages

### Input/Output Mapping
- Use `${variable}` syntax to map outputs from previous nodes as inputs to subsequent nodes
- Support for literal values and complex object mapping
- Flexible output storage (raw output under node ID or mapped to context variables)

### Error Handling & Retries
- Configurable maximum retry attempts per node
- Automatic retry with exponential backoff (leveraging Pi's retry system)
- Detailed error logging and tracking via Pi's message system

### Persistence
- All workflow definitions stored in `flows` table
- All workflow executions stored in `flow_runs` table
- Individual task executions stored in `task_instances` table
- Automatic table creation via `run_bootstrap` tool

### TUI Integration
- Session startup widget showing "🟢 Flow Engine Ready" when operational
- Extensible for future enhancement with detailed workflow visualization

## Installation

The extension is installed as a standard Pi extension:

```bash
# From anywhere, install the extension
pi install git:https://github.com/MauricioPerera/pi-extension-data-architect

# Or install locally if you have the repo cloned
pi install -l /path/to/pi-extension-data-architect
```

## Usage

Once installed, the extension provides several custom tools:

### flow_create
Store a new workflow definition.

**Parameters:**
- `name` (string, required): Unique name for the workflow
- `definition` (string, required): JSON string representing the workflow definition
- `description` (string, optional): Human-readable description
- `tags` (string, optional): Comma-separated tags for categorization
- `version` (string, optional): Semantic version (default: "1.0.0")

**Example:**
```bash
/tool:flow_create
{
  "name": "server_startup_check",
  "definition": "{\"nodes\":[{\"id\":\"check_server\",\"type\":\"tool\",\"tool\":\"bash\",\"inputMapping\":{\"command\":\"curl -s http://localhost:3000/public/tables || echo 'SERVER_DOWN'\"}},{\"id\":\"start_if_needed\",\"type\":\"agent\",\"agent\":\"server-watchdog\",\"inputMapping\":{\"command\":\"start\"},\"dependsOn\":[\"check_server\"]}]}",
  "description": "Check js-doc-store-server status and start if needed",
  "tags": "server,startup,automation",
  "version": "1.0.0"
}
```

### flow_list
List all stored workflow definitions.

**Parameters:**
- `tags` (string, optional): Comma-separated tags to filter by

**Example:**
```bash
/tool:flow_list
# Lists all workflows

/tool:flow_list
{
  "tags": "server,startup"
}
# Lists only workflows with server and startup tags
```

### flow_run
Execute a workflow by name.

From MauricioPerera/pi-extension-data-architect, MIT.

More from MauricioPerera/pi-extension-data-architect