# Batch

Use Batch to run independent workflow steps in parallel, such as fetching data from several APIs or sending messages through different channels.

## Overview

**Type:** `batch`

Every child receives the same parent context. Children cannot read one another's results. Choose `all` to collect the results or `race` to return the first settled result, including a failure.

## Configuration

### Steps

* **Type:** Array of Objects
* **Required:** Yes
* **Description:** List of steps to execute in parallel

Each step object is a standard workflow step definition with its own configuration.

```json
[
  {
    "type": "request",
    "output": "user_data",
    "config": {
      "url": "https://api.example.com/users/{{ userId }}",
      "method": "GET"
    }
  },
  {
    "type": "request",
    "output": "orders_data",
    "config": {
      "url": "https://api.example.com/orders/{{ userId }}",
      "method": "GET"
    }
  }
]
```

### Mode

* **Type:** Enum
* **Required:** No
* **Default:** `all`
* **Options:** `all`, `race`
* **Description:** Execution strategy

**Modes:**

* `all`: Wait for all steps to complete and return all results
* `race`: Return the first settled step result; the batch rejects if that step fails

### Max concurrency

* **Type:** Number
* **Required:** No
* **Min:** `1`
* **Max:** `100`
* **Description:** Maximum number of steps executing simultaneously

In `all` mode, omitting this setting starts all steps together. `race` mode starts all steps together and ignores `maxConcurrency`.

```json
{
  "maxConcurrency": 5
}
```

### Fail fast

* **Type:** Boolean
* **Required:** No
* **Default:** `false`
* **Description:** Reject the Batch on the first error in `all` mode

In `all` mode, `true` rejects the batch on the first error. Already running operations continue. When `false`, all steps complete and errors are included in results. `race` mode ignores this setting and rejects if the first settled step fails.

## Output

In `all` mode, the Batch step returns results keyed by their `output` names. Steps without an `output` still run, but their results are omitted:

```typescript
{
  [stepOutputName: string]: any  // Result from each step
}
```

If a step fails and `failFast` is `false`, the error is captured:

```typescript
{
  [stepOutputName: string]: {
    error: string  // Error message
  }
}
```

In `race` mode, the winning result is returned directly. If that result has `success: true`, its `output` field is returned instead. Other steps continue running.

### Accessing the output

```
{{ batch_step.user_data.data.name }}
{{ batch_step.orders_data.data }}
{{ batch_step.user_data.status }}
```

## Examples

These examples are steps to include in an [execution request](/docs/execute-workflows). Supply their referenced variables and credentials in context.

### Parallel API calls

```json
{
  "type": "batch",
  "output": "api_results",
  "config": {
    "steps": [
      {
        "type": "request",
        "output": "user",
        "config": {
          "url": "https://api.example.com/users/{{ userId }}",
          "method": "GET",
          "headers": {
            "Authorization": "Bearer {{ token }}"
          }
        }
      },
      {
        "type": "request",
        "output": "orders",
        "config": {
          "url": "https://api.example.com/orders/{{ userId }}",
          "method": "GET",
          "headers": {
            "Authorization": "Bearer {{ token }}"
          }
        }
      },
      {
        "type": "request",
        "output": "preferences",
        "config": {
          "url": "https://api.example.com/preferences/{{ userId }}",
          "method": "GET",
          "headers": {
            "Authorization": "Bearer {{ token }}"
          }
        }
      }
    ],
    "mode": "all"
  }
}
```

### Controlled concurrency

```json
{
  "type": "batch",
  "output": "processed_items",
  "config": {
    "steps": [
      {
        "type": "agent",
        "output": "analysis_1",
        "config": {
          "provider": "openai",
          "model": "gpt-4o-mini",
          "input": "{{ items[0] }}",
          "outputSchema": {
            "type": "object",
            "properties": {
              "summary": {
                "type": "string"
              }
            },
            "required": ["summary"]
          }
        }
      },
      {
        "type": "agent",
        "output": "analysis_2",
        "config": {
          "provider": "openai",
          "model": "gpt-4o-mini",
          "input": "{{ items[1] }}",
          "outputSchema": {
            "type": "object",
            "properties": {
              "summary": {
                "type": "string"
              }
            },
            "required": ["summary"]
          }
        }
      },
      {
        "type": "agent",
        "output": "analysis_3",
        "config": {
          "provider": "openai",
          "model": "gpt-4o-mini",
          "input": "{{ items[2] }}",
          "outputSchema": {
            "type": "object",
            "properties": {
              "summary": {
                "type": "string"
              }
            },
            "required": ["summary"]
          }
        }
      }
    ],
    "maxConcurrency": 2,
    "mode": "all"
  }
}
```

### Fail-fast mode

```json
{
  "type": "batch",
  "output": "validations",
  "config": {
    "steps": [
      {
        "type": "request",
        "output": "email_check",
        "config": {
          "url": "https://api.example.com/validate/email",
          "method": "POST",
          "data": {
            "email": "{{ user.email }}"
          }
        }
      },
      {
        "type": "request",
        "output": "phone_check",
        "config": {
          "url": "https://api.example.com/validate/phone",
          "method": "POST",
          "data": {
            "phone": "{{ user.phone }}"
          }
        }
      }
    ],
    "failFast": true,
    "mode": "all"
  }
}
```

### Multi-channel notifications

```json
{
  "type": "batch",
  "output": "notifications",
  "config": {
    "steps": [
      {
        "type": "email",
        "output": "email_sent",
        "config": {
          "to": "{{ user.email }}",
          "from": "noreply@company.com",
          "subject": "{{ notification.subject }}",
          "htmlPart": "{{ notification.body }}"
        }
      },
      {
        "type": "sms",
        "output": "sms_sent",
        "config": {
          "to": "{{ user.phone }}",
          "from": "{{ company.smsNumber }}",
          "body": "{{ notification.shortMessage }}"
        }
      }
    ],
    "failFast": false,
    "mode": "all"
  }
}
```

### Data enrichment from multiple sources

```json
{
  "type": "batch",
  "output": "enriched_data",
  "config": {
    "steps": [
      {
        "type": "request",
        "output": "company_info",
        "config": {
          "url": "https://api.example.com/companies?domain={{ company.domain }}",
          "method": "GET",
          "headers": {
            "Authorization": "Bearer {{ companyApiKey }}"
          }
        }
      },
      {
        "type": "request",
        "output": "social_profiles",
        "config": {
          "url": "https://api.example.com/company-enrichment",
          "method": "POST",
          "headers": {
            "Authorization": "Bearer {{ enrichmentApiKey }}"
          },
          "data": {
            "domain": "{{ company.domain }}"
          }
        }
      },
      {
        "type": "request",
        "output": "financial_data",
        "config": {
          "url": "https://api.example.com/financials/{{ company.ticker }}",
          "method": "GET"
        }
      }
    ],
    "maxConcurrency": 3,
    "failFast": false,
    "mode": "all"
  }
}
```

### Parallel AI analysis

```json
{
  "type": "batch",
  "output": "analyses",
  "config": {
    "steps": [
      {
        "type": "agent",
        "output": "sentiment",
        "config": {
          "provider": "openai",
          "model": "gpt-4o-mini",
          "input": "{{ feedback }}",
          "outputSchema": {
            "type": "object",
            "properties": {
              "sentiment": { "type": "string" },
              "score": { "type": "number" }
            }
          }
        }
      },
      {
        "type": "agent",
        "output": "categories",
        "config": {
          "provider": "openai",
          "model": "gpt-4o-mini",
          "input": "{{ feedback }}",
          "outputSchema": {
            "type": "object",
            "properties": {
              "categories": {
                "type": "array",
                "items": { "type": "string" }
              }
            }
          }
        }
      },
      {
        "type": "agent",
        "output": "keywords",
        "config": {
          "provider": "openai",
          "model": "gpt-4o-mini",
          "input": "{{ feedback }}",
          "outputSchema": {
            "type": "object",
            "properties": {
              "keywords": {
                "type": "array",
                "items": { "type": "string" }
              }
            }
          }
        }
      }
    ],
    "maxConcurrency": 3,
    "mode": "all"
  }
}
```

## Best practices

### When to use Batch

* **Independent operations**: Steps don't depend on each other's results
* **Multiple API calls**: Fetching data from different endpoints
* **Parallel processing**: Analyzing multiple items simultaneously
* **Multi-channel operations**: Sending notifications via multiple channels
* **Data enrichment**: Gathering information from various sources

### When not to use Batch

* **Sequential dependencies**: When step B needs step A's output
* **Shared resources**: When steps might conflict (writing to same resource)
* **Rate-limited APIs**: When you cannot keep parallel calls within the provider's limits
* **Order matters**: When execution order is important

### Concurrency management

* Set `maxConcurrency` to respect API rate limits
* Consider your system's resource capacity
* Set concurrency based on the API's request limit and observed request duration; concurrency alone does not enforce a requests-per-second limit
* Start conservative and increase based on monitoring

### Error handling strategy

* Use `failFast: true` for validation scenarios where one failure invalidates all
* Use `failFast: false` for best-effort scenarios (notifications, optional enrichment)
* Always check for errors in results when `failFast: false`
* Log failed steps for debugging and monitoring

### Performance optimization

* Group similar operations in same batch
* Use appropriate `maxConcurrency` to balance speed and resources
* Monitor execution times to identify bottlenecks
* Consider caching for repeated operations

### Cost management

* Be mindful of parallel API calls costs
* Set `maxConcurrency` to control spending rate
* Monitor token usage for parallel Agent steps
* Consider sequential execution for expensive operations

## Common use cases

* **User profile aggregation**: Fetch user data from multiple services
* **Multi-source data enrichment**: Combine data from various APIs
* **Parallel notifications**: Send email, SMS, and push simultaneously
* **Batch processing**: Process multiple items with AI analysis
* **Validation checks**: Run multiple validation rules in parallel
* **Multi-provider requests**: Collect results from several providers, or race them when the first settled result is acceptable
* **Data synchronization**: Update multiple systems at once
* **Report generation**: Gather data from multiple sources for reports
* **Webhook dispatching**: Notify multiple endpoints in parallel
* **A/B testing**: Execute multiple variants simultaneously

## Troubleshooting

### Rate limit errors

* Reduce `maxConcurrency`
* Consider spreading load over time

### Inconsistent results

* Verify all steps have unique `output` names
* Check for race conditions in shared resources
* Ensure steps are truly independent
* Review context data availability
* Test with smaller batches first

### Timeout issues

* Check the individual step timeout and any timeout in its configuration. Raising them does not extend the execute endpoint's response wait, which defaults to and is capped at 30 seconds.
* Reduce `maxConcurrency` to avoid resource contention
* Check for slow external services
* Consider breaking into smaller batches
* Monitor network latency
