# Agent

Use Agent to turn text into structured JSON with a large language model. Define the expected fields in `outputSchema` and choose whether to use one model or compare several models.

## Overview

**Type:** `agent`

The configuration schema exposes OpenAI. Configure its credentials in the service deployment or supply the `apiKey` override. With multiple configured models, choose consensus voting, the first settled result, or all results. Agreement between models does not establish factual correctness.

## Configuration

Put these fields in the step's `config` object. The examples below are configuration fragments; see [Execute workflows](/docs/execute-workflows) for the full request format. Supply their referenced input variables in context.

### Provider (legacy)

* **Type:** Enum
* **Required:** No
* **Schema default:** `openai`. Supply `provider` explicitly when not using `providers`.
* **Options:** `openai`
* **Description:** LLM provider for single-provider mode (legacy configuration)

### Model (legacy)

* **Type:** String
* **Required:** No
* **Schema default:** `gpt-4o-mini`. Supply `model` explicitly when not using `providers`.
* **Description:** Model identifier for single-provider mode
* **Examples:** `gpt-4o-mini`, `gpt-4o`, `gpt-4.1-mini`, `gpt-4.1`

### Providers

* **Type:** Array of Objects
* **Required:** No
* **Description:** Model configurations for single-model or multi-model execution

Each provider object contains:

**provider**

* **Type:** Enum
* **Required:** Yes
* **Options:** `openai`
* **Description:** Provider name

**model**

* **Type:** String
* **Required:** Yes
* **Description:** Model identifier

**weight**

* **Type:** Number
* **Required:** No
* **Default:** `1`
* **Description:** Weight for voting in consensus mode

**temperature**

* **Type:** Number
* **Required:** No
* **Description:** Override temperature for this specific provider

```json
[
  { "provider": "openai", "model": "gpt-4o-mini", "weight": 1 },
  { "provider": "openai", "model": "gpt-4o", "weight": 2 }
]
```

### Mode

* **Type:** Enum
* **Required:** No
* **Options:** `single`, `consensus`, `fastest`, `all`
* **Description:** Execution mode (auto-detected if not specified)

**Modes:**

* `single`: Execute with one provider (default when 1 provider configured)
* `consensus`: Execute all providers and vote on results (default when multiple providers configured)
* `fastest`: Return the first settled result, including a failure
* `all`: Execute all providers and return all results

### Voting

* **Type:** Object
* **Required:** No
* **Description:** Voting configuration for consensus mode

**strategy**

* **Type:** Enum
* **Required:** No
* **Default:** `majority`
* **Options:** `majority`, `weighted`, `unanimous`
* **Description:** Voting strategy

**minAgreement**

* **Type:** Number
* **Required:** No
* **Default:** `0.66`
* **Min:** `0`
* **Max:** `1`
* **Description:** Minimum average agreement across the schema's top-level fields, from 0 to 1

```json
{
  "strategy": "weighted",
  "minAgreement": 0.7
}
```

### System prompt

* **Type:** String
* **Required:** No
* **Templatable:** Yes
* **Multiline:** Yes
* **Description:** System instructions that define the agent's role and behavior

```
You are an expert customer service analyst.
Analyze conversations objectively and provide structured feedback.
```

### Input

* **Type:** String
* **Required:** Yes
* **Templatable:** Yes
* **Multiline:** Yes
* **Description:** User message or input to send to the agent

```
{{ conversationTranscript }}
Analyze this email: {{ emailContent }}
```

### Output schema

The converter supports object properties, required fields, string enums and lengths, numeric bounds, integers, booleans, arrays, and null. Other JSON Schema keywords are not enforced by this converter.

* **Type:** Object (JSON Schema)
* **Required:** Yes for execution. Although the configuration schema marks it optional, the current structured-output SDK requires a schema in every mode.
* **Description:** Expected output structure using JSON Schema format

**type**

* **Type:** String
* **Required:** Specify `object` explicitly. The schema converter uses `type` to build validation; the declared schema default is not applied automatically.
* **Description:** Schema type

**properties**

* **Type:** Object
* **Required:** Yes
* **Description:** Object properties definition

**required**

* **Type:** Array of Strings
* **Required:** No
* **Description:** List of required property names

```json
{
  "type": "object",
  "properties": {
    "sentiment": {
      "type": "string",
      "enum": ["positive", "neutral", "negative"]
    },
    "confidence": {
      "type": "number"
    },
    "tags": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["sentiment", "confidence"]
}
```

### Temperature

* **Type:** Number
* **Required:** No
* **Schema default:** `0.3`; supply it explicitly if needed. The handler does not apply this default at runtime.
* **Min:** `0`
* **Max:** `2`
* **Description:** Controls sampling randomness where supported by the selected model. Low values do not guarantee identical or correct outputs.

### Max tokens

* **Type:** Number
* **Required:** No
* **Schema default:** `2000`
* **Min:** `1`
* **Max:** `16000`
* **Description:** Declared token-limit setting. The current handler sends this as `maxTokens`, but the installed SDK expects `maxOutputTokens`, so this setting does not currently enforce an output limit.

### Timeout

* **Type:** Number
* **Required:** No
* **Default:** `60`
* **Min:** `10`
* **Max:** `300`
* **Description:** Per-provider timeout in seconds. The step timeout defaults to 30 seconds. The workflow request timeout (`options.timeout`) defaults to and is capped at 30 seconds. This setting does not extend either timeout.

### API key

* **Type:** String
* **Required:** No
* **Templatable:** Yes
* **Description:** Override API key (uses global configuration if not provided)

## Output

The Agent step returns different outputs based on the execution mode:

### Single mode

```typescript
{
  success: boolean,
  output: object,        // The structured result matching your schema
  metadata: {
    mode: "single",
    providers: string[], // Provider and model used
    duration: number     // Execution time in ms
  },
  result: object,        // Same as output
  usage: object,         // Token usage statistics
  model: string,         // Model identifier
  finishReason: string   // Completion reason (e.g., "stop")
}
```

### Consensus mode

```typescript
{
  success: boolean,
  output: object,        // Consensus result
  metadata: {
    mode: "consensus",
    providers: string[], // All providers used
    agreement: number,   // Overall agreement score (0-1)
    breakdown: {         // Per-field voting details
      [field: string]: {
        agreement: number,
        votes: object,
        winner: any
      }
    }
  }
}
```

### Fastest mode

```typescript
{
  success: boolean,
  output: object,
  metadata: {
    mode: "fastest",
    providers: string[], // Winning provider
    duration: number
  },
  result: object,
  usage: object,
  model: string,
  finishReason: string
}
```

### All mode

```typescript
{
  success: boolean,
  output: {
    [providerKey: string]: object  // Results from each provider
  },
  metadata: {
    mode: "all",
    providers: string[],
    durations: number[]
  }
}
```

### Accessing the output

```
{{ agent_step.output.sentiment }}
{{ agent_step.metadata.agreement }}
{{ agent_step.output.confidence }}
```

## Examples

### Simple sentiment analysis

```json
{
  "provider": "openai",
  "model": "gpt-4o-mini",
  "temperature": 0.3,
  "input": "{{ userReview }}",
  "outputSchema": {
    "type": "object",
    "properties": {
      "sentiment": {
        "type": "string",
        "enum": ["positive", "neutral", "negative"]
      },
      "confidence": { "type": "number" }
    },
    "required": ["sentiment", "confidence"]
  }
}
```

### Call analysis with system prompt

```json
{
  "provider": "openai",
  "model": "gpt-4o-mini",
  "temperature": 0.3,
  "systemPrompt": "You are an expert call quality analyst.\nAnalyze the conversation objectively and provide structured feedback.",
  "input": "{{ conversationTranscript }}",
  "outputSchema": {
    "type": "object",
    "properties": {
      "title": { "type": "string" },
      "summary": { "type": "string" },
      "sentiment": {
        "type": "string",
        "enum": ["positive", "neutral", "negative"]
      },
      "resolution": {
        "type": "string",
        "enum": ["Résolu", "Partiellement résolu", "Non résolu"]
      },
      "clarity": { "type": "string" },
      "error": { "type": "boolean" },
      "errors": {
        "type": "array",
        "items": { "type": "string" }
      }
    },
    "required": ["title", "summary", "sentiment", "resolution", "clarity", "error", "errors"]
  }
}
```

### Multi-model consensus

```json
{
  "providers": [
    { "provider": "openai", "model": "gpt-4o-mini", "weight": 1 },
    { "provider": "openai", "model": "gpt-4o", "weight": 2 }
  ],
  "mode": "consensus",
  "voting": {
    "strategy": "weighted",
    "minAgreement": 0.7
  },
  "temperature": 0,
  "input": "{{ emailContent }}",
  "outputSchema": {
    "type": "object",
    "properties": {
      "patientName": { "type": "string" },
      "appointmentDate": { "type": "string" },
      "examType": { "type": "string" },
      "urgency": {
        "type": "string",
        "enum": ["low", "medium", "high"]
      }
    },
    "required": ["patientName", "appointmentDate", "examType"]
  }
}
```

### Fastest response mode

```json
{
  "providers": [
    { "provider": "openai", "model": "gpt-4o-mini" },
    { "provider": "openai", "model": "gpt-4o" }
  ],
  "mode": "fastest",
  "input": "Summarize: {{ article }}",
  "outputSchema": {
    "type": "object",
    "properties": {
      "summary": { "type": "string" },
      "keyPoints": {
        "type": "array",
        "items": { "type": "string" }
      }
    },
    "required": ["summary"]
  }
}
```

### All providers mode

```json
{
  "providers": [
    { "provider": "openai", "model": "gpt-4o-mini" },
    { "provider": "openai", "model": "gpt-4o" }
  ],
  "mode": "all",
  "input": "{{ creativePrompt }}",
  "outputSchema": {
    "type": "object",
    "properties": {
      "idea": { "type": "string" },
      "reasoning": { "type": "string" }
    },
    "required": ["idea"]
  }
}
```

## Execution modes

### Single mode

Executes with one configured model. If `providers` is present, uses its first entry; otherwise uses `provider` and `model`.

**When to use:**

* Simple classification tasks
* Cost-effective operations
* Trusted single model is sufficient

### Consensus mode

Executes multiple models and votes on the results for each field. Agreement measures how consistently the configured models answered; it does not establish factual correctness.

**When to use:**

* Structured extraction requiring consistency checks
* Comparing outputs before human review or independent validation

**Voting strategies:**

* `majority`: Each model gets one vote, winner is most common value
* `weighted`: Models vote with their specified weight
* `unanimous`: All models must agree (100% agreement required)

### Fastest mode

Executes all models in parallel and returns the first settled result. If the first result is a failure, the step fails even if another provider could succeed. Losing results are discarded, but the underlying provider requests are not cancelled.

**When to use:**

* Time-sensitive operations
* Comparing response latency across configured models

### All mode

Executes all models and returns all results for comparison or ensemble processing.

**When to use:**

* A/B testing different models
* Comparative analysis
* Ensemble techniques
* Research and development

## Error handling

### Validation errors

**Missing providers for multi-provider modes:**

```typescript
{
  code: "FLUX_AGENT_VALIDATION_ERROR",
  message: "providers required for [mode] mode"
}
```

**Missing output schema for consensus:**

```typescript
{
  code: "FLUX_AGENT_VALIDATION_ERROR",
  message: "outputSchema required for consensus mode"
}
```

### Consensus errors

**Insufficient agreement:**

```typescript
{
  code: "FLUX_AGENT_CONSENSUS_ERROR",
  message: "Consensus failed: agreement 0.55 < 0.66"
}
```

**Unanimous voting failed:**

```typescript
{
  code: "FLUX_AGENT_CONSENSUS_ERROR",
  message: "Unanimous voting failed for field (agreement: 0.67)"
}
```

### Execution errors

**Timeout:**

```typescript
{
  code: "FLUX_AGENT_ERROR",
  message: "Agent execution failed: Agent execution timeout"
}
```

**General errors:**

```typescript
{
  code: "FLUX_AGENT_ERROR",
  message: "Agent execution failed: [error details]"
}
```

## Best practices

### Schema design

* Define clear, specific output schemas
* Use enums for classification fields
* Include required fields for critical data
* Keep schemas simple and focused
* Validate schema structure before deployment

### Prompt engineering

* Be specific and clear in your instructions
* Provide context in the system prompt
* Include examples when helpful
* Use consistent terminology
* Test prompts with different inputs

### Temperature settings

For models that support temperature, start with a low value for extraction or classification and test higher values for content generation. Check the model's accepted range. The configuration schema's 0-2 range does not establish what every model accepts, and lower values do not guarantee correctness or identical results.

### Multi-model strategy

* Use consensus to measure agreement, with independent validation for consequential decisions
* Set appropriate weights based on model capabilities
* Adjust `minAgreement` based on risk tolerance
* Use `fastest` mode for latency-sensitive operations
* Use `all` mode for testing and comparison

### Cost optimization

* Compare the cost and quality of models supported by your provider
* Account for every configured provider call in multi-provider modes
* Do not rely on `maxTokens` as an enforced cap in the current handler
* Use single mode when consensus isn't needed
* Monitor token usage in outputs

### Performance

* Set realistic timeout values
* Use `fastest` when latency matters and a failure from the first settled request is acceptable.
* Cache responses when appropriate
* Batch similar requests when possible

### Security

* Never expose API keys in logs
* Use environment variables for credentials
* Validate and sanitize user inputs
* Review agent outputs before using in production
* Limit how often callers execute workflows to stay within provider rate limits.

## Common use cases

* **Content moderation:** Classify user-generated content for policy violations
* **Data extraction:** Extract structured data from unstructured text
* **Sentiment analysis:** Analyze customer feedback and reviews
* **Call transcript analysis:** Evaluate customer service interactions
* **Document classification:** Categorize documents by type or topic
* **Support triage:** Classify urgency from customer requests
* **Email parsing:** Extract appointment details, contact info, action items
* **Document review assistance:** Extract clauses for a qualified reviewer
* **Product categorization:** Classify products from descriptions
* **Lead scoring:** Evaluate sales leads from communication
* **Resume screening:** Extract qualifications and match to requirements
* **Survey analysis:** Analyze open-ended survey responses

## Troubleshooting

### Schema validation failures

* Verify JSON Schema syntax is correct
* Ensure property types match expected data
* Check that required fields are specified
* Test schema with simple examples first

### Inconsistent outputs

* Lower temperature where the model supports it, then test whether output consistency improves.
* Improve system prompt clarity
* Add examples to guide the model
* Use consensus mode to compare model agreement, and validate the result independently.
* Increase `minAgreement` threshold

### Low agreement in consensus

* Review if models are interpreting instructions differently
* Simplify the output schema
* Make the system prompt more specific
* Check if input data is ambiguous
* Consider adjusting provider weights

### High costs

* Use smaller models when appropriate
* Use a provider-side output limit where available; the current handler does not enforce `maxTokens`
* Optimize system prompts to be concise
* Switch from consensus to single mode when acceptable
* Monitor usage patterns and adjust
