🚀 Agent Optimization Strategy

SuperOptiX provides universal optimization across all 6 major agent frameworks using GEPA (Genetic-Pareto) as the primary optimizer.

🌟 Key Achievement: The world's first framework-agnostic optimizer that delivers proven results across DSPy, OpenAI SDK, CrewAI, Google ADK, Microsoft Agent Framework, and DeepAgents!

Overview

Agent optimization in SuperOptiX follows a GEPA-first strategy that works seamlessly across all supported frameworks:

🔧 Universal Optimizer: Same GEPA optimizer works for all frameworks
📊 Proven Results: DSPy 37.5% → 80%, OpenAI/CrewAI 100% pass rates
⚡ Sample Efficient: Achieves improvements with just 3-10 scenarios
🎯 Framework Agnostic: Build with any framework, optimize with one tool
🔄 Consistent Workflow: Same commands work regardless of framework

🎯 GEPA: The Universal Optimization Strategy

Why GEPA-First?

GEPA (Genetic-Pareto) is SuperOptiX's universal optimizer that delivers consistent results across all frameworks:

Framework	Variables Optimized	Proven Results	Status
🔬 DSPy	10+ variables	37.5% → 80% (+42.5 pts)	Proven
🤖 OpenAI SDK	1 variable (instructions)	100% pass rate	Proven
👥 CrewAI	5 variables (role+goal+backstory+task)	100% pass rate	Proven
🔮 Google ADK	1 variable (instruction)	Ready for optimization	Ready
🏢 Microsoft	1 variable (instructions)	Ready for optimization	Ready
🌊 DeepAgents	1 variable (system_prompt)	Ready for optimization	Ready

GEPA's Advantages

🔧 Framework Agnostic: The ONLY optimizer that works across all major frameworks 📊 Sample Efficiency: Achieves significant improvements with just 3-10 scenarios 🎯 Domain Adaptable: Incorporates domain-specific feedback effectively 💡 Interpretable: Generates human-readable prompt improvements 🔄 Multi-Objective: Optimizes for multiple criteria simultaneously

🚀 Universal Optimization Workflow

Step 1: Choose Your Framework & Pull Agent

🔬 DSPy🤖 OpenAI SDK👥 CrewAI🔮 Google ADK🏢 Microsoft🌊 DeepAgents

super agent pull sentiment_analyzer
super agent compile sentiment_analyzer
super agent evaluate sentiment_analyzer

super agent pull assistant_openai
super agent compile assistant_openai
super agent evaluate assistant_openai

super agent pull researcher_crew
super agent compile researcher_crew
super agent evaluate researcher_crew

super agent pull assistant_adk
super agent compile assistant_adk
super agent evaluate assistant_adk

super agent pull assistant_microsoft
super agent compile assistant_microsoft
super agent evaluate assistant_microsoft

super agent pull research_agent_deepagents
super agent compile research_agent_deepagents
super agent evaluate research_agent_deepagents

Step 2: Optimize with GEPA (Same Command for ALL!)

# Universal GEPA command - works on ANY framework!
super agent optimize <agent_name> --auto medium

# Examples for each framework:
super agent optimize sentiment_analyzer --auto medium        # DSPy
super agent optimize assistant_openai --auto medium          # OpenAI SDK
super agent optimize researcher_crew --auto medium           # CrewAI
super agent optimize assistant_adk --auto medium             # Google ADK
super agent optimize assistant_microsoft --auto medium       # Microsoft
super agent optimize research_agent_deepagents --auto medium # DeepAgents

Step 3: Evaluate & Deploy

# Evaluate optimized version
super agent evaluate <agent_name>  # automatically loads optimized weights

# Run in production
super agent run <agent_name>

⚙️ GEPA Configuration Options

Automatic Mode (Recommended) ⭐

# Works for ANY framework!
super agent optimize <agent_name> --auto [light|medium|intensive]

Optimization Levels: - light: Quick optimization (2-3 iterations, ~5 minutes) - medium: Balanced optimization (5 iterations, ~10-15 minutes) ⭐ Recommended - intensive: Thorough optimization (10+ iterations, ~30+ minutes)

Manual Configuration

super agent optimize <agent_name> --optimizer gepa --max-iterations 5

Advanced Configuration (via playbook)

Works for all frameworks! Edit your agent playbook:

spec:
  optimization:
    optimizer:
      name: GEPA
      params:
        metric: answer_exact_match    # Evaluation metric
        auto: medium                  # Budget: light, medium, intensive
        reflection_lm: qwen3:8b       # Model for reflection
        reflection_minibatch_size: 3   # Examples per reflection
        skip_perfect_score: true      # Skip if already perfect

📊 Optimization Results by Framework

DSPy: Sentiment Analysis Agent

Framework: DSPy (Stanford Research Framework)
Variables Optimized: 10+ (signature instructions, field descriptions, reasoning steps, etc.)

Before GEPA:

Pass Rate: 37.5% (3/8 scenarios)

After GEPA Optimization (5 iterations, medium mode):

Pass Rate: 80.0% (6.5/8 scenarios)
Improvement: +42.5 percentage points 🏆

What GEPA Improved: - Better nuanced sentiment identification - Improved sarcasm and context handling - More accurate confidence scores - Clearer reasoning chains

OpenAI SDK: AI Assistant

Framework: OpenAI Agents SDK
Variables Optimized: 1 (instructions)

Before GEPA:

Pass Rate: 100% (4/4 scenarios)

After GEPA Optimization:

Pass Rate: 100% (4/4 scenarios)
Improvement: Maintained excellence ```

**What GEPA Improved**:
- Enhanced response quality and clarity
- Better instruction following
- More consistent behavior patterns
- Improved instruction structure

---

### **CrewAI: Research Crew (Phase 2)**

**Framework**: CrewAI (Multi-Agent Collaboration)  
**Variables Optimized**: 5 (role, goal, backstory, task description, expected output)

**Before GEPA**:

Pass Rate: 75% (¾ scenarios)

**After GEPA Optimization** (combined agent+task optimization):

Pass Rate: 100% (4/4 scenarios) Improvement: +25 percentage points ⭐

**What GEPA Improved**:
- Clearer role definitions
- Better goal alignment with tasks
- Improved agent-task coordination
- Enhanced task output quality
- Better multi-agent collaboration patterns

## 🎯 When to Use Alternative Optimizers

While GEPA is recommended for most cases, here are alternatives for specific scenarios:

### **SIMBA** (DSPy Only)
**Use When**: Large datasets, performance-critical applications
```bash
super agent optimize <agent> --optimizer simba

MIPROv2 (DSPy Only)

Use When: Instruction-following tasks, multi-step workflows

super agent optimize <agent> --optimizer miprov2

BootstrapFewShot (DSPy Only)

Use When: Getting started, limited training data

super agent optimize <agent> --optimizer bootstrapfewshot

📈 Best Practices

1. Always Establish Baseline

# CRITICAL: Always evaluate before optimizing
super agent evaluate <agent_name>

2. Start with Medium Budget

# Start with balanced optimization
super agent optimize <agent_name> --auto medium

# Increase if results justify cost
super agent optimize <agent_name> --auto intensive

3. Validate Improvements

# Always re-evaluate after optimization
super agent evaluate <agent_name>  # automatically loads optimized weights

4. Use Quality Training Scenarios

feature_specifications:
  scenarios:
    - name: comprehensive_test
      description: Cover main patterns and edge cases
      input:
        problem: "Well-defined, realistic problem"
      expected_output:
        answer: "Complete expected response with reasoning"

5. Monitor Resource Usage

Memory Requirements: - GEPA: ~25GB peak (main model + reflection model) - Local Models: Ollama works great for free optimization - Cloud Models: Monitor API usage for cost control

🔧 Troubleshooting

Common Issues

Issue: "Optimization failed" Solution: Check BDD scenarios are well-defined and evaluable

Issue: "Memory error" Solution: Use --auto light or switch to smaller models

Issue: "No improvement after optimization" Solution: Check evaluation metrics and scenario quality

Performance Tips

Local Models: Use Ollama for free, unlimited optimization
Cloud Models: Start with light budget to test effectiveness
Batch Processing: Run multiple agents in parallel for efficiency

🚀 Next Steps

After optimization:

Deploy: Use super agent run with optimized weights
Monitor: Track performance in production
Iterate: Re-optimize when adding new scenarios
Scale: Apply same workflow to other frameworks

Learn More: - GEPA Optimization Guide - Detailed GEPA documentation - Multi-Framework Support - Framework comparisons - Evaluation & Testing - Testing strategies