Ollama Integration Guide¶

CodeOptiX supports local Ollama models, allowing you to run evaluations without API keys! ✅ Now working correctly - generates code and provides proper security evaluations.

🎯 Why Ollama?¶

• ✅ No API key required - Perfect for open-source users
• ✅ Privacy-friendly - All processing happens locally
• ✅ Free to use - No cloud costs
• ✅ Works offline - No internet connection needed
• ✅ Flexible models - Choose from many open-source models

✅ Recent Updates¶

CodeOptiX now works correctly with Ollama! Recent fixes ensure:

• ✅ Proper code generation (not conversational responses)
• ✅ Accurate security evaluations (detects real issues)
• ✅ Meaningful scores (not always 100%)
• ✅ Full evaluation pipeline support

🚀 Try the Demo¶

Test the Ollama integration with our interactive demo:

python examples/ollama_demo.py

This demo shows Ollama generating code, detecting security issues, and providing proper evaluation scores.

📦 Installation¶

Step 1: Install Ollama¶

Visit https://ollama.com and install Ollama for your platform:

# macOS
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows
# Download from https://ollama.com/download

Step 2: Start Ollama Service¶

ollama serve

This starts the Ollama service on http://localhost:11434 (default).

Step 3: Pull a Model¶

Pull a model you want to use:

# Recommended models for CodeOptiX
ollama pull llama3.1:8b        # Fast, efficient (4.9 GB)
ollama pull qwen3:8b           # Alternative 8B model (5.2 GB)
ollama pull gpt-oss:120b       # Large, powerful (65 GB)
ollama pull gpt-oss:20b        # Medium model (13 GB)
ollama pull llama3.2:3b        # Lightweight (2.0 GB)

See available models:

ollama list

🚀 Quick Start¶

Basic Usage¶

codeoptix eval \
  --agent basic \
  --behaviors insecure-code \
  --llm-provider ollama

With Configuration File¶

Create ollama-config.yaml:

adapter:
  llm_config:
    provider: ollama
    model: llama3.1:8b  # Or gpt-oss:120b, qwen3:8b, etc.
    # No api_key needed!

evaluation:
  scenario_generator:
    num_scenarios: 2
    use_bloom: false

behaviors:
  insecure-code:
    enabled: true

Run:

codeoptix eval \
  --agent basic \
  --behaviors insecure-code \
  --config ollama-config.yaml \
  --llm-provider ollama

⚙️ Configuration¶

In Config File¶

adapter:
  llm_config:
    provider: ollama
    model: llama3.1:8b  # Choose your model
    # No api_key needed!

Via Environment Variable¶

export CODEOPTIX_LLM_PROVIDER=ollama
codeoptix eval --agent basic --behaviors insecure-code

Custom Ollama URL¶

By default, Ollama runs on http://localhost:11434. To use a different URL:

export OLLAMA_BASE_URL=http://localhost:11434
# Or use a remote Ollama instance
export OLLAMA_BASE_URL=http://remote-server:11434

📋 Available Models¶

Recommended Models¶

List Available Models¶

ollama list

Pull a Model¶

ollama pull <model-name>

💡 Usage Examples¶

Example 1: Try the Interactive Demo ⭐¶

See Ollama working in action:

python examples/ollama_demo.py

This demo shows code generation, security evaluation, and proper scoring.

Example 2: Basic Evaluation¶

codeoptix eval \
  --agent basic \
  --behaviors insecure-code \
  --llm-provider ollama

Example 3: With Custom Config¶

codeoptix eval \
  --agent basic \
  --behaviors insecure-code \
  --config examples/configs/ollama-insecure-code.yaml \
  --llm-provider ollama

Example 4: Multiple Behaviors¶

codeoptix eval \
  --agent basic \
  --behaviors insecure-code,vacuous-tests \
  --llm-provider ollama

Example 5: Verbose Output¶

codeoptix eval \
  --agent basic \
  --behaviors insecure-code \
  --llm-provider ollama \
  --verbose

Example 6: CI/CD Integration¶

# .github/workflows/codeoptix.yml
name: CodeOptiX Check
on: [pull_request]
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - name: Install Ollama
        run: |
          curl -fsSL https://ollama.com/install.sh | sh
          ollama pull llama3.1:8b
      - name: Install CodeOptiX
        run: pip install codeoptix
      - name: Run CodeOptiX
        run: |
          codeoptix ci \
            --agent basic \
            --behaviors insecure-code \
            --llm-provider ollama \
            --fail-on-failure

🔧 Troubleshooting¶

Ollama Not Running¶

Error: Failed to contact Ollama at http://localhost:11434

Solution:

# Start Ollama service
ollama serve

Model Not Found¶

Error: Model not available

Solution:

# Pull the model
ollama pull llama3.1:8b

# Verify it's available
ollama list

Connection Timeout¶

Error: Timeout when calling Ollama

Solution: - Large models (like gpt-oss:120b) may take longer - CodeOptiX uses a 300-second timeout by default - Ensure you have enough RAM for the model - Try a smaller model if timeouts persist

Custom Port¶

If Ollama is running on a different port:

export OLLAMA_BASE_URL=http://localhost:11435

🆚 Ollama vs Cloud Providers¶

Choose Ollama if: - You want privacy - You want to avoid API costs - You have sufficient local compute - You want to work offline

Choose Cloud Providers if: - You need the latest models - You don't have local compute - You need maximum speed - You're okay with API costs

⚠️ Known Limitations¶

Evolution Support¶

• Limited support for codeoptix evolve: The evolution feature uses GEPA optimization, which requires processing very long prompts. Ollama may fail with timeouts on complex evolution tasks.
• Recommendation: Use cloud providers (OpenAI, Anthropic, Google) for full evolution capabilities.

Performance Considerations¶

• Large models (e.g., gpt-oss:120b) require significant RAM and may be slow on consumer hardware
• Evolution tasks are computationally intensive and may not complete reliably with Ollama

For advanced features like evolution, consider cloud providers or contact us for tailored enterprise solutions.

📚 Next Steps¶

• Quick Start Guide
• Configuration Guide
• Behavior Specifications
• GEPA Optimization
• Bloom Evaluations

🐛 Issues?¶

If you encounter problems:

1. Check Troubleshooting above
2. Verify Ollama is running: ollama list
3. Check model is pulled: ollama list
4. Open an issue on GitHub