🤖 LLM Setup Guide

Welcome to SuperOptiX's LLM Setup Guide! This guide will help you configure and use local language models for your AI agents. We focus on local models for privacy, speed, and cost-effectiveness.

🚀 Quick Start

New to local models? Start with Ollama - it's the easiest option for beginners!

🎯 Overview

SuperOptiX supports multiple local model backends, each optimized for different use cases:

Backend	Best For	Platform	Ease of Use	Performance
🦙 Ollama	Beginners, All platforms	Cross-platform	⭐⭐⭐⭐⭐	⭐⭐⭐⭐
🍎 MLX	Apple Silicon users	macOS only	⭐⭐⭐⭐	⭐⭐⭐⭐⭐
🎮 LM Studio	Windows users	Windows/macOS	⭐⭐⭐	⭐⭐⭐⭐
🤗 HuggingFace	Advanced users	All platforms	⭐⭐	⭐⭐⭐⭐⭐

Production Inference Engines

vLLM, SGLang, and TGI are not included in the current version of SuperOptiX. These production-worthy inference engines are part of our enterprise offering.

🦙 Ollama (Recommended)

Ollama is the easiest way to run local models on any platform. Perfect for beginners!

🚀 Quick Setup

🍎 macOS🐧 Linux🪟 Windows

Bash

# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# Start Ollama (runs in background)
ollama serve

Bash

# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# Start Ollama
ollama serve

PowerShell

# Download from https://ollama.ai/download
# Or use winget
winget install Ollama.Ollama

# Start Ollama
ollama serve

📦 Install Models with SuperOptiX

Bash

# Install recommended models by tier
super model install llama3.2:1b      # Oracles tier - Small tasks, fast responses
super model install llama3.2:8b      # Genies tier - Complex reasoning, tools, memory
super model install llama3.2:3b      # Alternative small model
super model install qwen2.5:7b       # Great all-rounder

Show Output

Text Only

🚀 SuperOptiX Model Intelligence - Installing llama3.2:3b
🦙 Pulling model llama3.2:3b from Ollama...
⏳ This may take a few minutes depending on your internet connection and model size.

pulling manifest 
pulling dde5aa3fc5ff: 100% ▕██████████████████████████████████████████████▏ 2.0 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████████████████████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████████████████████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████████████████████████████████▏ 6.0 KB                         
pulling 56bb8bd477a5: 100% ▕██████████████████████████████████████████████▏   96 B                         
pulling 34bb5ab01051: 100% ▕██████████████████████████████████████████████▏  561 B                         
verifying sha256 digest 
writing manifest 
success 
✅ Model pulled successfully!

💡 You can now use it with SuperOptiX:
  super model dspy ollama/llama3.2:3b

📊 Model details:
  • Size: small
  • Task: chat
  • Parameters: 3B

🎉 Installation completed successfully!
🦙 Ollama running on http://localhost:11434 ready to use with SuperOptiX!

🖥️ Server Management

💡 Important: Ollama automatically starts its server when you run ollama serve or when you first use a model. You don't need to manually start the server unless you want custom configuration.

Bash

# Start Ollama server (runs on port 11434 by default)
ollama serve

# Or simply use a model - server starts automatically
ollama run llama3.2:1b

🔧 Custom Configuration: Only start the server manually if you need: - Different port: OLLAMA_HOST=0.0.0.0:8080 ollama serve - Custom model path: OLLAMA_MODELS=/custom/path ollama serve - GPU configuration: OLLAMA_GPU_LAYERS=35 ollama serve

✅ Automatic Detection: SuperOptiX automatically detects and connects to Ollama running on the default port (11434). No additional configuration needed!

📋 Manage Ollama Models

Bash

# List installed models
super model list --backend ollama

Example Output:

Text Only

                🚀 SuperOptiX Model Intelligence - 3 models                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━┓
┃ Model                   ┃  Backend  ┃    Status    ┃  Size   ┃   Task    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━┩
│ llama3.1:8b             │ 🦙 ollama │ ✅ installed │ medium  │   chat    │
│ llama3.2:1b             │ 🦙 ollama │ ✅ installed │  tiny   │   chat    │
│ nomic-embed-text:latest │ 🦙 ollama │ ✅ installed │ Unknown │ embedding │
└─────────────────────────┴───────────┴──────────────┴─────────┴───────────┘

Bash

# Get model information
super model info llama3.2:3b

# List all available models
super model list --all

🍎 MLX (Apple Silicon)

MLX is Apple's native machine learning framework, offering blazing-fast inference on Apple Silicon Macs.

Apple Silicon Only

MLX only works on Apple Silicon Macs (M1, M2, M3). If you're on Intel Mac, use Ollama instead.

🚀 Setup MLX

Bash

# Install MLX dependencies
pip install mlx-lm

# Or install with SuperOptiX
pip install "superoptix[mlx]"

📦 Install MLX Models

Bash

# Install popular MLX models
super model install -b mlx mlx-community/phi-2
super model install -b mlx mlx-community/Llama-3.2-3B-Instruct-4bit
super model install -b mlx mlx-community/Mistral-7B-Instruct-v0.2-4bit

🖥️ Start MLX Servers

Bash

# Start MLX server on specific port
super model server mlx phi-2 --port 8000
super model server mlx mlx-community/Llama-3.2-3B-Instruct-4bit --port 8000

Example Output:

Text Only

🍎 MLX Local Server
Starting MLX server for mlx-community_Llama-3.2-3B-Instruct-4bit on port 8000...
🚀 Starting MLX server...
📡 Server will be available at: http://localhost:8000
💡 Use this URL in your playbook's api_base configuration
🔧 Manual server startup command:
   python -m mlx_lm.server --model mlx-community_Llama-3.2-3B-Instruct-4bit --port 8000
📋 Example playbook configuration:
   language_model:
     provider: mlx
     model: mlx-community_Llama-3.2-3B-Instruct-4bit
     api_base: http://localhost:8000
🔄 Executing: /path/to/python -m mlx_lm.server --model mlx-community_Llama-3.2-3B-Instruct-4bit --port 8000
⏳ Server is starting... (Press Ctrl+C to stop)

📋 Manage MLX Models

Bash

# List MLX models
super model list --backend mlx

Example Output:

Text Only

                    🚀 SuperOptiX Model Intelligence - 1 models                     
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━┓
┃ Model                                    ┃ Backend ┃    Status    ┃ Size  ┃ Task ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━┩
│ mlx-community_Llama-3.2-3B-Instruct-4bit │ 🍎 mlx  │ ✅ installed │ small │ chat │
└──────────────────────────────────────────┴─────────┴──────────────┴───────┴──────┘

Bash

# Get model information
super model info mlx-community/phi-2
super model info mlx-community_Llama-3.2-3B-Instruct-4bit

# Models are ready to use with SuperOptiX agents

🎮 LM Studio

LM Studio provides a user-friendly interface for running local models, especially popular on Windows.

🚀 Setup LM Studio

1. Download LM Studio from https://lmstudio.ai
2. Install and launch LM Studio
3. Download a model through the interface
4. Start the server (default port: 1234)

📦 Install Models with SuperOptiX

Bash

# Install models (use the name from LM Studio)
super model install -b lmstudio llama-3.2-1b-instruct
super model install -b lmstudio llama-3.2-3b
super model install -b lmstudio your-model-name

🖥️ Start LM Studio Servers

Bash

# Start server with specific model
super model server lmstudio llama-3.2-1b-instruct --port 1234
super model server lmstudio llama-3.2-3b --port 1234

Example Output:

Text Only

🎮 LM Studio Local Server
Starting LM Studio server for llama-3.2-1b-instruct on port 1234...
🚀 Starting LM Studio server...
📡 Server will be available at: http://localhost:1234
💡 Use this URL in your playbook's api_base configuration
🔧 Manual server startup command:
   # Start server in LM Studio app first, then connect
📋 Example playbook configuration:
   language_model:
     provider: lmstudio
     model: llama-3.2-1b-instruct
     api_base: http://localhost:1234
⏳ Server is starting... (Press Ctrl+C to stop)

📋 Manage LM Studio Models

Bash

# List LM Studio models
super model list --backend lmstudio

Example Output:

Text Only

                  🚀 SuperOptiX Model Intelligence - 3 models                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━┓
┃ Model                          ┃   Backend   ┃    Status    ┃  Size  ┃ Task ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━┩
│ llama-3.2-1b-instruct          │ 🎮 lmstudio │ ✅ installed │ small  │ chat │
│ llama-3.3-70b-instruct         │ 🎮 lmstudio │ ✅ installed │ large  │ chat │
│ llama-4-scout-17b-16e-instruct │ 🎮 lmstudio │ ✅ installed │ medium │ chat │
└────────────────────────────────┴─────────────┴──────────────┴────────┴──────┘

Bash

# Get model information
super model info llama-3.2-1b-instruct

# Models are ready to use with SuperOptiX agents

🤗 HuggingFace

HuggingFace offers access to thousands of models, perfect for advanced users who want maximum flexibility.

🚀 Setup HuggingFace

Bash

# Install HuggingFace dependencies
pip install transformers torch fastapi uvicorn

# Or install with SuperOptiX
pip install "superoptix[huggingface]"

📦 Install HuggingFace Models

Bash

# Install popular models
super model install -b huggingface microsoft/Phi-4
super model install -b huggingface microsoft/DialoGPT-small
super model install -b huggingface microsoft/DialoGPT-medium
super model install -b huggingface meta-llama/Llama-2-7b-chat-hf

🖥️ Start HuggingFace Servers

Bash

# Start server with specific model
super model server huggingface microsoft/Phi-4 --port 8001
super model server huggingface microsoft/DialoGPT-small --port 8001
super model server huggingface microsoft/DialoGPT-medium --port 8001

Example Output:

Text Only

🤗 HuggingFace Local Server
Starting HuggingFace server for microsoft/DialoGPT-small on port 8002...
🚀 Starting HuggingFace server...
📡 Server will be available at: http://localhost:8002
💡 Use this URL in your playbook's api_base configuration
🔧 Manual server startup command:
   python -m superoptix.models.backends.huggingface_server microsoft/DialoGPT-small --port 8002
📋 Example playbook configuration:
   language_model:
     provider: huggingface
     model: microsoft/DialoGPT-small
     api_base: http://localhost:8002
Device set to use cpu
INFO:     Started server process [4652]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8002 (Press CTRL+C to quit)

📋 Manage HuggingFace Models

Bash

# List HuggingFace models
super model list --backend huggingface

Example Output:

Text Only

                🚀 SuperOptiX Model Intelligence - 2 models                
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━┓
┃ Model                    ┃    Backend     ┃    Status    ┃ Size  ┃ Task ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━┩
│ microsoft/DialoGPT-small │ 🤗 huggingface │ ✅ installed │ small │ chat │
│ microsoft/Phi-4          │ 🤗 huggingface │ ✅ installed │ small │ chat │
└──────────────────────────┴────────────────┴──────────────┴───────┴──────┘

Bash

# Get model information
super model info microsoft/Phi-4
super model info microsoft/DialoGPT-small

# Models are ready to use with SuperOptiX agents

🎯 Model Management Commands

🖥️ Server Commands

Bash

# Get help for server commands
super model server --help

Example Output:

Text Only

usage: super model server [-h] [--port PORT] {mlx,huggingface,lmstudio} model_name

🚀 Start local model servers for MLX, HuggingFace, or LM Studio. Examples: 
super model server mlx mlx-community/Llama-3.2-3B-Instruct-4bit 
super model server huggingface microsoft/DialoGPT-small --port 8001
super model server lmstudio llama-3.2-1b-instruct 

Backends: 
mlx Apple Silicon optimized (default: port 8000) 
huggingface Transformers models (default: port 8001) 
lmstudio Desktop app models (default: port 1234) 

Note: Ollama servers use 'ollama serve' command separately.

positional arguments:
  {mlx,huggingface,lmstudio}  Backend type
  model_name                   Model name to start server for

options:
  -h, --help                   show this help message and exit
  --port PORT, -p PORT         Port to run server on

📋 List and Explore Models

Bash

# List all installed models
super model list

Example Output:

Text Only

                           🚀 SuperOptiX Model Intelligence - 9 models                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━┓
┃ Model                                    ┃    Backend     ┃    Status    ┃  Size   ┃   Task    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━┩
│ llama-3.2-1b-instruct                    │  🎮 lmstudio   │ ✅ installed │  small  │   chat    │
│ llama-3.3-70b-instruct                   │  🎮 lmstudio   │ ✅ installed │  large  │   chat    │
│ llama-4-scout-17b-16e-instruct           │  🎮 lmstudio   │ ✅ installed │ medium  │   chat    │
│ llama3.1:8b                              │   🦙 ollama    │ ✅ installed │ medium  │   chat    │
│ llama3.2:1b                              │   🦙 ollama    │ ✅ installed │  tiny   │   chat    │
│ microsoft/DialoGPT-small                 │ 🤗 huggingface │ ✅ installed │  small  │   chat    │
│ microsoft/Phi-4                          │ 🤗 huggingface │ ✅ installed │  small  │   chat    │
│ mlx-community_Llama-3.2-3B-Instruct-4bit │     🍎 mlx     │ ✅ installed │  small  │   chat    │
│ nomic-embed-text:latest                  │   🦙 ollama    │ ✅ installed │ Unknown │ embedding │
└──────────────────────────────────────────┴────────────────┴──────────────┴─────────┴───────────┘

🔍 Discover more models: super model discover
📥 Install a model: super model install <model_name>

Bash

# List all available models (including uninstalled)
super model list --all

# Filter by backend
super model list --backend ollama
super model list --backend mlx
super model list --backend lmstudio
super model list --backend huggingface

# Verbose information
super model list --verbose

📊 Get Model Information

Bash

# Get detailed model info
super model info llama3.2:3b
super model info mlx-community/phi-2
super model info microsoft/Phi-4
super model info llama-3.2-1b-instruct

🎯 Choose Your Setup

🚀 Beginner (Recommended)

Bash

# 1. Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# 2. Install SuperOptiX
pip install superoptix

# 3. Install a model
super model install llama3.2:3b

# 4. Models are ready to use with SuperOptiX agents

🍎 Apple Silicon User

Bash

# 1. Install MLX dependencies
pip install mlx-lm

# 2. Install SuperOptiX
pip install superoptix

# 3. Install MLX model
super model install -b mlx mlx-community/phi-2

# 4. Start server
super model server mlx phi-2 --port 8000

# 5. Models are ready to use with SuperOptiX agents

🎮 Windows User

Bash

# 1. Install LM Studio from https://lmstudio.ai
# 2. Download a model in LM Studio
# 3. Start server in LM Studio

# 4. Install SuperOptiX
pip install superoptix

# 5. Connect to LM Studio
super model server lmstudio your-model-name --port 1234

# 6. Models are ready to use with SuperOptiX agents

🤗 Advanced User

Bash

# 1. Install HuggingFace dependencies
pip install transformers torch fastapi uvicorn

# 2. Install SuperOptiX
pip install superoptix

# 3. Install HuggingFace model
super model install -b huggingface microsoft/Phi-4

# 4. Start server
super model server huggingface microsoft/Phi-4 --port 8001

# 5. Models are ready to use with SuperOptiX agents

🔧 Advanced Configuration

🌐 Multiple Servers

Run multiple models simultaneously:

Bash

# Terminal 1: Ollama model
# Models are ready to use with SuperOptiX agents

# Terminal 2: MLX model (Apple Silicon)
super model server mlx phi-2 --port 8000
# Models are ready to use with SuperOptiX agents

# Terminal 3: HuggingFace model
super model server huggingface microsoft/Phi-4 --port 8001
# Models are ready to use with SuperOptiX agents

# Terminal 4: LM Studio model
super model server lmstudio llama-3.2-1b-instruct --port 1234
# Models are ready to use with SuperOptiX agents

🚨 Troubleshooting

Common Issues

Model Not FoundServer Connection FailedPort Already in UseApple Silicon RequiredMissing Python PackagesMissing CLI ToolsAuthentication Errors

Error: Model not found or Model does not exist

Solution:

Bash

# Check available models
super model list --all

# Use correct model name
super model install llama3.2:3b  # ✅ Correct
super model install llama3.2     # ❌ Wrong

Error: Connection refused or Cannot connect to server

Solution:

Bash

# Check if server is running
# For Ollama: ollama serve
# For MLX: super model server mlx phi-2 --port 8000
# For LM Studio: Start in LM Studio app
# For HuggingFace: super model server huggingface model --port 8001

Error: Address already in use

Solution:

Bash

# Use different port
super model server mlx phi-2 --port 8001
super model server huggingface microsoft/Phi-4 --port 8002

Error: MLX requires Apple Silicon

Solution:

Bash

# Use Ollama instead
super model install llama3.2:3b
super model dspy ollama/llama3.2:3b

Error: ModuleNotFoundError: No module named 'mlx_lm' or ModuleNotFoundError: No module named 'transformers'

Solution:

Bash

# Install MLX dependencies
pip install mlx-lm

# Install HuggingFace dependencies
pip install transformers torch fastapi uvicorn

# Or install with SuperOptiX extras
pip install "superoptix[mlx]"
pip install "superoptix[huggingface]"

Error: Command 'ollama' not found or Command 'lms' not found

Solution:

Bash

# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# Install LM Studio
# Download from https://lmstudio.ai
# Or use winget on Windows:
winget install LMStudio.LMStudio

# Verify installation
ollama --version
lms --version

Error: 401 Unauthorized or Repository Not Found

Solution:

Bash

# For HuggingFace models, login:
huggingface-cli login

# For MLX models, ensure you have access:
# Some models require accepting terms on HuggingFace website

# Use public models instead:
super model install -b mlx mlx-community/phi-2
super model install -b huggingface microsoft/Phi-4

🎉 Next Steps

Now that you have your local models set up:

1. 🚀 Quick Start Guide - Build your first agent with local models
2. 🤖 Create Your First Genies Agent - Step-by-step tutorial
3. 🏪 Marketplace - Discover pre-built agents
4. 🔍 Model Intelligence Guide - Advanced model management

💬 Need Help?

• 📖 Documentation - Comprehensive guides
• 🐛 Support Portal - Report bugs

---

🤖 Ready to Run Local Models?

🤗 HuggingFace

HuggingFace offers access to thousands of models, perfect for advanced users who want maximum flexibility.

🚀 Setup HuggingFace

Bash

# Install HuggingFace dependencies
pip install transformers torch fastapi uvicorn

# Or install with SuperOptiX
pip install "superoptix[huggingface]"

📦 Install HuggingFace Models

Bash

# Install popular models
super model install -b huggingface microsoft/Phi-4
super model install -b huggingface microsoft/DialoGPT-small
super model install -b huggingface microsoft/DialoGPT-medium
super model install -b huggingface meta-llama/Llama-2-7b-chat-hf

🖥️ Start HuggingFace Servers

Bash

# Start server with specific model
super model server huggingface microsoft/Phi-4 --port 8001
super model server huggingface microsoft/DialoGPT-small --port 8001
super model server huggingface microsoft/DialoGPT-medium --port 8001

Example Output:

Text Only

🤗 HuggingFace Local Server
Starting HuggingFace server for microsoft/DialoGPT-small on port 8002...
🚀 Starting HuggingFace server...
📡 Server will be available at: http://localhost:8002
💡 Use this URL in your playbook's api_base configuration
🔧 Manual server startup command:
   python -m superoptix.models.backends.huggingface_server microsoft/DialoGPT-small --port 8002
📋 Example playbook configuration:
   language_model:
     provider: huggingface
     model: microsoft/DialoGPT-small
     api_base: http://localhost:8002
Device set to use cpu
INFO:     Started server process [4652]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8002 (Press CTRL+C to quit)

📋 Manage HuggingFace Models

Bash

# List HuggingFace models
super model list --backend huggingface

Example Output:

Text Only

                🚀 SuperOptiX Model Intelligence - 2 models                
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━┓
┃ Model                    ┃    Backend     ┃    Status    ┃ Size  ┃ Task ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━┩
│ microsoft/DialoGPT-small │ 🤗 huggingface │ ✅ installed │ small │ chat │
│ microsoft/Phi-4          │ 🤗 huggingface │ ✅ installed │ small │ chat │
└──────────────────────────┴────────────────┴──────────────┴───────┴──────┘

Bash

# Get model information
super model info microsoft/Phi-4
super model info microsoft/DialoGPT-small

# Models are ready to use with SuperOptiX agents