Skip to content

Memory & Learning System

SuperQode learns from your feedback to improve over time. The memory system stores project-specific learnings that persist across QE sessions.

Overview

The Memory & Learning System enables SuperQode to:

  • Remember patterns: Track recurring issues and fixes
  • Learn from feedback: Improve based on your corrections
  • Suppress false positives: Automatically filter known false positives
  • Score file risk: Identify high-risk files based on history
  • Track role effectiveness: Measure which QE roles work best for your project

Components

MemoryStore

Persistent storage for project learnings:

  • User-local memory: ~/.superqode/memory/project-{hash}.json
  • Team-shared memory: .superqode/memory.json (in repo)

Memory is merged with user-local taking precedence.

FeedbackCollector

Collects and processes user feedback on findings:

  • Marks findings as valid/false positive
  • Records fix patterns
  • Updates role metrics
  • Integrates with ML predictor

What Gets Remembered

Issue Patterns

Recurring issues detected across sessions:

{
  "fingerprint": "abc123",
  "title": "SQL injection vulnerability",
  "category": "security",
  "severity": "critical",
  "occurrences": 5,
  "first_seen": "2025-01-10T10:00:00",
  "last_seen": "2025-01-15T14:30:00",
  "files_affected": ["src/api/users.py", "src/api/orders.py"],
  "avg_confidence": 0.85
}

Suppressions

False positive suppression rules:

{
  "pattern": "legacy authentication code",
  "pattern_type": "fingerprint",
  "scope": "project",
  "created_at": "2025-01-10T10:00:00",
  "created_by": "user",
  "expires_in_days": null
}

Fix Patterns

Successful fix patterns:

{
  "finding_type": "sql_injection",
  "fix_approach": "parameterized_queries",
  "success_rate": 0.95,
  "times_used": 10,
  "files_fixed": ["src/api/users.py", ...]
}

File Risk Scores

Risk scores per file based on finding history:

{
  "src/api/users.py": 0.85,
  "src/auth/login.py": 0.92,
  "src/utils/helpers.py": 0.15
}

Role Metrics

Effectiveness tracking per QE role:

{
  "security_tester": {
    "total_findings": 45,
    "confirmed_findings": 40,
    "false_positives": 5,
    "accuracy_rate": 0.89
  }
}

Feedback Collection

Mark Finding as Valid

superqe feedback finding-001 --valid

Updates: - Role metrics (increases confirmed findings) - Issue patterns (tracks recurring issues) - ML training data (if ML enabled)

Mark as False Positive

superqe feedback finding-002 --false-positive -r "Intentional for testing"

Creates: - Suppression rule - Updates role metrics - Adds to ML training data

Mark as Fixed

superqe feedback finding-003 --fixed -r "Applied suggested patch"

Records: - Fix pattern (approach used) - File risk score adjustment - Role effectiveness update

Storage Locations

User-Local Memory

Location: ~/.superqode/memory/project-{hash}.json

  • Per-user, per-project storage
  • Not committed to repo
  • Takes precedence over team memory

Team-Shared Memory

Location: .superqode/memory.json

  • Committed to repository
  • Shared across team
  • Merged with user-local memory

Merging Strategy

When both exist: 1. Load user-local memory 2. Load team-shared memory 3. Merge (user-local takes precedence) 4. Use merged result

Automatic Learning

Pattern Detection

The system automatically detects patterns:

  • Recurring issues: Same issue in multiple files
  • Common fixes: Successful fixes applied repeatedly
  • False positive patterns: Frequently suppressed findings

Risk Scoring

File risk scores update automatically:

  • Finding detected → increase risk
  • Fix applied → decrease risk
  • Multiple findings → higher risk

Role Metrics

Role effectiveness tracked automatically:

  • Finding confirmed → increase accuracy
  • False positive → decrease accuracy
  • Tracks over time

API Usage

Load Memory

from superqode.memory import MemoryStore

store = MemoryStore(project_root)
memory = store.load()

# Access data
issue_patterns = memory.issue_patterns
suppressions = memory.suppressions
file_risk = memory.file_risk_map

Collect Feedback

from superqode.memory import FeedbackCollector

collector = FeedbackCollector(project_root)

# Mark as valid
collector.mark_valid(
    finding_id="sec-001",
    finding_title="SQL injection",
    category="security",
    severity="critical",
    role_name="security_tester"
)

# Mark as false positive
collector.mark_false_positive(
    finding_id="sec-002",
    finding_title="False positive",
    finding_fingerprint="abc123",
    role_name="security_tester",
    reason="Intentional for testing"
)

Update File Risk

memory = store.load()
memory.update_file_risk("src/api/users.py", "critical")
store.save()

Use Cases

1. Reduce False Positives

# First time: mark as false positive
superqe feedback finding-001 --false-positive -r "Known pattern"

# Future sessions: automatically suppressed

2. Track Recurring Issues

The system tracks issues that appear multiple times:

Issue Pattern: SQL injection in user input
Occurrences: 5
Files: users.py, orders.py, products.py

3. Identify High-Risk Files

Files with high risk scores get more attention:

# Check file risk
memory = store.load()
risk = memory.file_risk_map.get("src/api/users.py", 0.0)
if risk > 0.8:
    print("High-risk file - focus testing here")

4. Measure Role Effectiveness

See which roles work best:

metrics = memory.role_metrics.get("security_tester")
print(f"Accuracy: {metrics.accuracy_rate:.2%}")
print(f"Confirmed: {metrics.confirmed_findings}")

Best Practices

1. Provide Regular Feedback

# After each QE session
superqe feedback finding-001 --valid
superqe feedback finding-002 --false-positive -r "reason"

2. Share Team Memory

Commit .superqode/memory.json to repository:

git add .superqode/memory.json
git commit -m "Update team memory"

3. Review Suppressions

Periodically review suppressions:

superqe suppressions

4. Monitor Role Metrics

Check which roles are most effective:

# High accuracy = role works well for your project
# Low accuracy = may need different approach

Troubleshooting

Memory Not Persisting

Check storage location: - User-local: ~/.superqode/memory/ - Team-shared: .superqode/memory.json

Suppressions Not Working

Verify pattern matching: - Pattern type (fingerprint vs. title) - Scope (project vs. file) - Expiration (if set)

Risk Scores Not Updating

Ensure feedback collected: 

superqe feedback finding-001 --valid

Next Steps