Skip to content

[v4.0.0] Phase 14: Structured Remediation Planner #285

Description

@utkarsh232005

Overview

Implement the structured Remediation Planner that converts verified consensus diagnoses into minimal, auditable remediation patches with rollback instructions.

Part of Milestone v4.0.0 — Phase 14 of the Multi-Agent SRE Architecture.


⚠️ IMPORTANT CONTRIBUTOR & PR INSTRUCTIONS

TARGET BRANCH: All Pull Requests implementing this phase MUST target branch v4.0.0 (DO NOT target main).
PR TITLE: feat(remediation): Phase 14 - Structured Remediation Planner
PR SCOPE: Remediation plan generation and diff preview.


1. Description of What Has to Be Done

Diagnosis without remediation leaves the SRE with manual work. However, automated fixes must never execute arbitrary or opaque commands. The Remediation Planner takes the verified ConsensusDiagnosis and generates an actionable RemediationPlan consisting of discrete, ordered steps, manifest diffs, and exact rollback commands.

Contributors must:

  1. Define RemediationPlan and RemediationStep data structures.
  2. Implement planners for core failure types:
    • OOMKilled: Calculate conservative memory limit increase (e.g. 256Mi -> 512Mi) and generate kubectl set resources or manifest patch.
    • CrashLoopBackOff: Suggest configuration/env updates and restart commands.
    • ImagePullBackOff: Suggest image tag fix or secret creation command.
  3. Generate exact rollback commands (e.g. kubectl rollout undo deployment/...).
  4. Assign an auditable risk_level (low, medium, high) based on whether the action causes downtime or pod restarts.

2. Desired Outcome & Expected Behavior

Expected Output

plan = planner.create_plan(consensus, bundle)
# Returns:
RemediationPlan(
    plan_id="plan-oom-001",
    target_workload="checkout-api",
    summary="Bump memory limit to 512Mi to prevent OOMKill restarts.",
    risk_level="low",
    steps=[
        RemediationStep(
            order=1,
            title="Update container memory limit",
            command="kubectl set resources deployment checkout-api --limits=memory=512Mi -n production",
            diff_preview="- memory: 256Mi\n+ memory: 512Mi",
            is_destructive=False
        )
    ],
    rollback_command="kubectl rollout undo deployment/checkout-api -n production"
)

3. The Implementation Plan & Architectural Blueprint

Files to Create and Modify

  • [NEW] agents/remediation/__init__.py: Package init.
  • [NEW] agents/remediation/types.py: RemediationPlan and RemediationStep.
  • [NEW] agents/remediation/planner.py: RemediationPlanner.
  • [NEW] agents/tests/test_remediation_planner.py: Planner unit tests.

4. Detailed Task Breakdown

  • Task 14.1: Implement Remediation Models (types.py)

    • Define RemediationStep with order, title, command, diff_preview, is_destructive.
    • Define RemediationPlan with plan_id, target_workload, summary, risk_level, steps, rollback_command.
  • Task 14.2: Implement RemediationPlanner (planner.py)

    • Match root cause patterns to predefined remediation templates.
    • Compute safe parameter bumps (e.g. 2x memory limit).
    • Generate corresponding kubectl commands and rollback instructions.
  • Task 14.3: Write Unit Tests (test_remediation_planner.py)

    • Test memory bump calculation logic.
    • Test rollback command generation.

5. Technical Specifications & Concrete Code Signatures

# agents/remediation/types.py
from dataclasses import dataclass
from typing import List, Optional

@dataclass
class RemediationStep:
    order: int
    title: str
    command: str
    diff_preview: Optional[str] = None
    is_destructive: bool = False

@dataclass
class RemediationPlan:
    plan_id: str
    target_workload: str
    summary: str
    risk_level: str
    steps: List[RemediationStep]
    rollback_command: Optional[str] = None

6. Verification & Acceptance Checklist

  • Run python3 -m unittest agents/tests/test_remediation_planner.py — all tests pass.
  • Safe rollback command generated for every plan.
  • PR targets branch v4.0.0.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions