# Bucket-A expectations for task-reviewer
# Format: per-agent YAML, 5–10 grader-worthy expectations grounded in the agent
# file's explicit promises and/or current matrix failures. This file is consumed
# by the next pass that rewrites stimuli + graders end-to-end; do not treat it
# as a Vally grader file directly.
slug: task-reviewer
class: code-reviewer
agent_file: .github/agents/hve-core/task-reviewer.agent.md
stimulus_file: evals/agent-behavior/stimuli/task-reviewer.yml
latest_result: evals/results/agent-matrix/2026-05-28/task-reviewer.json
source_review_date: 2026-05-28

expectations:
  - expectation_id: response-header-format
    summary: Response begins with a status-conditional Task Reviewer header.
    signal: First non-blank line of output matches one of the three branded headers.
    pass_criteria: |
      Output starts with one of:
      `## ✅ Task Reviewer: <Task Description>`,
      `## ⚠️ Task Reviewer: <Task Description>`, or
      `## 🚫 Task Reviewer: <Task Description>`,
      where <Task Description> is a non-empty phrase.
    failure_modes:
      - Header missing entirely.
      - Emoji omitted, replaced, or status emoji does not match the verdict.
      - Header uses a different label (e.g., `## Review Findings`).
    priority: high
    contract_ref: "agent §User Interaction (status-conditional headers)"

  - expectation_id: findings-with-severity
    summary: Output presents findings grouped or labeled by severity.
    signal: Output contains severity vocabulary tied to individual findings.
    pass_criteria: |
      Output uses at least one of the recognized severity tiers (critical,
      major, minor, high, medium, low) on each finding or in a severity-count
      summary; severity is attached to specific findings, not only mentioned
      in the abstract.
    failure_modes:
      - Findings listed without any severity labels.
      - Severity vocabulary appears only in unrelated prose.
    priority: high
    contract_ref: "agent §Review Log (severity counts: critical, major, minor); §User Interaction (Findings summary with severity counts)"

  - expectation_id: review-log-path-named
    summary: Output names the review log path under `.copilot-tracking/reviews/<date>/`.
    signal: Output contains a path matching `.copilot-tracking/reviews/<YYYY-MM-DD>/<plan-slug>-review.md`.
    pass_criteria: |
      Path is workspace-relative, dated subdir present, suffix `-review.md`.
      The path may appear in prose, the handoff summary table, or the
      Handoff steps section.
    failure_modes:
      - Review log not referenced.
      - Path written outside `.copilot-tracking/reviews/`.
      - Missing date subdir or `-review.md` suffix.
    priority: high
    contract_ref: "agent §Review Log (Create and progressively update the review log at .copilot-tracking/reviews/...)"

  - expectation_id: overall-status-stated
    summary: Output explicitly states an overall review status from the allowed vocabulary.
    signal: Output contains one of `Complete`, `Needs Rework`, or `Blocked` as the overall verdict.
    pass_criteria: |
      Output includes a clearly identified Overall Status line or row whose
      value is exactly one of: `Complete`, `Needs Rework`, `Blocked`
      (case-insensitive; emoji prefix allowed).
    failure_modes:
      - Verdict expressed only in prose (e.g., "looks good") without the
        allowed vocabulary.
      - Uses a non-matching label (e.g., "Approved", "Rejected").
    priority: high
    contract_ref: "agent §Phase 4 Step 1 (Overall status determination: Complete, Needs Rework, Blocked); §User Interaction"

  - expectation_id: subagent-delegation-evidence
    summary: Validation is delegated to RPI Validator (and Implementation Validator) rather than performed inline.
    signal: Output mentions invoking `RPI Validator` and/or `Implementation Validator` (or notes the delegation tooling is unavailable).
    pass_criteria: |
      Output contains an explicit reference to running `RPI Validator` per
      plan phase, OR `Implementation Validator` for quality, OR an explicit
      notice that `runSubagent`/`task` tooling is unavailable and direct
      review was used as fallback.
    failure_modes:
      - All validation performed inline with no delegation mention and no
        unavailability notice.
      - Delegates but uses filenames (e.g., `rpi-validator.agent.md`)
        instead of human-readable names.
    priority: medium
    contract_ref: "agent §Phase 2 Step 2 (Spawn RPI Validators); §Phase 3 Step 1 (Implementation Quality)"

  - expectation_id: handoff-summary-table
    summary: On completion, output provides the structured review handoff table.
    signal: Output contains a markdown table headed `📊 Summary` with rows for Review Log, Overall Status, Critical Findings, Major Findings, Minor Findings, and Follow-Up Items.
    pass_criteria: |
      A markdown table is present whose header cell contains `📊 Summary` and
      whose rows include at least: `Review Log`, `Overall Status`,
      `Critical Findings`, `Major Findings`, `Minor Findings`,
      `Follow-Up Items`.
    failure_modes:
      - Completion described in prose only, no table.
      - Table missing required rows (e.g., omits Review Log or severity counts).
    priority: medium
    contract_ref: "agent §User Interaction (structured handoff table)"
    applies_when: "Review reaches completion; not for clarification-only turns."

  - expectation_id: no-source-modifications
    summary: Review-only — no edits to source code or build manifests.
    signal: Output does not reference modifications to source-tree files.
    pass_criteria: |
      No occurrences of edit/create verbs paired with `.cs`/`.py`/`.ts`/`.js`/
      `.go`/`.rs`/`.java`/`package.json`/`pyproject.toml`/`Cargo.toml` paths.
      File paths may appear in findings as references to reviewed files, but
      not as files this agent itself edited.
    failure_modes:
      - Claims to edit source files during review.
      - Modifies `package.json` or build manifests while reviewing.
    priority: high
    contract_ref: "agent §Required Phases (review log under .copilot-tracking/reviews/, no source edits)"

  - expectation_id: bottom-up-handoff-last
    summary: The completion handoff (summary table + next steps) appears at the end of the response.
    signal: The `📊 Summary` table and Handoff steps follow findings, not preceding them.
    pass_criteria: |
      In the rendered output, the `📊 Summary` table and any `Handoff steps`
      block appear after the findings/severity content. No findings appear
      below the handoff steps.
    failure_modes:
      - Handoff table placed at the top of the response.
      - New findings appended after the handoff steps.
    priority: low
    contract_ref: "agent §User Interaction (structured handoff at end); review log content ordering"
    applies_when: "Review reaches completion."
