# AMSTAR-2 3-rater peer review consensus

**Compiled:** 2026-05-14

## Distribution of overall confidence by reviewer

| Rating | LLM Round 1 (Opus 4.7) | Sergio (breast surgeon, 22y) | David (plastic surgeon, 23y) | **Consensus** |
|---|---:|---:|---:|---:|
| High | 0 | 0 | 0 | **0** |
| Moderate | 2 | 0 | 0 | **5** |
| Low | 6 | 7 | 1 | **6** |
| Critically Low | 25 | 26 | 32 | **22** |
| **TOTAL** | **33** | **33** | **33** | **33** |

## Inter-rater agreement

- Average per-SR item agreement (all 3 raters): **63.8%** of 16 items
- Total per-SR-per-item discrepancies: **191** (out of 528 = 36.2%)
- Overall confidence κ — LLM-Sergio: 0.252 | LLM-David: 0.184 | Sergio-David: 0.208

## Files generated

- `per_sr/<author>_<year>.json` — per-SR detailed consensus + audit trail
- `three_rater_matrix.csv` — wide table for analysis
- `cohens_kappa.md` — full κ matrix (16 items + overall)
