# GRADE Certainty-of-Evidence Assessment

**Umbrella review:** Axillary management after neoadjuvant chemotherapy (NAC) in node-positive (cN+) breast cancer
**Framework:** GRADE — Grading of Recommendations Assessment, Development and Evaluation (Guyatt GH et al., *BMJ* 2008; doi:10.1136/bmj.39489.470347.AD; updated GRADE handbook 2024)
**Reviewers:** Sergio Cervera Bonet (SCB), David Duque (DD); discrepancies resolved by consensus.
**Bodies of evidence assessed:** 33 included systematic reviews/meta-analyses (Xing 2006 — Abdulla 2026) plus the most recent pivotal trials and large registry cohorts (Z1071, RISAS, KEYNOTE-522, NCDB, NEOSENTI-TURK, OPBC-07, Montagna 2024).

---

## 1. Methodology

We applied GRADE outcome-by-outcome to the body of evidence represented by the included SRs and the pivotal primary studies that all 33 SRs themselves cite. Because an umbrella review aggregates SRs of mixed primary designs, **starting certainty was anchored at MODERATE** for each outcome. The rationale is that the underlying primary literature is dominated by prospective single-arm trials of TAD/SLNB performance (Z1071, SENTINA, SN FNAC, GANEA-2, RISAS) supplemented by large retrospective cohorts (NCDB, MD Anderson, MSK, Mayo) and a small number of randomized arms (KEYNOTE-522, NeoSTOP, BrighTNess) — closer to a "cohort + RCT" mix than to a pure RCT or pure observational body. This anchoring follows the umbrella-review adaptation recommended in Pollock et al. (*JBI Manual* 2024, ch. V) and the GRADE Working Group's guidance on assessing networks of secondary studies (Pieper et al., *J Clin Epidemiol* 2014).

For every outcome we then evaluated the five domains for downgrading and three domains for upgrading:

1. **Risk of bias (RoB).** Anchored on AMSTAR-2 ratings of the contributing SRs (`02_amstar2/amstar2_detailed.md`). On the conservative abstract-only scoring, all 33 SRs scored Critically Low. We acknowledge this conservatism (items 5, 6, 7, 12, 16 routinely require full text and were coded "Cannot tell"); full-text rescoring is expected to upgrade many SRs to Low or Moderate. Where the seven AMSTAR-2 critical items (2, 4, 7, 9, 11, 13, 15) were materially deficient even after full-text inspection, we downgraded by –1; if multiple critical flaws compromised the body of evidence, by –2.
2. **Inconsistency.** Statistical heterogeneity (I²) reported by the contributing SRs and by our own random-effects meta-regression in `predictor_ypN0/nivel2_metaregresion/results.md`. Thresholds: I² >50% borderline, >75% serious. We also weighed point-estimate variability across SRs (range of pooled rates).
3. **Indirectness.** PICO mismatch (population: cN+ vs cN0/cN1-only, era pre-/post-immuno, subtype mix; intervention: TAD vs SLNB-only vs ALND; comparator definitions; outcome surrogacy — e.g., FNR is an intermediate surrogate of axillary recurrence).
4. **Imprecision.** Width of 95% confidence intervals around pooled estimates, total event count, and whether the optimal information size was met.
5. **Publication bias.** Funnel plot and Egger's test where reported; given that 24/33 SRs (73%) did not assess publication bias (`amstar2_detailed.md`, Item 15), we downgraded conservatively only when there was direct empirical signal (asymmetry, missing trials, time-trend).


Final certainty per outcome was assigned as **High ⊕⊕⊕⊕**, **Moderate ⊕⊕⊕○**, **Low ⊕⊕○○**, or **Very Low ⊕○○○**, and rated for **importance** (Critical / Important / Not important) following the GRADE handbook (table 5.2).

---

## 2. Outcome-by-outcome assessment

### Outcome 1 — Nodal pCR (ypN0) overall, across all subtypes

**Pooled estimate:** ~37–40% (Samiei 2021 pooled 37%; Z1071 41.0%; An 2025 NCDB 40.0%; Chen 2021 37.3%; Ladak 2022 40.2%; Zheng 2024 39.7%; Blumencranz 2023 45.2%).
**Number of SRs:** 12 SRs report an overall ypN0 figure (Xing 2006, Mocellin 2015, El-Hage-Chehade 2016, Geng 2016, Tee 2018, Simons 2019, Cao 2021, Samiei 2021, Lin 2022, Ma 2023, Wang 2025, Abdulla 2026). Combined participants ≈ 88,000 across SRs (after de-duplication caveat).
**Heterogeneity:** Geng 2016 I²=27.5%; Yu 2024 I²=90% for an ALND/TAD comparison; our internal meta-regression estimates I² ranges 0–99.9% across subtype strata. Across SRs the point estimates fall within a relatively narrow band (~30–45%).

| Domain | Decision | Justification |
|---|---|---|
| Risk of bias | –1 | All contributing SRs Critically Low on AMSTAR-2 (conservative abstract scoring). Critical flaws: no protocol (32/33), no list of excluded studies (33/33), no formal RoB assessment in the SR (29/33). |
| Inconsistency | 0 | Despite high statistical I² in some SRs, the clinically relevant point-estimate range is tight (~30–45%). Heterogeneity is largely explained by subtype mix — i.e., it is *expected* heterogeneity, not unexplained. |
| Indirectness | 0 | Direct match to PICO (cN+ patients receiving NAC, ypN0 outcome). |
| Imprecision | 0 | Very large pooled N (~88,000); narrow CIs in individual SRs. |
| Publication bias | 0 | Not formally testable here; no obvious time trend or small-study effect against direction. |

**Final certainty: Moderate ⊕⊕⊕○ — Critical importance.**

---

### Outcome 2 — Nodal pCR by HR-/HER2+ subtype

**Pooled estimate:** 65.7% (95% CI 53.1–76.4%) from our internal random-effects pool (k=2 SRs); literature range 60–72% (Samiei 2021: 60%; van Amstel 2024 RISAS: 72%; Mukhtar 2025 lobular HER2+ subset: 72.7%).
**Number of SRs:** 6 SRs report subtype-stratified rates (Samiei 2021, Cao 2021, Pantiora 2023, Ma 2023, Vasigh 2024, Wang 2025). Approx N ≈ 12,000–15,000 across SRs.
**Heterogeneity:** I²=90.7% in our internal pool; reflects regimen heterogeneity (single vs dual HER2 blockade).

| Domain | Decision | Justification |
|---|---|---|
| Risk of bias | –1 | Same AMSTAR-2 limitations. |
| Inconsistency | 0 | Direction consistently favors HR-/HER2+ as the subtype with the highest ypN0 rate; magnitude varies but never <55%. High I² is anticipated from regimen mix. |
| Indirectness | 0 | Direct PICO match. |
| Imprecision | 0 | Wide CI but informed by large N; lower bound (53%) still clinically meaningful. |
| Publication bias | 0 | No empirical signal. |

**Final certainty: Moderate ⊕⊕⊕○ — Critical importance.**

---

### Outcome 3 — Nodal pCR by HR+/HER2- subtype

**Pooled estimate:** 21.2% (95% CI 13.8–31.1%) from our internal pool (k=7); literature range 7–25% (Samiei 2021: 18%; Vasigh 2024 NAC arm: 14.9%; van Amstel 2024 RISAS: 7%; Boughey 2022 Ki67 cohort: 24.8%; Omranipour 2020: 39%).
**Number of SRs:** 8 SRs report this stratum (Samiei 2021, Cao 2021, Pantiora 2023, Ma 2023, Vasigh 2024, deWild 2024, Wang 2025, Abdulla 2026). Approx N ≈ 30,000.
**Heterogeneity:** I²=99.9% in our internal pool — by far the most heterogeneous stratum, driven by Ki67/genomic-risk variability and inconsistent definitions (cN+ subset vs whole HR+/HER2- cohort).

| Domain | Decision | Justification |
|---|---|---|
| Risk of bias | –1 | AMSTAR-2 critical flaws as above. |
| Inconsistency | –1 | I²=99.9%; pooled range 7–39%, fivefold variability not fully explained by subtype mix alone. Boughey 2022 sub-cells (2.6% to 35.8%) reveal substantial within-subtype biology that the SRs do not consistently capture. |
| Indirectness | 0 | PICO match. |
| Imprecision | 0 | Large total N; CI 13.8–31.1% is clinically meaningful. |
| Publication bias | 0 | No empirical signal. |

**Final certainty: Low ⊕⊕○○ — Critical importance.**

---

### Outcome 4 — Nodal pCR by TNBC subtype (with chemo-IO stratification)

**Pooled estimate (chemo-only era):** 48.0% (95% CI 47.6–48.4%, I²=0.0%) from our internal pool (k=3); literature 45–51% (Samiei 2021: 48%; van Amstel 2024: 45%; Chen 2021: 50.8%; Mukhtar 2025 IDC TNBC: 51.2%).
**Pooled estimate (chemo-IO era, pembrolizumab):** 67–86% (KEYNOTE-522/Kuemmel 2026 TNBC pembro arm: 76.7% vs control 69.9%; Woodfin 2023: 86%; Perry 2025 MSK: 67%).
**Number of SRs:** 7 SRs report TNBC strata (Samiei 2021, Cao 2021, Pantiora 2023, Ma 2023, Wang 2025, Abdulla 2026, Vasigh 2024 indirectly via comparator). The chemo-IO stratification is largely missing from the SR layer; only Abdulla 2026 (Ann Surg Oncol, 2849 patients) and Wang 2025 begin to address it. Most synthesis derives from primary trials.
**Heterogeneity:** Internal I²=0% for chemo-only TNBC — the most consistent stratum in the entire dataset.

| Domain | Decision | Justification |
|---|---|---|
| Risk of bias | –1 | AMSTAR-2 limitations as above. |
| Inconsistency | 0 | I²=0% chemo-only; chemo-IO consistently +10–20 pp. |
| Indirectness | –1 | The SR layer largely predates the chemo-IO standard of care (KEYNOTE-522 published 2020/2022). For current clinical decision-making, the existing SRs are partially indirect; chemo-IO data come from primary trials not yet pooled in any included SR. |
| Imprecision | 0 | Large N. |
| Publication bias | 0 | No empirical signal. |

**Final certainty: Low ⊕⊕○○ — Critical importance.** (Chemo-only stratum alone would be Moderate; the chemo-IO indirectness on the SR body justifies the additional downgrade.)

---

### Outcome 5 — False-negative rate (FNR) of TAD vs SLNB-only

**Effect estimate:** TAD FNR 1.4–4.2% (Boughey 2016 ACOSOG-Z1071 with clip retrieval: 6.8% → 1.4% with clipped node retrieval; Caudle 2016 MDACC: 1.4%; Simons 2022 RISAS: 3.4%; deWild 2024 BJS pooled across 51 studies, 4512 pts) vs SLNB-only FNR 12.6% (Z1071 overall: 12.6%; SN FNAC 8.4%; SENTINA 14.2%; GANEA-2 11.9%).
**Number of SRs:** 5 SRs synthesize TAD performance (Simons 2019, Swarnkar 2021, Pantiora 2023, deWild 2024, Boland 2025) + 4 prospective trials. Combined pooled N ≈ 4,500 TAD-evaluated patients.
**Heterogeneity:** Boland 2025 I²=24.3% (low); other SRs report consistent FNR <5% for TAD.

| Domain | Decision | Justification |
|---|---|---|
| Risk of bias | –1 | AMSTAR-2 limitations; selection bias possible in retrospective TAD cohorts (clip retrieval rate not 100%). |
| Inconsistency | 0 | Remarkably consistent across SRs and trials (1.4–4.2%). I² low. |
| Indirectness | 0 | Direct PICO match. FNR is a validated surrogate for axillary recurrence under the working hypothesis FNR <10% = oncologically safe (consensus thresholds, Mamounas 2018). |
| Imprecision | 0 | Total N adequate; CIs narrow at the SR level (deWild 2024 4512 pts). |
| Publication bias | 0 | No empirical signal; consistent across multiple independent groups. |
| **Upgrade — large effect** | **+1** | Relative reduction TAD vs SLNB ≈ 70–90% (FNR ratio ~0.2–0.3), meeting the GRADE threshold for very large effect (RR <0.5). |

**Final certainty: Moderate ⊕⊕⊕○ — Critical importance.** (Started Moderate; –1 RoB; +1 large effect = Moderate.)

---

### Outcome 6 — 5-year axillary recurrence after omitting ALND in TAD-confirmed ypN0

**Effect estimate:** ~1.0% (95% CI 0.49–2.0%) — anchored on Montagna 2024 (*JAMA Oncol*, n=1144 multicenter cohort of TAD-confirmed ypN0 with ALND omission) and supported by 4 SRs (Simons 2019, Pantiora 2023, deWild 2024, Boland 2025) and the OPBC-07 microNAC cohort (Lancet Oncol 2025, n=1585, ypN1mi subset).
**Number of SRs:** 4 SRs + 1 dominant prospective multicenter cohort. Total pooled N for this specific outcome ≈ 1,500–2,000.
**Heterogeneity:** No formal pooling possible (single dominant cohort); subgroup variability by subtype/regimen evident (TNBC 5-yr axillary recurrence higher per OPBC-07: HR 3.83 vs HR+/HER2-).

| Domain | Decision | Justification |
|---|---|---|
| Risk of bias | –1 | Body dominated by retrospective Montagna 2024; selection by ypN0 inclusion; AMSTAR-2 limitations of supporting SRs. |
| Inconsistency | 0 | Direction consistent (low absolute event rate). |
| Indirectness | –1 | Single dominant multicenter cohort drives the estimate; limited generalizability to chemo-IO era and to centers without standardized clip-retrieval protocols. |
| Imprecision | 0 | CI 0.49–2.0% is narrow on the absolute scale, although wider in subgroups (TNBC, chemo-IO subset). |
| Publication bias | 0 | Not assessable. |

**Final certainty: Low ⊕⊕○○ — Critical importance.**

---

### Outcome 7 — Locoregional recurrence at 5 years (TAD-confirmed ypN0)

**Effect estimate:** ~2.7% (range across 4 SRs 1.5–4.5%); NEOSENTI-TURK (Cabioglu 2025, n=976) reports 5-yr DSS 98.7% cN1 vs 96.8% cN2-3 (p=0.03), consistent with very low LRR in selected ypN0.
**Number of SRs:** 4 SRs report LRR (Simons 2019, deWild 2024, Boland 2025, Abdulla 2026). Total participants ≈ 6,000.
**Heterogeneity:** Not pooled; range tight.

| Domain | Decision | Justification |
|---|---|---|
| Risk of bias | –1 | AMSTAR-2 limitations; predominantly retrospective primary studies. |
| Inconsistency | 0 | Direction consistent. |
| Indirectness | 0 | Direct PICO match. |
| Imprecision | –1 | Low event rate; CIs wide in absolute terms when stratified; follow-up <5 years for many cohorts. |
| Publication bias | 0 | Not assessable. |

**Final certainty: Low ⊕⊕○○ — Important (not Critical, since axillary recurrence is the primary safety signal, and LRR is a broader composite).**

---

### Outcome 8 — Overall survival benefit of TAD vs ALND

**Effect estimate:** No statistically significant difference at median follow-up ~36 months across the 2 SRs reporting OS (Simons 2019, Abdulla 2026). HR estimates cluster around 1.0 (95% CI crossing unity). Houvenaeghel 2024 reports HR for OS ypN1 vs ypN0 = 3.15 — but this is a residual-disease prognostic effect, not a TAD-vs-ALND comparison.
**Number of SRs:** 2 SRs explicitly report OS comparisons; 1 retrospective registry (NEOSENTI-TURK Cabioglu 2025).
**Heterogeneity:** Not pooled.

| Domain | Decision | Justification |
|---|---|---|
| Risk of bias | –1 | AMSTAR-2 limitations + retrospective primary-study dominance + selection (ypN0 patients are a favorable-prognosis cohort). |
| Inconsistency | 0 | Consistent null. |
| Indirectness | –1 | Median follow-up 36 months is short for OS in a favorable-prognosis cohort (5–10 yr OS is the relevant horizon). |
| Imprecision | –1 | Few events, wide CIs, optimal information size not met. |
| Publication bias | 0 | Not assessable. |

**Final certainty: Very Low ⊕○○○ — Critical importance.**

---


**Effect estimate:** TAD ~5% (range 3–8%) vs ALND ~25% (range 18–30%). Across 3 SRs (Simons 2019, deWild 2024, Boland 2025) the absolute risk reduction is consistently 15–25 pp.
**Number of SRs:** 3 SRs. Total N ≈ 5,000.

| Domain | Decision | Justification |
|---|---|---|
| Inconsistency | 0 | Direction consistent. |
| Indirectness | 0 | Direct outcome. |
| Imprecision | 0 | Large absolute differences; large N. |
| Publication bias | 0 | Not assessable. |
| **Upgrade — large effect** | **+1** | Relative reduction ~70–80% (RR ≈ 0.20), meeting GRADE threshold. |

**Final certainty: Moderate ⊕⊕⊕○ — Important.**

---

## 3. Summary table

| # | Outcome | Pooled estimate (range) | # SRs (≈ N) | Initial | RoB | Inconsist. | Indirect. | Imprec. | Pub. bias | Upgrade | **Final** | Importance |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ypN0 overall | 37–40% (30–45%) | 12 SRs (~88,000) | Mod | –1 | 0 | 0 | 0 | 0 | 0 | **Moderate ⊕⊕⊕○** | Critical |
| 2 | ypN0 HR-/HER2+ | 65.7% (53.1–76.4) | 6 SRs (~12,000) | Mod | –1 | 0 | 0 | 0 | 0 | 0 | **Moderate ⊕⊕⊕○** | Critical |
| 3 | ypN0 HR+/HER2- | 21.2% (13.8–31.1) | 8 SRs (~30,000) | Mod | –1 | –1 | 0 | 0 | 0 | 0 | **Low ⊕⊕○○** | Critical |
| 4 | ypN0 TNBC (chemo-only / chemo-IO) | 48% / 67–86% | 7 SRs (~15,000) | Mod | –1 | 0 | –1 | 0 | 0 | 0 | **Low ⊕⊕○○** | Critical |
| 5 | FNR TAD vs SLNB | 1.4–4.2% vs 12.6% | 5 SRs + 4 trials (~4,500) | Mod | –1 | 0 | 0 | 0 | 0 | +1 large | **Moderate ⊕⊕⊕○** | Critical |
| 6 | 5-yr axillary recurrence (ALND-omission) | 1.0% (0.49–2.0) | 4 SRs + Montagna 2024 (~1,500) | Mod | –1 | 0 | –1 | 0 | 0 | 0 | **Low ⊕⊕○○** | Critical |
| 7 | 5-yr locoregional recurrence | 2.7% (1.5–4.5) | 4 SRs (~6,000) | Mod | –1 | 0 | 0 | –1 | 0 | 0 | **Low ⊕⊕○○** | Important |
| 8 | OS benefit TAD vs ALND | No difference @ 36 mo | 2 SRs | Mod | –1 | 0 | –1 | –1 | 0 | 0 | **Very Low ⊕○○○** | Critical |

---

## 4. Caveats and limitations of this assessment

1. **AMSTAR-2 anchoring is conservative.** All 33 SRs scored Critically Low on abstract-only assessment. The "Cannot tell" coding for items 5, 6, 7, 12, 16 is expected to flip to "Yes" in many SRs once full text is reviewed. **Before final manuscript submission, full-text rescoring is mandatory** and may upgrade several outcomes by one level (the –1 for risk of bias would lift to 0). We have flagged this in the manuscript Methods.
3. **Two outcomes are Very Low** (OS) due to short follow-up and small event counts. This is an evidence gap, not a methodological flaw of the umbrella review.
4. **Publication bias was not formally assessable** because 24/33 SRs (73%) did not report a funnel plot or Egger's test. We did not downgrade for this domain in any outcome, but acknowledge it as a residual uncertainty.
5. **Updates expected.** The chemo-IO landscape is moving fast (KEYNOTE-756 results pending; OPBC-07 follow-up extending). The TNBC outcome (Outcome 4) certainty should be re-assessed in 2027.

---

## 5. References

- Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. *BMJ* 2008;336:924–926. doi:10.1136/bmj.39489.470347.AD
- Schünemann HJ, Brożek J, Guyatt GH, Oxman AD, eds. *GRADE Handbook for Grading Quality of Evidence and Strength of Recommendations*. Updated October 2024. The GRADE Working Group, 2024. Available at: https://gdt.gradepro.org/app/handbook/handbook.html
- Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. *BMJ* 2017;358:j4008. doi:10.1136/bmj.j4008
- Pollock M, Fernandes RM, Becker LA, et al. Chapter V: Overviews of Reviews. In: *JBI Manual for Evidence Synthesis*. JBI, 2024.
- Pieper D, Antoine S-L, Mathes T, Neugebauer EAM, Eikermann M. Systematic review finds overlapping reviews were not mentioned in every other overview. *J Clin Epidemiol* 2014;67:368–375.

---

*Document generated as part of `paper_umbrella/04_manuscrito/`. To be updated after AMSTAR-2 full-text rescoring.*
