# Statistical Analysis Plan (SAP) — Nivel 3 ypN0 Predictor

**Document version:** 1.0  —  **Date:** 2026-05-14  —  **Pre-specified prior to data lock.**
**PI:** Dr. Sergio Cervera  —  **Statistician:** TBD

---

## 1. Objectives

### 1.1 Primary objective
Develop and internally validate a multivariable prediction model for ypN0 (binary outcome)
in cN+ breast cancer patients treated with neoadjuvant chemotherapy (NAC), and externally
validate the model in an independent cohort.

### 1.2 Secondary objectives
- Compare discrimination, calibration, and clinical utility against the existing literature-
  based Nivel 2 meta-regression model.
- Quantify variable importance (logistic regression coefficients + SHAP values for ML model).
- Subgroup analyses by molecular subtype, cN stage, and treatment era.

---

## 2. Study design

- Retrospective cohort, single-institution (development) + multicenter (external validation
  via collaboration).
- Design class per TRIPOD-AI: **Type 2 (development + external validation, different samples).**
- Reporting standard: **TRIPOD-AI 2024** checklist (Collins et al., BMJ 2024).

---

## 3. Population

### 3.1 Inclusion
- Female or male patients ≥18 yr.
- Invasive breast carcinoma, any subtype.
- cT1-cT3, cN1-cN3 confirmed by FNA/core biopsy at diagnosis.
- Received NAC with ≥80% of planned cycles.
- Underwent axillary surgery (SLNB, TAD, or ALND) at the institution.
- Final pathology available.

### 3.2 Exclusion
- cT4 inflammatory.
- True cN0 at baseline (no biopsy-confirmed nodal involvement).
- Distant metastases at diagnosis.
- Loss of follow-up before surgery.
- Concurrent second primary cancer requiring systemic therapy that may confound NAC effect.

---

## 4. Outcome

### 4.1 Primary (Y)
- **ypN0** = binary; 1 if final pathology shows ypN0 or ypN0(i+); 0 otherwise.
  - Note: micrometastases (ypN1mi) coded as 0 (consistent with surgical decision-making).

### 4.2 Secondary
- Axillary recurrence (time-to-event).
- Locoregional recurrence (time-to-event).

---

## 5. Predictors (X)

### 5.1 Pre-specified candidate predictors
| Variable | Type | Rationale |
|----------|------|-----------|
| Age at dx | continuous | Effect on response demonstrated in Z1071 |
| BMI | continuous | Effect on chemo efficacy |
| cT stage | ordinal (T1/T2/T3) | Tumor burden |
| cN stage | ordinal (N1 vs N2-3) | Higher nodal burden = lower ypN0 |
| Molecular subtype | nominal (4 levels) | Strongest predictor (Nivel 2 confirmed) |
| Ki67 | continuous | Proliferation marker |
| Anti-HER2 dual blockade (trastuzumab + pertuzumab) | binary | Major effect modifier in HER2+ |
| Pembrolizumab inclusion | binary | KEYNOTE-522 effect in TNBC |
| Carboplatin inclusion | binary | TNBC + HR+ pCR enhancement |
| Cycles received / planned ratio | continuous (0-1) | Treatment intensity |
| Clinical response (post-NAC clinical+US, RECIST) | ordinal (CR/PR/SD/PD) | Surrogate of axillary response |
| MRI residual size | continuous | Anatomical predictor |

### 5.2 Variable selection strategy
- **No automated stepwise selection** (consensus: this introduces bias and overfitting).
- All pre-specified variables enter the multivariable model regardless of univariable p-value.
- Optional supplementary analysis: backward elimination at p<0.157 (AIC-equivalent) for
  comparison with literature heuristics.

---

## 6. Data preparation

### 6.1 Data lock and freeze
- Data lock: **week 14** post-launch.
- All analyses use frozen dataset; any post-hoc additions documented in addendum.

### 6.2 Quality control
- Range checks (age 18–100, BMI 12–60, Ki67 0–100).
- Logical checks (NAC end > NAC start; surgery > NAC end; ypN coded only if axillary
  surgery performed).
- 10% sample re-abstracted by PI; require Cohen's kappa ≥ 0.85 per variable.

### 6.3 Missing data handling
- **Per-variable missingness** reported in flow diagram.
- If any single variable >40% missing → drop from analysis with sensitivity analysis.
- **<10% missingness:** complete-case analysis (primary).
- **10–40% missingness:** multiple imputation by chained equations (MICE), m=20 imputations,
  pooling via Rubin's rules. R package `mice`.
- Imputation model includes all predictors + outcome (to avoid bias).
- Sensitivity analysis: complete-case vs imputed comparison.

### 6.4 Continuous predictor modeling
- Linear by default; **fractional polynomials** (Royston-Sauerbrei) explored for non-linearity
  in age, BMI, Ki67.
- If non-linearity AIC-improvement Δ ≥10, retain transformation.

---

## 7. Descriptive statistics

### 7.1 Table 1 — Baseline characteristics
- Stratified by ypN0 outcome (yes/no).
- Continuous: median (IQR) or mean ± SD as appropriate (Shapiro-Wilk for normality).
- Categorical: n (%).
- Comparisons: chi-square / Fisher's exact (categorical), Wilcoxon / t-test (continuous).
- p-values reported but **not used for variable selection**.

---

## 8. Model development

### 8.1 Primary model — multivariable logistic regression
- Generalized linear model, family = binomial, link = logit.
- All pre-specified variables entered together.
- Interactions pre-specified:
  - subtype × pembrolizumab
  - subtype × pertuzumab
  - cN × subtype

### 8.2 Comparator model — XGBoost
- Hyperparameters tuned via grid search within nested 5-fold CV:
  - `max_depth` ∈ {3, 4, 5, 6}
  - `eta` ∈ {0.01, 0.05, 0.1}
  - `min_child_weight` ∈ {1, 3, 5}
  - `subsample`, `colsample_bytree` ∈ {0.7, 0.8, 0.9}
- `n_estimators` selected by early stopping (50 rounds without improvement).
- Class imbalance handled via `scale_pos_weight = N_negative / N_positive`.

### 8.3 Comparator model — Nivel 2 meta-regression
- Fixed coefficients from Nivel 2 model.
- Applied to institutional data without re-fitting.

### 8.4 Explainability
- For logistic: odds ratios with 95% CIs.
- For XGBoost: **SHAP values** (TreeSHAP) — global feature importance + local explanations
  for selected representative cases.

---

## 9. Internal validation

### 9.1 Cross-validation
- **Stratified 10-fold CV** (stratified by ypN0).
- Repeated 10 times → 100 fold-fits; mean ± SD reported.

### 9.2 Bootstrap optimism correction
- 1000 bootstrap resamples (with replacement).
- Optimism = (apparent performance) - (mean test-set performance).
- Optimism-corrected metric = apparent - optimism.

### 9.3 Performance metrics

| Metric | Target | Reporting |
|--------|--------|-----------|
| Discrimination (AUROC) | ≥0.75 (acceptable), ≥0.80 (good) | Mean + 95% CI (bootstrap) |
| Discrimination (AUPRC) | report alongside AUROC | Mean + 95% CI |
| Calibration slope | 0.85–1.15 | Mean + 95% CI |
| Calibration-in-the-large (intercept) | -0.20 to +0.20 | Mean + 95% CI |
| Brier score | <0.20 (lower is better) | Mean + 95% CI |
| Brier skill score | >0 vs null | Mean + 95% CI |

### 9.4 Calibration plots
- Smoothed (LOESS) calibration curves with 10-decile binning.
- Hosmer-Lemeshow test reported descriptively (not used as gatekeeper given known limitations).

---

## 10. External validation

### 10.1 Cohort
- Independent dataset from collaborator (Z1071 / SenTa / RISAS / Montagna; see
  `06_collaboration_outreach_template.md`).
- **No re-fitting** of model coefficients on external data.

### 10.2 Metrics
- All metrics from Section 9.3, reported on external set.
- 95% CIs by **bootstrap percentile** (1000 resamples).

### 10.3 Recalibration
- If calibration slope outside [0.85, 1.15], **report** but do not change primary model.
- Secondary analysis: recalibrated intercept-only update + recalibrated slope.

---

## 11. Decision-curve analysis (DCA)

- Compute net benefit across thresholds 5%–60% (typical for omitting ALND).
- Compare three strategies:
  1. Treat all (all patients undergo ALND).
  2. Treat none (no patient undergoes ALND).
  3. Treat by Nivel 2 model.
  4. Treat by Nivel 3 model.
- **Primary clinical question:** does Nivel 3 yield meaningful net benefit gain at
  thresholds 10–25% (clinically relevant zone for de-escalation decisions)?

---

## 12. Subgroup analyses (pre-specified)

| Subgroup | Cut | Justification |
|----------|-----|---------------|
| Molecular subtype | TNBC / HER2+ / HR+ HER2- | Strongest effect modifier |
| cN stage | cN1 vs cN2-3 | Different baseline event rate |
| Treatment era | <2020 vs ≥2020 | Pre/post-pembrolizumab era |
| Age | <50 vs ≥50 | Premenopausal biology |
| Anti-HER2 regimen | T alone vs T+P | Dual blockade effect |

For each subgroup: AUC, calibration slope, n; **caution**: subgroup AUCs unstable when n<80.

---

## 13. Sensitivity analyses

1. Complete-case vs multiple imputation.
2. Excluding patients with NAC <80% completion.
3. Excluding cT4 (in case any slipped through).
4. ITC counted as positive (ypN1mi in outcome) → re-estimate.
5. Restricted to single-tracer SLNB (vs dual-tracer TAD).
6. Pre-IO era only (sensitivity if external cohort lacks pembro data).

---

## 14. Software

- **R 4.4+** as primary (packages: `tidyverse`, `mice`, `rms`, `pROC`, `boot`,
  `riskRegression`, `dcurves`, `xgboost`, `SHAPforxgboost`, `pmsampsize`).
- **Python 3.11+** for XGBoost + SHAP if preferred (packages: `xgboost`, `scikit-learn`,
  `shap`, `pandas`).
- Code repository: GitHub (private during development → public at publication).
- Random seed: 20260514 (fixed for reproducibility).

---

## 15. Reporting

- TRIPOD-AI checklist completed and submitted as supplementary.
- Flow diagram (CONSORT-style) of patient inclusion/exclusion.
- All code + de-identified analytic dataset deposited (Zenodo) at acceptance.
- Pre-registration: OSF or `clinicaltrials.gov` (observational) at start.

---

## 16. Deviations from SAP

Any deviation logged in dated addendum, signed by PI + statistician, before unblinding.

---

## 17. References

1. Collins GS, Moons KGM, Dhiman P, et al. **TRIPOD+AI statement.** BMJ 2024;385:e078378.
   doi:10.1136/bmj-2023-078378
2. Steyerberg EW. **Clinical Prediction Models: a practical approach to development,
   validation and updating.** 2nd ed. Springer; 2019.
3. Steyerberg EW, Vickers AJ, Cook NR, et al. **Assessing the performance of prediction
   models: a framework for traditional and novel measures.** Epidemiology 2010;21:128-38.
   doi:10.1097/EDE.0b013e3181c30fb2
4. Vickers AJ, Elkin EB. **Decision curve analysis.** Med Decis Making 2006;26:565-74.
   doi:10.1177/0272989X06295361
5. Royston P, Sauerbrei W. **Multivariable Model-Building.** Wiley; 2008.
6. White IR, Royston P, Wood AM. **Multiple imputation using chained equations.**
   Stat Med 2011;30:377-99. doi:10.1002/sim.4067
7. Lundberg SM, Lee SI. **A unified approach to interpreting model predictions.**
   NeurIPS 2017. arXiv:1705.07874
