docs/experiments.md

# Experiments for the ICLR draft

The core question is whether a direct endpoint map improves intervention selection when the objective includes population structure and gene interactions. The initial results support a focused study of Norman combinations and population matching.

| Comparison | Executable workflow | Status |
|---|---|---|
| Corrected Norman response and pair nomination | `scripts/run_matrix.py --dataset norman --split combination` | Implemented; full runs pending |
| Mean control, average effect, additive, ridge, endpoint MLP, conditional MLP | `python -m pivot.cli evaluate --baseline NAME` | Implemented |
| Map/tangent/composition ablations | `scripts/run_matrix.py --ablations` | Implemented; full runs pending |
| Population-loss weights 2 and 10 | `configs/distribution_2.json`, `configs/distribution_10.json` | Implemented; full runs pending |
| Guidance versus exhaustive ranking | `evaluate --initialization random` or `best` | Implemented with projection diagnostics |
| Reward sweep | `evaluate --reward centroid`, `cosine`, `mmd`, or `wasserstein` | Implemented; choose on validation targets |
| Replogle single-gene holdout | `--dataset replogle_k562 --split gene` | Adapter implemented; corrected full run pending |
| GEARS on common features and outcome cells | Original GEARS scripts in archived source | New shared-protocol adapter pending |
| CellFlow distributional comparison | Published implementation, matched data representation | External adapter pending |
| PAIRING/PDGrapher inverse comparison | Requires task and action-space alignment | Secondary external integration |
| Independent pair interaction residuals | Reserved single-gene and pair responses | Analysis implementation pending |
| Prospective new combinations | New perturbation experiment | Future experimental validation |

Every `run_matrix.py` invocation also needs `--raw`, `--output`, and a device. See the README for the complete command. The matrix saves separate per-seed result files. `plot_results.py` exports each run's summaries as CSV and plots their measured values. Use the condition-level rows for paired comparisons; do not pool cells as independent replicates.

## Priority for the remaining week

1. Run the corrected Norman combination split with PIVOT, additive effects, ridge, and the matched cell-conditional MLP across three seeds. This is the main decision-quality comparison.
2. Repeat the map/tangent/composition and population-loss comparisons on those same splits. Select reward and hyperparameters on validation conditions.
3. Score selected actions against reserved measured responses. Report measured regret, Top-5, population discrepancy, and variation across seeds. Add the independent interaction-residual analysis.
4. Complete one strong external comparison. GEARS has an existing historical integration and is the first priority. CellFlow directly tests the population-model claim if a matched adapter is feasible.
5. Compare random-start guidance, warm-start guidance, and exhaustive ranking with initialization included in the budget. Preserve full target IDs and time both initial and amortized catalog scoring.

A larger unrelated benchmark is lower priority than these comparisons. The manuscript's blank tables already contain the necessary rows. Full-scale results should replace those cells only after validation against the saved condition-level records.

## Submission preparation

The compiled draft uses the supplied ICLR 2027 style and an anonymous author block. The official limit is nine main-text pages. The draft AI-use section is intentionally blank. Complete an accurate disclosure and confirm the author order before submission. The public Hugging Face archive contains author attribution and must be anonymized separately if supplied directly to reviewers.

Official requirements: [Author guidelines](https://iclr.cc/Conferences/2027/AuthorGuidelines), [AI policy](https://iclr.cc/Conferences/2027/AIPolicyForAuthors).