rarfileexe/X-Pathology-Colorectal-Tissue-Classifier-CTransPath-Backbone

X-Pathology — Colorectal Tissue Classifier (CTransPath Backbone)

A 9-class histopathology tissue classifier for H&E-stained colorectal biopsy tiles, built on CTransPath — a Swin Transformer pretrained specifically on histopathology images via Semantically-Relevant Contrastive Learning (SRCL), not on natural ImageNet photos.

97.33% test accuracy (Macro F1: 0.9615) on a fully independent, held-out patient cohort (CRC-VAL-HE-7K, n=3,590) — measured against the same benchmark where an ImageNet-pretrained EfficientNet-B1 baseline scored 92.7%.

You can find the full Training Notebook on my GitHub : https://github.com/Muhammad-Hassan12/XPathology-Colorectal-Tissue-Specialist

---

⚠️ Intended Use and Clinical Disclaimer

RESEARCH AND EDUCATIONAL USE ONLY.
This model is not a medical device, has not been clinically validated, has not undergone regulatory review (FDA, CE-IVD, etc.), and must never be used for clinical diagnosis, patient triage, or real-world treatment planning. It is distributed solely to facilitate open research, education, benchmarking, and interpretability studies in computational pathology.

• Appropriate Uses: Academic research, digital pathology algorithm benchmarking, educational demonstrations, interpretability & explainable AI (XAI) experiments.
• Inappropriate Uses: Primary or secondary diagnostic screening, clinical decision support, commercial patient management, or any unvalidated real-world medical workflow.

---

Model Architecture

Component	Specification
Backbone	CTransPath — Swin-Tiny with a custom convolutional patch-embedding stem (ConvStem)
Pretraining	Semantically-Relevant Contrastive Learning (SRCL) on ~15M histopathology patches from ~32K WSIs (PAIP + TCGA)
Backbone Parameters	~27.5M (total ~27.7M with head)
Feature Dimension	768
Classification Head	LayerNorm(768) → Dropout(0.3) → Linear(768→256) → GELU → Dropout(0.225) → Linear(256→9)
Input Resolution	224×224 RGB
Normalization	Mean: [0.485, 0.456, 0.406], Std: [0.229, 0.224, 0.225]
Output	9-class probabilities, Temperature-Calibrated (T \approx 0.5655)

---

Histological Taxonomy (9 Classes)

Abbreviation	Full Histological Classification	Description & Clinical Significance
ADI	Adipose Tissue	Fat tissue / subserosal adipose
BACK	Background	Glass slide / non-tissue areas
DEB	Debris & Necrosis	Necrotic tumour fragments and cellular debris
LYM	Lymphocytes	Immune cell infiltration / tumor-infiltrating lymphocytes (TILs)
MUC	Mucus	Mucin pools in colorectal mucosa
MUS	Smooth Muscle	Muscularis propria / muscularis mucosae
NORM	Normal Colon Mucosa	Healthy colonic glands and epithelial lining
STR	Cancer-Associated Stroma	Desmoplastic stroma surrounding invasive glands
TUM	Colorectal Adenocarcinoma	Malignant epithelial tumour tissue

---

Training Data & Non-Leakage Validation Split

Split	Source Dataset	Patches (n)	Role in Pipeline
Train	NCT-CRC-HE-100K	100,000	Used in full for Phase 1 (linear probe) & Phase 2 (fine-tuning)
Validation	CRC-VAL-HE-7K (50% split)	3,590	Model selection, early stopping monitoring, temperature calibration ($T$)
Test	CRC-VAL-HE-7K (50% split)	3,590	Untouched holdout cohort for final evaluation report

Why Cross-Cohort Splitting Matters: NCT-CRC-HE-100K and CRC-VAL-HE-7K are derived from distinct patient cohorts. Random tile-level splitting within a single dataset causes patient/slide leakage and artificially inflates accuracy to >99%. Every metric reported below is evaluated on a truly independent patient cohort never seen during training or tuning.

---

Training Procedure

A two-phase transfer learning regime with mixed precision (torch.amp), gradient clipping (max norm 1.0), and balanced class-weighted Cross-Entropy loss with 0.1 label smoothing:

Hyperparameter / Stage	Phase 1 — Linear Probe	Phase 2 — Fine-Tuning
Backbone State	Fully frozen	Top 2 of 4 Swin stages unfrozen
Trainable Parameters	~200K (head only)	Top 2 Swin stages + classification head
Learning Rate	1e-3	Differential: Backbone 1e-5, Head 2e-4
Optimizer & Scheduler	AdamW + CosineAnnealingLR	AdamW + CosineAnnealingLR
Epochs (Max / Executed)	8 / 8 epochs	15 / 8 epochs (Early stopping triggered)
Early Stopping	Patience 5, monitored on val loss	Patience 5, monitored on val loss
Hardware & Time	2× NVIDIA T4 (16GB), DataParallel	~91 minutes total wall-clock runtime

---

Evaluation Results

Evaluated on the independent CRC-VAL-HE-7K test cohort ($n=3,590$):

• Overall Test Accuracy: 97.33% (3,494 / 3,590 correct)
• Macro F1 Score: 0.9615
• Weighted F1 Score: 0.9734

Per-Class Performance Metrics

Class	Precision	Recall	F1-Score	Support (n)
ADI	0.9970	0.9925	0.9948	669
BACK	1.0000	1.0000	1.0000	423
DEB	0.9825	0.9941	0.9882	169
LYM	0.9937	0.9968	0.9953	317
MUC	0.9942	0.9942	0.9942	518
MUS	0.8889	0.8649	0.8767	296
NORM	0.9761	0.9919	0.9840	371
STR	0.8145	0.8531	0.8333	211
TUM	0.9934	0.9805	0.9869	616
Macro Average	0.9600	0.9631	0.9615	3,590
Weighted Average	0.9737	0.9733	0.9734	3,590

Benchmark Comparison

Model Architecture	Pretraining Domain	Test Accuracy (Holdout)	Macro F1
EfficientNet-B1 Baseline	ImageNet-1k (Natural images)	92.70%	0.8980
XPathology CTransPath (This Model)	Histopathology (PAIP + TCGA, ~15M patches)	97.33%	0.9615

Known Histological Ambiguity: MUS vs STR

Smooth muscle (MUS) and cancer-associated stroma (STR) share high morphological and eosinophilic textural similarity in H&E sections. The majority of minor misclassifications occur between this pair (40 MUS samples predicted as STR; 27 STR samples spread across MUS/MUC/NORM/TUM). This is a well-documented challenge in colorectal computational pathology.

---

Temperature Scaling & Confidence Calibration

Raw neural network softmax outputs often exhibit overconfidence. We calibrated logits using post-hoc Temperature Scaling ($T \approx 0.5655$), optimized by minimizing Negative Log-Likelihood (NLL) exclusively on the validation cohort:

$${\widehat{P}}_i=\frac{e^{z_i\mathrm{/}T}}{\sum _j{e}^{z_j\mathrm{/}T}}\backslash hat\{P\}\_i\; =\; \backslash frac\{e^\{z\_i\; /\; T\}\}\{\backslash sum\_\{j\}\; e^\{z\_j\; /\; T\}\}$$

• Uncalibrated Validation NLL: 0.1960
• Calibrated Validation NLL: 0.0937 (52.2% reduction in calibration error)

---

Explainability: Grad-CAM++ with Signed Feature Fix

This repository includes a specialized implementation of Grad-CAM++ adapted for vision transformers with LayerNorm feature maps:

1. Signed LayerNorm Adaptation: CTransPath's forward_features() ends in LayerNorm, creating zero-centered, signed representations. Grad-CAM++ closed-form weighting assumes non-negative activations; we compute $\alpha$-weights over positive clamped activations (feat_map.clamp(min=0.0)), preventing denominator collapse and flat heatmaps.
2. Degenerate Guard with Automatic Fallback: If an activation map exhibits collapsed dynamic range (\max - \min < 10^{-6}), the pipeline automatically falls back to vanilla linear-weighted Grad-CAM.

---

Quickstart & Usage

1. Installation

pip install torch torchvision timm opencv-python matplotlib pillow

2. Standalone Minimal Inference (Copy & Paste)

import torch
import torch.nn as nn
import torch.nn.functional as F
import timm
from timm.layers.helpers import to_2tuple
from PIL import Image
import numpy as np

# 1. Architecture definition
class ConvStem(nn.Module):
    def __init__(self, img_size=224, patch_size=4, in_chans=3, embed_dim=768, norm_layer=None, **kwargs):
        super().__init__()
        img_size = to_2tuple(img_size)
        patch_size = to_2tuple(patch_size)
        self.img_size, self.patch_size = img_size, patch_size
        stem = []
        in_dim, out_dim = in_chans, embed_dim // 8
        for _ in range(2):
            stem.extend([
                nn.Conv2d(in_dim, out_dim, kernel_size=3, stride=2, padding=1, bias=False),
                nn.BatchNorm2d(out_dim),
                nn.ReLU(inplace=True)
            ])
            in_dim = out_dim
            out_dim *= 2
        stem.append(nn.Conv2d(in_dim, embed_dim, kernel_size=1))
        self.proj = nn.Sequential(*stem)
        self.norm = norm_layer(embed_dim) if norm_layer else nn.Identity()

    def forward(self, x):
        return self.norm(self.proj(x).permute(0, 2, 3, 1))

class CTransPathColonClassifier(nn.Module):
    def __init__(self, num_classes=9, dropout_rate=0.3):
        super().__init__()
        self.backbone = timm.create_model("swin_tiny_patch4_window7_224", embed_layer=ConvStem, pretrained=False, num_classes=0)
        feat_dim = self.backbone.num_features
        self.head = nn.Sequential(
            nn.LayerNorm(feat_dim),
            nn.Dropout(dropout_rate),
            nn.Linear(feat_dim, 256),
            nn.GELU(),
            nn.Dropout(dropout_rate * 0.75),
            nn.Linear(256, num_classes),
        )
    def forward(self, x):
        feat_map = self.backbone.forward_features(x)
        return self.head(feat_map.mean(dim=(1, 2)))

# 2. Load model & weights
CLASS_NAMES = ["ADI", "BACK", "DEB", "LYM", "MUC", "MUS", "NORM", "STR", "TUM"]
TEMPERATURE = 0.56552676

model = CTransPathColonClassifier(num_classes=9)
model.load_state_dict(torch.load("xpathology_colon_ctranspath_v6.pt", map_location="cpu"))
model.eval()

# 3. Preprocess image
img = Image.open("test_tile.png").convert("RGB").resize((224, 224))
arr = np.array(img).astype(np.float32) / 255.0
mean, std = np.array([0.485, 0.456, 0.406]), np.array([0.229, 0.224, 0.225])
norm_arr = (arr - mean) / std
tensor = torch.from_numpy(norm_arr.transpose(2, 0, 1)).unsqueeze(0).float()

# 4. Predict with temperature calibration
with torch.no_grad():
    logits = model(tensor)
    calibrated_probs = F.softmax(logits / TEMPERATURE, dim=1)[0]

pred_idx = torch.argmax(calibrated_probs).item()
print(f"Prediction: {CLASS_NAMES[pred_idx]} | Calibrated Confidence: {calibrated_probs[pred_idx]:.2%}")

3. Using the Comprehensive Pipeline Script (predict_and_gradcam.py)

A fully-featured diagnostic script is included in the repository:

# Test a single image with Grad-CAM++ visualization:
python predict_and_gradcam.py --image "test/Colorectal Adenocarcinoma (Tumour).jpg"

# Target a specific class (e.g., Stroma):
python predict_and_gradcam.py --image "test/Cancer-Associated Stroma.png" --target_class STR

# Batch process an entire directory of tiles:
python predict_and_gradcam.py --dir test --output_dir gradcam_output

---

Citation

If you use this model or code in your research, please cite the underlying CTransPath architecture and dataset:

@article{wang2022transformer,
  title={Transformer-based unsupervised contrastive learning for histopathology image classification},
  author={Wang, Xiyue and Yang, Sen and Zhang, Jun and Wang, Minghui and Zhang, Jing and Yang, Wei and Huang, Junzhou and Han, Xiao},
  journal={Medical Image Analysis},
  volume={81},
  pages={102559},
  year={2022},
  publisher={Elsevier},
  doi={10.1016/j.media.2022.102559}
}

@article{kather2018100000,
  title={100,000 histological images of human colorectal cancer and healthy tissue},
  author={Kather, Jakob Nikolas and Halama, Niels and Marx, Alexander},
  journal={Zenodo},
  year={2018},
  doi={10.5281/zenodo.1214456}
}

---

License

This model and repository are licensed under GPL-3.0 (Non-Commercial Research and Educational Use), consistent with the upstream licensing terms of the original CTransPath model (Xiyue-Wang/TransPath).

---

Project Context & Author

Part of X-Pathology, an open-source initiative in computational pathology. (Research and Educational! not Clinically Verified or Affiliated!)

• Author: Syed Muhammad Hassan (Portfolio · GitHub · LinkedIn)