Senum/plaid-meddialog-lora

Plaid-1B MedDialog LoRA

LoRA adapter fine-tuned on the OpenMed/MedDialog patient-doctor conversation dataset on top of the Plaid-1B continuous diffusion language model. Final-year-project artifact.

Research demo only. The outputs are not medical advice. Always consult a qualified clinician for real health questions.

What's in this repo

File	Purpose
lora_model/adapter_config.json	PEFT LoRA config (rank, targets, ...)
lora_model/adapter_model.safetensors	LoRA weights (~200 MB)
embedding_matrix.pt	Custom token embedding matrix saved at checkpoint time
noise_schedule.pt	Learned diffusion noise schedule (if present)
gamma_bounds.pt	(gamma_0, gamma_1) bounds (if present)
training_info.json	Hyperparameters, step count, dataset split, etc.

The ~5.1 GB Plaid-1B base weights are not included - they are distributed separately by the Plaid authors. At inference time, load the base weights first, then apply this LoRA adapter on top.

Training

• Base model: Plaid-1B (continuous-latent diffusion transformer, ~1B params)
• Adapter: LoRA, rank 64, targeting attn_qkv, attn_out, fc1, fc2, and the embedding matrix.
• Dataset: OpenMed/MedDialog (patient <-> doctor dialogues).
• Hardware: 1x NVIDIA RTX 3090 (24 GB), float64 precision.
• Epochs: 10 (this checkpoint = epoch 8, picked for best val loss).

Usage

This adapter is intended to be loaded in the full Plaid training environment (torch, mup, peft, apex, flash_attn). A runnable interactive demo that wires everything together is available in the FYP repo fyp-final-demo (FastAPI backend + Streamlit frontend + cloudflared tunnel).

from huggingface_hub import snapshot_download

path = snapshot_download(repo_id="Senum/plaid-meddialog-lora")
# then load with sample_meddialog.load_model(
#     weights_path="/path/to/plaid1b_weights",
#     checkpoint_path=path,
#     ...
# )

Citation

If you use this artifact in academic work, please also cite the Plaid paper (continuous diffusion language models) and the MedDialog dataset.