UniCharacter

UniCharacter is a collection of character-specific checkpoints for Customized Multimodal Role-Play (CMRP), introduced in the paper Towards Customized Multimodal Role-Play.

Project Page | GitHub | Paper

The model is designed to customize a character's persona, dialogue style, and visual identity so that the character can respond consistently across text and image generation settings. Using a unified multimodal model, UniCharacter employs a two-stage training framework containing Unified Supervised Finetuning (Unified-SFT) and character-specific group relative policy optimization (Character-GRPO).

Repository Contents

This repository contains separate checkpoint folders for multiple characters. Each character directory includes sharded safetensors weights and a model.safetensors.index.json file.

Available character folders include:

• Adrien_Brody, Bo, Butin, Chandler, Coco, Furina, Gao_Qiqiang, Hermione, Ichihime, Joey, Leonardo, Mam, Miki_Nikaidou, Mydieu, Pikachu, Rin_Tohsaka, Saber, Will_In_Vietnam, Wukong, YuiYagi

Quick Usage Example

To use these checkpoints, please follow the installation instructions in the official repository. Below is an example of the unified inference interface:

from inference import create_unicharacter_inference
from pathlib import Path

# Initialize the unified inference (modify paths according to your environment)
inference = create_unicharacter_inference(
    model_path="models/BAGEL-7B-MoT",
    checkpoint_path="<checkpoint_path>",
    vit_checkpoint_path="<vit_checkpoint_path>",
    max_mem_per_gpu="40GiB",
    seed=42,
)

out_dir = Path("test_images/outputs")
out_dir.mkdir(parents=True, exist_ok=True)

# 1) Text-to-image generation (Role T2I)
res = inference.generate_image("Ichihime chasing a butterfly")
res["image"].save(out_dir / "t2i_ichihime.png")

# 2) Visual understanding / VQA
res = inference.visual_understanding(
    "data/personalized_data/train/Mahjong Soul-Ichihime/1.png",
    "What's the color of Ichihime's hair?",
)
print("VQA:", res["text"])

# 3) Knowledge QA
res = inference.knowledge_qa("When do you born?")
print("Knowledge QA:", res["text"])

# 4) Multimodal role-play
res = inference.role_play(
    character_name="Ichihime",
    description="",
    opening="",
    user_text="Hi, Ichihime. How are you?",
)
print("Role-play:", res["response"])

Download

Download the full repository:

huggingface-cli download Tangc03/UniCharacter --local-dir UniCharacter

Download a single character checkpoint folder:

huggingface-cli download Tangc03/UniCharacter --include "Hermione/*" --local-dir UniCharacter

Citation

If you use UniCharacter, please cite:

@article{tang2026towards,
  title={Towards Customized Multimodal Role-Play},
  author={Tang, Chao and Wu, Jianzong and Shi, Qingyu and Tian, Ye and Zhang, Aixi and Jiang, Hao and Zhang, Jiangning and Tong, Yunhai},
  journal={arXiv preprint arXiv:2605.08129},
  year={2026}
}