CUAStepBench leaderboard judge outputs

Raw judge outputs behind every row of the CUAStepBench leaderboard, one folder per row. CUAStepBench itself (goals, screenshots, actions and the human-annotated-zzh ground truth) comes from huggingface.co/datasets/ZJUSCL/CUAStepBench.

Each folder holds a README.md, a metrics.json, a judge_name_mapping.json inventory, and a CUAStepBench/ tree mirroring the released dataset layout:

CUAStepBench/<benchmark>/<agent>/<task_id>/judge/<run>/result.json

Using them

# merge one row (or all seven) into a CUAStepBench checkout
rsync -a 04-seekjudge-9b/CUAStepBench/ /path/to/CUAStepBench/

# analyse it with the SeekJudge release code
REWARD_SUBCRITERIA_MODEL_PATH=./configs/subcriteria/fitted_kfold_P84_xgb_md2_ft_3bench.pkl \
python -m seekjudge.analysis.analyse_judge \
    --data-dir /path/to/CUAStepBench \
    --gt-judge-name human-annotated-zzh \
    --target-judge-name seekjudge-9b \
    --step-9class2class-config configs/subcriteria_step/9classmap.yaml

The leaderboard numbers use the fitted_kfold_P84_xgb_md2_ft_3bench.pkl scorer.

The run directories keep the _rand1 .. _rand5 suffix convention, which is how analyse_judge finds the repeated samples of one configuration. Passing the base name picks up every repeat.