google
/

flan-t5-xxl

text2text-generation

text-generation-inference

Model card Files Files and versions

Config does not align with original paper

#57

by cbock90 - opened Aug 7, 2023

•

edited Aug 7, 2023

Hey there,

I see that the config significantly differs from the one on the paper:

dmodel: Config: 4096, Paper: 1024
#heads: Config: 64, Paper: 128
d_ff: Config: 10240, Paper: 65536
dkv: Config: 64, Paper: 128

Are there any insights why there is this difference in the checkpoint?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment