Ignesso Lesias's picture

Ignesso Lesias

Dustyang111
ยท

AI & ML interests

None yet

Recent Activity

repliedto merve's post about 1 month ago
Don't sleep on new AI at Meta Vision-Language release! ๐Ÿ”ฅ https://huggingface.co/collections/facebook/perception-encoder-67f977c9a65ca5895a7f6ba1 https://huggingface.co/collections/facebook/perception-lm-67f9783f171948c383ee7498 Meta dropped swiss army knives for vision with A2.0 license ๐Ÿ‘ > image/video encoders for vision language modelling and spatial understanding (object detection etc) ๐Ÿ‘ > The vision LM outperforms InternVL3 and Qwen2.5VL ๐Ÿ‘ > They also release gigantic video and image datasets The authors attempt to come up with single versatile vision encoder to align on diverse set of tasks. They trained Perception Encoder (PE) Core: a new state-of-the-art family of vision encoders that can be aligned for both vision-language and spatial tasks. For zero-shot image tasks, it outperforms latest sota SigLIP2 ๐Ÿ‘ > Among fine-tuned ones, first one is PE-Spatial. It's a model to detect bounding boxes, segmentation, depth estimation and it outperforms all other models ๐Ÿ˜ฎ > Second one is PLM, Perception Language Model, where they combine PE-Core with Qwen2.5 LM 7B. it outperforms all other models (including InternVL3 which was trained with Qwen2.5LM too!) The authors release the following checkpoints in sizes base, large and giant: > 3 PE-Core checkpoints (224, 336, 448) > 2 PE-Lang checkpoints (L, G) > One PE-Spatial (G, 448) > 3 PLM (1B, 3B, 8B) > Datasets Authors release following datasets ๐Ÿ“‘ > PE Video: Gigantic video datasete of 1M videos with 120k expert annotations โฏ๏ธ > PLM-Video and PLM-Image: Human and auto-annotated image and video datasets on region-based tasks > PLM-VideoBench: New video benchmark on MCQA
View all activity

Organizations

None yet