Is it possible to train a Regional Conditioning controlnet?
Forge couple implementation tried to do the same thing but it is very inconsistent and can't to complex spatial generation since it targets only the cross attention layer which isn't great for spatial understanding. Like the inpainting model, it accepts a mask to localize the focus area, i was thinking if it was possible to do a regional conditioning model? With the control image being as color blobs denoting where the character/object should generate. But it would also require to map the specific prompt to that specific color blob. What are your thoughts?
I understand that this isn't inpainting, but rather conditioning to generate something in a specific area. It might be possible using LLLite with a mask, where the same subject is always present in the masked area.
However, currently, LLLite's training code only supports random masks, so I think the training code would need to be updated.
Thanks for replying. I think it would be nice to have some strong regional conditioning method, since previous ones aren't very consistent.
oh that would b really great, Im currently struggling a lot with regional conditioning