Config structure prevents AutoModel compatibility (Bottleneck for T5Gemma-2 migration)
This encoder only extraction contains info about the decoding in the config file meaning if it is loaded and not explicitly told not to use the decoder, transformers may look for something that doesn't exist.
Even with is_encoder_decoder set to false, the presence of the decoder configuration block causes AutoModel to attempt a full model initialization and fail. This forces users to use the specific T5GemmaEncoderModel class rather than the standard AutoModel factory, which breaks cross-compatibility with newer T5Gemma-2 based architectures.
Yes, the code needs to be refactored to ensure better compatibility and versatility. I will try to work on this in the coming weeks.
I want to note that using this with T5Gemma2 or other models will require creating a new adapter model from scratch, as they are incompatible with the previous version.
I stand corrected, AutoModel refuses to load the T5Gemma encoder at all referring to the specific T5GemmaEncoder class.
Yes, the code needs to be refactored to ensure better compatibility and versatility. I will try to work on this in the coming weeks.
I want to note that using this with T5Gemma2 or other models will require creating a new adapter model from scratch, as they are incompatible with the previous version.
I am aware, please take a look at the github repo for the comfy ui node for llm to sdxl. I already made some suggestions and laid the groundwork for T5Gemma2.