Detect 2.0 vs 2.1 ZImageControlNetModel

[hlky]

What does this PR do?

In DIFFUSERS_DEFAULT_PIPELINE_PATHS z-image-turbo-controlnet-2.x is changed for separate z-image-turbo-controlnet-2.0 and z-image-turbo-controlnet-2.1. z-image-turbo-controlnet-2.x is kept for CHECKPOINT_KEY_NAMES, there is no key difference between the 2 versions. The detection comes from another key and checking whether the weight is all zeros: torch.all(checkpoint["control_noise_refiner.0.before_proj.weight"] == 0.0). This works due to zero_module when before_proj is initialized - these layers remained untrained in 2.0. We also account for the possibility of control_noise_refiner being removed from the checkpoint as done in the Diffusers version.

diffusers/src/diffusers/models/controlnets/controlnet_z_image.py

Line 343 in 55463f7

self.before_proj = zero_module(nn.Linear(self.dim, self.dim))

With PR:

controlnet = ZImageControlNetModel.from_single_file(
    hf_hub_download(
        "alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.0",
        "Z-Image-Turbo-Fun-Controlnet-Union-2.0.safetensors",
    ),
    torch_dtype=torch.bfloat16,
-   config="hlky/Z-Image-Turbo-Fun-Controlnet-Union-2.0",
)

Passing config is no longer required.

import torch
from diffusers import ZImageControlNetModel
from huggingface_hub import hf_hub_download


controlnet = ZImageControlNetModel.from_single_file(
    hf_hub_download(
        "alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.0",
        "Z-Image-Turbo-Fun-Controlnet-Union-2.0.safetensors",
    ),
    torch_dtype=torch.bfloat16,
)
assert controlnet.control_noise_refiner is None

controlnet = ZImageControlNetModel.from_single_file(
    hf_hub_download(
        "alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.0",
        "Z-Image-Turbo-Fun-Controlnet-Union-2.1.safetensors",
    ),
    torch_dtype=torch.bfloat16,
)
assert controlnet.control_noise_refiner is not None

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.