How to Install ControlNet Extension in Stable Diffusion (A1111)

In this blog post, we will explore the process of installing ControlNet for Stable Diffusion (A1111). ControlNet is a neural network interface structure that enhances the control over stable diffusion models by adding additional constraints. It allows you to generate better and more controlled outputs. We'll provide a step-by-step guide to help you through the installation process.

To install the ControlNet extension, open the web UI interface and follow these steps:

• Navigate to the "Extensions" tab.
• Click on "Install from URL" and
• Paste the Git URL: https://github.com/Mikubill/sd-webui-controlnet
• Click "Install"
• Once done, Click on "Apply and restart UI" or Close Stable Diffusion web UI and restart it.

When successfully installed, you should be able to see the ControlNet expansion panel in both the 'txt2img' and 'img2img' tabs. It should look like this when the expansion panel is expanded:

With the ControlNet extension installed, we need to download the pre-trained models. The original pre trained models can be found on the huggingface website.

Make sure to download at least one model (file ending with .pth), but it's suggested to have all ControlNet models installed. Once downloaded, place the models in following folder location: "extensions/sd-web-ui/ControlNet/models" within the Stable Diffusion folder. Example below:

You also have the option to download the .safetensors pre-trained models, which consume less storage space.

Each model needs to be paired with the appropriate pre-processor. For example, if you're using the canny preprocessor, pair it with the original or pre-trained canny model. Example:

The same goes for depth, HED, mlsd, normal map, open pose, scribble, and segmentation models. Ensure that the correct combination is selected. Although the specific use of ControlNet is beyond the scope of this blog post, successfully installing it is the primary focus.

Some users may encounter errors related to Gradio when generating images with Control Net. To resolve this issue, upgrade the Gradio version to 3.16.2. You can do this by opening the command line within the Stable Diffusion folder. On Windows, right-click and select "Open in Terminal." Then, enter the command pip install gradio==3.16.2 to initiate the installation.

Congratulations! You have successfully installed Control Net for Automatic 1111's Web UI in Stable Diffusion. By following this comprehensive guide, you now have the necessary tools and knowledge to enhance your control over diffusion models and generate better, more controlled outputs.