Vibrant Sketch Colorization: Stable Diffusion & ControlNet

Step into a realm where sketches come alive in brilliant color. Discover how we use Stable Diffusion and ControlNet to transform plain drawings into vibrant artworks. Let's explore the magic of blending technology and art to make static sketches tell vivid visual stories.

To start adding appealing colors to your lovely sketches or line drawings, it's important to set up ControlNet correctly with the pre-trained models, Lineart, and Shuffle. If you haven't installed ControlNet yet, don't worry, we've put together a detailed guide that provides clear instructions on how to get it up and running. You can find the tutorial down below:

How to Install ControlNet Extension in Stable Diffusion (A1111)

Once ControlNet is installed correctly, make sure you have a captivating sketch or a carefully crafted line drawing that you're ready to infuse with color. If you haven't prepared one yet, no need to be concerned – we've a guide that walks you through the process of creating sketch art using Stable Diffusion

In the "txt2img" section, we'll choose for the revAnimated checkpoint. When it comes to the sampling method, we'll select DDIM and set the sampling steps to 30. For the dimensions of the output image, we'll make sure to resize it to match your sketch's dimensions. In our case, this translates to a resolution of 6:19, a width of 400 pixels, and a height of 712 pixels.

If the need arises to upscale the image, rest assured that we can easily utilize the Ultimate Upscale SD script for that purpose.

• Sampling Method: DDIM
• Sampling Steps: 30
• Width and Height: Resize to match the size of your sketch image; for us, it's a resolution of 6:19 with a width of 392 and a height of 704.
• CFG: 7
• Seed: -1

ControlNet Unit 0

For the first ControlNet configuration, place your prepared sketch or line art onto the canvas through a simple drag-and-drop action. Next, let's move forward by adjusting the following settings listed down below:

• Enable ControlNet
• Enable Pixel Perfect
• Control Type: Lineart
• Control Weight: 2 (to retain visible lines in the colorization output)
• Control Mode: ControlNet is more important

Note: In place of selecting "lineart" as the control type, you also have the alternative of opting for "Canny" as the control type.

ControlNet Unit 1

For the second ControlNet unit, we'll introduce a colorized image that represents the color palette we intend to apply to our initial sketch art. Below, you'll find the settings required for this process:

• Enable ControlNet
• Enable Pixel Perfect
• Enable Allow Preview
• Control Type: Shuffle
• Control Weight: 1
• Control Mode: Balanced

With all the elements aligned, it's time to click the "Generate" button and witness the transformation of your incredible sketch art into a vibrant masterpiece, adorned with stunning colors.

Once ControlNet has completed the rendering process for your image, you'll have the opportunity to observe the remarkable results it produces. Feel free to explore and experiment with various color palettes to achieve the desired appearance for your sketch or line art.

our sketches have blossomed from mere lines into vibrant tales, all thanks to Stable Diffusion and ControlNet. As you experiment and click "Generate," remember that your artistic exploration has just begun. Unleash your creativity, and let your sketches tell endless stories in vibrant color.

We're excited to have embarked on this journey with you, and we eagerly anticipate welcoming you to our next tutorial. Until then, keep exploring, keep creating, and keep infusing life into your art. Your creative potential knows no bounds.