How to Color Your Sketch Art with Stable Diffusion (A1111)

Welcome, fellow artists and creative souls!

Are you ready to bring your sketches to life with a burst of color? If you've been looking for a fun and easy way to transform your black-and-white sketches into vibrant, eye-catching masterpieces, then you're in the right place! Today, I'm thrilled to introduce you to the magic of Stable Diffusion (A1111). This amazing tool has completely revolutionized the way I color my sketches, and I can't wait to share all the tips and tricks I've learned with you.

Grab your favorite sketch, fire up Stable Diffusion (A1111), and let's dive into the colorful world of digital art together. Trust me, by the end of this guide, you'll be amazed at how effortlessly you can turn your sketches into stunning works of art. Let's get started and unleash our creativity!

To jumpstart our colorization process for sketch art, you'll need a sketch ready to go. In our last post, we delved into the thrilling process of turning an existing image into breathtaking sketch art! If you missed it, no worries—you can catch up by following the link below.

Transform Your Images into Sketch Art with Stable Diffusion (A1111)

With your initial sketch art piece ready, let's dive into the requirements for colorization with stable diffusion!

To start adding appealing colors to your lovely sketches or line drawings, it's important to set up ControlNet correctly with the pre-trained models, Lineart, and Shuffle.

If you haven't installed ControlNet yet, don't worry! We've put together a detailed guide that provides clear instructions on how to get it up and running. You can find the tutorial down below:

How to Install ControlNet Extension in Stable Diffusion (A1111)

Once you’ve installed ControlNet, it’s time to download the pre-trained models for "Lineart" and "Shuffle." Head over to the pre-trained models page on HuggingFace and download these specific models:

• control_v11e_sd15_shuffle_fp16.safetensors
• control_v11p_sd15_lineart_fp16.safetensors

Next, place these files in the following directory within the ControlNet extension: "extensions/sd-web-ui/ControlNet/models" inside your Stable Diffusion folder. Let's get those models set up so we can dive into the fun part – bringing your sketches to life with vibrant colors!

Open Stable Diffusion and navigate to the txt2img tab. First, scroll down to the ControlNet panel. In the first ControlNet panel (0), paste your initial sketch art and apply the following settings:

Controlnet Unit 0 (Lineart)

• Select "Enable" checkbox
• Select "Pixel Perfect" checkbox
• Control Type: Lineart
• Preprocessor: lineart_standard (from white bg & black line)
• Model: control_v11p_sd15_lineart_fp16.safetensors
• Control Weight: 1 (default)
• Control Mode: Balanced (default)

Controlnet Unit 1 (Shuffle)

Next, we'll open another ControlNet unit (1) and upload a color palette image to the canvas, using the settings listed below:

• Select "Enable" checkbox
• Select "Pixel Perfect" checkbox
• Control Type: Shuffle
• Preprocessor: shuffle
• Model: control_v11e_sd15_shuffle [526bfdae]
• Control Weight: 0.5 (default)
• Control Mode: Balanced (default)

Note: If the colors are too bright in the final rendered image, try increasing the control weight.

Scroll to the top, where we'll select the checkpoint for our image inference. We'll choose a checkpoint named "Photon." We'll leave the prompt empty, but feel free to add some keywords to highlight specific colors if you'd like! Since we've already provided a color palette image within the Shuffle ControlNet panel, leaving it empty works perfectly.

Below, you’ll find the basic settings we need to tweak before we click the magical "Generate" button and watch our black and white sketch burst into vibrant colors!

• Checkpoint: Photon
• Sampling Method: Euler a
• Sampling Steps: 30
• Width & Height: 888 x 496 (16:9 ratio, this should be the same as our intial sketch ratio)
• CFG Scale: 7
• Seed: -1

Once all the settings are configured, we can click on the "Generate" button!

After the render completes, we've transformed our initial sketch art into a vibrant colorized piece using Stable Diffusion and ControlNet (LineArt & Shuffle). Below, you can see a collage featuring a variety of colorized sketches, each using different color palettes with the Shuffle ControlNet panel.

Congratulations! You've now mastered the art of effortlessly coloring your sketch art using Stable Diffusion (A1111). By following this step-by-step guide, you've learned how to transform your black-and-white sketches into vibrant, eye-catching artworks. With the power of ControlNet (LineArt & Shuffle), you can now explore endless possibilities in digital artistry, experimenting with different color palettes and settings to bring your creative visions to life.