How to Master Lighting Control in Stable Diffusion Photography

Lighting plays a pivotal role in photography, influencing not only the visibility of subjects but also the emotional tone and overall quality of images. In the realm of digital photography, especially in technologies like Stable Diffusion, the ability to manipulate lighting can dramatically enhance the visual storytelling of your creations. This blog post aims to delve into three powerful techniques that will empower you to master lighting control in your Stable Diffusion photography: the use of lighting keywords, regional prompting, and ControlNet for advanced lighting manipulation.

Lighting is a fundamental aspect of photography, capable of transforming an image's quality, mood, and impact. Understanding how to effectively control lighting is essential for photographers seeking to create visually stunning and captivating compositions.

Adding Keywords to the Prompt:

Keywords serve as the key to unlocking desired lighting effects in Stable Diffusion photography. Learn how to strategically incorporate specific keywords into your prompts to control lighting and enhance the overall composition. We will guide you through the process of selecting and utilizing the right keywords to achieve various lighting effects, providing a solid foundation for your creative endeavors.

Let's begin and generate some images without any lighting-related keywords, to observe the impact that incorporating such keywords can have. You can see the starting image below:

Example 1 (No lighting keywords):

The above image were generated by using these settings:

• Positive Prompt: best quality, ultra high res, (photorealistic:1.4), 1girl, (asian girl:1.3), off-shoulder black shirt, skirt, black choker, (full body portrait:1.3), (faded ash gray hair:1.4), slim, (medium breasts:1), looking at viewer, outside, (cleavage:1.2)
• Negative Prompt: (worst quality:1.2), (low quality:1.2), (lowres:1.1), (monochrome:1.1), (greyscale), multiple views, comic, sketch, (((bad anatomy))), (((deformed))), (((disfigured))), watermark, multiple_views, mutation hands, mutation fingers, extra fingers, missing fingers, watermark,
• Sampling Steps: 30
• Sampler: DDIM
• CFG scale: 8
• Size: 768x432
• Model: ICBINP
• Clip Skip: 2

Now let's explore some examples where we will apply some lightning keywords.

Example of Different Lighting Effects:

Discover a plethora of lighting effects that can be achieved by using the right combination of keywords. Let's explore some examples:

• Volumetric lighting, Atmospheric illumination, God rays: These lighting keywords refer to the presence of distinct light beams within an image, employed in photography to enhance the perception of depth and volume. When adding these keywords to your prompt you will get the following effect:

Witness the transformative power of the lighting keywords and gain inspiration for incorporating them into your own photographic projects.

Tips for Using Lighting Keywords:

Unlock the full potential of lighting keywords with our expert tips and recommendations:

• Give greater importance to the keyword if the desired effect is not evident. For example use (god rays:1.5) instead of just "god rays".
• Note that not all lighting keywords yield consistent results; try generating a few images simultaneously for testing purposes.

While lighting keywords can affect the entire image, sometimes it is necessary to control lighting in specific regions. This can be achieved with the Regional Prompter extension. This tool allows users to designate specific areas of the image for varying lighting effects, providing a more tailored approach to illumination.

Steps to Use Regional Prompting:

1. Install the Regional Prompter extension.
2. Access the txt2img page and unfold the Regional Prompter section.
3. Activate the Regional Prompter and select the following options:
   • Active: Yes
   • Use common prompt: Yes
   • Split mode: Vertical
   • Divide Ratio: 1:1
4. Create prompts for different regions using the keyword BREAK to separate them.
5. Experiment with adjusting the intensities of lighting keywords for each region.

Lets explain the following prompt:

We have three prompts above seperated by a BREAK

• Common prompt = Portrait image of an asian women
• Prompt for region 0 = (god rays:1.2), (volumetric lighting:1.2)
• Prompt for region 1 = (dimly lit:1.4)

Example Output & Explanation:

With the Regional Prompter extension, you gain the ability to create distinct visual and lighting effects in different regions of your image. This extension empowers you to customize and control the lighting composition, allowing for unique and captivating visuals throughout various areas of your photograph.

Tips for Regional Prompting

• Adjust keyword weights for desired effects. Instead of just adding "dimly lit" you can add (dimly lit: 1.4).
• Regional prompting may require multiple iterations for optimal results.

ControlNet offers a precise method for controlling illumination in your photography. To utilize this advanced technique, you will need to have the ControlNet extension installed. If you haven't installed it yet, follow the provided tutorial for step-by-step instructions on installing the ControlNet extension.

Once installed, you will unlock the ability to achieve precise control over the lighting in your images, allowing you to create stunning and visually captivating compositions.

ControlNet Settings (Img2Img):

1. On the img2img page, navigate to the ControlNet section.
2. Upload an Portrait or Photography to ControlNet Unit 0.
3. Use the following ControlNet settings:

• Enable: Yes
• Pixel Perfect: Yes
• Allow preview: Yes
• Control Type: Depth
• Preprocessor: depth_zoe
• Model: control_depth-fp16
• Control Weight: 0.6

The uploaded portrait image was created with txt2img:

- Positive Prompt: best quality, ultra high res, (photorealistic:1.4), 1girl, (asian girl:1.3), off-shoulder black shirt, skirt, black choker, (full body portrait:1.3), (faded ash gray hair:1.4), slim, (medium breasts:1), looking at viewer, inside, (cleavage:1.2)

• Negative Prompt: (worst quality:1.2), (low quality:1.2), (lowres:1.1), (monochrome:1.1), (greyscale), multiple views, comic, sketch, (((bad anatomy))), (((deformed))), (((disfigured))), watermark, multiple_views, mutation hands, mutation fingers, extra fingers, missing fingers, watermark,
• Sampling Steps: 40
• Sampler: DDIM
• CFG scale: 8
• Size: 768x432
• Model: ICBINP
• Clip Skip: 2

After generating an image with txt2img you can send it to img2img so that the prompt and dimensions will be replicated inside the img2img tab:

4. img2img canvas settings

• Upload a lightning image to the img2imag canvas (so replace the original image which appeared when you clicked the "send to img2img" button from the txt2img tab).

You can find these images just by googling something like: "spotlight" or "god rays".

• Set the resize mode to "Just Resize".

• Set denoising strength to 0.95.

- Press the "Generate" button.

You should get images with the light source behind the girl. See examples below:

In summary, mastering the art of lighting control in Stable Diffusion photography is a transformative skill that can elevate your images significantly. By employing the techniques discussed—using lighting keywords, harnessing the Regional Prompter extension, and utilizing the advanced capabilities of ControlNet—photographers can gain enhanced control over their visual narratives. These methods not only empower your creative expression but also enable the creation of compelling and emotionally resonant images that can captivate any audience.