DreamBooth: Easy AI Model Training in Stable Diffusion (A1111)

DreamBooth is an incredible tool that allows you to train your own AI models effortlessly. In this step-by-step guide, we will explore the process of setting up DreamBooth and training your first stable diffusion model. You can either train a person or an object/style. Get ready to dive into the world of AI-powered creativity!

Before we can start training our AI models, we need to set up DreamBooth correctly. Here are the essential steps:

Installing DreamBooth Extension:

• Navigate to the "Extension" tab.
• Select "Available" and click on the "Load from:" option.
• Search for "DreamBooth" and click on "Install".

This will start the installation process. Please be patient as it may take some time. If everything is installed you should see the new "sd_dreambooth_extension" folder inside the "stable-diffusion\stable-diffusion-webui\extension" folder:

Restarting Stable Diffusion:

After completing the installation, close the command window. Then, double-click on the webui-user.bat file once more to restart Stable Diffusion. This will ensure that the new DreamBooth extensions tab becomes visible within the stable diffusion user interface, confirming that all the changes have been applied.

Now that we have set up DreamBooth, let's dive into training your first stable diffusion model. Follow these steps to kickstart the training process:

Creating a DreamBooth Model:

In the DreamBooth interface, navigate to the "Model" section and select the "Create" tab. Choose a descriptive "Name" for your model and select the source checkpoint. For stable diffusion models, it is recommended to use version 1.5 pruned EMA. You can also experiment with other versions if you prefer.

Proceed by clicking on the "Create model" button. Please note that this process may require some time, and you can monitor the progress in the "Output" section. Once this is finished you can see your model when you navigate to the "Model" section and select the "Select" tab. In this tutorial i called the model: "FirstDreamBooth".

Configuring DreamBooth Training Parameters:

To optimize your training, it's essential to configure the training parameters correctly. To change some settings, navigate to the "Input" section and select the "Settings" tab. Now open up the "Basic" expansion panel to adjust some settings:

In the "Basic" expansion panel look for the "Intervals" section where we change the following:

• Change the Training Steps Per Image (Epochs) to 150 (This is the total number of training steps that will be performed on each instance image).
• The rest of the "Interval" parameters can be left at 0

Scroll down to the "Tuning" section and change the following:

• Set the "Optimizer" to "8bit AdamW"
• Set the "Mixed Precision" to "fp16"
• Set "Memory Attention" to "xformers"

The "Basics" Settings are now configured properly. Just to be sure you can click on "Save Settings" at the top of the page before we move on.

Setting Image Concepts and Prompts:

Next we need to bring in our actual concepts (our images). The image concepts and prompts play a crucial role in shaping the training process. So go to the "Input" section and click on the "Concepts" tab.

Specify the "Dataset Directory" for the input images you want to train on. The images should be in PNG format, and the dimensions should be 512 x 512 pixels. I have prepared a dataset of the super Mario game character:

Now you can copy the address where your data set is located. This can be done by right-clicking on the file location and click "Copy address".

Now paste the address link in the "Dataset Directory". This way DreamBooth can train the model based on your dataset:

Now scroll down to the "Training Prompts" section. Here you can specify an "Instance Prompt" which describes the dataset. My prompt is: "a photo of mariodream character". Make sure that if you describe your dataset, to use a word that doesn't exists yet. I choose for the word "mariodream". This word can be seen as a token which can eventually be used as a prompt word when testing our model in the "txt2img" tab later on.

The last setting will be in the "Saving" tab. Here we will check the "Half Model" checkbox under the "Checkpoints" section. This will generate a model with fp16 precision and will result in a smaller checkpoint with minimal loss in quality.

Starting the Training:

With all the parameters set, it's time to initiate the training process. First click "Save Settings" and after that click on the "Train" button. DreamBooth will start training your model. Depending on your GPU and settings, the training process can take some time. Sit back, relax, and let DreamBooth work its magic!

Once the training is complete, it's time to utilize the trained model and explore its capabilities. To make sure the model has been properly trained, you can check if there is a model file inside the "stable-diffusion\stable-diffusion-webui\models\Stable-diffusion" folder.

You should see a .safetensors file in the newly created folder.

Now it is time to test our model! For this we will use the "txt2img" tab. Here are the steps to test the created DreamBooth model:

• Select your newly created Checkpoint
• Apply a prompt and make sure you use the chosen token word when you created the model. I chose the word "mariodream", so that is the word i have to implement inside the prompt for the model to work.
• Click "Generate" to let the magic happen

Example 1:

Example 2:

DreamBooth opens up a world of possibilities for AI model training and creativity. In this guide, we have explored the process of setting up DreamBooth, training your first stable effusion model, and utilizing the trained model to generate stunning outputs. Unleash your creativity and enjoy the exciting journey of AI-driven artistry with DreamBooth!