Fast Video Generation with AnimateDiff & LCM LoRA's (A1111)

In the ever-evolving realm of video production, efficiency and quality are paramount. If you have been following the advancements in animation technology, you might be aware of the powerful tools available within Stable Diffusion. One such tool is the Animate Diff extension, which, when combined with LCM LoRA's, can drastically speed up your video animation rendering process. This blog post aims to guide you through the essential steps to leverage these tools effectively, enhancing both the speed and quality of your video content.

Before we can speed up the process of creating videos or GIFs using LCM LoRA, there are a few things we need to have in place. Check out the list below to make sure we meet the necessary requirements.

Requirement 1: AnimateDiff Extension

• Go to the "Extensions" tab located within Stable Diffusion.
• Select the "Available" subtab.
• Click the "Load from" button.
• Look for "AnimateDiff" and proceed to click on the "Install" option.

To maximize the benefits of the AnimateDiff Extension, acquire a Motion module by downloading it from the Hugging Face website. Visit the official Hugging Face website and find the AnimateDiff Motion files.

• Download either the "mm_sd_v14.ckpt", "mm_sd_v15.ckpt", "mm_sd_v15_v2.ckpt" ore the latest version 3 file labeled as: "v3_sd15_mm.ckpt".
• Once you have the Motion Module, relocate the file to the designated directory structure: "stable-diffusion-webui" > "extension" > "sd-web-ui-animatediff" > "model."

Once the extension and motion module installation is finished, navigate to the "Installed" tab, and select "Apply and restart UI." Additionally, I highly recommend performing a complete restart of your Stable Diffusion for a seamless experience and to address any potential errors.

Requirement 2: Latent Consistency Model (LCM) LoRA

For faster video generation in Stable Diffusion, a single LCM LoRA is sufficient and can be integrated directly into the prompt, similar to regular LoRAs. Explore our comprehensive guide on harnessing the power of LCM LoRA's for accelerated image generation.

Downloading LCM LoRA

Depending on whether you are using SDXL or SD 1.5, various LCM LoRAs are available for download. Below, we will provide a list of where you can obtain the "pytorch_lora_weight.safetensors" file from either of these pages.

• SDXL LCM LoRa: LCM SDXL LoRA
• SD 1.5 LCM LoRa: LCM SD 1.5 LoRa

LCM LoRa File Names & Location

Just like with standard LoRA's, move the downloaded file to the LoRa models folder, situated at: "stable-diffusion-webui\models\Lora"

I recommend customizing the name of the LCM LoRA to align with your version. For instance, following the download of the SD 1.5 LCM LoRA mentioned earlier, I modified the file name to "LCM_SD1.5.safetensors." Similarly, for the SDXL LCM LoRA, I opted for "LCM_SDXL.safetensors.

Once all the installations are configured, we can move forward with configuring the AnimateDiff settings. We'll utilize AnimateDiff to craft captivating animations, available in either GIF or MP4 format. Let's launch the Stable Diffusion UI and go to the "txt2img" tab. Scroll down to find the AnimateDiff dropdown menu, where we'll proceed to configure the settings as follows:

• Motion module: mm_sd_v15_v2.ckpt (The latest Motion Module avaiable at the moment)
• Save format: MP4 (Moreover, you have the option to save it as a GIF as well)
• "Enable" AnimateDiff
• Number of frames: 16 (The SD1.5 motion modules are trained using 16 frames, ensuring optimal results when the number of frames is set to 16)
• FPS (Frame Per Second): 8

The remaining settings can remain in their default state. However, if you desire more information about the AnimateDiff extension, you can visit the AnimateDiff GitHub page.

If all the configurations are properly set within the AnimateDiff Menu, we can proceed by scrolling to the top. Initially, we will select our checkpoint. In our scenario, as we aim for a realistic video animation, we will choose the "Realistic Vision" checkpoint.

Txt2img Prompt Settings (LCM LoRa)

Subsequently, we will incorporate the downloaded LCM LoRa file. In our instance, we are using SD 1.5 and will include our LCM LoRa within the positive prompt.

• Navigate to the Lora subtab within the "txt2img" tab.
• Select the correct LCM LoRa from the available Lora options.
• Provide a positive prompt, incorporating the included LCM LoRA, and provide a detailed description of your desired final video animation.
• Create a negative prompt

### Txt2Img Settings

Next, we'll progress to the global settings within the "txt2img" tab, playing a pivotal role in improving rendering speed when employing with the LCM LoRA, Here are the settings to enhance the speed of your video generation:

- Sampling Method: LCM (If you cant select the LCM sampling method, go to extensions tab, then click on "Check for Updates" and Apply and restart UI")

• Sampling Steps: 8 (Set between 2-8, Sampling Steps of 8 yields the highest quality results).
• CFG Scale: 2 (Choose between 1-2, CFG Scale of 2 will include the negative prompt and CFG Scale of 1 will exclude the negative prompt).
• Width & Height: Select your preferred ratio; in our instance, we'll generate a video with a 9:16 ratio.

Having set all the precise configurations, proceed by scrolling to the top and click on the crucial "Generate" button. This action will enable you to craft remarkable videos or GIFs, depending on the settings you specified in the AnimateDiff option menu.

In conclusion, the integration of Animate Diff and LCM LoRA's within Stable Diffusion introduces a transformative shift in video animation rendering. By following the outlined steps and optimizing your settings, you can achieve remarkable efficiency and quality in your video production efforts. The synergy of these tools empowers creators to move beyond the constraints of traditional workflows, enabling them to explore new creative horizons. As you embark on your journey with these powerful extensions, anticipate a faster, more fluid animation process that supports your artistic vision and elevates your content. Embrace the technology at your fingertips, and let your animations captivate audiences like never before.