How to Upscale any Video with AI for Free (Stable Diffusion)

Do you want to take your videos to the next level without a hefty price tag? Desiring crisper visuals and smoother motion? You're in luck! This step-by-step guide will show you how to upscale your videos, amping up the frames per second for that buttery-smooth result. The best part? It won't cost you a penny, and you don’t need a NASA computer to achieve it. So, let's dive in and transform your videos effortlessly, all for free!

To upscale and increase the video's FPS we will use Stable Diffusion and Flowframes. These 2 programs are free to use and super easy to set up.

• To start things off we will need to install Stable Diffusion, if you don’t have this already we have a step-by-step guide on how to install Stable Diffusion on Windows with automatic updates.
• We will also utilize the NextView extension for Stable Diffusion, this extension helps us to transform any video into a Sequence of images and vice versa. Here is a detailed video on how to install the NextView extension.
• Next, we will need Flowframes (optional), a program for rapid video interpolation on any GPU. This means it can generate additional frames between existing frames, effectively increasing the video's frame rate and making the video appear smoother and more fluid. More information on how to install Flowframes can be found later.
• Lastly, we of course will need an initial video to enhance; it can be any video, whether one you filmed yourself or one created with AI. The choice is yours, as the techniques we'll explore work seamlessly with all types of content.

To start off our upscaling journey we will first need to transform our video into a sequence of images, for this we will use the NextView extension. Simply drag and drop your video into the “Video 2 Image Sequence” section and press “Generate Image Sequence”.

When your video has been processed you will find the Image Sequence Location at the bottom. Copy this location by clicking the copy button and then open the folder by pressing on the folder icon.

Before we start upscaling entire video, it’s crucial that we do some testing first. If you don’t, you might end up wasting a lot of time and have to restart.

1. Head to the “Extras” tab inside your A1111 interface, and on the “Single Image” tab upload any frame from the folder we just opened with the NextView extension.
2. Next, set "Resize" to your preferred value, I recommend starting with resizing 2 times.
3. For "Upscaler 1" we have a few different options. This is the model that will be used by the AI upscaler. Stable Diffusion comes with a lot of default models, I will list the best options for your use case here:

• Realism: R-ESRGAN-4x+ or LDSR (Slower)
• Paintings: ESRGAN_4x
• Anime: R-ESRGAN 4x+ Anime6B

I will select R-ESRGAN-4x+ since my video is realistic. (Alternatively you can download other upscale models from OpenModelDB, but I won’t go into detail on that for now.)

If your video features faces in it I highly recommend enabling CodeFormer, this is a tool for face restoration that offers a trade-off between quality and fidelity and works literal wonders when it comes to upscaling. This is also what we want to test on our single frame. Set "CodeFormer visibility" to 0,75 and "CodeFormer weight" to 0,25. Leave the rest of the settings untouched and press Generate! If you sense that the face stands out in terms of quality compared to the rest of the image, fine-tune visibility and weight settings to achieve your desired result. Keep in mind that the CodeFormer weight is reversed (0 = maximum effect, 1 = minimum effect).

More information on the other settings can be found in the FAQ.

Once you’re satisfied with the outcome of your single frame, we can begin by upscaling the entire image sequence.

• To do this, start by navigating to the “Batch from Directory” tab under the “Extras” tab.
• For the “Input directory” we will paste our Image Sequence Location that we copied earlier.
• For the “Output directory” I recommend you create an empty folder anywhere on your pc, I will create mine in the extras-images folder so I can locate it easily.
• Copy the path to this folder and paste it in the Output directory. S:\stable-diffusion\stable-diffusion-webui\outputs\extras-images\Beach_Girl_Upscaled
• The settings that were last used will be copied over so we don’t need to adjust those.

Now you’re all set to Generate, this might take a while depending on the amount of frames and the speed of your GPU. So while you wait, go grab a cup of coffee. Speaking of coffee, did you know you can support us by buying us a coffee? Hosting a website isn’t cheap and we rely on your support to keep things running, you can also support us by subscribing to the Next Diffusion YouTube channel!

When the upscaling of your image sequence is complete, we can start the process of merging the individual frames back into a video. For this we will use the NextView extension once again. Head to the “NextView” tab, but this time we will use the right side of the extension; Image Sequence 2 Video. Paste the folder location with your upscaled images in the "Image Sequence Location". Next, set the Frames Per Second to match your original video, this can be found in the “Result Information” or by going to the properties of your original video and going to “Details”. Now press “Generate Video”. Congratulations! You’ve successfully upscaled your video.

After your video has been generated you can click on the folder icon to quickly locate your video.

To even further enhance our video we will boost the frames per second of our video with interpolation. To do this we will use a free program called Flowframes. You can download Flowframes for free on the official itch.io website. Once you’ve downloaded the installer, open it and select all the default options for the installation. Default > Auto-Detect > Auto-Detect > Auto-Detect > Don’t pre-install AI models > Next

After the installation has finished go to the Interpolation tab.

• For Interpolation AI select RIFE, if you have an Nvidia GPU select the Nvidia only version.
• For your Input Video, click "Browse (Video)" and choose your upscaled video.
• Select a folder for the Output Directory, then paste the path here or leave it at the default path.
• When configuring Output FPS and Speed, I recommend to go with x2 or x3 and select Normal Speed.

Leave the rest of the settings at their default values and press “Interpolate!” At the bottom left corner.

The first time interpolating your video will take a bit longer than usual, this is because Flowframes has to download the Interpolation AI first.

Take a look at the noticeable improvements in your video. The increased frames per second and enhanced details are clearly visible. See for yourself how these upgrades make a significant difference in the video quality.

In conclusion, the journey from a standard video to an upscaled masterpiece has never been more accessible. With Stable Diffusion, Flowframes, and our step-by-step guide, you've harnessed the power of AI to transform your videos, all for free. Your visuals are crisper, motion smoother, and overall quality greatly improved. The results are impressive and within reach for everyone, without the need for a high-end computer. So, go ahead and share your enhanced content with the world, knowing that you've unlocked the potential of upscaling with ease.