Open Source AI Upscaler for VHS footage

[FreakyNobleGas]

Hello! I'm working on a home project to upscale my VHS tapes that were converted to digital using an AI upscaler and I was curious if anyone in the community has tried this with any success?

 

I've tried a few AI upscalers from Github (ex. k4yt3x/video2x), but haven't had much success. Many of these upscalers are created specifically for Anime, so I think the AI models are not trained to handle the visual noise that is common in VHS footage.

 

Feel free to let me know if you have any suggestions! I think I might need to create a custom model for this project.

[Alex Atkin UK]

You're paying for the hours and hours of model training with commercial software so its one of the cases where I do not think you're going to find a good Open Source solution I'm afraid.  I wanted one myself but ended up getting Topaz Video Enhance, though the results from VHS material is by far the worst as there isn't really any detail to work with.

[jaslion]

17 minutes ago, Alex Atkin UK said:

You're paying for the hours and hours of model training with commercial software so its one of the cases where I do not think you're going to find a good Open Source solution I'm afraid.  I wanted one myself but ended up getting Topaz Video Enhance, though the results from VHS material is by far the worst as there isn't really any detail to work with.

If you have a nvidia gpu the nvidia scaler works quite well

[Alex Atkin UK]

2 minutes ago, jaslion said:

If you have a nvidia gpu the nvidia scaler works quite well

That's a good point as it didn't exist when I bought Topaz.  Is there any software that can use that to permanently save the output as its probably better than nothing?

 

That said, anything real-time is still not going to be as good as a slower model.

[jaslion]

1 hour ago, Alex Atkin UK said:

That's a good point as it didn't exist when I bought Topaz.  Is there any software that can use that to permanently save the output as its probably better than nothing?

 

That said, anything real-time is still not going to be as good as a slower model.

Like vlc does it

 

Theres others too but I havent much looked into it

 

 

[Alex Atkin UK]

9 hours ago, jaslion said:

Like vlc does it

 

Theres others too but I havent much looked into it

I always forget VLC has an encode option.

[wanderingfool2]

I've always been curious of this.

 

I mean while I do agree that it likely could require a lot of data...at the same time parts of the VHS quality issues I find are things that might be within the realm of non AI fixing, and then using a video upscaler after that.

[Franck]

22 hours ago, Alex Atkin UK said:

Is there any software that can use that to permanently save the output as its probably better than nothing?

I do similar upscale recording the janky way but it works.

- Activate Nvidia upscaler

- DP out GPU, in capture card

- OBS record from capture card

 

Why I do that is because OBS doesn't record the Nvidia upscaling and i just didn't feel like finding why so I did that 15 min work around and it works perfectly.

[Raytsou]

Since this is the programming section, I'm gonna assume you can code.

OpenCV has an AI upscaling function. Tutorials here: https://docs.opencv.org/4.x/d8/df8/tutorial_table_of_content_dnn_superres.html

[FreakyNobleGas]

Thank you for the insight everyone! Nvidia's super resolution capability is very interesting, but it would be great to have a CLI tool that I can automate and execute in the background.

 

I've been going through various Github repositories on Upscaling and Denoising video. I had some partial success with Nihui's realsr-ncnn-vulkan upscaler. I'll definitely report back here if I find anything interesting!

 

@Raytsou, thanks for the link! I'll check that out. I'm also digging into Intel® Open Image Denoise project.