Weird FPS drop in all games

[LethoX]

I built my own PC for the first time and now I'm getting these weird FPS drops. Like for example in LoL, sometimes when I click the minimap it takes a second before it actually reacts. In CSGO sometimes when I shoot it also takes a second before reacting. It's not always when I'm like clicking, it happens randomly it seems like. I have asked in another forum for help, but I've gotten no results so far. So I thought I would try a few more forums to see if there's something I haven't tried yet. Here's some things I've changed / tried to do but with no success: Swapped SSD for another one (new) | Swapped RAM with my brother(I was still getting those weird drops and he wasn't) | Swapped graphic card | Updated BIOS | Updated drivers | Did a memtest with my RAM that showed nothing was out of order. Off the top of my head that's everything I believe. Any help is much appreciated as I've been trying to fix this for a couple of weeks now. Specs are following:

PSU: Corsair RM550x

CPU: I5-10400f

SSD: Samsung 970 EVO

Motherboard: ASUS TUF GAMING B460-PLUS

RAM: Corsair Vengeance LPX DDR4-3200 C16 / 2x8 GB

GPU: EVGA GTX 1080 SC