RobRdz

You should also consider the 1660 Super, as it gets a nice boost (sometimes really noticeable) compared to the regular 1660 with its GDDR6 memory. That being said, the 1660 (non-Super) will be on par or barely faster than the 580 in almost all cases, so unless you find a 580 for way cheaper, or really don't care at all about Nvidia Shadowplay, I'd suggest going with the 1660.
 
When it comes to bottlenecking, especially in recent CPU intensive titles, your i5 will bottleneck either of the two GPUs you've mentioned by a fair bit, especially if you're running at something like 1080p. But again, how much your GPU would be bottlenecked will vary on a game by game basis.

i5 3rd gen + RX 580 won't be too bad.
Depends on the game we're talking about, but most will run fine.

You have an Intel system. You will need a new motherboard and RAM to use a Ryzen CPU. The GPU may work, but you may have power and thermal issues in a prebuilt like that. 

The motherboard on its own should support the RX 5700, because it's still just PCIE. A newer version.. But PCIE is back/forwards compatible.
It won't work with the CPU, since it's an entirely different socket. That CPU will also need DDR4, not DDR3 like you have not.
 
At that point you're replacing half your PC (only left with case, PSU and drives), so I might just suggest to build a new tower in general. Usually the prebuilts have weird internal design too..