joem2201plays

I play both of these games with an older i5 and a 970 just fine...?
 
(btw brick rigs and BeamNG are probably the best games ever)

There were firmware problems with this card originally.  Your tech is good so as long as your coolers are free of dust and there arn't any funny programs grabbing at the card Id say update the firmware and tweek a little so see if you can get the two to work together better.

GPU usage probably goes up because the CPU bottleneck is removed and the frame rate increases.

While true, OP asked if he should replace his iPhone 5 with a 4 simply because the iPhone 4 had a better DAC and AMP compared to the 5. 
 
The answer is simply no. 

The iPhone 4 would still serve as a decent enough iPod. No need for multi-core CPUs for music playback.

From my experience with my Audio Technica ATH-M50X's, they are amazing for professional use, and they are also great for music and gaming, but I have found that the highs in those headphones are really high, and almost piercing depending on how high you go. I am really happy with them though, so yeah, I think they can be used as daily drivers, I guess it just depends on portability as well.
Hope that helps

They can be used for sure. I’m not sure what specific headphones you are talking about, but here’s the logic on both sides. Monitoring headphones are supposed to be very flat with a narrow sound stage, the people who say monitoring headphones are bad for music say this because extremely flat headphones are often boring and most people won’t enjoy a perfectly flat headphone for music. The people who say they are good for music are people who want the music as close to the original source as possible. 
 
What particular headphones are you talking about.