Low gpu usage and gpu usage drops

[Moritschac]

Specs: gtx 1060/ryzen 1600 
In most games (for example Rust/Fallout 4) my gpu rarely ever goes over 50% gpu usage resulting in pretty bad framerates. On top Im also getting fps drops caused by gpu usage dropping for a split second from for example 40% to 15% dropping my fps hard. How is this possible? Gpu isn't overheating http://imgur.com/a/dOXgu (right percentage is gpu usage/left is power consumption) No components have been overclocked (gpu factory oc) and Im using the latest Nvidia drivers
Playing Fallout 4 I also noticed that looking at certain things/areas dropped my gpu usage hard to like 30 both, turning around looking at a wall or something my gpu usage actually goes up.