Nvidia control panel and HDMI / Displayport

[Zeler]

Not sure if this is the right place to post this, if not please move it to the appropriate place.
 
There is an issue I have with nvidia control panel that seems to only happen through HDMI or Displayport. The problem is that it will not display colours correctly using those ports (makes the colours look quite "washed"), the short term soloution is a little "tweaking" in the control panel every time I start the computer (or seems to reset like it does on startup, if a game goes in fullscreen mode).
 
The way of "fixing" it short term is to ajust the hue or digitital vibrance in the nvidia contol panel, then move it back to where it was, but the colours on the screen will look much better than before. I'd love to show a screenshot of the difference, however that is not possible due to the nature of this bug.
I could take a picture of my screen with my shitty phone, if that is needed.
 
From what I know it seems to be because HDMI and Displayport have the ability to transfer 10bit, insted of the usual 8bit, but that is just a guess. I've had this problem since april because that's when I had a screen able to use HDMI or Displayport, however I think this problem have been going on for quite some time (unless I'm doing something very wrong here).
This problem have been sitting through every driver since april. I'm using gigabyte gtx 670.
 
After a long time of searching I have found a program that someone made that solves this at every startup (not sure if it fixes it for the fullscreen thing), however that seems to only work with HDMI and not with Displayport. As then the program seems to somehow disable my keyboard which is very inconvenient as you might imagine.

 

If anyone know a way to fix this permanently it would be greatly appreciated!