Game mode is a good feature (in theory) especially for people with lower end systems. Microsoft's marketing has been a little bit misleading, and they havent quite got it right just yet. But it is going to improve over time, I am sure. Not only are system recources dedicated and favoured towards any running game. In certain situations, code can be processed without having to go through the normal abstraction layers which naturally take up recourses. But like people have already said, you will only see the benefit of this on lower end systems. Microsoft also have the advantage that they can manipulate parts of the operating system that third party 'boost' software can't access.
Love or hate Microsoft and Windows, there is one thing that cannot be argued. That they seriously know what they are doing when it comes to programming software. The Windows NT kernal that powers both Windows and the Xbox, has grown in functunality and features over the years, while reducing the amount of processing power that it needs to run.
Windows runs on a million different combinations of hardware, that flexibility comes at the price of performance. They do not have that issue with a console as they know the specs of every single Xbox being sold. So software can be wrote with that in mind, giving extra performance. Game mode is partly about giving the performance benefits of a single spec'd machine, on a Platform that supports millions of combinations, They will never get it down to the level on consoles and they havent got it quite right at the moment. But it will get better. This is a good thing though, as it is going to bring more gamers to PC.
Try and run a given game on a PC with the same spec as an Xbox one. The performance will be way lower on the PC than the Xbox, because of the streamlined paths consoles can use.