Its not that you cant game on then, but as I'm sure you know Xeon are workstation/server CPU's. This means they typically have higher core count at lower clock speeds, which is great for applications that can utilize a large number of cores.
However games typically don't, even in 2021 a 6 core is all you really need, the core clock of the CPU matters more when comparing your 4770k and e5-2623 for gaming, the max clock of the Xeon is 3.5 ghz, while I'm unsure of your cooling setup its pretty easy to achieve a 4-4.2ghz clock sustained for gaming on a 4770k, which would likely explain your 15% delta.