I think I know this one, try disable AMD's ULPS. https://www.sevenforums.com/tutorials/316913-ulps-ultra-low-power-state-disable-amd-crossfirex.html
The tool they included was down forever but you can edit the registry manually. Just search ULPS and set every entry you found to 0.
MSI Afterburner have option to disable it as well iirc.