Well, it depends how much do you like the rest of your hardware?
I dont think less voltage could really break things, but if thats already happening what else could happen.
Since this should NOT really happen to be honest. Most good PSU's sense that the voltage is WAY to low and shut down the PSU completely.
So yes getting a new one fast is the best to keep the PC running. (Even if its just to save the other hardware from breaking).