da_KOWZ

There are a few things to be mindful of.
#1 Temperatures, you don't want your card to get too toasty as that affects a number of things including the stability of your overclock and the health of the PCB components. You'd be surprised to hear this but graphics cards often fail due to one of the PCB components failing rather than the GPU itself.

#2 Voltage, this is important because the more voltage you apply to your graphics cards the hotter it will get. Also there's a safe over-volting limit which differes depending on the manufacturing process used to make the GPU. For 28nm you're pretty much safe all the way to 1.3-1.4v as long as you keep the temperatures in check.

Nvidia GPUs will not allow you to go over 1.225v-1.28v depending on the card, while you can apply any sort of voltage you like to AMD GPUs via a small MSI Afterburner tweak.
I personally wouldn't recommend going substantially above 1.3v as you'll begin to hit diminishing returns.
 
#3 Overall stability, you want to make sure that your card is 100% stable so that you won't suffer from artifacts, crashes, black screens, blue screens or hangs.
What you want to do is to vigorously test each overclocking profile you set up in the games you play, if you notice any of the above or you notice performance degradation you'll either have to pull back on the frequency or apply additional voltage, this will depend on the sort of temperatures you're getting.

I'd personally consider anything below about 80c to be good, below 70c is optimal as the card will actually require slightly less voltage to maintain the overclock at lower temperatures.