Are you sure you can overclock this one? Almost none of the Pentiums except the G3258 are overclockable. And you could overclock your CPU if you had one with the stock cooler, but heat output would rise because, it's stock, obviously.
You can't overclock that CPU.
Back in the day you didn't use the multiplier to overclock. You had a base (usually between 6-11) and then the FSB and base frequency (800 = 200, 1066 = 266, 1333 = 333). Base / FSB was a 1:4 ratio so for 4.5GHz on a chip that came at 1333 FSB with a multi of 9 (so 3GHz stock) you'd go for 2000 FSB (500 on the bus). Just felt like clearing that up as a lot of people aren't aware of this, multiplier OC only became 'relevant' as of Sandybridge (unlocked multipliers were pretty much extreme edition exclusive). So here is a locked Core2Duo E8400 @4.5GHz. We have this to thank for 'extreme oc/gaming' boards becoming a thing because back then it actually mattered when overclocking
Anyway to answer the OPs question you can most likely do this on stock cooling unless your chip is a pig. I got my E2200 from 2.2GHz to 3GHz without any extra voltage. Your chip is on a multiplier of 11 and an FSB of 1066 (266 base speed). Simply go into the BIOS and tweak that 266 to 300. Then reboot into the system and stress test it. Keep an eye on temps too. ~65c is about as high as you want to go.
You're also in luck. E6500s were known to OC quite well. See the average OC on hwbot.
You'll also need to keep an eye on your RAM divider. A 1:1 ratio is the best but not always possible. Try adjusting either RAM frequency or divider (depends how sophisticated your BIOS is) to keep as close to spec as possible (e.g. 667mhz-800).
If you're lazy and want to OC via Windows there's a few things you could use. My nForce 680i SLI has a nVidia tuning tool, ASUS had a thing I believe and there's always the PLL (however that's more fuss than the BIOS).