Yes, you buy bitcoins from stock exchanges.
I personally use Coinbase, which is fairly good. You can also use it as your wallet if you aren't going to store huge amounts of bitcoins or make lots of transactions.
If your goal is to pay for stuff using bitcoin, I wouldn't recommend storing a lot of it in your wallet, since its value fluctuates (by a lot, sometimes), and you might find that your $300 worth of bitcoin are suddenly only worth $250. Though it could also be the other way around.