If you're just starting out, Ubuntu is probably your best bet. There's even an option called WUBI to install it through windows, allowing you to uninstall it through windows at any time.
EDIT: I looked into it, WUBI is difficult to find. The official guide recommends you to get it from here. The guide for it can be found here and probably isn't too bad of a read.