As already stated, you need some sort of splitter/switch.
Most people go with the AVR-solution, that is buying a home theatre receiver which handles audio and the HDMI switching that you want. On the plus-side you can connect a couple of speakers to it and not have to listen to the microspeakers in your TV. With a reciver you just connect all your peripherals by HDMI, and then run one single HDMI to your TV. If you've got HDMI ARC (Audio return channel) in your TV, audio will also travel downstream from your TV to the receiver. This is nice if your for example use your TV's built in Netflix App or something similar.
Example of a home theater receiver: http://www.bestbuy.com/site/denon-avr-5-2-ch-4k-ultra-hd-and-3d-pass-through-a-v-home-theater-receiver-black/5756606.p?skuId=5756606
And some speakers: http://www.bestbuy.com/site/definitive-technology-procinema-600-5-1-channel-home-theater-speaker-system-black/8206201.p?skuId=8206201