amplifier hiss

[Laniakea]

I have a class ab amplifier circuit that I salvaged from a HP sound bar that had headphone jacks and I made it into a pocket USB amplifier. Now it is hard for me to always find an USB power source to be able to use it everywhere so I put two 1.5 volt batteries and a votage amplifier (MT3608) in it and directly connected it to the usb power input of the amp, controlled with a switch. I configured it to 4.93 volts but the amplifier still works with voltages down to  4.5 that I've tried so far. When I plug the amp to my laptop there's so little hissing that I can ignore it. normally I plug it to phone wall charging adaptors but the hiss just kills my ears when I do it. My main question is there's way more hiss with the battery circuit that I've built inside it and it's unpleasent to use, I know it's from the power source, so how can I prevent it? Maybe with a capacitor I thought but my electronics knowledge ends there and I just dunno how. And one more question, what will happen if I accidentally turn on the switch of the battery circuit when I have the amp connected to laptop's usb port and it's powering from there? 

[FAZIN]

It depends whether its is a hissing sound or a buzzing sound. A hissing sound could indicate noise on your audio line. This could be helped with the right capacitor across your audio signal. A buzzing sound could indicate a ground loop somewhere in your circuit.

 

If you were to close the battery switch while connected to your laptop the voltage amp may sink current and be damaged.

[LogicalDrm]

-> Moved to Troubleshooting

[Laniakea]

Can anyone offer any new information? I'm kind of stuck.

 

And one more question that I'm having that might let me switch between power sources but I need to connect both grounds from the two power sources for this and change the 5V inputs, would this work? I'm planning on connecting the negative of the batteries with the usb ground and switching between the amp's 5V output and usb's 5V. So something like this: