Yeah you are going to blow that speaker to pieces. Most of those bluetooth spearkers are meant for having a source coming from a phone. Those jacks are good for ±3V at just a few miniamps.
EDIT: Found some more info. Some cases they are good for ±1.4-2v.
EDIT2: Found more info: https://source.android.com/devices/accessories/headset/jack-headset-spec
Most speakers are going to be around that if they are bluetooth. So, it would be safe to say 20v source is going to turn things into a small mushroom cloud.