The Visual Microphone: Passive Recovery of Sound from Video

[DuckDodgers]

 

 

When sound hits an object, it causes small vibrations of the object’s surface. We show how, using only high-speed video of the object, we can extract those minute vibrations and partially recover the sound that produced them, allowing us to turn everyday objects—a glass of water, a potted plant, a box of tissues, or a bag of chips—into visual microphones. We recover sounds from highspeed footage of a variety of objects with different properties, and use both real and simulated data to examine some of the factors that affect our ability to visually recover sound. We evaluate the quality of recovered sounds using intelligibility and SNR metrics and provide input and recovered audio samples for direct comparison. We also explore how to leverage the rolling shutter in regular consumer cameras to recover audio from standard frame-rate videos, and use the spatial resolution of our method to visualize how sound-related vibrations vary over an object’s surface, which we can use to recover the vibration modes of an object.

 

Project page

[Unclescar]

I find this ridiculously fascinating and neat.

[bob345]

This is awesome. Its a simple concept really, but to see this kind of thing actually working still amazes me. I would think that resolution may also help with the amount of recoverable data too like if you could get a better frequency response with 4k video over 1080p 

[StingBull]

Amazing. The concept makes sense, when you think about it. All that is needed is good software. Maybe in a few years, this can be done on our PCs and even smartphones.

[Thorium19]

Very clever. This could certainly be put to use in a number of areas and possibly save a fair amount of money. I like.

[Entropy]

This form of technology has been used since the Cold War era.

[FlamingDefibs]

Haven't I seen this in some movie where an AI hears what someone is saying from the vibrations from a coffee cup on a table?

[DuckDodgers]

Haven't I seen this in some movie where an AI hears what someone is saying from the vibrations from a coffee cup on a table?

http://www.imdb.com/title/tt1059786/