Neural Networks and Machine Learning are getting better at interpreting what you see.

[WMGroomAK]

In a step forward for the human-machine interface (or in a scary step to losing your privacy) a study has been published on a new approach for a Neural Network using Machine Learning to interpret functional Magnetic Resonance Images (fMRI) of the visual cortex into actual images.  From the MIT Technology Review:

 

https://www.technologyreview.com/s/604332/mind-reading-algorithms-reconstruct-what-youre-seeing-using-brain-scan-data/?set=604340

 

Quote

The difficulty, of course, is finding ways to efficiently process the data from functional magnetic resonance imaging (fMRI) scans. The task is to map the activity in three-dimensional voxels inside the brain to two-dimensional pixels in an image.

 

That turns out to be hard. fMRI scans are famously noisy, and the activity in one voxel is well known to be influenced by activity in other voxels. This kind of correlation is computationally expensive to deal with; indeed, most approaches simply ignore it. And that significantly reduces the quality of the image reconstructions they produce.  

 

So an important goal is to find better ways to crunch the data from fMRI scans and so produce more accurate brain-image reconstructions.

 

Today, Changde Du at the Research Center for Brain-Inspired Intelligence in Beijing, China, and a couple of pals, say they have developed just such a technique. Their trick is to process the data using deep-learning techniques that handle nonlinear correlations between voxels more capably. The result is a much better way to reconstruct the way a brain perceives images.

 

Changde and co start with several data sets of fMRI scans of the visual cortex of a human subject looking at a simple image—a single digit or a single letter, for example. Each data set consists of the scans and the original image.

 

The task is to find a way to use the fMRI scans to reproduce the perceived image. In total, the team has access to over 1,800 fMRI scans and original images.

 

They treat this as a straightforward deep-learning task. They use 90 percent of the data to train the network to understand the correlation between the brain scan and the original image.

 

They then test the network on the remaining data by feeding it the scans and asking it to reconstruct the original images.

 

The big advantage of this approach is that the network learns which voxels to use to reconstruct the image. That avoids the need to process the data from them all.

 

It also learns how the data from these voxels is correlated. That’s important because if the correlations are ignored, they end up being treated like noise and discarded. So the new approach—the so-called deep generative multiview model—exploits these correlations and distinguishes them from real noise.    

 

To evaluate the deep generative multiview model, Changde and co compare its results from those of a number of other brain image reconstruction techniques. They do this using standard image comparison methods to see how closely the reconstructed images match the originals..

 

Looking at the DGMM images, they definitely appear to be the closest to the original presented images...  Just hope that this doesn't turn into an Orwellian situation.

 

https://www.engadget.com/2017/05/09/neural-network-recreates-brain-images/

 

[randomhkkid]

11 hours ago, WMGroomAK said:

In a step forward for the human-machine interface (or in a scary step to losing your privacy) a study has been published on a new approach for a Neural Network using Machine Learning to interpret functional Magnetic Resonance Images (fMRI) of the visual cortex into actual images.  From the MIT Technology Review:

 

https://www.technologyreview.com/s/604332/mind-reading-algorithms-reconstruct-what-youre-seeing-using-brain-scan-data/?set=604340

 

 

Looking at the DGMM images, they definitely appear to be the closest to the original presented images...  Just hope that this doesn't turn into an Orwellian situation.

 

https://www.engadget.com/2017/05/09/neural-network-recreates-brain-images/

 

Wow this is pretty fantastic! Like a whole other level above the classic MNIST handwriting problem. Interesting that they'd only use 90% of the data rather than doing cross validation.

 

Hopefully we'll be able to eliminate the human output modality bottleneck in the next 10-15 years!