Markos Athanasiadis: Attack based identification of most informative patterns in fMRI visual stimuli classification

Abstract 

 Classifier-based decoders have traditionally been used to link neuronal activity features to cognitive functions and behavior. The black-box nature of these algorithms results in the lack of information regarding the decision hyperplane geometry, which in turn hampers the interpretability of the classifier and hinders our understanding of its neuronal basis. Recently, methods have been developed that are capable of probing the decision hyperplanes of classifiers, with the generation of adversarial patterns prone to misclassifications.

 We propose a method to identify the most-informative directions in linear and non-linear, binary classification tasks, taking advantage of such adversarial patterns. We validate our method against multidimensional synthetic datasets, and observe that we outperform estimating the weight vector by bootstrapping the training process on multiple subsamplings for high levels of class overlap.

 Furthermore we explore fMRI activity patterns in the brain of human subjects during an object-location memory task, with pairs of visual stimuli, repeatedly presented at distinct screen locations. We use a linear neural network to examine if the stimulus category can be decoded for two sets of activity patterns, one with clear category distinction between the presented stimuli (fruits vs animals) and one with stimuli from the same category but for which distinct associations were learned (objects associated with fruits vs objects associated with animals). Using our approach, we visualize the decision hyperplane to directly identify the neuronal basis of the linear ANN, and observe that few features contribute to the task consistently during recall but not encoding, across subjects and conditions. 

About the speaker and his research

Hosted by Vladislav Ivantaev