Predicting brain activation patterns associated with individual lexical concepts based on five sensory-motor attributes


While major advances have been made in uncovering the neural processes underlying perceptual representations, our grasp of how the brain gives rise to conceptual knowledge remains relatively poor. Recent work has provided strong evidence that concepts rely, at least in part, on the same sensory and motor neural systems through which they were acquired, but it is still unclear whether the neural code for concept representation uses information about sensory-motor features to discriminate between concepts. In the present study, we investigate this question by asking whether an encoding model based on five semantic attributes directly related to sensory-motor experience – sound, color, visual motion, shape, and manipulation – can successfully predict patterns of brain activation elicited by individual lexical concepts. We collected ratings on the relevance of these five attributes to the meaning of 820 words, and used these ratings as predictors in a multiple regression model of the fMRI signal associated with the words in a separate group of participants. The five resulting activation maps were then combined by linear summation to predict the distributed activation pattern elicited by a novel set of 80 test words. The encoding model predicted the activation patterns elicited by the test words significantly better than chance. As expected, prediction was successful for concrete but not for abstract concepts. Comparisons between encoding models based on different combinations of attributes indicate that all five attributes contribute to the representation of concrete concepts. Consistent with embodied theories of semantics, these results show, for the first time, that the distributed activation pattern associated with a concept combines information about different sensory-motor attributes according to their respective relevance. Future research should investigate how additional features of phenomenal experience contribute to the neural representation of conceptual knowledge.

Fernandino, L, Humphries, CJ, Seidenberg, MS, Gross, WL, Conant LL, Binder, JR (2015) Predicting brain activation patterns associated with individual lexical concepts based on five sensory-motor attributes. Neuropsychologia; doi:10.1016/j.neuropsychologia.2015.04.009

Concept Representation Reflects Multimodal Abstraction: A Framework for Embodied Semantics


Recent research indicates that sensory and motor cortical areas play a significant role in the neural representation of concepts. However, little is known about the overall architecture of this representational system, including the role played by higher level areas that integrate different types of sensory and motor information. The present study addressed this issue by investigating the simultaneous contributions of multiple sensory-motor modalities to semantic word processing. With a multivariate fMRI design, we examined activation associated with 5 sensory-motor attributes—color, shape, visual motion, sound, and manipulation—for 900 words. Regions responsive to each attribute were identified using independent ratings of the attributes’ relevance to the meaning of each word. The results indicate that these aspects of conceptual knowledge are encoded in multimodal and higher level unimodal areas involved in processing the corresponding types of information during perception and action, in agreement with embodied theories of semantics. They also reveal a hierarchical system of abstracted sensory-motor representations incorporating a major division between object interaction and object perception processes.

Fernandino, L, Binder, JR, Desai, RH, Pendl, SL, Humphries, CJ, Gross, WL, Conant, LL, Seidenberg, MS (2015) Concept Representation Reflects Multimodal Abstraction: A Framework for Embodied Semantics. Cerebral Cortex; doi: 10.1093/cercor/bhv020

SfN Poster: Global signal regression improves fMRI prediction of language outcome after left anterior temporal lobectomy

Gross, W.L., Zhou, Y.Q., Binder, J.R. (2014) Global signal regression improves fMRI prediction of language outcome after left anterior temporal lobectomy. Presented at the annual meeting of the Society for Neuroscience, Washington DC


Noise segregation is a critical component of fMRI analysis. Because many noise sources are distributed evenly across the brain image, global signal regression (GSR; removing spatially averaged signal components) has become a popular method of noise removal. Although this can be done to improve SNR in any fMRI analysis, it is particularly relevant to functional connectivity analyses, where spatial correlations are key.

Current literature on GSR cites multiple theoretical issues with this methodology. Because calculation of global signal (GS) includes voxels of interest, it is weakly correlated with signals of interest (Murphy et al., 2009) and seems to reduce the magnitude and extent of activation (Aguirre et al., 1998). It produces negative correlations in connectivity analyses, which some observers propose are factitious (Murphy et al., 2009). Because of these effects, many have argued against using GSR. However, it is not clear if these effects are purely artifactual, as there are no external standards, and most arguments are based on theoretical conjecture. A few studies have compared fMRI to electrical recordings in the brain (Scholvinck et al., 2010; Keller et al., 2013) and found that, although the GS is present in neural activity, GSR improves the correspondence of fMRI with high gamma power.

To test the validity of GSR, we applied it to an fMRI analysis that predicts neuropsychological outcome after left anterior temporal lobe surgery (Sabsevitz et al., 2003; Binder et al., 2008). We took the ability to predict outcome as an externally valid standard. If predictions improve after GSR, SNR must be increasing in the data. This analysis included 36 patients with intractable epilepsy who underwent left anterior temporal lobectomy and preoperative fMRI mapping. The maps were used to produce lateralization indices of language, which were correlated with outcome. GS was calculated by taking the spatial average across all voxels in the brain, and was applied to the analysis either as a regressor, or through proportional scaling (Gavrilescu et al., 2002). Voxel counts were also calculated within ROIs known to be associated with the task, along with the rest of the brain.

On all models tested, adding GSR (either as a regressor, or through proportional scaling) increased the R2 of the outcome prediction model. Interestingly, it also consistently reduced the voxel count in every ROI examined. It appears that GSR decreases noise in the data, and improves information content, although this is not reflected in the raw voxel counts. These data provide support for the use of GSR and caution against making strong conclusions about data quality based only on voxel counts.

Alternative Thresholding Methods for FMRI Data Optimized for Surgical Planning


Current methods for thresholding functional magnetic resonance imaging (fMRI) maps are based on the well-known hypothesis-test framework, optimal for addressing novel theoretical claims. However, these methods as typically practiced have a strong bias toward protecting the null hypothesis, and thus may not provide an optimal balance between specificity and sensitivity in forming activation maps for surgical planning. Maps based on hypothesis-test thresholds are also highly sensitive to sample size and signal-to-noise ratio, whereas many clinical applications require methods that are robust to these effects. We propose a new thresholding method, optimized for surgical planning, based on normalized amplitude thresholding. We show that this method produces activation maps that are more reproducible and more predictive of postoperative cognitive outcome than maps produced with current standard thresholding methods.

Gross, W. L., & Binder, J. R. (2014). Alternative thresholding methods for fMRI data optimized for surgical planning. NeuroImage, 84C, 554–561. doi:10.1016/j.neuroimage.2013.08.066

NLC Poster: The Neural Representation of Event Nouns

Pendl, S., Humphries, C.J., Gross, W.L., Binder, J.R. (2012) The Neural Representation of Event Nouns. Presented at the annual meeting of the Society for the Neurobiology of Language, San Sebastian, Spain

Event nouns (e.g., circus) differ from non-event nouns (e.g., dandelion) in their reference to dynamic temporal and spatial configurations. We asked whether the dynamic nature of event concepts leads to recruitment of specific brain regions. Fifteen volunteers underwent event-related fMRI, during which they read individual nouns and indicated with button presses whether or not they could experience the indicated entity with their senses. The 40 event nouns and 40 non-event nouns were matched on word frequency, imageability, length, orthographic neighborhood size, constrained bigram frequency, and constrained trigram frequency. A mix of abstract words and pseudowords served as filler items. Event words more strongly activated the left posterior middle temporal gyrus (pMTG; -59, -50, 1), left angular gyrus (AG; -32, -77, 37), and left inferior frontal gyrus (IFG; -40, 7, 26). While all of these regions have been identified previously as belonging to the semantic system, these results suggest a unique neural representation for the semantic content of event words. Event concepts depend on knowledge of thematic relationships, spatial relationships, temporal sequences, actions, and causality. A region of the left pMTG overlapping with the focus we observed here was recently linked with processing of causality. Regions including the left AG have been implicated in processing thematic relationships between words in comparison to taxonomic relationships. The present results, together with these previous findings, suggest that portions of the left MTG and left AG are involved in processing specific kinds of knowledge that are unique to event concepts.