Visual and semantic determinants of mental and neural representations of objects
Visual processing of sensory signals and our semantic knowledge about our visual world are heavily intertwined. As a consequence, our knowledge about the relative contribution of visual-perceptual and semantic dimensions to our mental and neural representations of objects and thus our ability to form categories has remained piecemeal. To address this challenge, we will carry out a detailed comparison of the mental and neural representations underlying the visual processing of object images and the representation of object words. By using large-scale online crowdsourcing of human similarity judgments, computational modeling of the underlying representational dimensions, and densely-sampled functional MRI paired with encoding models of these dimensions, this project promises to reveal the precise nature of the interplay between visual-perceptual and semantic information in brain and behavior.