Visual and semantic determinants of mental and neural representations of objects

Visual processing of sensory signals and our semantic knowledge about our visual world are heavily intertwined. As a consequence, our knowledge about the relative contribution of visual-perceptual and semantic dimensions to our mental and neural representations of objects and thus our ability to form categories has remained piecemeal. To address this challenge, we will carry out a detailed comparison of the mental and neural representations underlying the visual processing of object images and the representation of object words. By using large-scale online crowdsourcing of human similarity judgments, computational modeling of the underlying representational dimensions, and densely-sampled functional MRI paired with encoding models of these dimensions, this project promises to reveal the precise nature of the interplay between visual-perceptual and semantic information in brain and behavior.

new project-related publications

Hansen, H., & Hebart, M. N. (2022). Semantic features of object concepts generated with GPT-3. arXiv preprint arXiv:2202.03753. find paper

Hebart, M. N., Contier, O., Teichmann, L., Rockter, A. H., Zheng, C. Y., Kidder, A., Corriveau, A., Vaziri-Pashkam, M., & Baker, C. I. (2023). THINGS-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior. Elife, 12, e82580. find paper