How do neural representations along the ventral stream relate to different similarity spaces?

 

In this project, we related neural responses to real-world objects to different behavioral tasks and found changes in the similarity space as we moved from more posterior to more anterior regions of the ventral stream.

 

Here's a video of a talk I gave on this project (virtual-VSS, June 2020).

Investigating the nature of the neural representation of objects in high-level visual cortex

Inanimate object representation in the occipitotemporal cortex follows a large-scale organization along the dimensions of real-world size (Konkle and Oliva, 2012). 

 

But, what do we mean by size? Do neural responses reflect 1) an abstract interpretation of a stimulus as being big or small, or 2) does it reflect something more visual in nature, such as the typical shapes of big and small objects?

To explore the first possibility, we contrasted real-world size to motor-relevance. Motor-relevance is a higher-level, non-visual property that covaries with size in the world: small objects tend to be more often hand-held and manipulated, while big objects tend to be less so. 

 

We collected a stimulus set in which the two dimensions were pitted one against the other, showed them to participants in an fMRI scanner, and explored the degree to which each explained neural representations in high-level visual cortex. 

 

You can see a poster of this project here (presented at VSS 2016).

To explore the second possibility, we related real-world size to a mid-level visual feature that correlates with it in the real world: curvature. In the world, small objects tend to be curvier, while big objects tend to be boxier. It is possible that curvature alone might drive the organization by size of inanimate objects in the ventral stream; alternatively, real-world size information might be preserved even when correcting for curvature. 

Using a similar approach as before, we collected a 2 x 2 stimulus set in which the two dimensions were pitted one against the other, and presented them to participants while lying on the scanner.

You can see what we found here (VSS 2019).