https://www.youtube.com/embed/ph--BKi6iWk?si=oc9Kcjz9_jiZLYQd

The spatial ventriloquism illusion occurs when a sound is heard as coming from a visible source—such as a puppet’s moving mouth—even though the sound actually comes from somewhere else – the puppeteer’s mouth. This happens when the sound occurs at the same time as a clear, attention-grabbing visual movement that matches the sound in some way, such as the lip movements matching those that would normally be expected to produce that sound. It is a popular illusion, as numerous ventriloquist shows attest. Although its workings seem fairly straightforward, ventriloquism demonstrates a number of complex scientific phenomena.

When observing the illusion, the information our brains receive is perplexing. Our ears receive information that would, by itself, indicate that the speaker’s voice is coming from the puppeteer, but our eyes receive information that would, by itself, indicate that the lip movements is coming from the puppet. Our brains have a difficult decision to make—who is it that is talking? The fact that we perceive the illusion that the puppet is talking, attests to the fact that our senses interact and can even influence one another (Stein, 2012).

The reason we perceive the puppet to be talking, and not the puppeteer, resides within the theory of visual dominance (Colavita, 1974). The theory suggests a larger influence of visual stimuli than auditory stimuli upon an individual’s overall perceptual experience. This is only true, however, if the judgement to be made regards the spatial location of the source (a “where” judgement). If the judgement regards the timing of multisensory signals (a “when” judgement), auditory stimuli tend to be more influential than visual ones (Slutsky & Recanzone, 2001). Given that, in the ventriloquism case, our brains’ task is to decide where the audio-visual stimuli are originating, the visual information is more influential and attracts the perceived location of the sound.

Why don’t we just perceive the information as coming from two different sources? To maintain coherence among our perceptions of the external environment, our brains merge information presented to our different senses. One rule which determines whether the information we receive is merged or not is whether this information spatially and temporally aligns. If our brains receive visual and auditory information originating from the same location, presented at the same time, it’s likely that this information comes from the same source. Our brains, therefore, assume that different stimuli that align temporally and/or spatially are from the same source and integrate them. These rules are flexible, in that these spatial and temporal windows can be slightly broadened (Chen & Vroomen, 2013). This means that if information does not come from exactly the same location, or at exactly the same time, it may still be integrated. Ventriloquism beautifully illustrates the span of the spatial windows.

As previously mentioned, the phenomenon described here is spatial ventriloquism. A lesser known phenomenon is illusory temporal ventriloquism. In this illusion, temporal aspects of a visual stimulus, such as duration, onset, or interval, can be shifted if merged with a temporally asynchronous auditory stimulus (Alias & Burr, 2004). An example of this is the Sound-Induced Flash Illusion.