"Long before it's in the papers"
August 03, 2010

RETURN TO THE WORLD SCIENCE HOME PAGE

Paintings really can be heard, scientist says

Sept. 7, 2006
Courtesy University College London
and World Science staff

The Rus­sian art­ist Was­si­ly Kan­din­sky was­n’t talk­ing non­sense when he claimed his paint­ings could be heard, a sci­ent­ist says. In fact, he adds, we all link sound and col­or men­tal­ly at some le­vel—and tend to do so in con­sist­ent ways, which art­ists can exploit.

Kandin­sky's "Composition VIII, 1923," in the Sol­o­mon R. Gug­gen­heim Mu­se­um, New York. 
Some peo­ple con­scious­ly rea­lize the cross­o­ver of sen­ses in their brains, said the re­search­er, Ja­mie Ward of Uni­ver­si­ty Col­lege Lon­don. These in­di­vi­du­als have a rare con­di­tion in which the senses min­gle, called syn­aes­the­sia.

They may real­ly feel they hear paint­ings, for in­s­tance. But the con­di­tion does­n’t oc­cur on­ly with sound and sight: the most bi­zarre forms of it have been re­ported. 

A stu­dy in the Aug. 22 is­sue of the re­search jour­nal Con­scious­ness and Cog­ni­tion, for ex­am­ple, found that some peo­ple link time and space. One de­s­cribed De­cem­ber as a red ar­e­a lo­cat­ed at arm’s length to the left of their bod­y.

Ward detailed find­ings of his own new stu­dies on syn­aes­the­sia at a talk at the Bri­t­ish As­so­ci­a­tion for the Ad­vance­ment of Sci­ence Fes­ti­val of Sci­ence in Nor­wich, U.K. this week. His work fo­cused on the sound-col­or-link­age of which Kan­din­sky spoke.

His found that most of us tend to agree with syn­aes­thetes on which im­ages match which sounds, and that we pre­fer them com­bined rath­er than alone.

Kan­din­sky, who lived from 1866 to 1944, “wanted to make vis­u­al art more like mu­sic—more ab­strac­t,” Ward said. “He al­so hoped that his paint­ings would be ‘heard’ by his au­di­ences. This seems more achiev­a­ble now that we have found such a strong link be­tween vi­sion and hear­ing.”

“Although in­for­ma­tion from the world en­ters our heads vi­a dif­fer­ent sen­so­ry or­gans—the eyes and ears in this in­stance—once they are in the brain they are in­ti­mate­ly con­nect­ed with each oth­er. Im­pres­sive­ly, they are con­nect­ed in non-random ways, so that some com­bi­na­tions of sound and vi­sion go to­geth­er bet­ter than oth­ers.”

In ex­per­i­ments, Ward said he asked six synaes­thetes to draw and de­scribe their vis­u­al ex­pe­ri­ences of mu­sic from the New Lon­don Or­ches­tra. Six non-syn­aes­thetes were asked to do the same. Also, an an­i­ma­tor crea­ted films com­bin­ing the mu­sic and drawn im­ages. These were shown to vi­si­tors at Lon­don’s Sci­ence Mu­se­um. 

Furthermore, 100 im­ages were presented to over 200 peo­ple. They were asked to choose the im­age that best fit the mu­sic. They con­sist­ently chose im­ages drawn by synaes­thetes over others, Ward re­ported. 

This shows, he said, that while non-syn­aes­thetes can’t hear a paint­ing or see mu­sic lit­er­ally, they do sense the cross­o­ver and tend to choose the “cor­rect” im­age. “All of us have links be­tween our hear­ing and vi­sion—even if we don’t really realise it,” Ward said.

That’s not to say that synaes­thetes hear pre­cise­ly the same sounds in “lis­ten­ing” to a Kan­din­sky paint­ing.

De­scrib­ing the art­ist’s “Com­po­si­tion VIII, 1923,” Ward re­ported, one synaes­thete said: “The jum­bled mass of lines gave var­i­ous tones, which changed as my eyes trav­elled round the pic­ture. When look­ing at the large mul­ti­coloured pow­er­ful cir­cle at up­per left, I get a pure tone which can be too much, so to re­lieve my mind of this I trav­el back to the ca­coph­o­ny of jum­bled lines and shapes. This paint­ing there­fore is a good bal­ance of con­trast­ing noise—pure tones and ca­coph­o­ny—which was a de­light to see.”

Anoth­er de­scribed it as fol­lows: “There is a huge splurge of sound left-hand top—boom­ing and vul­gar! Be­low it is a mousy lit­tle meee sound which then trans­lates in­to ‘o­h’s and ‘ah’s and pops at the var­i­ous cir­cles. The lines are sharp and are mov­ing to the right with the sound of steel—like blades scrap­ing against one anoth­er. The tri­an­gle and boom­er­ang shape are sur­prised and pop up laugh­ing with a ‘whooo’.”

The next stage of the re­search will use brain scans to mon­i­tor the brains of synaes­thetes when Kan­din­sky trig­gers sound or when sound trig­gers a vi­sion, Ward said.

* * *

Send us a comment on this story, or send it to a friend

 

Sign up for
e-newsletter
   
 
subscribe
 cancel

On Home Page         

LATEST

  • F­ungus-treat­ed vio­lin beats St­rad in bli­nd te­st

  • Show­er­heads may spray ger­ms in your fa­ce

EXCLUSIVES

  • Re­port: cells “from space” have un­usual make­up

  • Dol­phins and the evo­lution of teach­ing

  • D­rug may tri­ck body into “think­ing” you ex­ercised

  • Tit-for-tat: bi­rds found to re­pay war­­time help

  • Musi­cal genes may be com­ing to light

MORE NEWS

  • R­ock-hurl­ing zoo chimp stock­ed am­mo in ad­vance: study

  • Fai­th found to re­duce er­rors on psycho­logical test

  • Dood­ling gets its due: ti­ny art­works may aid mem­ory

  • From or­al to mor­al? Dirty deeds may prompt “bad taste” reac­tion

The Russian artist Wassily Kandinsky wasn’t talking nonsense when he claimed his paintings could be heard, a scientist said: In fact, we all link sound and color subconsciously, and follow consistent patterns in doing so. A tiny percentage of people is consciously aware of the crossover of senses in our brains, said Jamie Ward of University College London, speaking at the British Association for the Advancement of Science Festival of Science in Norwich, U.K. this week. These people, known as synaesthetes, have a rare condition in which the senses mingle. It doesn’t happen necessarily only with sound and color. The most bizarre forms of synaesthesia have been reported. A study in the Aug. 22 issue of the research journal Consciousness and Cognition found that some people link time and space: one person experienced December as a red area located at arm’s length to the left of their body. Ward’s studies focused on the better known color-sound linkage of which Kandinsky spoke. His results show that most of us prefer image and sound combined, rather than either in isolation, Ward said. We also tend to agree on which images match particular sounds. This could have implications for how we understand art and develop art forms that combine visual images with sound—such as ballet, opera, visual jockeying and animation. Kandinsky, who lived from 1866 to 1944, “wanted to make visual art more like music—more abstract,” Ward said. “He also hoped that his paintings would be ‘heard’ by his audiences. This seems more achievable now that we have found such a strong link between vision and hearing. “Although information from the world enters our heads via different sensory organs—the eyes and ears in this instance—once they are in the brain they are intimately connected with each other. Impressively, they are connected in non-random ways, so that some combinations of sound and vision go together better than others.” During a series of experiments, Ward said he asked six synaesthetes to draw and describe their visual experiences of music played by the New London Orchestra. A control group of six people without the condition were asked to do the same. Animated films, combining the music and drawn images were created by an animator, Sam Moore of the University of Wolverhampton, and shown to the public visiting London’s Science Museum. Also, 100 images were shown to over 200 people. They were asked to choose the image that provided the best fit to the music. Respondents consistently chose the images drawn by synaesthetes over control images, Ward reported. This shows that while people without synaesthesia are not able to hear a painting or see a piece of music in a literal sense, they are able to sense the crossover and tend to choose the ‘correct’ image. “While some synaesthetes can actually hear a Kandinsky in a very real way, the rest of us don’t have such a pronounced crossover of senses. But, this research shows that all of us have links between our hearing and vision—even if we don’t really realise it,” he said. That’s not to say that synaesthetes hear precisely the same sounds in “listening” to a Kandinsky painting. Describing the artist’s “Composition VIII, 1923,” Ward reported, one synaesthete said: “The jumbled mass of lines gave various tones, which changed as my eyes travelled round the picture. When looking at the large multicoloured powerful circle at upper left, I get a pure tone which can be too much, so to relieve my mind of this I travel back to the cacophony of jumbled lines and shapes. This painting therefore is a good balance of contrasting noise—pure tones and cacophony—which was a delight to see.” Another synaesthete described it as follows: “There is a huge splurge of sound left-hand top—booming and vulgar! Below it is a mousy little meee sound which then translates into ‘oh’s and ‘ah’s and pops at the various circles. The lines are sharp and are moving to the right with the sound of steel—like blades scraping against one another. The triangle and boomerang shape are surprised and pop up laughing with a ‘whooo’.” The next stage of the research will use brain scans to monitor the brains of synaesthetes when Kandinsky triggers sound or when sound triggers a Kandinsky-like vision, Ward said.