Researcher Highlights
The Rhythm and Melody of Language

Associate Professor Kiwako Ito is tuning in to the rhythm of speech to understand its complexities.

Associate Professor Kiwako Ito studies the effect of prosody, or rhythm and melody in speech and how people respond to it. Research on prosody is an important focus in many disciplines including Linguistics, Speech Pathology, Education, Psychology and Computer Science. Ito’s own empirical research into how people respond to the emphasis in speech is focused on children with developmental disorders, aging adults and hearing impaired individuals. Kiwako was named by The Australian as leader of the field of Language and Linguistics, with the highest number of citations from papers published in the last five years in the 20 top journals in the field.

One of her latest studies has delivered exciting results that disprove the stereotype that children with autism are generally not sensitive to prosody and ‘joint attention cues’.

The study focused on children aged one to three years old and involved them watching a video where an actor asks ‘Where is the pig,’ then turns their head to look at the pig. Associate Professor Ito used an eye tracking device to monitor the child’s gaze and test whether the children would respond to the speech cues and joint attention cues of the actor turning their head and pointing.

Associate Professor Ito says that what she found went against the stereotypical belief that children with autism don’t respond to speech and social cues.

“A joint attention cue such as pointing, a head turn or a shift of gaze is what we use to draw attention of others to the same direction. Typically developing children are expected to follow the gaze of the actor to the thing that is pointed to. The prediction for this experiment was that children with autism would have low sensitivity to these cues,” she said.

“The data from this study proves that wrong. The children we tested were actually very sensitive and did shift their gaze according to the joint attention cues and it happened much faster than expected. They are also sensitive to the emphasis in speech.”

Lab for Applied Language Sciences

The eye tracking device Associate Professor Ito uses is the key instrument in a new Lab for Applied Language Sciences at the University of Newcastle. Associate Professor Ito plans to develop the lab as a collaborative space where she can work with researchers from other disciplines on a broad range of applications.

“The eye tracker is set up to chase the eye movements of anyone from infants to people playing a board game or using a computer. What I manipulate is the speech input. In most of my experiments I show the subject something and then I test reactions to the placement of emphasis in speech,” Associate Professor Ito said.

“This can tell us how quickly the mind processes the speech input and how quickly we make decisions accordingly. Our eye movements are one of the fastest muscle responses in our body. It’s part of our natural survival system and we can’t help moving our eyes around if there is some informative input such as speech,” she said.

The lab offers opportunities for collaboration with pathology clinicians, medical researchers, psychologists and educators. Ito expects that research will include speech perception in children, adults and people with clinical conditions, grammatical development in school age children and children with developmental disorders and the effect of specific types of training on second/foreign language processing.

“I keep hearing from educators that the anxiety issue is really prevalent at all stages of development, so that is another area I am aiming to cultivate in the lab.”

“My study of prosody also incorporates the tone of voice, which means I can run experiments to see what kind of tone of voice can help people focus or get their attention without making them anxious,” she said.

“We can also incorporate the presence or absence of gesture. When people talk they use facial expressions, eye gaze and body language, so we can experiment to see what kind of combination is most effective and a good mode for communication.”

How is grammar heard?

A new research project that Ito is commencing involves speech pathology and education collaborators. Their research aims to get a baseline of how typically developing children hear and process grammar.

“We then hope to extend this process and assess how children with hearing disabilities process speech and respond to speech stimuli,” Associate Professor Ito said.

“The experiment we are setting up has to do with how people hear or don’t hear the ‘s’ in a sentence. For example take the sentence: ‘The alligators that slowly moved through the swamp hear the frog.’ If you hear the ‘s’ at the beginning of the sentence on ‘alligators’ you expect not to hear the ‘s’ on the verb ‘hear’.”

“If they can’t hear the ‘s’ they can’t process the grammatical function. People are not aware of this process they are doing daily, but the eye-tracker can capture the moment they respond to the ‘s’ or not respond.”

This type of research will create a database that informs us what age school age children start responding to this type of stimuli and process grammar.

“We can compare data between children and adults and draw the trajectory of grammar and language development,” Associate Professor Ito said.

“We can then test children with hearing disabilities, autism, and memory problems and compare it to the baseline data. This will show us how speech perception and memory span interacts and affects the ability to process grammar.”

Associate Professor Ito says her research is ideal for industry applications and there may be organisations who are interested in supporting this research to produce a communication device that is specific to the clinical condition.

“Through this research we will find out what kind of cues are processed better by people with hearing disabilities and by people who have attention disorders or memory deficits. This can also be applied to aging populations who have both hearing and memory declination.”