An article for InventUM | University of Miami Miller School of Medicine
READ ON THE MIAMI MILLER SCHOOL OF MEDICINE WEBSITE
BY ROCHELLE BRODER-SINGER
In individuals with mild cognitive impairment, speech behavior following pauses is different than in healthy individuals. Machine learning algorithms can use this behavior to screen for cognitive impairment. University of Miami Miller School of Medicine researchers have published the first research showing how post-pause speech in certain tasks can play an important role in identifying mild cognitive impairment.
“Speech is an easy-to-collect behavior, and computer analysis of specific speech tasks offers a minimally invasive way to help identify those with mild cognitive impairment (MCI),” said Michael Kleiman, Ph.D., research assistant professor of neurology at Miller School, member of the University of Miami Comprehensive Center for Brain Health and the article’s first author. “These findings suggest that tracking how people talk in specific tasks could become a simple way to spot early signs of Alzheimer’s or other cognitive problems.”
Around 80% of patients who have MCI are not diagnosed until after they’ve progressed to clinical dementia.
“Early detection of cognitive impairment offers patients the possibility of opportunities for early intervention with medication, participation in clinical trials and advanced care planning, but most patients in the U.S. are not diagnosed until the moderate stage,” explained the article’s senior author, James Galvin, M.D., M.P.H., Alexandria and Bernard Schoninger Endowed Chair in Memory Disorders and professor of neurology, psychiatry and behavioral sciences at the Miller School. “While there are many reasons for this later diagnosis, one important reason is the lack of sensitive tools for early detection. Further research is needed, but the use of speech patterns analyzed with artificial intelligence potentially provides a novel path forward.”
Dr. James Galvin, the study’s senior author, is hopeful speech pattern analysis will open up avenues for early detection of cognitive decline.
The article was published in the Journal of Alzheimer’s Disease online and will be published in the journal’s print edition.
How Mild Cognitive Impairment Affects Speech
Some of the earliest impacts of MCI often occur in the brain regions associated with language and speech production. Prior research has shown speech changes in people with mild cognitive impairment may include slower speech, use of words with fewer syllables and changes in pauses. The research by Dr. Kleiman and Dr. Galvin found that the words immediately following a pause (post-pause speech) are affected, too.
Pauses are a normal part of human speech. They can be unfilled (extended silences between words) or filled with utterances such as “uh,” “um” and “er.” Filled pauses are generally considered “searching” pauses and signal longer cognitive delays than unfilled pauses. During searching pauses, the speaker may be searching for the correct answer to a question or the correct thing to say, searching for something new to describe or searching for their next words. Dr. Kleiman and Dr. Galvin found that, in certain circumstances, individuals with MCI have meaningfully different behavior after filled pauses
Dr. Michael Kleiman is using AI to analyze speech patterns and predict cognitive impairment.
“People with mild cognitive impairment show subtle changes in their speech, such as using simpler words after pauses and taking longer to resume speaking, especially during demanding tasks like storytelling,” Dr. Kleiman noted.
These differences are significant enough that algorithms can use them to identify which individuals are likely to have MCI and which are likely healthy.
Analyzing Speech with AI and Machine Learning
The study by Dr. Kleiman and Dr. Galvin included 53 total participants, each older than 60. The 14 participants with MCI and 39 healthy controls are part of the Healthy Brain Initiative, a longitudinal study of brain health and cognition at the Comprehensive Center for Brain Health.
First, the researchers administered four speech-focused tasks to participants:
• An immediate narrative-recall task, during which a story was read and visually presented to participants. They were asked to recall the story immediately.
• A delayed-recall task, asking participants to recall the narrative 15 to 20 minutes later, after they had been distracted with other speech tasks.
• A picture-description task, giving participants 90 seconds to describe an image.
• A free-response task, in which participants were asked to describe their typical morning routines.
Responses were recorded. Leading and trailing silence and background noise such as room tone and other voices were removed. Those audio files were processed into text transcripts using the OpenAI Whisper Large-v2 model, then manually corrected as needed by trained research staff. Finally, the transcripts were parsed using a script that incorporated a variety of speech analyses. The speech of healthy controls and those with MCI was compared.
The researchers controlled for age in their analysis, using it as a covariate in all comparisons between participants with and without MCI. There were no differences in performance between groups based on gender, years of education, race, ethnicity, vulnerability or resilience.
Pauses and Post-pause Speech
Several characteristics distinguished the speech of individuals with MCI from the healthy controls. The two most significant differences were found following filled pauses:
• Those with MCI had longer latencies during the delayed narrative recall task.
• Those with MCI used more high-frequency language during the free-response task.
Individuals with MCI demonstrated a number of other differences, including:
• Significantly more filled pauses, especially using “uh”
• Longer latencies between any type of pause and their next word
• Lower total word count in every speech task
• Less use of adverbs after unfilled pauses in the free-response task
• Use of less-complex syntax to describe the picture
“The most important takeaway is that you can analyze speech with a bunch of different tasks, and each tells you something different,” Dr. Kleiman said. “They all give us a small piece of the puzzle. Only by combining them all together may we be able to identify mild cognitive impairment.”
Screening for Mild Cognitive Impairment
The predictive model developed in this study does a good job of distinguishing between individuals with and without mild cognitive impairment. The two most effective screening measures – looking for increased use of more-common words after pauses during less-demanding tasks and post-filler latency in highly demanding tasks – accurately predict MCI with area-under-the-curve (AUC) accuracy of 79.1%.
This particular study is a stepping stone. The ultimate goal is to build a speech-based detection algorithm that can identify mild cognitive impairment, maybe even pre-Alzheimer’s disease.
Dr. Michael Kleiman
The analysis developed by Dr. Kleiman and Dr. Galvin has a very low rate of false positives, correctly identifying individuals who do not have mild cognitive impairment with a specificity of 94.6%. However, the model has a sensitivity of just 43.6%, meaning it missed more than half of cases with impairment present. This makes the model useful for screening purposes. It’s good at ruling out nonimpaired individuals but it is less reliable for diagnosing impairment.
“Post-pause metrics of latency and use of common language would be an excellent addition to any machine-learning model that utilizes speech behavior and seeks to identify healthy individuals,” Dr. Kleiman said.
Expanding the Model to Address Limitations
The study is ongoing, and these early results had a number of limits the researchers are addressing, including:
• A small sample size
• Far more healthy controls than individuals with MCI
• Primarily non-Hispanic, white participants
• Exclusively English-speaking participants
• A lack of biomarker data
• Examination of only three pause fillers (“uh,” “um” and “er”)
Dr. Kleiman is adding participants’ biomarker data into the model and is recruiting more participants with mild cognitive impairment, as well as more racially and ethnically diverse individuals, including those who primarily speak Spanish. He has a grant specifically focused on collecting speech data from underserved populations, such as those who speak with accents or use dialects such as African American Vernacular English. He currently has nearly 300 participants, compared to 53 in this study.
“Our lab is really trying to make our cohort as diverse as possible. We’re aiming for no more than 50% of our cohort to be non-Hispanic white,” Dr. Kleiman said. “The thing about machine learning models is that you need them to be representative, because if they don’t represent everyone, they’re not able to be accurate for everyone.”
If non-Hispanic white people are overrepresented in the data, he explained, screening will be less accurate for individuals in other populations.
Early Diagnosis of Cognitive Impairment
The article only covers a small portion of the data the researchers have collected. They continue to build a more diverse corpus of data to improve the algorithm and expand its use to other languages.
“This particular study is a stepping stone,” Dr. Kleiman said. “The ultimate goal is to build a speech-based detection algorithm that can identify mild cognitive impairment, maybe even pre-Alzheimer’s disease, using a two-to-10-minute speech test or interview.”
This research was funded by grants from the National Institute on Aging, the Alzheimer’s Association, the Evelyn F. McKnight Brain Research Foundation and the Harry T. Mangurian Jr. Foundation.
