TORONTO, January 23, 2022
Ensuring that digital tools accurately measure what they are designed to measure is a key step in the validation process of novel clinical tools. Typically referred to as analytical validation, this process is noted by the Digital Medicine Society (DiMe) in their highly cited V3 validation framework as an essential milestone and best practice that will help move the field of digital measurement forward.
With that goal in mind, we have developed an in-house pipeline that measures the performance of our natural language processing (NLP) and machine learning tools that extract linguistic characteristics of speech. To do this, we collected data from expert linguists who annotated the linguistic characteristics of speech transcripts. The annotations included the parts of speech like nouns and verbs, the dependencies between words, the syntactic relationships, and more. Our linguists provided these gold standard linguistic labels for English, French, Spanish, and German transcripts, from healthy and non-healthy participants, enabling us to validate the accuracy of our tools in all four languages and across therapeutic areas.
We then implemented these gold standards into our processing pipeline so that we can continuously quantify how our automatically calculated features compare to the linguists’ gold standards. Because it is embedded in our pipeline, we can validate performance whenever we upgrade our tools or add new ones. We will continue to expand to our gold standard dataset to include other characteristics of speech (e.g. acoustics), to other languages, and across more therapeutic areas.
This effort has been many months in the making. We’re grateful for the work of the dedicated team of linguists and Winterlight team members who worked so hard on this foundational project!
Winterlight Labs is commercializing a proprietary language-based diagnostic system that analyzes natural speech to detect and monitor dementia, Alzheimer’s, aphasia, and various other cognitive conditions. Winterlight's scalable platform uses short recorded speech samples to analyze hundreds of linguistic cues, and can detect dementia and other conditions with a high level of accuracy. This is a major improvement over current pencil-and-paper tests which are time-consuming, costly, and difficult to administer. The platform has applications in drug trials, long-term and primary care, and speech-language pathology.
Since its founding, Winterlight Labs has seen strong interest from pharmaceutical companies and other potential partners who view the technology as a major improvement over current methods of detection or screening. The company has gained support from the pan-Canadian AGE-WELL Network of Centres of Excellence (NCE), the Ontario Brain Institute, Ontario Centres of Excellence, and the University of Toronto Banting and Best Centre for Innovation and Entrepreneurship. For more information, visit www.winterlightlabs.com.
Media Inquiries:
Winterlight Labs
Liam Kaufman, CEO
liam [at] winterlightlabs.com