Recently, our research team completed a study validating digital cognitive assessments administered with BrainCheck against their respective traditional pen and paper counterparts. Here, we share the results of this study.
Research Focus and Participants
At BrainCheck, we use a wide range of assessments to measure a person’s cognitive function. Some of these assessments are based on well-established measures, which have traditionally been completed using pen and paper. In these cases, it is important that the individual BrainCheck assessments be consistent with these traditional pen and paper neuropsychological tests (NPTs) commonly used in clinical practice.
Our research team tested 27 subjects (ages ranging from 12 to 88, mean age of 46.9). Each participant took a digital cognitive assessment using BrainCheck and corresponding pen and paper NPTs. The pen and paper NPTs included Raven’s Matrices, digit-symbol modulation, the Stroop Color and Word Test, the Trail Making Test (parts A and B), balance examination, and immediate/delayed Hopkins Verbal Learning Test.
Validation criteria included correlations of greater than 0.6 between the BrainCheck-administered versions of the tests and the corresponding traditional test, as well as a p-value below 0.05, which we chose to indicate a statistically significant correlation.
Results indicate consistent performance across most assessments
Of the tests, the BrainCheck-administered matrix, digit-symbol, Stroop, and Trails A & B tests passed the validation criteria, while the balance and immediate/delayed recall failed to reach validation criteria.
Notably the balance assessment within the BrainCheck platform uses the iPad’s on-board hardware to measure participant motor skills and coordination. The traditional comparison uses a full-body balance test. This was likely not a well matched comparison, explaining the lack of correlation.
BrainCheck-administered immediate and delayed recognition also differs from the traditional free recall tests. In the case of recognition, a cue is provided, whereas with recall there is no cue. The brain circuits involved with these tests differ in important ways. This likely explains the lack of correlation.
Advantages of Computerized Testing
Traditional NPTs take up vast clinical resources. They are time consuming and require administration/interpretation by trained neuropsychologists. Also, shorter screening tools are not eligible for reimbursement from insurance payors.
Our results demonstrate some of the individual BrainCheck assessments may serve as suitable options for computerized NPTs. Additionally, use of computerized testing has several advantages, including eliminating the floor and ceiling effects commonly associated with pen and paper tests, as well as automated scoring/interpretation and integration into the electronic health record.
What is perhaps most important is the ability to build assessments to correlate with patient function and biological changes rather than the ability to do cognitive tests, which often does not have a real world application.
(As a provider, I’m not concerned with how well a patient can match letters and symbols on paper. I am concerned about how that patient can complete activities of daily living.)
These traditional tests have served us well and provide some reasonable estimates. However, we ultimately need to measure patients’ ability function in their everyday lives.
BrainCheck: Innovation that Improves Care
At BrainCheck, we are constantly trying to improve our platform to help clinicians accurately measure patients’ cognitive function and ability to complete everyday tasks. We’ve taken important lessons from established NPTs. But we’ve also expanded on them, using technology to create new and improved tests and automated clinical decision support.
Learn more about the science behind BrainCheck »
Additional contributions to this article by Huy Phi.
