CogAI

Research
The CogAI assessment is evidence based. Development of the CogAI Psycholobot commenced in early 2020. By the end of 2021 the CogAI psycholbot was fully developed and the CogAI was ready to go.

In 2022, three research students (2 Psychology Honours students and 1 Clinical Psychology Masters student) from Australian Catholic University (ACU) conducted two clinical trials on the CogAI and other research looking at the reliability and validity of the CogAI. The clinical trials were conducted under a Clinical Trials Notification (CTN) with the Australian Therapeutic Goods Administration (TGA) and were approved by the ACU Human Research Ethic Committee (ACU-HREC). The research was supervised by Dr Kirsten Chalinor (ACU), Dr Megan Willis (ACU) and Dr Tony Florio (ACU, Florio Research and developer of the CogAI).

In brief these studies found evidence for reliability and validity of the CogAI:

Katrina Swavley (Psych Hons candidate), in a one week test-retest reliability trial found a reliability co-efficient of 0.76 (95% CI: .65 to .84, n=82). Using linear regression, this study also found the relation ship between 2 CogAI assessments 1 week apart had slope of 1 and an intercept of 0, indicating that on average there was perfect agreement. Comparing predictive classifications (no dementia next 5 years, yes/no) made by the two CogAIs there was 83% agreement and a Kappa of .65.
Anastasia Serafimovska (Clin Psych Masters candidate), conducted a concurrent validity trial of the CogAI, a comparison of the CogAI (administered by a psycholobot) with the TICS-M test (administered by a psychologist, with a one week interval between assessments, blind assessment and tests presentation in a randomised counterbalanced order. This study found a validity correlation of r = .81 (95% CI: .73 to .87, n=100). Using linear regression, this study also found the relation ship between the CogAI and TICS-M assessments 1 week apart had slope of 1 and an intercept slightly larger than 0, indicating that on average there was near perfect agreement. Comparing predictive classifications (no dementia next 5 years, yes/no) made by the two assessments (CogAI and TICS-M) there was 88% agreement and a Kappa of .73.
Alice Ao (Psych Hons candidate) conducted a construct validity study of the CogAI by replicating 5 previous studies which used the TICS-M and examing if the findings were concordant or not. The dataset was created by pooling CogAI records (n=181) from the previous reliability (n=81) and validity studies(n=100). Some of the previous TICS-M findings were replicated with CogAI, but others were not. However the CogAI sample was much younger than the samplres used in the previous TICS-M studies and the CogAI sample did not contain participants with cognitive disorders (such as MCI or Dementia), whereas the TICS-M study samples did. The pattern of concordance and discordance of findings in this construct validity study make sense, once sample differences are taken into account.

The results of these studies have been presented to professional peers for scrutiny at the Australian Psychological Society, Conference of Clinical Neuropsychology, held at Manly NSW, early November 2022.

The three studies are currently in the process of being submitted for publication in a peer reviewed scientific journal. Further details to be posted here when they are available.

The two links below are of the CogAi presentation at the APS Clinical Neuropsychology Conference

Watch our presentation on video - 25 minutes Look at our presentation slides

The CogAI was developed at the Florio Research Cognitive Science Research Laboratory at Abbotsford NSW.