External Validation of an AI mHealth Tool for Gingivitis Detection among Older Adults at Daycare Centers: A Pilot Study

OBJECTIVES: Periodontal disease is a significant public health concern among older adults due to its relationship with tooth loss and systemic health disease. However, there are numerous barriers that prevent older adults from receiving routine dental care, highlighting the need for innovative screening tools at the community level. This pilot study aimed first, to evaluate the accuracy of GumAI, a new mHealth tool that uses AI and smartphones to detect gingivitis, and the user acceptance of personalized oral hygiene instructions provided through the new tool, among older adults in day-care community centers.

METHODS: Participants were invited from 3 day-care community centers. Intraoral photographs were captured and assessed by both GumAI (test) and a panel consisting of 2 calibrated periodontists and a dentist (benchmark). Mean sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and F1 score were calculated to determine GumAI's diagnostic performance in comparison to the benchmark. User acceptance with this tool was assessed using 2 Rasch Theory-based 5-point Likert-type questions.

RESULTS: 44 participants were recruited out of 80 invited older adults. GumAI demonstrated a sensitivity of 0.93 and specificity of 0.50 compared to the panel's assessments, with a PPV of 0.90 and NPV of 0.56. The accuracy and F1 scores were 0.85 and 0.91, respectively. All participants expressed high acceptance of the process.

CONCLUSION: GumAI demonstrates high sensitivity, PPV, accuracy, and F1 score compared to the panel's assessments but falls relatively short in specificity and NPV. Despite this, the tool was highly accepted by older adults, indicating its potential to enhance gingivitis detection and oral hygiene management in community settings. Further refinements are necessary to improve specificity and validate usability measures.

CLINICAL RELEVANCE: This study may pave the way for broader applications of mHealth systems in community settings, enabling greater health coverage and addressing oral health disparities.