Treating old data with new tricks: Exploring reliability between humans and artificial intelligence

Paulas et al. (2013) noted that technological advances like transcription tools and data analytic software are reshaping qualitative methodological practices. They noted that qualitative analytic software with generative artificial intelligence (AI) tools has raised further issues for qualitative researchers (Paulus & Marone, 2024). Sun, Kim, Kim, & Choi, (2025) compared findings from qualitative researchers’ coding with those generated from AI-augmented coding using ChatGPT and revealed that although many of the ChatGPT themes closely aligned with the researchers, there were significant differences, as the researchers identified important themes missed by AI. Gillen (2024) cautions using AI qualitative analytic tools given they lack a human perspective which fails to incorporate the researcher’s positionality, a cornerstone of qualitative methodology. This presentation will highlight the results of a reanalysis of qualitative data using self-reported reasons why individuals diagnosed with severe mental illness were not receiving any treatment to those obtained from AI-enhanced analytic software.