Comparable signs can complicate seizure prognosis, prompting researchers to check whether or not AI can establish helpful clues in routine medical notes earlier than confirmatory monitoring.
Examine: Early epilepsy detection from digital well being data with massive language fashions. Picture Credit score: Only_NewPhoto / Shutterstock
In a current examine printed within the journal npj Digital Drugs, researchers describe the event and validation of “EpiScreen”, a novel synthetic intelligence (AI) system designed to differentiate epilepsy from psychogenic non-epileptic seizures (PNES) utilizing scientific notes from routinely documented digital well being data (EHRs).
The retrospective, two-institution examine used 13,633 affected person data to evaluate whether or not fine-tuned massive language fashions (LLMs) may precisely differentiate epilepsy from PNES.
EpiScreen outperformed the comparability fashions, attaining an space below the receiver working attribute curve (AUC) of as much as 0.992 in within-institution testing and 0.980 in cross-institutional testing. AUC measures a mannequin’s potential to differentiate between the 2 circumstances throughout completely different classification thresholds.
In a retrospective case-review evaluation, EpiScreen improved clinicians’ classification accuracy, supporting its potential in future epilepsy screening functions.
Background
Epilepsy is clinically outlined as a power neurological dysfunction characterised by recurring, unprovoked seizures. Seizures come up from sudden, irregular electrical discharges within the mind, with public well being data estimating that roughly 50 million people stay with the situation globally.
Epilepsy’s prognosis will be difficult by PNES, additionally termed practical or dissociative seizures. These contain paroxysmal occasions (sudden, momentary seizure-like episodes) that resemble epileptic seizures however aren’t brought on by epileptic electrical exercise.
Since epilepsy and PNES have completely different underlying mechanisms and administration wants, well timed differentiation and correct diagnoses are essential for applicable care. Misdiagnosis stays widespread, with prior research indicating that 20%–30% of sufferers referred to tertiary facilities for presumed drug-resistant epilepsy obtain a PNES prognosis.
Extended inpatient video-electroencephalography (vEEG) is the diagnostic gold normal for distinguishing epileptic seizures from PNES. The strategy data synchronized mind waves alongside behavioral knowledge. However vEEG is dear and time-consuming, usually costing over US$15,000 per admission in the US, with reported wait instances of 8 to 12 weeks that fluctuate throughout healthcare techniques.
Past direct bills, diagnostic delays can result in pointless remedy and delay applicable care. Researchers are investigating whether or not routinely collected scientific notes can assist earlier screening for each circumstances.
Routine scientific notes include in depth diagnostic indicators, however seizure semiology (the signs, indicators, and sequence of a seizure-like occasion) overlaps significantly throughout problems. Earlier narrative-based approaches have proven restricted diagnostic efficiency.
Concerning the examine
The researchers tailored present LLMs to handle this diagnostic bottleneck and differentiate between epilepsy and PNES utilizing solely unstructured scientific narratives.
The system was named “EpiScreen” and was educated and evaluated utilizing a collated dataset comprising 3,451 data from the Beth Israel Deaconess Medical Middle’s Medical Data Mart for Intensive Care IV (MIMIC-IV) database and 10,182 data from the College of Minnesota (UMN) Scientific Information Repository. Labels mirrored clinicians’ last documented diagnoses, and we processed notes to approximate info obtainable earlier than prognosis. Sufferers with coexisting epilepsy and PNES had been excluded.
EpiScreen was developed by instruction fine-tuning open-source LLMs, together with Qwen-2.5-32B, DeepSeek-R1-32B, Llama-3.3-70B, and Med42-70B, utilizing Quantized Low-Rank Adaptation (QLoRA). This technique reduces the reminiscence and coaching calls for of adapting massive fashions.
To clarify the mannequin, the examine used sentence-level Built-in Gradients (IG), a mathematical approach that computes attribution scores to focus on the sentences most influential within the algorithm’s choices throughout 11 predefined scientific function classes.
Lastly, the examine carried out a retrospective clinician-AI evaluation through which 4 medical groups (six neurology residents and two senior attending physicians) evaluated 300 MIMIC-IV take a look at circumstances. Every pair reached choices by consensus.
Examine findings
EpiScreen persistently outperformed standard supervised baselines, together with ClinicalBERT, attaining AUCs of 0.891 on MIMIC-IV and 0.976 on UMN for its Med42-based variant. Qwen- and DeepSeek-based variants reached 0.992 on UMN.
Cross-institutional evaluations additionally outperformed the comparability fashions, with the very best AUCs reaching 0.875 on MIMIC-IV and 0.980 on UMN. These evaluations examined fashions on data from establishments that didn’t provide their coaching knowledge.
Within the clinician evaluation, standalone EpiScreen achieved 81.0% diagnostic accuracy, in contrast with unaided senior attendings (81.7%) and unaided neurology residents (76.3%–79.3%). These variations weren’t statistically vital. Offering clinicians with AI predictions and sentence attributions improved resident-group accuracy to 87.7%–89.0% and attending accuracy to 89.3%. The biggest enchancment among the many reported group accuracies was about 12.7 proportion factors.
Lastly, error-pattern analyses confirmed that AI help decreased resident Group A’s false negatives (missed epilepsy circumstances) from 36 to 14, whereas the attending pair decreased false positives from 37 to 19.
Not all diagnostic labels had been vEEG-confirmed, however sensitivity analyses in confirmed circumstances supported the findings. Documentation bias stays doable, and different seizure mimics, together with syncope and sleep problems, weren’t evaluated.
Conclusions
The findings assist additional analysis of EpiScreen as a neurological decision-support software. The authors counsel utilizing it throughout or after neurology referral, earlier than confirmatory vEEG, to assist earlier evaluation and assist clinicians prioritize pressing monitoring. The authors suggest decreased diagnostic delays, prices, and pointless antiseizure medicine publicity, however these stay advantages quite than measured outcomes.
Future investigations should prioritize potential, real-time trials to verify how this collaborative AI system performs in real-world scientific settings utilizing minimally curated notes. EpiScreen is meant to assist scientific judgment alongside definitive testing.
Sources:
Journal reference:
