Comparing AI-generated and human-written A3/A4 medical licensing questions
Official title: Scientific Validity Assessment and Optimization of AI-Generated A3/A4 Type Questions for the Chinese Medical Licensing Examination: An Empirical Analysis Based on the "Answering-Generating" Closed Loop and Transformation of Competency Assessment
Run by Guangdong Provincial People's Hospital · NCT07505862 · Data updated 2026-11-12
The project will test whether AI-created A3/A4‑type questions perform like human-written ones for medical students in standardized residency training preparing for the Chinese Medical Licensing Examination.
Quick facts
| Study type | Observational |
|---|---|
| Ages | 18 Years to 60 Years |
| Sex | All |
| Enrollment | 20 (estimated) |
| Sponsor | Guangdong Provincial People's Hospital Academic / other |
| Locations | 1 site (Guangzhou, Guangdong) |
| Status | Recruiting |
| Started | October 1, 2025 |
| Main results expected | March 30, 2026 |
| Study ends | March 30, 2026 |
| Registry record updated | April 1, 2026 |
| Trial ID | NCT07505862 on ClinicalTrials.gov |
Is this trial a fit for you?
✓Likely a good fit
Ideal participants are medical students enrolled in a Standardized Residency Training program who hold a Bachelor of Medicine degree or higher and can use digital platforms to complete assessments.
✕Probably not a fit
Those involved in training the AI or creating the human question bank, or individuals unable to complete digital assessments due to impairments or severe illness, are unlikely to benefit from participation.
A plain-English reading of the study's rules. Only the study team can confirm whether you qualify.
What this trial studies
This cross-sectional, two-stage project combines quantitative analysis with qualitative review to compare AI model outputs to human-authored A3/A4 exam items. Phase I performs a randomized model performance comparison, and Phase II conducts an item quality evaluation in which AI-generated and human-written items are interspersed for blinded testing. Single-blinding, randomization, and standardized procedures are used to reduce bias and improve reproducibility. Outcomes include item performance metrics, participant scoring, and qualitative feedback to guide optimization of AI-generated content.
Why it matters
Potential benefit: If successful, AI could help produce high-quality licensing exam items faster, easing test development and expanding practice resources for trainees.
How similar studies have gone: Similar work on AI-generated exam questions has shown mixed results with some comparable performance but remaining quality gaps, so the approach is partially validated but still evolving.
Who can join: the full rules
Show full inclusion / exclusion criteria
Inclusion Criteria: * 1.Professional Status: Medical students currently enrolled in a Standardized Residency Training (SRT) program. 2.Educational Background: Holders of a Bachelor of Medicine degree or higher, with foundational clinical knowledge. 3.Informed Consent: Voluntarily participate in the study and provide written informed consent. 4.Technical Competency: Proficient in using digital platforms to complete assessments and scoring. Exclusion Criteria: * 1.Conflict of Interest: Individuals involved in the AI model training, prompt engineering, or the creation of the human-authored question bank for this study. 2.Inability to Complete: Presence of visual/auditory impairments or severe illness that precludes completion of the assessment within the specified time. 3.Investigator's Discretion: Any other condition that, in the opinion of the investigator, renders the participant unsuitable for the study.
Where it's running
-
Guangdong Provincial People's Hospital
Guangzhou, Guangdong, China Recruiting
Study contacts
- Study coordinator: Zhuoyi Chen MD
- Email: 18737552662@163.com
- Phone: +86 18737552662
Similar studies
Other studies for the same condition or treatment. Recruiting studies are listed first.
-
AI support in emergency medicine helps doctors make differential diagnoses
This study tests whether a generative AI tool can help emergency doctors arrive at more accurate differential diagnoses for adult patients at Karolinska University Hospital.
-
AI-based communication practice with stories from real people for pediatric nursing students
Aimed at helping nursing students test a new teaching method that uses AI conversation simulations plus real-person narratives to improve clinical communication, empathy, and emotion regulation.
-
AI-assisted cardiac CT to predict coronary artery disease risk
This project will try using AI-enhanced heart CT scans to better predict future coronary events in adults who are already getting a clinical cardiac CT.
-
AI estimation of length, weight, and head size in newborns and infants up to 6 months
This project will test whether AI can estimate babies' length, weight, and head circumference from photos or videos for infants from birth to 6 months old.
-
AI tool to measure jaw, neck, and shoulder movement for oral cancer care
This project tests an AI system that automatically measures jaw, neck, and shoulder movement for people with or recovering from oral cancer and compares its readings to manual measurements.
-
Predicting ICL postoperative vault and anterior chamber tomography from preoperative corneal scans
This will see if an AI model can use preoperative corneal topography to predict postoperative ICL vault and generate anterior chamber images for people planning ICL surgery.
How to take part
- Review the eligibility criteria above with your treating physician.
- Check the official listing on ClinicalTrials.gov for the latest contact details and recruitment status.
- Contact the study coordinator or principal investigator to ask for pre-screening. Pre-screening is free and never obligates you to enroll.