← Podcasts & articles

Construct Validation of a 3D Printed Neonatal Thoracoscopic Simulator: Can it Measure Expertise?

articles · StayCurrentMD · Apr 04, 2021

Abstract

Background

Acquiring technical expertise for neonatal thoracoscopy is challenging. To address this, we designed a fully synthetic thoracoscopic simulator for which we established its construct validity.

Methods

Three thoracoscopic tasks were assessed: ring transfer, needle pass and incision of a blind upper esophageal pouch (EA cut). Participants watched instructional videos with accompanying written instructions for each task before having their attempt video recorded. All tasks were marked by three blinded paediatric surgeons using a modified Objective Structured Assessment of Technical Skills (OSATS). Scores were assessed using appropriate statistical analysis and inter-rater reliability was analysed by interclass correlation coefficient (ICC).

Results

23 participants completed the ring transfer and needle pass and 21 the EA cut: 5 experts (consultant surgeons), 5 intermediate (registrars on a training program) and 13 novices (medical students, house surgeons or non-training registrars). All three tasks distinguished between novice and intermediate/expert (ring transfer p=0.00001, needle pass p=0.0004 and EA cut p=0.0014 respectively). Interrater reliability was good for ring transfer and needle pass but poor for EA cut.

Conclusion

The tasks distinguished between novice and intermediate/expert but not between expert and intermediate. In needle pass and EA cut, there was a trend for the experts to score higher than intermediate participants. Ring transfer and needle pass tasks achieved construct validity, had good interrater reliability and were found to be useful in assessing a novice surgeon's progression towards the intermediate level. Distinguishing between intermediate and expert may require assessment of more complex tasks such as intracorporeal suturing and tying.

Level of evidence

2A

Open