A virtual reality simulator for hysteroscopic placement of tubal sterilization micro-inserts: the face and construct validity
© Springer-Verlag Berlin Heidelberg 2013
Received: 29 January 2013
Accepted: 28 March 2013
Published: 18 April 2013
This study investigated the validity of a virtual reality simulator for hysteroscopic tubal sterilization. Initially performed laparoscopically, the hysteroscopic sterilization method is becoming increasingly popular. An adequate training model could enhance one’s skills prior to the start of performing the procedure on the real patient. This prospective study (Canadian Task force II-2) enrolled 69 residents and gynecologists who were divided into three groups, based on vaginoscopic hysteroscopy and Essure® experience level: novices (N = 17), intermediates (N = 35), and experts (N = 17). Participants completed two cases on a virtual reality simulator (EssureSim™) in which four Essure® placements were performed. A questionnaire was completed to assess face validity, and reality scores were given on a 5-point Likert scale. Construct validity was represented by the ability of six simulator-derived parameters to significantly differentiate between different hysteroscopic experience levels. Reality of the sterilization procedure was scored with a median of 5.00 points on a 5-point Likert scale by all participants with prior sterilization experience. Of these participants, 95.5 % indicated the simulator as a useful preparation for real-time Essure® placement. The expert and intermediate group performed both cases significantly faster than novices (p = .001). The novices had a significantly longer path length in comparison to the other groups (p = .006). Analysis of the remaining parameters did not show a persistent ability to differentiate between experience levels. Satisfactory validity was demonstrated for the EssureSim™ by high reality scores and moderate ability to distinguish between different performance levels.
Female sterilization is the most common method of contraception worldwide . More than 600,000 tubal sterilizations are performed annually in the USA . Initially performed laparoscopically, the hysteroscopic sterilization method is becoming increasingly popular [3, 4].
Hysteroscopy is generally considered as a safe procedure with a low complication rate [5, 6]. Its practice ranges from diagnostics in an outpatient setting to a surgical alternative for many gynecological problems. Teaching hysteroscopy skills traditionally has been based on a mentored model, where trainees are exposed to procedures with the guidance of an experienced teacher. However, in recent years, the surgical volume has been limited by restrictions on resident working hours and less highly skilled teachers are available [7, 8]. This results in difficulties in acquiring sufficient skills in advanced endoscopic surgery [9, 10]. Effective usage of simulation and training models is a possible solution to this problem [9–11].
Development and validation research on training models and simulators has been mainly focused on laparoscopy. Training models allow a surgeon to safely overcome the learning curve of a new technique before practicing on a patient [9, 12, 13]. Virtual reality (VR) simulators especially, allow more independent instruction and objective immediate feedback for more reliable, unbiased assessment of psychomotor skills [9, 12, 14]. In addition, it allows for repeated practice without any risk to patients. Training on a VR system bypasses the ethical concerns associated with practice on animals or cadavers. Besides, many VR systems allow for practice at varying levels of difficulty and across a wide range of scenarios, thus accommodating trainees at many levels [14, 15].
Prior to implementation of a new training tool in a curriculum, evaluation and validation of the simulator and its parameters are mandatory [16–19]. Validity measures whether a simulator is actually teaching or measuring what it is intended to teach or measure . Different aspects of validity exist. Face validity refers to whether the model resembles the task or procedure it is aiming to train for, by determining the opinion of users on realism of the simulation. Objective approaches consist of construct and predictive validity. Construct validity refers to whether the model measures the quality or ability it is supposed to measure . In this regard, the simulator must be able to differentiate between the experienced and the inexperienced surgeon, or in addition, measure improvement in novices’ performance by training. Predictive validity is the extent to which the simulator predicts future performance by assessing whether the skills acquired on a simulator actually result in improved skills in patients in the real-time clinical setting [17, 20].
Excellent data are available to support the validity and effectiveness of VR training of surgical skills in general surgery [21–23], urology  as well as in gynecology [24, 25]. VR training leads to more efficient movements and less errors, which translates into less operating time and improved patient safety.
In comparison to laparoscopy, little work has been done regarding hysteroscopy training despite its upcoming use and applicability during the last decades. Several training methods have been designed, focusing mainly on the development of physical models and box trainers [26–28]. A collaboration between gynecologists and technicians in Switzerland led to the development of the Hysteroscopic Surgery Simulator System (HystSim™)—a VR simulator for hysteroscopic interventions. Face and construct validity have been established for a diagnostic training module [29, 30]. Recently, a new procedural training module became available by which the Essure® sterilization method can be practiced (EssureSim™).
The hysteroscopic sterilization method by Essure® Permanent Birth Control system (Conceptus; Mountain View, CA, USA) was approved in 2001 by the European Health Office and in 2002 by the U.S. Food and Drug Administration. Micro-inserts placed in both the tubal ostia cause a sterile inflammatory response of the intramural and isthmic parts of the Fallopian tube, thereby occluding the tubes within 3 months. Since the introduction of this method, it is performed by gynecologists around the world and has become an accepted alternative to laparoscopic sterilization. Initially taught with significant hands-on supervision, the EssureSim™ is developed to train gynecologists who want to start performing this procedure in a more efficient manner and without risks for the patient.
The aim of this study is to determine the face and construct validity of this VR training module for the hysteroscopic placement of tubal sterilization micro-inserts.
Between June 2010 and April 2011, 25 ob-gyn residents and 44 consultant gynecologists (N = 69) were randomly recruited at the Annual Meeting of the Dutch Society of Obstetrics and Gynecology and from a university hospital and a major teaching hospital in the Netherlands.
Given that hysteroscopic sterilization is performed as a type of therapeutic vaginoscopic hysteroscopy, without use of a speculum and tenaculum, three groups were made. This division was based on a combination of Essure® experience level and experience level in therapeutic vaginoscopic hysteroscopy. “Novices” (N = 17): never performed an Essure® placement nor a therapeutic vaginascopic hysteroscopy, “experts” (N = 17): performed >25 Essure® placements and >25 therapeutic vaginascopic hysteroscopies, “intermediates” (N = 35): any experience varying between a novice and expert. The assessment of the participants’ experience was made by self-estimated numbers of both procedures.
Participants completed a questionnaire immediately after completing the cases on the simulator. It included questions about participants’ demographics and experience level in hysteroscopy training, several hysteroscopy procedures, and hysteroscopic sterilization. The opinion of each participant was assessed with 14 questions about the simulator and sterilization module. These questions concerned the realism of the simulation and training capacities, and were presented on a 5-point Likert scale . Additionally, two statements were proposed for further opinion inquiry. These were answered with “agree,” “disagree,” or “no opinion.” Face validity was determined by analyzing the opinion of the participants with prior Essure® experience. In this manner, realism and training capacity of the simulator was evaluated only by the participants who had knowledge of the real-time procedure and who could make a comparison between both environments.
To investigate construct validity, the participants performed tasks on the simulator. To all participants, a standard introduction of the simulator and sterilization procedure was given. A familiarization with the VR simulator was executed, consisting of one tubal micro-insert placement in a uterus with normal tubes. In the first case (case 1), the participant performed a bilateral sterilization in a uterus with normal tubes, as shown in the animation (Online Resource 1). The second case (case 2) comprised a bilateral placement in a uterus of a more difficult level, because of the thickened endometrium of this uterus, decreased visibility, and slightly more lateral insertion of the tubes (Online Resource 2). All participants were supervised by one supervisor (J.A.J.), who gave answers to questions and gave instructions if one was not able to proceed.
Description of all parameters used
Time of the total procedure, from insertion of scope into cervix to removal of scope, in seconds
Path length of the tip of the hysteroscope in millimeters
Cumulative number of contacts of the scope with the cervix and uterine wall
Combination of number of trauma and the distension pressure of the fluids exerted on the uterine wall, given on a 10-point scale, 1 = extremely uncomfortable, 10 = no discomfort at all
1 to 8 coils of the micro-insert need to be visible after placement
Distension fluid used
The amount of distension fluid used in milliliters
Use of statistics
Data were analyzed using the statistical software package SPSS 17.0 (SPSS, Inc., Chicago, IL). Differences between the general demographics and performances between the three groups were analyzed using the Kruskal–Wallis test for nonparametric data. If the Kruskal–Wallis test resulted in a significant difference, then a comparison between two separate groups was done using the Mann–Whitney U test with post hoc Dunn’s (Bonferroni) correction. To verify the minimum sample size, a power analysis was performed. A total sample of 69 subjects achieves a power of >.80 with the Kruskal–Wallis test with a target significance of .05. The average within-group standard deviation assuming the alternative distribution is 1.0 (PASS 2008 NCSS; LCC, Kayville, UT). A p value of <.05 was considered to be statistically significant. Values are presented as medians with interquartile ranges unless stated otherwise.
Baseline characteristics of all participants
All participants (N = 69)
Novices (N = 17)
Intermediates (N = 35)
Experts (N = 17)
Age, median in years (IQR)
Gender, % male/female
Handedness, % right/left
Status, % resident/consultant
Hysteroscopy training courses, in hours (%)
Experience with virtual reality in general, in hours (%)
Experience with HystSim (%)
Number of therapeutic vaginoscopic hysteroscopies performed (%)
Number of Essure® placements performed (%)
Results face validity
Participants with prior Essure® experience (N = 22)
What is your opinion about the realism of the following items? (1 = not realistic…5 = very realistic)
In- and outflow valves
Quality of images
What is your opinion about the training capacity of the following items? (1 = very bad…5 = very good)
Training capacity in general
Statement 1: the HystSim™ offers procedural training of hysteroscopic skills
Agree: 100.0 %
Disagree: 0.0 %
No opinion: 0.0 %
Statement 2: the EssureSim™ offers a useful preparation for the real-time Essure® sterilization procedure
Agree: 95.5 %
Disagree: 4.5 %
No opinion: 0.0 %
Results of construct validity for each group
Novices (N = 17)
Intermediates (N = 35)
Experts (N = 17)
203.40, IQR 172.65–302.65
161.90, IQR 123.00–180.40
118.70, IQR 103.80–146.60
Path length (mm)
647.30, IQR 583.55–946.05
498.60, IQR 412.60–553.80
462.20, IQR 383.05–545.60
11.00, IQR 5.50–17.00
7.00, IQR 2.00–9.00
6.00, IQR 3.00–8.50
Patient comfort (10-point scale)
6.90, IQR 6.80–7.80
7.10, IQR 6.70–7.70
7.30, IQR 6.75–7.70
Distension medium (mL)
558.90, IQR 267.15–791.40
600.20, IQR 449.95–905.55
489.60, IQR 193.08–571.15
Correct placement left/right (percentage)
224.90, IQR 180.30–279.25
177.60, IQR 150.30–197.40
136.70, IQR 120.35–174.30
Path length (mm)
665.40, IQR 586.40–837.85
538.70, IQR 504.90–626.20
561.40, IQR 495.95–677.75
8.00, IQR 4.50–28.00
10.00, IQR 5.00–28.00
13.00, IQR 6.50–25.50
Patient comfort (10-point scale)
7.90, IQR 6.60–8.10
7.20, IQR 6.20–8.00
7.30, IQR 7.00–8.10
Distension medium (mL)
671.45, IQR 180.55–831.40
836.50, IQR 503.45–1141.63
635.55, IQR 438.48–873.38
Correct placement left/right (percentage)
The parameter task time was able to differentiate significantly between all groups in both cases. The novice group performed both cases significantly slower in comparison to the other groups (p = .001 for both cases). In addition, all groups required more time to finish the second case, a uterus of a more difficult level, in comparison to the first case.
Similarly, the parameter path length showed significant differences between groups in both cases. The novices had a significantly longer path length in comparison to the intermediate and expert group (case 1, p = .001 in both groups; case 2, p = .006 in comparison with the intermediate group).
In the first case, the parameter trauma displayed a significant difference between the novices and the intermediate group and a similar trend in comparison to the expert group. However, in the second case, a reversed (nonsignificant) effect is observed. The novice group achieved a median score of 8 contacts in comparison to 13 in the expert group. A similar contradictory trend in both cases is seen for the parameter patient comfort.
The analysis of the parameter distension medium did not show significant results, while the intermediate group used the largest amount of fluid in both cases. The last parameter, the number of correctly placed devices, did not differ significantly between the three groups and no specific trend could be observed. Both inexperienced and experienced participants were able to position the sterilization micro-inserts in a correct manner.
The aim of this study was to determine the validity of a new training module by which the Essure® sterilization method can be practiced on a commercially available VR simulator. We assessed the realism of the simulator by questionnaires (face validity) and determined the capacity of the simulator to distinguish between experienced and inexperienced hysteroscopists (construct validity). Face validity was established with high scores, while construct validity showed moderate results.
The study was preceded by a power analysis and contained a sufficient number of participants. One supervisor coached all participants to limit inter-supervisor bias.
According to the fact that hysteroscopic sterilization is usually performed as a type of therapeutic vaginoscopic hysteroscopy [32, 33], participants were grouped by their experience in both procedures.
In general, gynecologists with ample experience in performing hysteroscopies are considered experts. In the absence of generally accepted criteria for the classification of experience levels, we applied the arbitrary number of 0 and 25 therapeutic procedures to form three levels. Both the novice and expert group were of similar size (N = 17), whereas the intermediate group consisted of clearly more participants (N = 35), indicating that the majority of our study population had some or more therapeutic vaginoscopic hysteroscopy experience. Face validity was assessed by taking into account only the opinion of those participants who had tubal sterilization experience (performed ≥1 Essure®). In this manner, realism was evaluated only by those participants who had knowledge of the real-time procedure.
Not all performance parameters measured by the simulator were able to differentiate between participants with varying hysteroscopy experience. We hereby confirm findings of previous studies by Bajka et al.  and Panel et al. , who investigated the face and construct validity of the diagnostic and sterilization module on this hysteroscopy simulator, respectively. Both studies found that less than half of all used parameters significantly correlated with hysteroscopy experience.
Possible reasons for the current study could be the fact that an active coaching strategy was adopted, by which the supervisor was easily accessible for questions and practical advice. It should be emphasized that this might have reduced possible differences between experienced and inexperienced participants. Another reason may possibly be the lack of haptic feedback, which might impair especially the experienced hysteroscopist. Not only the visual aspect but also haptic feedback gives guidance to the operator for efficient and safe hysteroscopies. Both parameters trauma and patient comfort, which is a combination of number of trauma and the distension pressure of the fluids exerted on the uterine wall, might not be able to differentiate in a consequent manner between novices and experts as a result.
Also, the parameter distension medium should be interpreted with caution due to a number of missing data, as the simulator tended not to register fluid use during all placements. Further refinement of the software and scoring systems is therefore necessary. Incorrectly placed devices were mainly caused by placing them too deeply into the tubes, whereby the coils were not visible in the uterine cavity after deployment. The fact that novices scored high percentages for correctly placed devices might be explained by the observation that those participants without any hysteroscopy experience tended to adhere more closely to any practical advice given during device placement, in contrast to the more experienced groups. In addition, one needs to realize that the assessment of the participants’ experience was self-reported and therefore is subject to recall bias. Also, the division into three levels of experience could be seen as a potential source of bias since the norm of both sufficient hysteroscopy and sterilization experience must be met to be classified as an expert.
One could ask oneself in general if a slower performance with more use of distension medium is not preferred when a higher correct placement rate is achieved with better patient comfort. The parameters used by this simulator might not be the only measures of hysteroscopy performance. For procedural exercises, one could design a global rating scale (GRS), which is a scoring system that is built on certain clinically relevant performance parameters [35, 36].
In conclusion, this simulator received the highest scores regarding both procedure realism and training capacity. It was able to differentiate between subjects with varying hysteroscopy experience for two out of six parameters. We consider this study as an essential basic step in the validation cascade of a VR simulator for training operative hysteroscopies and for hysteroscopic sterilization in particular. Also, we believe this simulator could be suitable for future training of hysteroscopic sterilization skills, after further refinement of the software. The next important step would be the investigation of the learning curve, with concurrent use of a clinically relevant GRS. The learning curve is a vital part of construct validity and in addition addresses implementation of the simulator in hysteroscopy training curricula. The learning curve could possibly indicate the necessary number of training sessions contributing to efficient and safe daily practice. Assessing predictive validity would be a last and ideal step in the validation cascade, providing data to which extent the simulation can predict real-life hysteroscopic performance.
We would like to thank the residents and consultants who voluntarily participated in this study. We would like to thank Ass. Prof. M.J.C. (René) Eijkemans from the Julius Centre for Health Sciences and Primary care, University of Utrecht, the Netherlands for his help with the statistical analysis. No funding resources or compensation disclosure is applicable.
Potential conflict of interest
J.A.J. has nothing to disclose. S.V. has received honorariums for training sessions on the Essure® device for Conceptus and was involved in the development of the EssureSim™. F.J.B. has nothing to disclose. H.W.R.S. has nothing to disclose. No funding was received for this study.
- Peterson HB (2008) Sterilization. Obstet Gynecol 111:189–203PubMedView ArticleGoogle Scholar
- Chan LM, Westhoff CL (2010) Tubal sterilization trends in the United States. Fertil Steril 94:1–6PubMedView ArticleGoogle Scholar
- Levie M, Chudnoff MS (2011) A comparison of novice and experienced physicians performing hysteroscopic sterilization: an analysis of an FDA-mandated trial. Fertil Steril 96:643–648PubMedView ArticleGoogle Scholar
- Shavell VI, Abdallah ME, Shade GH Jr, Diamond MP, Berman JM (2009) Trends in sterilization since the introduction of Essure hysteroscopic sterilization. J Minim Invasive Gynecol 16:22–27PubMedView ArticleGoogle Scholar
- Jansen FW, Vredevoogd CB, van Ulzen K, Hermans J, Trimbos JB, Trimbos-Kemper TC (2000) Complications of hysteroscopy: a prospective, multicenter study. Obstet Gynecol 96:266–270PubMedView ArticleGoogle Scholar
- Aydeniz B, Gruber IV, Schauf B, Kurek R, Meyer A, Wallwiener D (2002) A multicenter survey of complications associated with 21,676 operative hysteroscopies. Eur J Obstet Gynecol Reprod Biol 104:160–164PubMedView ArticleGoogle Scholar
- Blanchard MH, Amini SB, Frank TM (2004) Impact of work hour restrictions on resident case experience in an obstetrics and gynecology residency program. Am J Obstet Gynecol 191:1746–1751PubMedView ArticleGoogle Scholar
- Van Dongen H, Kolkman W, Jansen FW (2006) Hysteroscopic surgery: perspectives on skills training. J Minim Invasive Gynecol 13:121–125PubMedView ArticleGoogle Scholar
- Gallagher AG, Cates CU (2004) Virtual reality training for the operating room and cardiac catheterization laboratory. Lancet 364:1538–1540PubMedView ArticleGoogle Scholar
- Shoukrey MN, Fathulla BI, Al Samarrai T (2008) Hysteroscopy training in the UK: the trainees’ perspective. Gynecol Surg 5:213–219View ArticleGoogle Scholar
- Chudnoff SG, Liu CS, Levie MD, Bernstein P, Banks EH (2010) Efficacy of a novel educational curriculum using a simulation laboratory on resident performance of hysteroscopic sterilization. Fertil Steril 94:1521–1524PubMedView ArticleGoogle Scholar
- Chou B, Handa VL (2006) Simulators and virtual reality in surgical education. Obstet Gynecol Clin North Am 33:283–296PubMedView ArticleGoogle Scholar
- Ahlberg G, Enochsson L, Gallagher AG et al (2007) Proficiency-based virtual reality significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193:797–804PubMedView ArticleGoogle Scholar
- Palter VN, Grantcharov T (2010) Virtual reality in surgical skills training. Surg Clin North Am 90:605–617PubMedView ArticleGoogle Scholar
- Neary PC, Boyle E, Delaney CP, Senagore AJ, Keane FB, Gallagher AG (2008) Construct validation of a novel hybrid virtual–reality simulator for training and assessing laparoscopic colectomy; results from the first course for experienced senior laparoscopic surgeons. Surg Endosc 22:2301–2309PubMedView ArticleGoogle Scholar
- Verdaasdonk EG, Stassen LP, Monteny LJ, Dankelman J (2006) Validation of a new basic virtual reality simulator for training of basic endoscopic skills: the SIMENDO. Surg Endosc 20:511–518PubMedView ArticleGoogle Scholar
- McDougall EM (2007) Validation of surgical simulators. J Endourol 21:244–247PubMedView ArticleGoogle Scholar
- Schout BM, Hendrikx AJ, Scheele F, Bemelmans BL, Scherpbier AJ (2010) Validation and implementation of surgical simulators: a critical review of present, past, and future. Surg Endosc 24:536–546PubMedView ArticleGoogle Scholar
- Gallagher AG, Ritter EM, Satava RM (2003) Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc 17:1525–1529PubMedView ArticleGoogle Scholar
- Schout BM, Ananias HJ, Bemelmans BL et al (2009) Transfer of cysto-urethroscopy skills from a virtual–reality simulator to the operating room: a randomized controlled trial. BJUI Int 106:226–231View ArticleGoogle Scholar
- Seymour NE, Gallagher AG, Roman SA et al (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–463PubMedView ArticleGoogle Scholar
- Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91:146–150PubMedView ArticleGoogle Scholar
- Van Sickle KR, Ritter EM, Baghai M et al (2008) Prospective, randomized, double-blind trial of curriculum-based training for intracorporeal suturing and knot tying. J Am Coll Surg 207:560–568PubMedView ArticleGoogle Scholar
- Larsen CR, Soerensen JL, Grantcharov TP et al (2009) Effect of virtual reality training on laparoscopic surgery: randomized controlled trial. BMJ 338:b1802. doi:10.1136/bmj.b1802 PubMedView ArticleGoogle Scholar
- Schreuder HW, van Dongen KW, Roeleveld SJ, Schijven MP, Broeders IA (2009) Face and construct validity of virtual reality simulation of laparoscopic gynecologic surgery. Am J Obstet Gynecol 200:540.e1-8PubMedView ArticleGoogle Scholar
- Burchard ER, Lockrow EG, Zahn CM, Dunlow SG, Satin AJ. Simulation training improves resident performance in operative hysteroscopic resection techniques. Am J Obstet Gynecol 197:542.e1-4.Google Scholar
- Kingston A, Abbott J, Lenart M, Vancaillie T (2004) Hysteroscopic training: the butternut pumpkin model. J Am Assoc Gynecol Laparosc 11:256–261PubMedView ArticleGoogle Scholar
- Wallwiener D, Rimbach S, Bastert G (1994) The HysteroTrainer, a simulator for diagnostic and operative hysteroscopy. J Am Assoc Gynecol Laparosc 2:61–63PubMedView ArticleGoogle Scholar
- Bajka M, Tuchschmid S, Streich M, Fink D, Székely G, Harders M (2009) Evaluation of a new virtual–reality training simulator for hysteroscopy. Surg Endosc 23:2026–2033PubMedView ArticleGoogle Scholar
- Bajka M, Tuchschmid S, Fink D, Székely G, Harders M (2010) Establishing construct validity of a virtual–reality training simulator for hysteroscopy via a multimetric scoring system. Surg Endosc 24:79–88PubMedView ArticleGoogle Scholar
- Mattel MS, Jacoby J (1972) Is there an optimal number of alternatives for Likert-scale items? J App Psychol 56:506–509View ArticleGoogle Scholar
- Miño M, Arjona JE, Cordón J, Pelegrin B, Povedano B, Chacon E (2007) Success rate and patient satisfaction with the Essure sterilisation in an outpatient setting: a prospective study of 857 women. BJOG 114:763–766PubMedView ArticleGoogle Scholar
- Ubeda A, Labastida R, Dexeus S (2004) Essure: a new device for hysteroscopic tubal sterilization in an outpatient setting. Fertil Steril 82:196–199PubMedView ArticleGoogle Scholar
- Panel P, Bajka M, Le Tohic A, Ghoneimi AE, Chis C, Cotin S (2012) Hysteroscopic placement of tubal sterilization implants: virtual reality simulator training. Surg Endosc. doi:10.1007/s00464-011-2139-6 PubMedGoogle Scholar
- Reznick RK (1993) Teaching and testing technical skills. Am J Surg 165:358–361PubMedView ArticleGoogle Scholar
- Martin JA, Regehr G, Reznick R et al (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278PubMedView ArticleGoogle Scholar