Hysteroscopy training and learning curve of 30° camera navigation on a new box trainer: the HYSTT

Abstract

Despite the upcoming use of hysteroscopy and increased applicability during the last decades, little work has been done regarding the development of hysteroscopic training models in comparison to laparoscopy. Camera navigation is often perceived to be an easy task, but it is far from an innate ability, especially when an angled optic is used. This study investigated the learning curve of hysteroscopic 30° camera navigation on a new box trainer: Hysteroscopic Skills Training and Testing (HYSTT). This prospective study (Canadian Task Force II-2) enrolled 30 novices (medical students) and ten experts (gynecologists who had performed >100 diagnostic 30° hysteroscopies). All participants performed nine repetitions of a 30° camera exercise on the HYSTT. Novices returned after 2 weeks and performed a second series of five repetitions to assess retention of skills. The parameter procedure time and structured observations on performance using the Global Rating Scale provided measurements for analysis. The learning curve is represented by improvement per procedure. Two-way repeated-measures analysis of variance was used to analyze learning curves. Effect size (ES) was calculated to express the practical significance of the results (ES ≥0.50 indicates a large learning effect). For both parameters, significant improvements were found in novice performance within nine repetitions. Moderate to large learning effects were established (p < 0.05; ES 0.44–0.71). Retention of skills and prolonged learning curves were observed. The learning curve, established in this study, of hysteroscopic 30° camera navigation skills on the HYSTT box trainer, indicates a good training capacity and provides the first step towards recommended implementation into a training curriculum.

Background

The exponential growth of minimally invasive surgery and the limited resident availability for educational endeavors by work hour restrictions are recent changes in modern medicine that have led to the increasing demand of valid simulation training models [1]. For developing endoscopic skills, simulators offer a safe, effective and attractive way of repeatedly training these skills without causing discomfort or harming patients [2, 3]. Thus far, the beneficial results of simulator training are mainly derived from studies on laparoscopy [4, 5].

Hysteroscopy is generally considered as a safe procedure with a low complication rate [6, 7]. Its practice ranges from diagnostics in an outpatient setting to a surgical alternative in the operation room for many gynecological problems. Despite the upcoming use of hysteroscopy and increased applicability during the last decades, little work has been done regarding the development of hysteroscopic training models in comparison to laparoscopy.

Recently, the Hysteroscopic Skills Training and Testing (HYSTT) method has been developed under auspices of the European Academy of Gynaecological Surgery (Leuven, Belgium). This box trainer aims at practicing camera navigation skills with a 30° angled hysteroscope. Camera navigation is often perceived to be an easy task, but it is far from an innate ability. Psychomotor skills need to be learned to overcome the barriers that are known for endoscopic skills in general, namely the fulcrum effect, a two-dimensional environment, a fixed access point, and decreased range of motion [8]. In addition, skills unique to camera navigation include maintaining a correct horizontal axis while centering the operative field and holding a steady image. Further dexterity and knowledge is required for correct use of the additional degrees of freedom afforded by an angled scope [8–10], which is used routinely in many hysteroscopic procedures [9, 11]. In a national UK survey among gynecologists, a disappointing percentage of 25.8% of all responders that use a 30° hysteroscope showed understanding of the principles of 30° angled view [9].

Camera navigation is a basic and essential skill for performing hysteroscopic procedures, especially when an angled scope is used. The HYSTT box trainer might provide a simple, effective and feasible answer to the need for training this skill.

The objective of the present study was to investigate the effectiveness of repetitive training on the HYSTT. This was sought to be achieved by determining the learning curve of novice participants and by investigating whether novices can improve and retain their skills, and whether they can approximate the expert level.

Methods

Participants

From April to June 2012, 30 novices and ten experts voluntarily conducted a series of repetitions on the HYSTT. Medical students of the University of Utrecht participated as novices, during or after their gynecology internship. The novices had a basic understanding of hysteroscopy but had never previously performed nor assisted in a hysteroscopic procedure. All novices were invited to participate via oral and written means, and all agreed. Ten gynecologists served as experts to set a reference for novice performance. For the present study, a gynecologist was considered a hysteroscopy expert after performing >100 diagnostic hysteroscopies with a 30° scope and still practicing diagnostic hysteroscopy on a weekly base. All gynecologists were personally invited and all agreed to participate. None had any experience of performing hysteroscopy on this box trainer.

The study was exempt from Institutional Review Board approval, since no potential harm could be done to humans or nonhumans. All participants gave oral consent prior to the start of the study.

Box trainer

The HYSTT has been developed under auspices of the European Academy of Gynaecological Surgery (Leuven, Belgium) and consists of a plastic uterus model in which 14 numbers and characters are placed at 14 anatomical locations, known as: isthmus anterior/posterior/left/right, mid anterior/posterior/left/right, fundus anterior/posterior, cornua left/right and tubal ostium left/right. Six models are available in which each location contains a different number or character (model A–F). This plastic uterus is placed in a silicone model of a vulva, which in turn is situated in a plastic model of a female pelvis (Fig. 1). A 30° hysteroscope (Karl Storz diagnostic continuous-flow) connected to a video-camera, light source and monitor (Telepack, Karl Storz) were used.

Fig. 1

Set-up Hysteroscopic Skills Training and Testing (HYSTT) box trainer

Exercises

Beforehand, a survey was administered to obtain baseline characteristics. The novices received a short standardized oral introduction on hysteroscopy, the box trainer and study protocol. The experts received a standardized oral introduction on the box trainer and study protocol. Figure 2 displays the scheme of exercises per group. One-minute practice time was given to each participant to obtain familiarization with the HYSTT model. One investigator (C.J.T.) supervised all tests to limit inter-supervisor bias. Both groups conducted a series of nine exercises. In detail, the exercise was as follows: the supervisor read out an anatomical location (e.g., fundus posterior), after which the participant had to navigate to that specific location and visualize the associated number or character within a black circle with a diameter of 2.5 cm. Once this was correctly and readably visualized and named, the next command was read out. Video 1 (Supplemental Material) shows a short display of the model and exercise. During each repetition, the participants had to identify as many targets in a correct manner with a maximum of 14. Each repetition ended after 3 min, after which the total number of correctly visualized objects was noted. We chose to end each repetition after 3 min, as specified by the European Academy and because we wanted to limit the training duration per participant to 30 min per session, to optimize the concentration of the subjects. If a participant identified all 14 objects within 3 min, the time to finish the exercise was marked. Each repetition contained a completely different sequence of commands and uterus model A was switched to model B after five repetitions for every participant. This switch was performed since five different command sequences were available per model.

Fig. 2

Scheme of exercises. Note: of all repetitions, the even-numbered repetitions were meant for training by the supervisor; during the odd-numbered repetitions no training was provided because these performances were used solely for data analysis

Of all nine repetitions, the first, third, fifth, seventh and ninth repetitions were used for data analysis. The other repetitions were meant for training by the supervisor and consisted of answering questions of the participants and giving tips and tricks on the procedure. For this reason, the even-numbered repetitions had a different duration and goal and were consequently excluded from further analysis. During the odd-numbered repetitions, no questions could be asked nor were any tips given because these performances were used solely for data analysis. To assess retention of skills, after 2 weeks, novices returned for a second series of five repetitions (model C). The first, third and fifth repetitions were used for data analysis. The other repetitions were meant for training by the supervisor and were excluded from further analysis.

Outcome measures

The main outcome parameter was procedure time, measured as time needed to finish a repetition, by identifying all 14 signs with a limit of 3 min. If less than 14 signs were identified in 3 min, the total number of correctly visualized signs (n) in 3 min was recorded by the investigator. This score was then converted to the parameter time using the following formula: 180 s × (14/n) = score (in seconds). Participants were not assessed only by procedure time because performing a procedure very fast does not necessarily mean it is performed properly and/or with good results. For that reason, a 5-point Global Rating Scale (GRS) was used to assess competence from another (clinical) perspective. The GRS was adjusted for hysteroscopic 30° camera navigation training (Fig. 3) and has not yet been validated [12]. Aspects that were rated included respect for tissue, handling of the hysteroscope, time and motion, flow and forward planning and procedure knowledge. Blinding was not possible due to the clear differences between age and status of the groups and the necessity to score both the simulator screen and the participant behavior.

Fig. 3

Global Rating Scale, adjusted for hysteroscopic 30° camera navigation training

Expert opinion

To investigate the expert opinion on this new box trainer, the experts completed a questionnaire at the end of the training session. The experts rated six statements on a 5-point Likert scale, concerning the applicability of the HYSTT for testing and training camera navigation, for training residents and/or medical students and for learning anatomy. The experts also valued the realism of the HYSTT in simulating a diagnostic hysteroscopy.

Statistical analysis

Data were analyzed using commercially available software (SPSS version 20.0; SPSS, Inc., Chicago, IL, USA). No power analysis was performed prior to the study. To analyze the improvement within the novice group, a sample size of 30 was considered sufficient.

The independent t-test and chi-square test were used to compare general demographic data of the experts and novices. Two-way repeated measures analysis of variance was used to analyze learning curves. The between-subject factor group was added to investigate novice and expert performance separately. Retention of skills was investigated by within-subject contrasts and was assessed by comparing the last repetition of both series; a significant improvement by repetitive training was defined as a prolonged learning curve [13]. A p value of <0.05 was considered as statistically significant for all tests. Means and 95% confidence intervals (CI) were used to compare data for the learning curves because these are applicable to the analysis of variance.

While statistical significance provides information on evidence of any effect at all, the practical significance of the results was quantified by the effect size (ES), which indicates whether a learning effect is meaningful or important [14]. The ES is independent of sample size and a scale-free index. The ES was extracted from the analysis of variance output in SPSS. ES of 0.10, 0.30 and 0.50 were considered to indicate small (negligible), medium (moderate) and large (crucial) effects, respectively. ES was considered relevant only if a significant (p < 0.05) result was found.

Findings

Demographic data

General demographic data of novices and experts are given in Table 1. As expected, there is a significant difference between gender (p < 0.001), age (p < 0.001) and prior hysteroscopy experience (p < 0.001). To assess retention of skills, novices returned after a median of 14 days (range, 11–19 days) for a second series of repetitions. None of the experts had previous experience of performing exercises on HYSTT.

Table 1 Baseline characteristics of all participants

Learning curve

The main outcome measure was procedure time. The secondary outcome parameter was clinical performance, which was assessed by the mean GRS score. Results of novice performance in both series of repetitions are given in Table 2 (original measurements). A graphic presentation of the novice learning curve with the expert performance as a reference curve is shown in Fig. 4.

Table 2 Results of novice performance in both series

Fig. 4

Learning curve for novices (blue) and experts (green) in the first series of exercises and for novices in the second series

Novices showed a significant and moderate learning effect for the time needed to complete a repetition (p < 0.05, ES = 0.44). A large difference was observed between experts and novices in procedure time, in favour of the experts (experts, mean 215.8 s, 95% CI 154.9 – 276.7 s; novices, 869.4 s, 95% CI 570.1–1,168.6 s). As recognized in the graph, novices progressed towards expert level in time and reached a plateau phase at the seventh repetition, while experts performed stable after their first exercise. Both plateau phases did not coincide (experts, mean 108.3 s, 95% CI 87.8–128.8 s; novices, mean 154.3 s, 95% CI 130.3–178.3 s).

For mean GRS score, the novice group demonstrated a significant and large learning effect (p < 0.05, ES = 0.71). The expert group received higher GRS scores from the start (experts, mean 3.4, 95% CI 3.0 – 3.8; novices, mean 1.8, 95% CI 1.6 – 2.1) and the difference between both groups only moderately decreased. No plateau phase was recognized in the novice learning curve.

Retention of skills

For both procedure time and GRS, analysis of novice performance after 2 weeks showed retention of skills. Comparing the last repetitions of both series, no significant decrease in performance was found. Instead, a significant improvement of performance parameters was observed by repetitive training (p < 0.05, ES = 0.35–0.65), indicating a prolonged learning curve (Table 2).

Expert opinion

All experts completed the questionnaire concerning the realism and training capacity of the HYSTT. Table 3 summarizes the scores awarded by the experts. The ability to test and train camera navigation skills was scored with a median of 5.00 points on a 5-point Likert scale. The experts indicated the HYSTT to be very applicable in training residents (median 4.90). The box trainer was considered less applicable for training medical students (median 3.50) and for learning anatomy (median 3.00). The realism of the HYSTT in simulating a diagnostic hysteroscopy was awarded a median of 3.00 points.

Table 3 Results of expert opinion

Discussion

The present study assessed the learning curve for performance of hysteroscopic 30° camera navigation skills using the HYSTT, a new box trainer for diagnostic hysteroscopy. For all parameters, significant improvements were found in novice performance within nine repetitions. Retention of skills was demonstrated and a prolonged learning curve was established. These results indicate an adequate training capacity of the HYSTT and the effectiveness of repetitive training. One or more training sessions substantially improve the acquisition of 30° camera navigation skills on the HYSTT.

Strong points of this study are its realistic study design, additional assessment of retention of skills, and that one investigator supervised all tests in both groups. Furthermore, the use of ES adds information as to whether significant learning curves can be translated into meaningful and important learning effects. Besides procedure time, a clinical parameter (GRS) was used to assess competence from another perspective. Performing a procedure very fast does not necessarily mean it is performed properly and/or with good results.

Differences between the performance levels of both groups give an indication of construct validity, which is an important investigation before implementation of a model into a training curriculum [16]. The results imply that the HYSTT is indeed able to teach and evaluate those skills that are intended to be taught and measured, by differentiating between levels of expertise. However, the present study was not designed to investigate construct validity or powered to differentiate. Therefore, the indication of construct validity should be interpreted with caution.

The GRS was adjusted for hysteroscopic 30° camera navigation training and has not yet been validated, though similar rating scales have been implemented and validated for use in a general hysteroscopic training program and diagnostic cystoscopy in urology [17, 18].

It can be argued whether the first repetition(s) should be used for data analysis in a study assessing the learning curve, considering the improvement in performance parameters for both groups. The time needed to become familiar with a simulation model could influence results. The chosen study design reflects reality and integrates the possibility of feedback and training in the process [19]. The expert curve functions as a reference, and possible improvement during the first cases is likely to represent reality. Experts also must become accustomed to the new environment because this is by definition a deduction of reality.

The learning curve for GRS shows that the experts continue to improve their score until the seventh repetition, which might be a later plateau phase than one expects. A possible explanation could be that the white plastic HYSTT model does not resemble the uterus in a very close matter, as indicated by the expert opinion. The anatomical terms were considered confusing for several uterine locations. For example, "cornua" implies the region where a fallopian tube enters the uterine cavity. The HYSTT model contains specific spots for both "cornua" and "tubal ostium", which are located very close to each other and without any further clues concerning the differentiation between these anatomical terms. These locations were frequently mixed up by several experts throughout the repetitions, which led to lower GRS scores. Also, in the HYSTT model, the cervical canal is absent and therefore the hysteroscope can slip out of the uterus without visual warning.

To improve the use and applicability of the HYSTT, the realism of the model might be enhanced by improving the visual aspects of the uterine cavity, e.g., silicone material with realistic colour effects, augmented uterine shape with real tubal ostia and a cervical canal. A different possibility might be to remove the cornual locations and change the anatomical terms into more general terms, e.g., front/mid/rear, combined with anterior/posterior/left/right.

A recent study by the current authors investigated the learning curve of hysteroscopic sterilization on a virtual reality (VR) simulator [20]. The curves of both training models show similar shapes, indicating an adequate training capacity of repetitive training for both exercises and models. Concerning procedure time, the novices learned somewhat faster and reached a plateau phase within nine repetitions on the camera navigation box trainer in comparison to the VR sterilization simulator. Regarding GRS score, the novice group showed a slightly greater improvement of clinical skills on the VR sterilization simulator; a prolonged learning curve was observed for both training models.

Enabling a good training model to be successful in daily practice, it should be implemented into a well-thought-out training curriculum [21, 22]. This curriculum preferably contains training sessions taking place on an interval basis rather than massed into a short period of extensive practice. In addition, expert performance should be used to provide a proficiency criterion [21]. This gives trainees an objectively established goal they would have to reach before progressing to a next level or the operating room. Furthermore, feasibility is important to consider, for easy employment of the training model in a curriculum. The present box trainer showed promising results for interval training and the expert level was determined to provide an indication of the proficiency criterion. Also, the HYSTT is portable and reusable and the instruments are available at any institution providing hysteroscopic procedures.

Assessment of predictive validity is a further important step in determining the applicability of a simulator in a training curriculum. The predictive validity indicates the extent to which the training model predicts future performance and there remains a paucity of predictive validity testing of gynecologic simulators at this time, especially for hysteroscopy.

Conclusions

In conclusion, the learning curve of the present box trainer for hysteroscopic 30° camera navigation skills indicate a good training capacity because large improvements were made for novice training on this box trainer. Furthermore, retention of skills and prolonged learning curves were observed for both parameters procedure time and GRS. In addition, experts awarded the HYSTT the highest scores for training camera skills and applicability in residency training. Improvements can be made on the realism and anatomy of the uterus model.