Grading surgical skills curricula and training facilities for minimally invasive surgery

Abstract

In teaching hospitals all over the world, skills laboratories have been set up in order to train and assess minimally invasive surgical (MIS) skills. However, there are no generally accepted standards as to what an MIS skills laboratory should look like and how the training should be conducted. This study is an attempt to develop an international and consensus-based set of quality criteria for a skills laboratory for training MIS. Three quality domains for skills laboratory were defined: ‘personnel and resources’, ‘trainee motivation’ and ‘curriculum’. A list of consensus-based criteria, nine items per domain, was made. Twenty-three recognised experts in MIS rated each item on a 0 to 3 scale in the level of importance. The most important items per domain were: ‘personnel and resources’, the presence of a box trainer, a laparoscopic expert and the availability of financial resources; ‘trainee motivation’, mandatory training supervised by laparoscopic experts; and ‘curriculum’, the presence of a structured skills curriculum, dedicated time for training and a yearly evaluation of the skills of the resident. This consensus list can be used when setting up a skills laboratory, but also for verifying the quality of an existing laboratory. From there, the focus for new developments can be chosen.

Background

In teaching hospitals all over the world, skills laboratories have been set up in order to train and assess minimally invasive (e.g. laparoscopic) surgical skills outside the operating room in a safe, reproducible environment [1]. This development is driven by quality and patient safety concerns, a restriction in resident working hours and increasing costs of operating room time [2]. Simulator-acquired skills are proven to be transferable to the actual operations on patients, leading to a faster operating time and, more important, to fewer errors [3, 4].

However, no guideline exists on how to design and use a minimally invasive surgical (MIS) skills laboratory nor has a well-recognised standard been defined. The lack of consensus on the appropriate equipment is one of the most common impediments [5]. Furthermore, a well-equipped skills laboratory does not automatically generate skilled surgeons. Simulation centres are underutilized, with minimal voluntary use of the models outside the realm of research studies or a structured mandatory training curriculum [6, 7]. Nevertheless, there is agreement on at least the need for properly implemented, monitored and evaluated training curricula for MIS skills [8–10].

This study is an attempt to develop an international and consensus-based set of quality criteria for a skills laboratory for training MIS. These criteria include aspects of the design of the skills laboratory and the training curriculum. Quality criteria may help current and future designers and clinicians to implement skills laboratories in their hospitals.

Methods

In order to develop a criteria framework for rating skills laboratories for laparoscopic surgery, the recognised consensus-based Delphi approach was used [11]. This approach enables integrating empirical evidence where it exists with the views of experts.

First, three quality domains were defined: personnel and resources, trainee motivation and curriculum. These domains were inspired on the study of Stefanidis et al. who explored the evidence in the surgical literature regarding laparoscopic curriculum development and who tried to identify the factors that influence the successful incorporation of simulator training into a resident's curriculum [12]. Regarding trainee motivation, external motivation of the trainee is addressed, which refers to interventions aimed at modifying behaviour, because the individual internal motivation seems difficult to influence [12].

Additionally, three authors (EH, HS and FWJ) independently searched the current literature for criteria that a skills laboratory should meet and categorized these per domain. For this search, the electronic databases MEDLINE, EMBASE, Current Contents, Science Citation Index and the Cochrane database were used. In a consensus meeting between the three authors, the lists of criteria were discussed, and an integrated consensus list was formed.

Next, the consensus list was sent electronically to known worldwide experts in training of MIS skills, or a paper version of the consensus list was given to them if he/she visited the Annual Meeting of the European Society of Gynaecologic Endoscopy. An expert in MIS was defined as a gynaecologist, who is well-recognised as an expert in advanced laparoscopic surgery, who has three or more publications on MIS-related topics and is actively involved in the organisation of MIS training programme in his/her teaching hospital. They were asked to rate each criterion on a 0 to 3 scale in the level of importance for a skills laboratory. The definitions of this scale are displayed in Table 1. The experts were also instructed to add missing criteria to the list if considered necessary.

Table 1 Definitions of rating scale quality criteria

Findings

The consensus list contained nine criteria per domain (Table 2). In total, 23 experts were selected from 14 countries in Europe, North and South America and Australia. They were either electronically, or in person, asked to fill out the consensus list. All 23 agreed to participate and have rated the nine criteria per domain (personnel and resources, trainee motivation and curriculum). None of the respondents added a new criterion to the list. The results per criterion are displayed as bar charts (Figs. 1, 2 and 3).

Table 2 Ranked consensus list of quality criteria (median scores of 23 experts)

Fig. 1

Expert opinion domain personnel and resources

Fig. 2

Expert opinion domain trainee motivation

Fig. 3

Expert opinion domain curriculum

In the domain ‘personnel and resources’ (Fig. 1), the presence of a lab technician was considered the least essential for a skills laboratory since it was rated with a median score of 1. The three criteria considered most important were the presence of a curriculum director (laparoscopic expert), the presence of a box trainer and the availability of financial resources. All these criteria received a median score of 3: indispensable for a good laboratory.

In the domain ‘trainee motivation’ (Fig. 2), the fact that the training should be mandatory is considered the most important. Thereafter, supervision of training by a laparoscopic expert and residents not allowing to perform surgery if the predefined skills level is not reached was considered of importance.

In the domain ‘curriculum’ (Fig. 3), the presence of over-training facilities (i.e. training after the initially required level of proficiency is achieved) was considered least important (median score 1). Four criteria were rated with a median score of 3 by the responding experts: the presence of a structured skills curriculum, time dedicated for skills training, maintenance of skills and a yearly evaluation of the progress in laparoscopic skills of the resident. As a result, a ranked list of quality criteria is presented, with the ranking based on the median scores of the 23 experts (Table 2).

Discussion

For the setting of a laparoscopic skills laboratory in a (teaching) hospital, the bottom line is that a box trainer model and financial resources are required. The training has to be mandatory, to be supervised by a laparoscopic expert, and residents should not perform (supervised) in vivo laparoscopic surgery if the predefined skills level is not reached. Skills training should be imbedded in a structured curriculum with time scheduled for training. Finally, maintenance of skills and a yearly evaluation of the skills level are recommended. Our detailed consensus list can be used when setting up an MIS skills laboratory. Furthermore, it gives cues for verifying the quality of an already existing laboratory, just by using the list of ranked quality criteria as a checklist. From there, the focus for improvement or new developments can be chosen.

In the domain personnel and resources, the presence of a box trainer is considered relatively more important than the presence of a virtual reality (VR) trainer. This finding is consistent with recent results of Palter et al., who found in their inventory that residents prefer box trainers above VR simulators for training the more advanced laparoscopic skills [13]. Regarding the assessment of laparoscopic skills, both trainer models have an equally good correlation [14]. VR trainers have the advantage as they allow solitary training while the supervisor can monitor the resident's skills level electronically. On the other hand, the presence of a supervisor required during box training has the advantage that surgical knowledge can be transmitted. Furthermore, the presence of a laboratory technician is rated low. This could be explained by the fact that an enthusiastic laparoscopic expert can fulfil this role. However, in our opinion, the presence of a permanent availability of a technician gives a professionalizing of skills laboratory, with all its advantages.

In parallel with the importance of setting the training mandatory, it was found that most residents do not reach the performance standards of basic laparoscopic skills if the skills training is voluntary [15]. Furthermore, training up till a predefined level of skills is superior over training based on the time spent. In fact, the time required varies and training till a certain level induces an external motivation. Ideally, the training should be proficiency-based [16] and supervised by a laparoscopic expert. Training exercises should not be based on time only, and a score for precision should be added [17]. It can be argued whether the exercises should have an increasing level of difficulty. On one hand, this may keep the trainees motivated throughout their entire specialty training; on the other hand, basic laparoscopic skills should be acquired as early as possible in residency after which residents can expand their proficiency in the operating room in learning anatomy, pathology and operating techniques, while maintenance of the basic skills is all there is left to do [16].

In the third domain, curriculum, there is a clear consensus about incorporating the skills training for MIS in a proficiency-based training curriculum. It is important to dedicate time for skills training during working hours and organize repetitive training sessions. Overall, the presence of a mandatory, structured and competency-based skills training curriculum is the key to success [12, 18, 19].

With the increasing pressure on guaranteed skilfulness of surgeons, many MIS specialty teaching hospitals feel the need to implement training facilities outside the OR. Although it is essential to define the purpose and to identify resources early in the development of a skills laboratory, the reality is often the other way around [1]. As a result, many hospitals have designed laboratories based on an individual trainer's ideas and preferences. Besides, curriculum development is lagging [4]. The strength of this study is that a consensus-based rating system has been developed with the agreement of laparoscopic experts all over the world. However, the selection of the 23 experts is a factor that may induce bias. In the first place, the chosen definition of an expert is debatable. In the second place, the currently used definition partly depended on gynaecologists’ reputation in their peers' field.

The European Academy of Gynaecological Surgery has recently, in conjunction with the European Society of Gynaecologic Endoscopy, elaborated a validated programme for training and certification of MIS [20]. This programme includes knowledge and skills training modules. The validated box training modules can easily be incorporated in any skills laboratory. A generally accepted set of criteria potentiates a system of accreditation for laparoscopic skills laboratories. The American College of Surgeons has developed a system for accreditation of skills laboratories regarding general surgical skills in institutes [21]. These criteria are used to determine whether an institute meets the minimum requirements for accreditation as a level II (basic education) or a level I (comprehensive education) institute. In parallel, our set of criteria can be used as a framework useful in daily practice and possibly for accreditation purposes in the future. More in detail, a skills laboratory can be assessed rating the presence of a criterion with the corresponding median score of our ranked quality criteria list. In that way, criteria that are considered more relevant according to our expert panel receive higher ratings. As a result, an MIS skills laboratory with an MIS skills curriculum can obtain at maximum 62 points (20 points for personnel and resources, 21 points for trainee motivation and 21 points for curriculum). This total score can be used to choose the focus for future developments. Additionally, a practical application might be that a basic MIS laboratory should have at least the indispensible criteria with a median score 3, while a comprehensive MIS laboratory should also have all criteria with a median score of 2 for the certification.

Conclusion

This rating list can be used to set up and maintain a minimally invasive skills laboratory. In a skills laboratory, at least a box trainer has to be present with a proficiency-based training programme. The training should be incorporated in a formal curriculum which is obliged prior to attendance of real in vivo surgery in order to enhance patient safety.