Grading surgical skills curricula and training facilities for minimally invasive surgery
© Springer-Verlag Berlin Heidelberg 2012
Received: 28 July 2012
Accepted: 8 October 2012
Published: 23 October 2012
In teaching hospitals all over the world, skills laboratories have been set up in order to train and assess minimally invasive surgical (MIS) skills. However, there are no generally accepted standards as to what an MIS skills laboratory should look like and how the training should be conducted. This study is an attempt to develop an international and consensus-based set of quality criteria for a skills laboratory for training MIS. Three quality domains for skills laboratory were defined: ‘personnel and resources’, ‘trainee motivation’ and ‘curriculum’. A list of consensus-based criteria, nine items per domain, was made. Twenty-three recognised experts in MIS rated each item on a 0 to 3 scale in the level of importance. The most important items per domain were: ‘personnel and resources’, the presence of a box trainer, a laparoscopic expert and the availability of financial resources; ‘trainee motivation’, mandatory training supervised by laparoscopic experts; and ‘curriculum’, the presence of a structured skills curriculum, dedicated time for training and a yearly evaluation of the skills of the resident. This consensus list can be used when setting up a skills laboratory, but also for verifying the quality of an existing laboratory. From there, the focus for new developments can be chosen.
In teaching hospitals all over the world, skills laboratories have been set up in order to train and assess minimally invasive (e.g. laparoscopic) surgical skills outside the operating room in a safe, reproducible environment . This development is driven by quality and patient safety concerns, a restriction in resident working hours and increasing costs of operating room time . Simulator-acquired skills are proven to be transferable to the actual operations on patients, leading to a faster operating time and, more important, to fewer errors [3, 4].
However, no guideline exists on how to design and use a minimally invasive surgical (MIS) skills laboratory nor has a well-recognised standard been defined. The lack of consensus on the appropriate equipment is one of the most common impediments . Furthermore, a well-equipped skills laboratory does not automatically generate skilled surgeons. Simulation centres are underutilized, with minimal voluntary use of the models outside the realm of research studies or a structured mandatory training curriculum [6, 7]. Nevertheless, there is agreement on at least the need for properly implemented, monitored and evaluated training curricula for MIS skills [8–10].
This study is an attempt to develop an international and consensus-based set of quality criteria for a skills laboratory for training MIS. These criteria include aspects of the design of the skills laboratory and the training curriculum. Quality criteria may help current and future designers and clinicians to implement skills laboratories in their hospitals.
In order to develop a criteria framework for rating skills laboratories for laparoscopic surgery, the recognised consensus-based Delphi approach was used . This approach enables integrating empirical evidence where it exists with the views of experts.
First, three quality domains were defined: personnel and resources, trainee motivation and curriculum. These domains were inspired on the study of Stefanidis et al. who explored the evidence in the surgical literature regarding laparoscopic curriculum development and who tried to identify the factors that influence the successful incorporation of simulator training into a resident's curriculum . Regarding trainee motivation, external motivation of the trainee is addressed, which refers to interventions aimed at modifying behaviour, because the individual internal motivation seems difficult to influence .
Additionally, three authors (EH, HS and FWJ) independently searched the current literature for criteria that a skills laboratory should meet and categorized these per domain. For this search, the electronic databases MEDLINE, EMBASE, Current Contents, Science Citation Index and the Cochrane database were used. In a consensus meeting between the three authors, the lists of criteria were discussed, and an integrated consensus list was formed.
Definitions of rating scale quality criteria
Not important for rating a skills laboratory
Optional criterion for a skills laboratory
Criterion that expresses good quality of a skills laboratory
Indispensible for a good skills laboratory
Ranked consensus list of quality criteria (median scores of 23 experts)
Rating consensus list
Median score (range)
Personnel and resources
Availability 24 h a day
Space for at least 4 trainees to train simultaneously
Presence of a lab technician
Presence of a curriculum director (a laparoscopic expert)
Presence of a box (video) trainer
Presence of a virtual reality trainer
Effective instruction material for the use of the trainer(s) (e.g. video)
Presence of an animal lab
Availability of financial resources for the skills lab
Training sessions are supervised by a laparoscopic expert
Training sessions are supervised by a lab technician
A proficiency (i.e. expert) based training goal has been set
The training goal is based on time and precision
Training is mandatory
Residents are not allowed to perform surgery if predefined skills level is not reached
Awards are given for good attendance
Presence of tasks of increasing level of difficulty
Variability is present in the laparoscopic tasks
Presence of a structured skills curriculum
Time is dedicated for skills training in the residency curriculum
Monthly training sessions are organized
Presence of ‘over-training’ (i.e. better than training goal) facilities
Repetitive training over various training sessions
Maintenance of training
Retention of skills is established every 12 months
Training goal increases with progression in residency
Progress in laparoscopic skills is incorporated in yearly evaluation of resident
In the domain ‘personnel and resources’ (Fig. 1), the presence of a lab technician was considered the least essential for a skills laboratory since it was rated with a median score of 1. The three criteria considered most important were the presence of a curriculum director (laparoscopic expert), the presence of a box trainer and the availability of financial resources. All these criteria received a median score of 3: indispensable for a good laboratory.
In the domain ‘trainee motivation’ (Fig. 2), the fact that the training should be mandatory is considered the most important. Thereafter, supervision of training by a laparoscopic expert and residents not allowing to perform surgery if the predefined skills level is not reached was considered of importance.
In the domain ‘curriculum’ (Fig. 3), the presence of over-training facilities (i.e. training after the initially required level of proficiency is achieved) was considered least important (median score 1). Four criteria were rated with a median score of 3 by the responding experts: the presence of a structured skills curriculum, time dedicated for skills training, maintenance of skills and a yearly evaluation of the progress in laparoscopic skills of the resident. As a result, a ranked list of quality criteria is presented, with the ranking based on the median scores of the 23 experts (Table 2).
For the setting of a laparoscopic skills laboratory in a (teaching) hospital, the bottom line is that a box trainer model and financial resources are required. The training has to be mandatory, to be supervised by a laparoscopic expert, and residents should not perform (supervised) in vivo laparoscopic surgery if the predefined skills level is not reached. Skills training should be imbedded in a structured curriculum with time scheduled for training. Finally, maintenance of skills and a yearly evaluation of the skills level are recommended. Our detailed consensus list can be used when setting up an MIS skills laboratory. Furthermore, it gives cues for verifying the quality of an already existing laboratory, just by using the list of ranked quality criteria as a checklist. From there, the focus for improvement or new developments can be chosen.
In the domain personnel and resources, the presence of a box trainer is considered relatively more important than the presence of a virtual reality (VR) trainer. This finding is consistent with recent results of Palter et al., who found in their inventory that residents prefer box trainers above VR simulators for training the more advanced laparoscopic skills . Regarding the assessment of laparoscopic skills, both trainer models have an equally good correlation . VR trainers have the advantage as they allow solitary training while the supervisor can monitor the resident's skills level electronically. On the other hand, the presence of a supervisor required during box training has the advantage that surgical knowledge can be transmitted. Furthermore, the presence of a laboratory technician is rated low. This could be explained by the fact that an enthusiastic laparoscopic expert can fulfil this role. However, in our opinion, the presence of a permanent availability of a technician gives a professionalizing of skills laboratory, with all its advantages.
In parallel with the importance of setting the training mandatory, it was found that most residents do not reach the performance standards of basic laparoscopic skills if the skills training is voluntary . Furthermore, training up till a predefined level of skills is superior over training based on the time spent. In fact, the time required varies and training till a certain level induces an external motivation. Ideally, the training should be proficiency-based  and supervised by a laparoscopic expert. Training exercises should not be based on time only, and a score for precision should be added . It can be argued whether the exercises should have an increasing level of difficulty. On one hand, this may keep the trainees motivated throughout their entire specialty training; on the other hand, basic laparoscopic skills should be acquired as early as possible in residency after which residents can expand their proficiency in the operating room in learning anatomy, pathology and operating techniques, while maintenance of the basic skills is all there is left to do .
In the third domain, curriculum, there is a clear consensus about incorporating the skills training for MIS in a proficiency-based training curriculum. It is important to dedicate time for skills training during working hours and organize repetitive training sessions. Overall, the presence of a mandatory, structured and competency-based skills training curriculum is the key to success [12, 18, 19].
With the increasing pressure on guaranteed skilfulness of surgeons, many MIS specialty teaching hospitals feel the need to implement training facilities outside the OR. Although it is essential to define the purpose and to identify resources early in the development of a skills laboratory, the reality is often the other way around . As a result, many hospitals have designed laboratories based on an individual trainer's ideas and preferences. Besides, curriculum development is lagging . The strength of this study is that a consensus-based rating system has been developed with the agreement of laparoscopic experts all over the world. However, the selection of the 23 experts is a factor that may induce bias. In the first place, the chosen definition of an expert is debatable. In the second place, the currently used definition partly depended on gynaecologists’ reputation in their peers' field.
The European Academy of Gynaecological Surgery has recently, in conjunction with the European Society of Gynaecologic Endoscopy, elaborated a validated programme for training and certification of MIS . This programme includes knowledge and skills training modules. The validated box training modules can easily be incorporated in any skills laboratory. A generally accepted set of criteria potentiates a system of accreditation for laparoscopic skills laboratories. The American College of Surgeons has developed a system for accreditation of skills laboratories regarding general surgical skills in institutes . These criteria are used to determine whether an institute meets the minimum requirements for accreditation as a level II (basic education) or a level I (comprehensive education) institute. In parallel, our set of criteria can be used as a framework useful in daily practice and possibly for accreditation purposes in the future. More in detail, a skills laboratory can be assessed rating the presence of a criterion with the corresponding median score of our ranked quality criteria list. In that way, criteria that are considered more relevant according to our expert panel receive higher ratings. As a result, an MIS skills laboratory with an MIS skills curriculum can obtain at maximum 62 points (20 points for personnel and resources, 21 points for trainee motivation and 21 points for curriculum). This total score can be used to choose the focus for future developments. Additionally, a practical application might be that a basic MIS laboratory should have at least the indispensible criteria with a median score 3, while a comprehensive MIS laboratory should also have all criteria with a median score of 2 for the certification.
This rating list can be used to set up and maintain a minimally invasive skills laboratory. In a skills laboratory, at least a box trainer has to be present with a proficiency-based training programme. The training should be incorporated in a formal curriculum which is obliged prior to attendance of real in vivo surgery in order to enhance patient safety.
The authors thank all experts for their voluntarily contribution to the consensus list.
Conflict of interest
The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
- MacRae HM, Satterthwaite L, Reznick RK (2008) Setting up a surgical skills center. World J Surg 32:189–195PubMedView ArticleGoogle Scholar
- Gould JC (2006) Building a laparoscopic surgical skills training laboratory: resources and support. JSLS 10:293–296PubMedGoogle Scholar
- Larsen CR, Soerensen JL, Grantcharov TP et al (2009) Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 338:b1802PubMedView ArticleGoogle Scholar
- Stefanidis D, Hope WW, Korndorffer JR Jr, Markley S, Scott DJ (2010) Initial laparoscopic basic skills training shortens the learning curve of laparoscopic suturing and is cost-effective. J Am Coll Surg 210:436–440PubMedView ArticleGoogle Scholar
- Korndorffer JR Jr, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ (2005) Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg 201:23–29PubMedView ArticleGoogle Scholar
- Chang L, Petros J, Hess DT, Rotondi C, Babineau TJ (2007) Integrating simulation into a surgical residency program: is voluntary participation effective? Surg Endosc 21:418–421PubMedView ArticleGoogle Scholar
- van Dongen KW, van der Wal WA, Rinkes IH, Schijven MP, Broeders IA (2008) Virtual reality training for endoscopic surgery: voluntary or obligatory? Surg Endosc 22:664–667PubMedView ArticleGoogle Scholar
- Korndorffer JR Jr, Stefanidis D, Scott DJ (2006) Laparoscopic skills laboratories: current assessment and a call for resident training standards. Am J Surg 191:17–22PubMedView ArticleGoogle Scholar
- Park A, Witzke DB (2002) Training and educational approaches to minimally invasive surgery: state of the art. Semin Laparosc Surg 9:198–205PubMedView ArticleGoogle Scholar
- Schijven MP, Schout BM, Dolmans VE, Hendrikx AJ, Broeders IA, Borel Rinkes IH (2008) Perceptions of surgical specialists in general surgery, orthopaedic surgery, urology and gynaecology on teaching endoscopic surgery in The Netherlands. Surg Endosc 22:472–482PubMedView ArticleGoogle Scholar
- Elwyn G, O’Connor A, Stacey D et al (2006) Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ 333(7565):417PubMedView ArticleGoogle Scholar
- Stefanidis D, Heniford BT (2009) The formula for a successful laparoscopic skills curriculum. Arch Surg 144:77–82PubMedView ArticleGoogle Scholar
- Palter VN, Orzech N, Aggarwal R, Okrainec A, Grantcharov TP (2010) Resident perceptions of advanced laparoscopic skills training. Surg Endosc 24:2830–2834PubMedView ArticleGoogle Scholar
- Newmark J, Dandolu V, Milner R, Grewal H, Harbison S, Hernandez E (2007) Correlating virtual reality and box trainer tasks in the assessment of laparoscopic surgical skills. Am J Obstet Gynecol 197:546.e1-4PubMedView ArticleGoogle Scholar
- Kolkman W, Wolterbeek R, Jansen FW (2005) Gynecological laparoscopy in residency training program: Dutch perspectives. Surg Endosc 19:1498–1502PubMedView ArticleGoogle Scholar
- Korndorffer JR Jr, Scott DJ, Sierra R et al (2005) Developing and testing competency levels for laparoscopic skills training. Arch Surg 140:80–84PubMedView ArticleGoogle Scholar
- Smith CD, Farrell TM, McNatt SS, Metreveli RE (2001) Assessing laparoscopic manipulative skills. Am J Surg 181:547–550PubMedView ArticleGoogle Scholar
- Gallagher AG, Ritter EM, Champion H et al (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241:364–372PubMedView ArticleGoogle Scholar
- McClusky DA III, Smith CD (2008) Design and development of a surgical skills simulation curriculum. World J Surg 32:171–181PubMedView ArticleGoogle Scholar
- Campo R, Molinas CR, de Wilde RL et al (2012) Are you enough for your patients? The European certification model in laparoscopic surgery. FFV ObGyn 4:95–101Google Scholar
- American College of Surgeons, Division of Education, Accredited Education Institute (2011) http://www.facs.org/education/accreditationprogram. Accessed 25 July 2012