Non-technical factors influence laparoscopic simulator performance among OBGYN residents
© Springer-Verlag 2012
Received: 3 March 2012
Accepted: 1 April 2012
Published: 10 May 2012
In addition to technical skills, nontechnical factors appear to influence surgical results. This study aims to analyze how visuospatial ability, self-efficacy, and flow are associated with simulated laparoscopic performance of residents in obstetrics and gynecology (OBGYN). In this cohort study, 28 residents in obstetrics and gynecology were tested for visuospatial ability and self-efficacy prior to simulator training. All participants subsequently conducted a basic set of tasks in the simulator. Self-efficacy, once again, and flow were assessed after training. Nineteen of the subjects then completed a 2-day course with identical simulator tasks, although now to a predefined credential level. Visuospatial ability correlated with simulator performance in the technically most advanced simulator task in the basic set (“total time,” r = −0.40, p = 0.039). Flow correlated with: “right instrument pathway”(r = −0.40, p = 0.004) in that same task and with the 2-day overall training results (r = −0.56, p = 0.017). Self-efficacy correlated with the 2-day result (r = −0.56, p = 0.013) and significantly improved after training (p = 0.011). When constructing a curriculum for OBGYN residents, visuospatial abilities and non-technical factors like flow and self-efficacy should be considered.
The lack of structured training programs for residents performing surgery is jeopardizing patient safety [1–3]. Simulators are becoming important training tools in preparing for various real-life scenarios. For example, computerized simulations appear to be beneficial for enhancing the technical performance among surgery and obstetrics and gynecology (OBGYN) residents [4–6].
In addition to technical skills, nontechnical factors appear to be important when constructing a surgical curriculum. Previous studies have reported behavior-rating systems for surgeons based on skills, such as situation awareness, decision making, communication, and leadership. Such systems appear to be beneficial for safe surgical practice [7–10].
Our research group has focused on trying to identify nontechnical factors that might influence the learning process during image-guided simulator training. Visuospatial ability, commonly defined as the capacity to generate “a mental representation of a two- or three-dimensional structure and then assessing its properties or performing a transformation of the representation”  appears to influence operative performance. We have previously reported that visuospatial abilities correlate with simulated laparoscopic performances among surgical novices and for consultants in OBGYN [12–14]. Since simulator training appears to improve surgical performance, testing residents for visuospatial abilities could improve the design of individual training programs.
Another important feature of the learning process is self-efficacy, i.e., one’s belief in one’s ability to succeed in specific situations . For example, Maschuw et al.  reported that low self-efficacy correlates with poor simulator performance among surgical residents. Flow, i.e., “to move or run smoothly with unbroken continuity and concentration and complete absorption in what one does” , is another factor that appears to be of importance for task performance. A recent study shows that repeated team training of cardiopulmonary resuscitation in a virtual world increase concentration, as well as self-efficacy in medical students . Assessments of self-efficacy and flow might therefore be useful for evaluating a training program’s or an instrument’s effectiveness and feasibility.
The aim of the present study was to explore visuospatial ability, self-efficacy, and flow in OBGYN residents in relation to laparoscopic simulator performance, in preparation for constructing a gynecological curriculum.
The study group consisted of 28 OBGYN residents from 21 different hospitals in Sweden. Exclusion criteria were more than ten independently performed laparoscopic tubal occlusions or attendance at a simulator course prior to the study. The study intended to investigate the abilities among laparoscopic novices with no prior laparoscopic simulator training.
The study was conducted at the Center for Advanced Medical Simulation and Training (CAMST), at Karolinska University Hospital, Stockholm, Sweden, during a week in 2009. The regional Research Ethics Committee of Stockholm approved the study.
In order to test for visuospatial ability, all participants first completed the redrawn Vandenberg and Kuse Mental rotations test, version A (MRT-A) . The test consists of 24 items each containing a target figure to the left with four stimulus figures to the immediate right of the target figure. The task for the test subject is to mentally rotate the stimulus figures around the vertical axis. Participants had to identify both of two correct alternatives in order to get a score of “1.” Thus the maximum score was 24. Each participant was given the items organized into two subsets consisting of 12 items each. Three minutes were given to complete each subset. There was a 1-min break between each part. Instructions, procedures, and scoring were identical to those of Peters and collaborators .
For laparoscopic gynecological simulation, we used the LapSimGyn® VR (virtual reality) (Surgical Science AB, Gothenburg, Sweden) simulator, which is an advanced laparoscopic simulator with proven construct validity [20–22]. The system consists of software that runs on a Xeon-1.8 GHz processor using the Microsoft Windows® XP (Microsoft Corporation, Redmond, WA, USA) operating system. The computer is equipped with 256 MB internal RAM, a NVIDIA Quadro2 EX graphics card (NVIDIA Corporation, Santa Clara, CA, USA), a 15-inch monitor, and a virtual laparoscopic interface manufactured by Immersion, Inc. (San Jose, CA, USA).
All participants did one set of basic procedures to get familiar with the simulator and then one to two more sets to complete the basic simulator training.
Additionally, 19 of the subjects completed a 2-day course, which consisted of the same set of tasks. The subjects trained until they reached a predefined credential level in the simulator . The simulator tasks performed were “cutting,” “clip applying,” “lifting and grasping,” “tubal occlusion,” and “salpingectomy.” The most demanding task in this set was “lifting and grasping,” and it was therefore selected in subsequent analyses. This skill, as well as the other in the set, has previously been reported to discriminate between novices and experts, i.e., to demonstrate construct validity . In this task, you need to move a small needle with great precision and use a short distance (instrument path length), a small angle (instrument angular path length), and speed (total time) to reach the credential level.
The simulator parameters analyzed in the task “lifting and grasping” were “total time” (seconds), left and right “instrument angular path length” (degrees), and “instrument path length” (meters). The performance during the 2-day training course was measured by the number of trials until reaching the credential level in the task “lifting and grasping” and the sum of trials of all tasks until reaching credential level.
Individual self-efficacy is a capability in which cognitive, social, emotional, and behavioral skills must be organized and effectively orchestrated to serve several different purposes. Self-efficacy is not concerned with the number of skills you have, but what you believe you can do with the skills you have. Banduras’ cognitive theory suggests that, while successful performance enhances, repeated failures reduce perceived self-efficacy .
In this study, self-efficacy was assessed using a three-item questionnaire where each item was rated on a 7-grade Likert-type scale. The subjects completed the questionnaire before and after basic training as well as after the 2-day training course. Self-efficacy was calculated as the sum of all items .
Flow experience is reported to facilitate learning, encourage one to attempt a difficult task, promote the pace of creativity, increase the joy during an activity, and create meaning of involvement in an activity [25, 26].
Four categories of flow  were self-assessed using a 0–10 visual analog scale (enjoyment, four items; concentration, four items; control, three items; exploratory use, four items). A flow score was calculated as the sum of all items. Flow was assessed after the basic simulator set and after the 2-day simulator training.
Data analyses were carried out using JMP® version 9.0.0 (SAS Institute, Inc., Cary, NC, USA) for Mac OS X® version 10.5.7 (Apple, Inc., Cupertino, CA, USA). All the variables were tested for normality. Pearson’s correlation was used to analyze the relationship between visuospatial ability, self-efficacy, and flow, respectively, and simulated laparoscopic performance. Student’s t test was used to investigate differences in self-efficacy mean scores between the different sessions of simulator training. A level of p < 0.05 was considered to be statistically significant.
The basic simulator set
The 28 subjects (23 females and five males; mean age of 34 years ± 5 SD) performed a basic set of tasks in the simulator. The first of two sets was performed to allow the subjects to familiarize themselves with the simulator.
We analyzed the performance of the second session of the task “lifting and grasping.”
No significant correlation was observed between self-efficacy, before or after the basic simulator set, and the simulator performance.
Correlation between basic simulator performance and flow
Lifting and grasping
Right instrument angular path length (°)
Left instrument angular path length (°)
Right instrument path length (m)
Left instrument path length (m)
The 2-day simulator training
We next investigated whether visuospatial ability and self-efficacy had an impact on the simulator performance when subjects were trained. For this purpose, the 19 subjects (15 females and four males) who attended the 2-day course and who then all passed the pre-evaluated credential levels were further studied. Whereas the average number of sessions to reach the credential level in the task “lifting and grasping” was 31, the other tasks only required between four and 15 sessions, suggesting that “lifting and grasping “was the most demanding task. The average number of trials of all tasks, to reach credential level, was 69. When assessing simulator data from the 2-day course, we analyzed the number of trials until reaching the credential level in the task “lifting and grasping,” as well as the sum of trials of all tasks until reaching credential level. The sum of trials of all tasks to reach credential level was calculated to generate an overall result from the 2-day training.
We also wanted to analyze the association between extensive simulator training and flow.
Correlations between simulator performance, self-efficacy, and flow
Simulator performance after training
Sessions of lifting and grasping until passed course
Total sessions until passed course
MRT-A, however, did not correlate with the simulator performance when subjects were trained.
Because of the gender imbalance, we also subanalyzed the female group. Most correlations became stronger, although when excluding the males, self-efficacy before the 2-day training and flow after did not significantly correlate to lifting and grasping. The correlations with the overall training result were still significant though.
Prior to the study, 14 of the residents had performed zero to one tubal occlusion, five of the residents had performed two to five, and ten of the subjects had performed six to ten laparoscopic tubal occlusion independently.
In this study, we report correlations between visuospatial ability, flow, and self-efficacy, respectively, and the performance in image guided simulation, among residents in OBGYN. We also report an increase in self-efficacy after intense simulator training. To the best of our knowledge, these parameters have not previously been evaluated in OBGYN residents.
We chose to analyze the most difficult task in the simulator, “lifting and grasping” , in relation to self-efficacy, flow, and visuospatial ability. For this task, the average number of trials to reach the credential level was 31, whereas the typical gynecological procedures, “tubal occlusion” and “salpingectomy,” were not as challenging, requiring only 4 and 8 trials, respectively. These procedures were, however, included when we analyzed the overall result, i.e., the sum of all tasks, to reach the credential level.
In this study, visuospatial ability correlated with the outcome of simulator performance only in the technically more demanding “lifting and grasping” in the basic simulator set. This is in line with the study by Westman et al.  and Hedman et al.  who demonstrated that visuospatial abilities correlate positively with performance in the technically more demanding simulated colonoscopy and arthroscopy, respectively. Moreover, we previously reported that the visuospatial abilities of consultants in OBGYN correlated with the more demanding tasks during laparoscopic simulator training . In the present study, visuospatial ability did not, however, correlate with simulator performance after training intensely during 2 days. These findings suggest that this type of training can compensate for a lesser visuospatial ability in novices. Testing the visuospatial ability can thus identify individuals who might benefit from such supplementary training and who need support in specific surgical tasks. It, however, remains to be established if visuospatial abilities also influence the actual surgical performance.
Others have reported that nontechnical factors play important roles in performing and learning the technical skills in surgery [8–10, 28]. Artino et al. reported that medical students who are bored and anxious achieve less theoretically. In their study, self-efficacy was positively correlated with enjoyment and negatively associated with boredom . This is in line with Pain et al.  who reported that the performance of youth soccer players was negatively influenced by anxiety and boredom. Similarly, the findings of Maschuv et al.  suggest that surgical residents with low self-efficacy perform poorly during simulated laparoscopy.
In the present study, those who reported a high flow score, i.e., being able to concentrate and being engaged in and enjoying the task, performed better in the simulator, after the basic set as well as after extensive training, than those who reported low flow scores. This observation suggests that the simulator and the tasks are feasible and user friendly. One can speculate that the more the simulator task simulates an authentic operation, i.e., exhibit face validity, the greater the enjoyment, concentration, and performance will be. Simulators’ accessibility and feasibility thus could be evaluated according to the flow experience. Future studies should therefore investigate the relationship between the face validity of simulator tasks and the flow experience.
Self-efficacy scores correlated positively with simulator performance suggesting that a high self-confidence level is important for the simulated surgical result. Moreover, self-efficacy increased after intense training, indicating a gain in confidence from the training. This is important because a higher confidence might lead to a better result in the next training session.
The majority of tasks for most OBGYN practitioners are outside of the operating room (OR). Because the tasks performed in the OR are discipline specific, procedures among surgical residents cannot always be generalized to OBGYN residents. We therefore think it is important to distinguish residents in OBGYN from the residents in surgery, when investigating performance in the simulator, as well as the nontechnical skills that might impact the surgical performance, and furthermore when designing a training program.
One limitation in this study was the gender imbalance. This reflects the situation in at least a Swedish gynecological setting. The gender aspect should be further explored in a future study.
The findings from this exploratory study should be replicated by also using other validated instruments for assessing self-efficacy and flow, as well as a control group design in order to analyze for cause–effect relationships.
When building a curriculum for residents in OBGYN, factors like enjoyment and concentration, as well as confidence, need to be considered. Flow and self-efficacy should be further investigated in relation to the actual performance in the operating theater.
The authors wish to express their gratitude to Mr. Lars-Gunnar Andersson and Mr. Tomas Askerud, Department for Biomedical Engineering and CAMST, at Karolinska University Hospital, Stockholm, for excellent technical assistance. We thank Dr. Christian R. Larsen for letting us use his proficiency criteria setting in the simulator. We furthermore thank Sue and Steve Anderson for editing grammar and language in our manuscript. This study was supported by research grants from the Stockholm County Council (Pick Up and ALF).
Disclosure of interest
None of the authors have any conflicts of interest to disclose.
- Scott DJ (2006) Patient safety, competency, and the future of surgical simulation. Simul Healthc 1(3):164–170PubMedView ArticleGoogle Scholar
- Scott DJ, Dunnington GL (2008) The new ACS/APDS Skills Curriculum: moving the learning curve out of the operating room. J Gastrointest Surg 12(2):213–221PubMedView ArticleGoogle Scholar
- Stefanidis D, Arora S, Parrack DM, Hamad GG, Capella J, Grantcharov T, Urbach DR, Scott DJ, Jones DB (2012) Association for Surgical Education Simulation Committee. Research priorities in surgical simulation for the 21st century. Am J Surg 203(1):49–53PubMedView ArticleGoogle Scholar
- Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, Schroeder TV, Ottesen BS (2009) Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 338:b1802PubMedView ArticleGoogle Scholar
- Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193(6):797–804PubMedView ArticleGoogle Scholar
- Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236(4):458–463, discussion 463–4PubMedView ArticleGoogle Scholar
- Carthey J (2003) The role of structured observational research in health care. Qual Saf Health Care 12 (Suppl 2):ii13–ii16Google Scholar
- Yule S, Flin R, Paterson-Brown S, Maran N, Rowley D (2006) Development of a rating system for surgeons’ non-technical skills. Med Educ 40(11):1098–1104PubMedView ArticleGoogle Scholar
- Crossley J, Marriott J, Purdie H, Beard JD (2011) Prospective observational study to evaluate NOTSS (Non-Technical Skills for Surgeons) for assessing trainees’ non-technical performance in the operating theatre. Br J Surg 98(7):1010–1020PubMedView ArticleGoogle Scholar
- Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S (2008) Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 32(4):548–556PubMedView ArticleGoogle Scholar
- Just MA, Carpenter PA (1985) Cognitive coordinate systems: accounts of mental rotation and individual differences in spatial ability. Psychol Rev 92(2):137–172PubMedView ArticleGoogle Scholar
- Ahlborg L, Hedman L, Murkes D, Westman B, Kjellin A, Felländer-Tsai L, Enochsson L (2011) Visuospatial ability correlates with performance in simulated gynecological laparoscopy. Eur J Obstet Gynecol Reprod Biol 157(1):73–77PubMedView ArticleGoogle Scholar
- Westman B, Ritter EM, Kjellin A, Törkvist L, Wredmark T, Felländer-Tsai L, Enochsson L (2006) Visuospatial abilities correlate with performance of senior endoscopy specialist in simulated colonoscopy. J Gastrointest Surg 10(4):593–599PubMedView ArticleGoogle Scholar
- Hedman L, Ström P, Andersson P, Kjellin A, Wredmark T, Felländer-Tsai L (2006) High-level visual–spatial ability for novices correlates with performance in a visual-spatial complex surgical simulator task. Surg Endosc 20(8):1275–1280PubMedView ArticleGoogle Scholar
- Kneebone R (2005) Evaluating clinical simulations for learning procedural skills: a theory-based approach. Acad Med 80(6):549Google Scholar
- Maschuw K, Schlosser K, Kupietz E, Slater EP, Weyers P, Hassan I (2011) Do soft skills predict surgical performance?: a single-center randomized controlled trial evaluating predictors of skill acquisition in virtual reality laparoscopy. World J Surg 35(3):480–486PubMedView ArticleGoogle Scholar
- Nakamura J, Csikszentmihalyi M (2002) The concept of flow. In: Snyder CR, Lopez SJ (eds) Handbook of positive psychology. Oxford University Press, Oxford, pp. 89–105Google Scholar
- Creutzfeldt J, Hedman L, Medin C, Heinrichs WL, Felländer-Tsai L (2010) Exploring virtual worlds for scenario-based repeated team training of cardiopulmonary resuscitation in medical students. J Med Internet Res 12(3):e38PubMedView ArticleGoogle Scholar
- Peters M, Laeng B, Latham K, Jackson M, Zaiyouna R, Richardson C (1995) A redrawn Vandenberg and Kuse mental rotations test: different versions and factors that affect performance. Brain Cogn 28(1):39–58PubMedView ArticleGoogle Scholar
- Cosman PH, Hugh TJ, Shearer CJ, Merrett ND, Biankin AV, Cartmill JA (2007) Skills acquired on virtual reality laparoscopic simulators transfer into the operating room in a blinded, randomised, controlled trial. Stud Health Technol Inform 125:76–81PubMedGoogle Scholar
- Larsen CR, Grantcharov T, Aggarwal R, Tully A, Sørensen JL, Dalsgaard T, Ottesen B (2006) Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator. Surg Endosc 20(9):1460–1466PubMedView ArticleGoogle Scholar
- van Dongen KW, Tournoij E, van der Zee DC, Schijven MP, Broeders IAMJ (2007) Construct validity of the LapSim: can the LapSim virtual reality simulator distinguish between novices and experts? Surg Endosc 21(8):1413–1417PubMedView ArticleGoogle Scholar
- Bandura A (1997) Self-efficacy: the exercise of control, 1st edn. Worth Publishers, New YorkGoogle Scholar
- Pintrich PR, Smith DAF, Garcia T, Mckeachie WJ (1993) Reliability and predictive validity of the motivated strategies for learning questionnaire (Mslq). Educ Psychol Meas 53(3):801–813View ArticleGoogle Scholar
- Csıkszentmihályi M (1978) Beyond boredom and anxiety: experiencing flow in work and play. Jossey-Bass, San FranciscoGoogle Scholar
- Webster J, Trevino L (1993) Computers in human behavior: the dimensionality and correlates of flow in human-computer interactions. Comput Human Behav 9, 411–426Google Scholar
- Ghani JA, Deshpande SP (1994) Task characteristics and the experience of optimal flow in human—computer interaction. J Psychol 128(4):381–391View ArticleGoogle Scholar
- Schaafsma BE, Hiemstra E, Dankelman J, Jansen FW (2009) Feedback in laparoscopic skills acquisition: an observational study during a basic skills training course. Gynecol Surg 6(4):339–343PubMedView ArticleGoogle Scholar
- Artino AR, La Rochelle JS, Durning SJ (2010) Second-year medical students’ motivational beliefs, emotions, and achievement. Med Educ 44(12):1203–1212PubMedView ArticleGoogle Scholar
- Pain MA, Harwood C (2007) The performance environment of the England youth soccer teams. J Sports Sci 25(12):1307–1324PubMedView ArticleGoogle Scholar