Open Access

Non-technical factors influence laparoscopic simulator performance among OBGYN residents

  • Liv Ahlborg1, 2, 3Email author,
  • Leif Hedman1, 4,
  • Carsten Rasmussen7,
  • Li Felländer-Tsai1, 2, 6 and
  • Lars Enochsson1, 2, 5
Gynecological SurgeryEndoscopic Imaging and Allied Techniques20129:748

https://doi.org/10.1007/s10397-012-0748-2

Received: 3 March 2012

Accepted: 1 April 2012

Published: 10 May 2012

Abstract

In addition to technical skills, nontechnical factors appear to influence surgical results. This study aims to analyze how visuospatial ability, self-efficacy, and flow are associated with simulated laparoscopic performance of residents in obstetrics and gynecology (OBGYN). In this cohort study, 28 residents in obstetrics and gynecology were tested for visuospatial ability and self-efficacy prior to simulator training. All participants subsequently conducted a basic set of tasks in the simulator. Self-efficacy, once again, and flow were assessed after training. Nineteen of the subjects then completed a 2-day course with identical simulator tasks, although now to a predefined credential level. Visuospatial ability correlated with simulator performance in the technically most advanced simulator task in the basic set (“total time,” r = −0.40, p = 0.039). Flow correlated with: “right instrument pathway”(r = −0.40, p = 0.004) in that same task and with the 2-day overall training results (r = −0.56, p = 0.017). Self-efficacy correlated with the 2-day result (r = −0.56, p = 0.013) and significantly improved after training (p = 0.011). When constructing a curriculum for OBGYN residents, visuospatial abilities and non-technical factors like flow and self-efficacy should be considered.

Keywords

Gynecological laparoscopySimulator trainingNontechnical skills

Background

The lack of structured training programs for residents performing surgery is jeopardizing patient safety [13]. Simulators are becoming important training tools in preparing for various real-life scenarios. For example, computerized simulations appear to be beneficial for enhancing the technical performance among surgery and obstetrics and gynecology (OBGYN) residents [46].

In addition to technical skills, nontechnical factors appear to be important when constructing a surgical curriculum. Previous studies have reported behavior-rating systems for surgeons based on skills, such as situation awareness, decision making, communication, and leadership. Such systems appear to be beneficial for safe surgical practice [710].

Our research group has focused on trying to identify nontechnical factors that might influence the learning process during image-guided simulator training. Visuospatial ability, commonly defined as the capacity to generate “a mental representation of a two- or three-dimensional structure and then assessing its properties or performing a transformation of the representation” [11] appears to influence operative performance. We have previously reported that visuospatial abilities correlate with simulated laparoscopic performances among surgical novices and for consultants in OBGYN [1214]. Since simulator training appears to improve surgical performance, testing residents for visuospatial abilities could improve the design of individual training programs.

Another important feature of the learning process is self-efficacy, i.e., one’s belief in one’s ability to succeed in specific situations [15]. For example, Maschuw et al. [16] reported that low self-efficacy correlates with poor simulator performance among surgical residents. Flow, i.e., “to move or run smoothly with unbroken continuity and concentration and complete absorption in what one does” [17], is another factor that appears to be of importance for task performance. A recent study shows that repeated team training of cardiopulmonary resuscitation in a virtual world increase concentration, as well as self-efficacy in medical students [18]. Assessments of self-efficacy and flow might therefore be useful for evaluating a training program’s or an instrument’s effectiveness and feasibility.

The aim of the present study was to explore visuospatial ability, self-efficacy, and flow in OBGYN residents in relation to laparoscopic simulator performance, in preparation for constructing a gynecological curriculum.

Methods

The study group consisted of 28 OBGYN residents from 21 different hospitals in Sweden. Exclusion criteria were more than ten independently performed laparoscopic tubal occlusions or attendance at a simulator course prior to the study. The study intended to investigate the abilities among laparoscopic novices with no prior laparoscopic simulator training.

The study was conducted at the Center for Advanced Medical Simulation and Training (CAMST), at Karolinska University Hospital, Stockholm, Sweden, during a week in 2009. The regional Research Ethics Committee of Stockholm approved the study.

Visuospatial ability

In order to test for visuospatial ability, all participants first completed the redrawn Vandenberg and Kuse Mental rotations test, version A (MRT-A) [19]. The test consists of 24 items each containing a target figure to the left with four stimulus figures to the immediate right of the target figure. The task for the test subject is to mentally rotate the stimulus figures around the vertical axis. Participants had to identify both of two correct alternatives in order to get a score of “1.” Thus the maximum score was 24. Each participant was given the items organized into two subsets consisting of 12 items each. Three minutes were given to complete each subset. There was a 1-min break between each part. Instructions, procedures, and scoring were identical to those of Peters and collaborators [19].

Simulator performance

For laparoscopic gynecological simulation, we used the LapSimGyn® VR (virtual reality) (Surgical Science AB, Gothenburg, Sweden) simulator, which is an advanced laparoscopic simulator with proven construct validity [2022]. The system consists of software that runs on a Xeon-1.8 GHz processor using the Microsoft Windows® XP (Microsoft Corporation, Redmond, WA, USA) operating system. The computer is equipped with 256 MB internal RAM, a NVIDIA Quadro2 EX graphics card (NVIDIA Corporation, Santa Clara, CA, USA), a 15-inch monitor, and a virtual laparoscopic interface manufactured by Immersion, Inc. (San Jose, CA, USA).

All participants did one set of basic procedures to get familiar with the simulator and then one to two more sets to complete the basic simulator training.

Additionally, 19 of the subjects completed a 2-day course, which consisted of the same set of tasks. The subjects trained until they reached a predefined credential level in the simulator [4]. The simulator tasks performed were “cutting,” “clip applying,” “lifting and grasping,” “tubal occlusion,” and “salpingectomy.” The most demanding task in this set was “lifting and grasping,” and it was therefore selected in subsequent analyses. This skill, as well as the other in the set, has previously been reported to discriminate between novices and experts, i.e., to demonstrate construct validity [21]. In this task, you need to move a small needle with great precision and use a short distance (instrument path length), a small angle (instrument angular path length), and speed (total time) to reach the credential level.

The simulator parameters analyzed in the task “lifting and grasping” were “total time” (seconds), left and right “instrument angular path length” (degrees), and “instrument path length” (meters). The performance during the 2-day training course was measured by the number of trials until reaching the credential level in the task “lifting and grasping” and the sum of trials of all tasks until reaching credential level.

Self-efficacy

Individual self-efficacy is a capability in which cognitive, social, emotional, and behavioral skills must be organized and effectively orchestrated to serve several different purposes. Self-efficacy is not concerned with the number of skills you have, but what you believe you can do with the skills you have. Banduras’ cognitive theory suggests that, while successful performance enhances, repeated failures reduce perceived self-efficacy [23].

In this study, self-efficacy was assessed using a three-item questionnaire where each item was rated on a 7-grade Likert-type scale. The subjects completed the questionnaire before and after basic training as well as after the 2-day training course. Self-efficacy was calculated as the sum of all items [24].

Flow

Flow experience is reported to facilitate learning, encourage one to attempt a difficult task, promote the pace of creativity, increase the joy during an activity, and create meaning of involvement in an activity [25, 26].

Four categories of flow [27] were self-assessed using a 0–10 visual analog scale (enjoyment, four items; concentration, four items; control, three items; exploratory use, four items). A flow score was calculated as the sum of all items. Flow was assessed after the basic simulator set and after the 2-day simulator training.

Study design

The 28 OBGYN residents were first tested regarding visuospatial ability using MRT-A. They completed the self-efficacy evaluation before and after basic simulator training. The 19 participants taking the 2-day training course were additionally evaluated after the course. Flow was assessed following both basic and extensive simulator training (Fig. 1).
Fig. 1

Study design

Statistical analyses

Data analyses were carried out using JMP® version 9.0.0 (SAS Institute, Inc., Cary, NC, USA) for Mac OS X® version 10.5.7 (Apple, Inc., Cupertino, CA, USA). All the variables were tested for normality. Pearson’s correlation was used to analyze the relationship between visuospatial ability, self-efficacy, and flow, respectively, and simulated laparoscopic performance. Student’s t test was used to investigate differences in self-efficacy mean scores between the different sessions of simulator training. A level of p < 0.05 was considered to be statistically significant.

Findings

The basic simulator set

The 28 subjects (23 females and five males; mean age of 34 years ± 5 SD) performed a basic set of tasks in the simulator. The first of two sets was performed to allow the subjects to familiarize themselves with the simulator.

We analyzed the performance of the second session of the task “lifting and grasping.”

We first tested if visuospatial abilities and self-efficacy influenced the simulator performance. For this purpose, MRT-A scores were correlated with the parameter “total time” in the task “lifting and grasping.” These two parameters were indeed inversely correlated with each other (r = −0.40, p = .039; Fig. 2). The visuospatial ability and self-efficacy were not intercorrelated.
Fig. 2

Correlation between the visuospatial ability and the simulator performance in the basic set

No significant correlation was observed between self-efficacy, before or after the basic simulator set, and the simulator performance.

Next, we assessed whether the experience of flow correlated with the basic simulator performance. Flow correlated with four of the basic “lifting and grasping” simulator performance parameters analyzed (Table 1).
Table 1

Correlation between basic simulator performance and flow

Simulator performance

Flow

Lifting and grasping

r

p value

Right instrument angular path length (°)

−0.43

0.026

Left instrument angular path length (°)

−0.45

0.019

Right instrument path length (m)

−0.40

0.037

Left instrument path length (m)

−0.42

0.028

The 2-day simulator training

We next investigated whether visuospatial ability and self-efficacy had an impact on the simulator performance when subjects were trained. For this purpose, the 19 subjects (15 females and four males) who attended the 2-day course and who then all passed the pre-evaluated credential levels were further studied. Whereas the average number of sessions to reach the credential level in the task “lifting and grasping” was 31, the other tasks only required between four and 15 sessions, suggesting that “lifting and grasping “was the most demanding task. The average number of trials of all tasks, to reach credential level, was 69. When assessing simulator data from the 2-day course, we analyzed the number of trials until reaching the credential level in the task “lifting and grasping,” as well as the sum of trials of all tasks until reaching credential level. The sum of trials of all tasks to reach credential level was calculated to generate an overall result from the 2-day training.

We also wanted to analyze the association between extensive simulator training and flow.

Performance in “lifting and grasping” and the overall result correlated with flow and self-efficacy, assessed after the course. Furthermore, the overall simulator results correlated with self-efficacy scores assessed before training (Table 2).
Table 2

Correlations between simulator performance, self-efficacy, and flow

 

Self-efficacy before

Self-efficacy after

Flow

Simulator performance after training

r

p value

r

p value

r

p value

Sessions of lifting and grasping until passed course

−0.40

0.089

−0.61

0.007

−0.48

0.042

Total sessions until passed course

−0.56

0.013

−0.74

0.001

−0.56

0.017

Self-efficacy was assessed before and after the 2-day simulator course. Flow was assessed after the 2-day simulator course

MRT-A, however, did not correlate with the simulator performance when subjects were trained.

A comparison of self-efficacy scores before basic training and after the 2-day simulator training suggested improvement in self-efficacy after intensive simulator training (Fig. 3).
Fig. 3

Self-efficacy score before and after the 2-day training

Because of the gender imbalance, we also subanalyzed the female group. Most correlations became stronger, although when excluding the males, self-efficacy before the 2-day training and flow after did not significantly correlate to lifting and grasping. The correlations with the overall training result were still significant though.

Prior to the study, 14 of the residents had performed zero to one tubal occlusion, five of the residents had performed two to five, and ten of the subjects had performed six to ten laparoscopic tubal occlusion independently.

Discussion

In this study, we report correlations between visuospatial ability, flow, and self-efficacy, respectively, and the performance in image guided simulation, among residents in OBGYN. We also report an increase in self-efficacy after intense simulator training. To the best of our knowledge, these parameters have not previously been evaluated in OBGYN residents.

We chose to analyze the most difficult task in the simulator, “lifting and grasping” [21], in relation to self-efficacy, flow, and visuospatial ability. For this task, the average number of trials to reach the credential level was 31, whereas the typical gynecological procedures, “tubal occlusion” and “salpingectomy,” were not as challenging, requiring only 4 and 8 trials, respectively. These procedures were, however, included when we analyzed the overall result, i.e., the sum of all tasks, to reach the credential level.

In this study, visuospatial ability correlated with the outcome of simulator performance only in the technically more demanding “lifting and grasping” in the basic simulator set. This is in line with the study by Westman et al. [13] and Hedman et al. [14] who demonstrated that visuospatial abilities correlate positively with performance in the technically more demanding simulated colonoscopy and arthroscopy, respectively. Moreover, we previously reported that the visuospatial abilities of consultants in OBGYN correlated with the more demanding tasks during laparoscopic simulator training [12]. In the present study, visuospatial ability did not, however, correlate with simulator performance after training intensely during 2 days. These findings suggest that this type of training can compensate for a lesser visuospatial ability in novices. Testing the visuospatial ability can thus identify individuals who might benefit from such supplementary training and who need support in specific surgical tasks. It, however, remains to be established if visuospatial abilities also influence the actual surgical performance.

Others have reported that nontechnical factors play important roles in performing and learning the technical skills in surgery [810, 28]. Artino et al. reported that medical students who are bored and anxious achieve less theoretically. In their study, self-efficacy was positively correlated with enjoyment and negatively associated with boredom [29]. This is in line with Pain et al. [30] who reported that the performance of youth soccer players was negatively influenced by anxiety and boredom. Similarly, the findings of Maschuv et al. [16] suggest that surgical residents with low self-efficacy perform poorly during simulated laparoscopy.

In the present study, those who reported a high flow score, i.e., being able to concentrate and being engaged in and enjoying the task, performed better in the simulator, after the basic set as well as after extensive training, than those who reported low flow scores. This observation suggests that the simulator and the tasks are feasible and user friendly. One can speculate that the more the simulator task simulates an authentic operation, i.e., exhibit face validity, the greater the enjoyment, concentration, and performance will be. Simulators’ accessibility and feasibility thus could be evaluated according to the flow experience. Future studies should therefore investigate the relationship between the face validity of simulator tasks and the flow experience.

Self-efficacy scores correlated positively with simulator performance suggesting that a high self-confidence level is important for the simulated surgical result. Moreover, self-efficacy increased after intense training, indicating a gain in confidence from the training. This is important because a higher confidence might lead to a better result in the next training session.

The majority of tasks for most OBGYN practitioners are outside of the operating room (OR). Because the tasks performed in the OR are discipline specific, procedures among surgical residents cannot always be generalized to OBGYN residents. We therefore think it is important to distinguish residents in OBGYN from the residents in surgery, when investigating performance in the simulator, as well as the nontechnical skills that might impact the surgical performance, and furthermore when designing a training program.

One limitation in this study was the gender imbalance. This reflects the situation in at least a Swedish gynecological setting. The gender aspect should be further explored in a future study.

The findings from this exploratory study should be replicated by also using other validated instruments for assessing self-efficacy and flow, as well as a control group design in order to analyze for cause–effect relationships.

Conclusions

When building a curriculum for residents in OBGYN, factors like enjoyment and concentration, as well as confidence, need to be considered. Flow and self-efficacy should be further investigated in relation to the actual performance in the operating theater.

Declarations

Acknowledgments

The authors wish to express their gratitude to Mr. Lars-Gunnar Andersson and Mr. Tomas Askerud, Department for Biomedical Engineering and CAMST, at Karolinska University Hospital, Stockholm, for excellent technical assistance. We thank Dr. Christian R. Larsen for letting us use his proficiency criteria setting in the simulator. We furthermore thank Sue and Steve Anderson for editing grammar and language in our manuscript. This study was supported by research grants from the Stockholm County Council (Pick Up and ALF).

Disclosure of interest

None of the authors have any conflicts of interest to disclose.

Authors’ Affiliations

(1)
Center for Advanced Medical Simulation and Training (CAMST), Karolinska University Hospital
(2)
Department of Clinical Science, Intervention And Technology (CLINTEC), Karolinska Institutet at Karolinska University Hospital
(3)
Division of Obstetrics and Gynecology, Södertälje Hospital
(4)
Department of Psychology, Umeå University
(5)
Divisions of Surgery, Karolinska University Hospital
(6)
Divisions of Orthopaedics, Karolinska University Hospital
(7)
Divisions of Obstetrics and Gynecology, Karolinska University Hospital

References

  1. Scott DJ (2006) Patient safety, competency, and the future of surgical simulation. Simul Healthc 1(3):164–170PubMedView ArticleGoogle Scholar
  2. Scott DJ, Dunnington GL (2008) The new ACS/APDS Skills Curriculum: moving the learning curve out of the operating room. J Gastrointest Surg 12(2):213–221PubMedView ArticleGoogle Scholar
  3. Stefanidis D, Arora S, Parrack DM, Hamad GG, Capella J, Grantcharov T, Urbach DR, Scott DJ, Jones DB (2012) Association for Surgical Education Simulation Committee. Research priorities in surgical simulation for the 21st century. Am J Surg 203(1):49–53PubMedView ArticleGoogle Scholar
  4. Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, Schroeder TV, Ottesen BS (2009) Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 338:b1802PubMedView ArticleGoogle Scholar
  5. Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193(6):797–804PubMedView ArticleGoogle Scholar
  6. Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236(4):458–463, discussion 463–4PubMedView ArticleGoogle Scholar
  7. Carthey J (2003) The role of structured observational research in health care. Qual Saf Health Care 12 (Suppl 2):ii13–ii16Google Scholar
  8. Yule S, Flin R, Paterson-Brown S, Maran N, Rowley D (2006) Development of a rating system for surgeons’ non-technical skills. Med Educ 40(11):1098–1104PubMedView ArticleGoogle Scholar
  9. Crossley J, Marriott J, Purdie H, Beard JD (2011) Prospective observational study to evaluate NOTSS (Non-Technical Skills for Surgeons) for assessing trainees’ non-technical performance in the operating theatre. Br J Surg 98(7):1010–1020PubMedView ArticleGoogle Scholar
  10. Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S (2008) Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 32(4):548–556PubMedView ArticleGoogle Scholar
  11. Just MA, Carpenter PA (1985) Cognitive coordinate systems: accounts of mental rotation and individual differences in spatial ability. Psychol Rev 92(2):137–172PubMedView ArticleGoogle Scholar
  12. Ahlborg L, Hedman L, Murkes D, Westman B, Kjellin A, Felländer-Tsai L, Enochsson L (2011) Visuospatial ability correlates with performance in simulated gynecological laparoscopy. Eur J Obstet Gynecol Reprod Biol 157(1):73–77PubMedView ArticleGoogle Scholar
  13. Westman B, Ritter EM, Kjellin A, Törkvist L, Wredmark T, Felländer-Tsai L, Enochsson L (2006) Visuospatial abilities correlate with performance of senior endoscopy specialist in simulated colonoscopy. J Gastrointest Surg 10(4):593–599PubMedView ArticleGoogle Scholar
  14. Hedman L, Ström P, Andersson P, Kjellin A, Wredmark T, Felländer-Tsai L (2006) High-level visual–spatial ability for novices correlates with performance in a visual-spatial complex surgical simulator task. Surg Endosc 20(8):1275–1280PubMedView ArticleGoogle Scholar
  15. Kneebone R (2005) Evaluating clinical simulations for learning procedural skills: a theory-based approach. Acad Med 80(6):549Google Scholar
  16. Maschuw K, Schlosser K, Kupietz E, Slater EP, Weyers P, Hassan I (2011) Do soft skills predict surgical performance?: a single-center randomized controlled trial evaluating predictors of skill acquisition in virtual reality laparoscopy. World J Surg 35(3):480–486PubMedView ArticleGoogle Scholar
  17. Nakamura J, Csikszentmihalyi M (2002) The concept of flow. In: Snyder CR, Lopez SJ (eds) Handbook of positive psychology. Oxford University Press, Oxford, pp. 89–105Google Scholar
  18. Creutzfeldt J, Hedman L, Medin C, Heinrichs WL, Felländer-Tsai L (2010) Exploring virtual worlds for scenario-based repeated team training of cardiopulmonary resuscitation in medical students. J Med Internet Res 12(3):e38PubMedView ArticleGoogle Scholar
  19. Peters M, Laeng B, Latham K, Jackson M, Zaiyouna R, Richardson C (1995) A redrawn Vandenberg and Kuse mental rotations test: different versions and factors that affect performance. Brain Cogn 28(1):39–58PubMedView ArticleGoogle Scholar
  20. Cosman PH, Hugh TJ, Shearer CJ, Merrett ND, Biankin AV, Cartmill JA (2007) Skills acquired on virtual reality laparoscopic simulators transfer into the operating room in a blinded, randomised, controlled trial. Stud Health Technol Inform 125:76–81PubMedGoogle Scholar
  21. Larsen CR, Grantcharov T, Aggarwal R, Tully A, Sørensen JL, Dalsgaard T, Ottesen B (2006) Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator. Surg Endosc 20(9):1460–1466PubMedView ArticleGoogle Scholar
  22. van Dongen KW, Tournoij E, van der Zee DC, Schijven MP, Broeders IAMJ (2007) Construct validity of the LapSim: can the LapSim virtual reality simulator distinguish between novices and experts? Surg Endosc 21(8):1413–1417PubMedView ArticleGoogle Scholar
  23. Bandura A (1997) Self-efficacy: the exercise of control, 1st edn. Worth Publishers, New YorkGoogle Scholar
  24. Pintrich PR, Smith DAF, Garcia T, Mckeachie WJ (1993) Reliability and predictive validity of the motivated strategies for learning questionnaire (Mslq). Educ Psychol Meas 53(3):801–813View ArticleGoogle Scholar
  25. Csıkszentmihályi M (1978) Beyond boredom and anxiety: experiencing flow in work and play. Jossey-Bass, San FranciscoGoogle Scholar
  26. Webster J, Trevino L (1993) Computers in human behavior: the dimensionality and correlates of flow in human-computer interactions. Comput Human Behav 9, 411–426Google Scholar
  27. Ghani JA, Deshpande SP (1994) Task characteristics and the experience of optimal flow in human—computer interaction. J Psychol 128(4):381–391View ArticleGoogle Scholar
  28. Schaafsma BE, Hiemstra E, Dankelman J, Jansen FW (2009) Feedback in laparoscopic skills acquisition: an observational study during a basic skills training course. Gynecol Surg 6(4):339–343PubMedView ArticleGoogle Scholar
  29. Artino AR, La Rochelle JS, Durning SJ (2010) Second-year medical students’ motivational beliefs, emotions, and achievement. Med Educ 44(12):1203–1212PubMedView ArticleGoogle Scholar
  30. Pain MA, Harwood C (2007) The performance environment of the England youth soccer teams. J Sports Sci 25(12):1307–1324PubMedView ArticleGoogle Scholar

Copyright

© Springer-Verlag 2012

Advertisement