Patient safety risk factors in minimally invasive surgery: a validation study
© The Author(s) 2011
Received: 3 August 2011
Accepted: 5 November 2011
Published: 25 November 2011
This study was conducted to adapt and validate a patient safety (PS) framework for minimally invasive surgery (MIS) as a first step in understanding the clinical relevance of various PS risk factors in MIS. Eight patient safety risk factor domains were identified using frameworks from a systems approach to patient safety. A questionnaire was drafted containing 34 questions. Three experts in the field of patient safety critically reviewed the questionnaire on clinical relevance and completeness. The questionnaire was distributed among known patient safety experts in person and also sent electronically. A total of 41 questionnaires were distributed and the response rate was 71%. The intraclass correlation coefficient was 0.42 representing moderate agreement. For seven of nine risk domains, Cronbach’s alpha was sufficient (α > 0.7). Mean scores of the risk domains showed the following order of influence on patient safety from high to low: surgeon’s experience [6.6, standard deviation (SD) 0.5], technical skills surgeon (6.6, SD 0.7), technology (5.9, SD 1.1), complications (5.9, SD 1.2), social interaction (5.0, SD 1.0), leadership surgeon (5.4, SD 1.2), blood loss (5.0, SD 1.2), length of surgery (5.0, SD 1.3), surgical team (4.9, SD 1.3), fallibility (4.9, SD 1.3), patient (4.5, SD 1.5), safety measures (4.4, SD 1.5), and finally environment(3.9, SD 1.5). This study is an initiative to give insight into clinical relevance of the maze of PS risk factors in MIS. All investigated risk domains were considered to be of noticeable influence on PS. Nevertheless, it is possible to prioritize various risk domains. In fact, experience and technical skills of the surgeon, technology, and complications are rated as the most important risk factors, closely followed by social interaction and leadership of the surgeon. Patient, safety measures, and environment are rated as the least important risk factors.
Over the past years, society has become more quality driven. Also, in health care, the demand for the highest quality possible has become an important center of attention, especially in the operating room. This urgent request for high quality has made patient safety (PS), as a quality parameter, a very important research topic. Research concerning “patient safety” has expended explosively since 1999 and “patient safety” in the “operating room” has been increasing since 2004. Research concerning “patient safety” combined with “laparoscopy” only recently started to increase, whereas research on “laparoscopy” alone has been increasing fairly stable. All the while, research with the keyword “operating room” as well as the total number of yearly publications has been stable for years.
Recently, many PS issues considering minimally invasive surgery (MIS), especially laparoscopy, have been brought to our attention. In The Netherlands, PS issues about MIS (especially laparoscopy) brought to the attention by the Dutch Inspectorate of Healthcare even led to a discussion in the Dutch parliament about the desirability of MIS . Despite the great amount of research about PS and laparoscopy in general, only a few publications concern both items.
In general, patient safety is a complex multidimensional concept and the clinical relevance of its various dimensions is not clearly understood. To comprehend the clinical relevance of various PS risk factors in surgery, the degree to which a patient is exposed to all of these risk factors during surgery should be studied. Yet, studies that try to measure risk factors during surgery usually focus on specific parts of the multidimensional concept. Nonetheless, it has been widely accepted that a wide range of factors influence PS in surgery and a number of important studies [2–5] have addressed this by developing frameworks according to the systems approach to quality and safety in surgery. The aim of this study is to adapt and validate these frameworks for MIS as a first step in understanding the clinical relevance of various PS risk factors in MIS.
Surgeon: risk factors regarding functioning of the surgeon,
Surgical team: risk factors regarding functioning of the scrub or circulating nurse,
Technology: risk factors regarding the availability and functioning of technology,
Social interaction: risk factors regarding teamwork and communication,
Environment: risk factors that potentially cause distraction or disruptions of the surgical process,
Patient: patient-related risk factors,
Fallibility: risk factors regarding factors that influence the fallibility of the surgeon,
Safety measures: items regarding (compliance of) safety protocols, and
Result: items regarding the result of the procedure.
Categorization of risk factors
Experience of the surgeon
Technical skills of the surgeon
Leadership of the surgeon
Experience of the scrub nurse
Scrub nurse’s knowledge of the procedure
Experience of the circulating nurse
All instruments are present
All instruments work properly
It is known how to handle all instruments
All equipment is present
All equipment works properly
The OR team knows how to handle all equipment
The surgeons knows how to handle all equipment
Communication between OR team members
Failure of professional communication
Communication of important issues at shift changes
Collaboration between OR team members
Disruptions of the surgical process
Distractions (e.g., questions not relating to the patient)
Number of people in the OR
Patient’s ASA score
Time of day surgery takes place (e.g., daytime, nighttime)
Number of procedure (e.g., first or last of that day)
Universal safety protocols
Briefing according to WHO checklist
Compliance of safety measures (protocol)
Amount of blood loss
Length of surgery
Next, a sample of international gynecological PS experts was asked to rate all PS risk factors. A PS expert was defined as a gynecologist specialized in laparoscopic surgery, who is well recognized by their peers and that had either published a minimum of two articles on PS-related topics and/or is actively involved in a commission on PS. All experts were members of the European Society for Gynaecological Endoscopy and/or the American Association of Gynecologic Laparoscopists. The questionnaire was distributed internationally both directly among PS experts during a gynecological MIS conference (N = 12) and electronically by email (N = 29). The risk factors were rated according to their potential impact on PS in MIS on a 6-cm Visual Analogue Scale (VAS) with at the endpoints “no impact” and “huge impact”. Because of technological restrictions, the electronic version of the VAS was converted to a 13-point scale with, similar to the VAS, at the endpoint 0 “no impact” and at the endpoint 12 “huge impact” and every intermediate point corresponded with half a centimeter on the VAS. For conventional reasons and ease of interpretation, both scales were converted to a seven-point scale after all data were collected. The mean scores of the VAS and 13-point scale were first analyzed separately showing similar results for both rating scales. As such, it was decided to combine the two rating scales to report the results of this study.
All data were analyzed with SPSS 16.0 software package (SPSS, Chicago, IL, USA). An overall inter-rater agreement was determined for the complete questionnaire with the intraclass correlation coefficient (ICC). The ICC is exactly identical to the weighted kappa with quadratic weights  and is an appropriate method to determine an inter-rater agreement between multiple raters of multiple questions with a large rating scale as is the case in the current study. The ICC values were interpreted according to Landis and Koch’s  guidelines for the interpretation of kappa (kappa values <0 poor; 0 to 0.2 slight; 0.2 to 0.4 fair; 0.4 to 0.6 moderate; 0.6 to 0.8 substantial; and >0.8 almost perfect agreement).
For each PS risk factor domain (containing a number of separate risk factors), internal consistency was determined using Cronbach’s alpha. If internal consistency of a PS risk factor domain was accepted (Cronbach’s alpha >0.7) , further analysis was performed for that domain as a whole. If internal consistency was insufficient (Cronbach’s alpha <0.7), further analysis was performed for every risk factor separately. Mean scores and 95% confidence intervals (95% CI) were calculated per PS risk domain (or single risk factors if internal consistency was insufficient) and plotted. Statistical significant difference between PS risk domains was determined with one-way analysis of variance (ANOVA) and Bonferroni post hoc analysis (p value < 0.05).
Cronbach’s alpha per risk domain
Patient safety risk domain
There were no missing risk factors reported in the free space section. There were a few comments, which will be further discussed in the next section.
In this study, an adapted framework of PS risk factors in MIS was validated as a first step in understanding the clinical relevance of various PS risk factors in MIS. All investigated risk domains were rated of noticeable influence on PS, confirming the multidimensionality of the concept PS. This also implies that in order to completely assess PS in MIS, all of the proposed risk factors should be considered.
The results of this study should be interpreted carefully. The selection of PS experts comprises only gynecologists who specialize in MIS; therefore, the results can only be interpreted as valid for gynecological MIS. Further research is needed to examine whether these results can be extrapolated to other specialisms and whether other professionals that participate in MIS (i.e., anesthesiologists and OR nurses) share this opinion. Furthermore, it should be taken into account that there are differences in agreement on the importance of various risk factors. This is illustrated by the differences in width of the 95% CI of mean (Fig. 1). A small 95% CI should be interpreted as strong agreement and a wide 95% CI should be interpreted as weak agreement. As such, agreement on the importance of technology, experience of the surgeon, and social interaction is strongest and therefore the average rating can be interpreted as more reliable than the average rating of length of surgery, leadership of the surgeon, complications, and blood loss, of which the 95% CIs are wider (e.g., agreement is less strong). Yet, overall agreement of the complete questionnaire was moderate (ICC = 0.42) and the widest 95% CI is 1.0 point wide. Together with the response rate of 71%, this is considered satisfactory.
The free space section in the questionnaire gave experts the possibility to comment on the questionnaire. There were no missing risk factors reported; however, there were few relevant comments made. There was one comment on the importance of the first assistant’s experience by directly affecting safety and by adding experience to a difficult procedure. We agree, however, we considered this as part of the qualified staff. Another expert commented that the 13-point scale in the electronic version of the questionnaire gives too much choice and could lead to clustering of the answers in the middle of the scale. Because of software restrictions, converting the VAS to a Likert-type scale for the electronic questionnaire was imperative. We chose to let every half a centimeter on the VAS correspond to a point on the Likert-type scale, hence the origin of the 13-point scale. Furthermore, the results show an arrangement of the risk domains in given rating; therefore, it can be stated that clustering of the answers due to the large scale is no issue in the current questionnaire.
It is possible to prioritize certain risk domains over others. As such, experience and technical skills of the surgeon, technology, and complications have the highest ratings, closely followed by social interaction and leadership of the surgeon. The use of advanced technology in MIS easily explains the experts’ view on the importance of good functioning instruments and equipment. The magnitude of technology-related problems during MIS has previously been illustrated in several studies [9, 10]. Also, in gynecological MIS procedures, many technology-related problems, with potentially dangerous consequences, have been observed . In these studies, technology-related incidents mainly led to delay and extra work. The importance of technology in MIS is supported by the fact that the experts rate this domain among the most important risk domains in MIS. Surgeons could be facilitated in the optimal use of advanced technology in laparoscopic operating units by checklists. Checklists have already been shown to improve the use of the available technology and reduce technology-related incidents [9, 12]. Problems in social interaction (such as communication and teamwork) have also been shown to occur frequently during surgery [13, 14]. For example, communicational difficulties have been reported to occur in approximately 30% of team exchanges . About a third of these communicational failures resulted in visible effects that can influence PS . The importance of social interaction for PS in MIS is supported by the fact that the experts also rate this domain among the most important risk domains in MIS.
Many studies concerning PS in the operating room have been focusing on disruptions and distractions from the surgical process (environment) because they are believed to influence the surgeon’s concentration or are perceived as stressful events [15–20]. Remarkably, the experts rated this risk domain (environment) as the least influential of all risk domains. The results of an observational study are in line with the experts’ opinion as they found frequent disruptions, however, the stressfulness of disruptions were least severe compared to other (e.g., technical) incidents .
This study is an initiative to give insight into clinical relevance of the maze of PS risk factors in MIS. All investigated risk domains were considered to be of noticeable influence on PS. Nevertheless, it is possible to prioritize various risk domains. In fact, experience and technical skills of the surgeon, technology, and complications are rated as the most important risk factors, closely followed by social interaction and leadership of the surgeon. Although in general, patient safety measures and environment are among the most widely studied risk factors, experts in the field rate them as the least important risk factors. Although the effect of environmental factors should not be underestimated, experts in the field indicate that it should not be overestimated as well.
More research is needed to give us better insight in the pitfalls of patient safety as perceived in clinical practice to guide us in future research. As the results of this study can only be interpreted as valid for gynecological MIS, it would be interesting to examine whether experts of other specialisms (i.e., surgeons, anesthesiologists, and OR nurses) share the same view on patient safety. A focal group session including experts from different specialisms could give us more insight in the real need and concerns of experts on patient safety in minimally invasive surgery.
The authors are grateful to all experts that participated in this study.
Conflicts of interest
The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
- Inspectie voor de gezondheidszorg (2007) Risico’s minimaal invasieve chirurgie onderschat, kwaliteitssysteem voor laparoscopische operaties ontbreekt. IGZ, Den HaagGoogle Scholar
- Calland JF, Guerlain S, Adams RB, Tribble CG, Foley E, Chekan EG (2002) A systems approach to surgical safety. Surg Endosc 6:1005–1014View ArticleGoogle Scholar
- Vincent C, Moorthy K, Sarker SK, Chang A, Darzi AW (2004) Systems approaches to surgical quality and safety: from concept to measurement. Ann Surg 4:475–482View ArticleGoogle Scholar
- Leake PA, Urbach DR (2010) Measuring processes of care in general surgery: assessment of technical and nontechnical skills. Surg Innov 17(4):332–339PubMedView ArticleGoogle Scholar
- Dankelman J, Grimbergen CA (2005) Systems approach to reduce errors in surgery. Surg Endosc 8:1017–1021View ArticleGoogle Scholar
- Steiner DL, Norman GR (2008) Health measurement scales: a practical guide to their development and use. Oxford University Press, New YorkGoogle Scholar
- Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 1:159–174View ArticleGoogle Scholar
- Nunnaly JC (1967) Psychometric theory. McGraw Hill, New YorkGoogle Scholar
- Verdaasdonk EG, Stassen LP, Hoffmann WF, van der Elst M, Dankelman J (2008) Can a structured checklist prevent problems with laparoscopic equipment? Surg Endosc 10:2238–2243View ArticleGoogle Scholar
- Wubben I, van Manen JG, van den Akker BJ, Vaartjes SR, van Harten WH (2010) Equipment-related incidents in the operating room: an analysis of occurrence, underlying causes and consequences for the clinical process. Qual Saf Health Care 19:1–7View ArticleGoogle Scholar
- Courdier S, Garbin O, Hummel M, Thoma V, Ball E, Favre R, Wattiez A (2009) Equipment failure: causes and consequences in endoscopic gynecologic surgery. J Minim Invasive Gynecol 1:28–33View ArticleGoogle Scholar
- Buzink SN, van Lier L, de Hingh I, Jakimowicz JJ (2010) Risk-sensitive events during laparoscopic cholecystectomy: the influence of the integrated operating room and a preoperative checklist tool. Surg Endosc 8:1990–1995View ArticleGoogle Scholar
- Lingard L, Regehr G, Espin S, Whyte S (2006) A theory-based instrument to evaluate team communication in the operating room: balancing measurement authenticity and reliability. Qual Saf Health Care 6:422–426View ArticleGoogle Scholar
- Lingard L, Espin S, Whyte S, Regehr G, Baker GR, Reznick R, Bohnen J, Orser B, Doran D, Grober E (2004) Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Saf Health Care 5:330–334View ArticleGoogle Scholar
- Wiegmann DA, ElBardissi AW, Dearani JA, Daly RC, Sundt TM III (2007) Disruptions in surgical flow and their relationship to surgical errors: an exploratory investigation. Surgery 5:658–665View ArticleGoogle Scholar
- Arora S, Hull L, Sevdalis N, Tierney T, Nestel D, Woloshynowych M, Darzi A, Kneebone R (2010) Factors compromising safety in surgery: stressful events in the operating room. Am J Surg 1:60–65View ArticleGoogle Scholar
- Healey AN, Primus CP, Koutantji M (2007) Quantifying distraction and interruption in urological surgery. Qual Saf Health Care 2:135–139View ArticleGoogle Scholar
- Sevdalis N, Forrest D, Undre S, Darzi A, Vincent C (2008) Annoyances, disruptions, and interruptions in surgery: the Disruptions in Surgery Index (DiSI). World J Surg 8:1643–1650View ArticleGoogle Scholar
- Sevdalis N, Healey AN, Vincent CA (2007) Distracting communications in the operating theatre. J Eval Clin Pract 3:390–394View ArticleGoogle Scholar
- Zheng B, Martinec DV, Cassera MA, Swanstrom LL (2008) A quantitative study of disruption in the operating room during laparoscopic antireflux surgery. Surg Endosc 10:2171–2177View ArticleGoogle Scholar