Defining competencies in robotic thyroidectomy: development of a model assessing an expert operator’s intraoperative performance skills and cognitive strategies
Hyejin Kim1#, Hyeong Won Yu2#
, Jong-hyuk Ahn3, Tae Seon Lee4,5, Kyu Eun Lee6
Contributions: (I) Conception and design: TS Lee, KE Lee; (II) Administrative support: HW Yu, KE Lee; (III) Provision of study materials or patients: HW Yu, JH Ahn, KE Lee; (IV) Collection and assembly of data: H Kim; (V) Data analysis and interpretation: H Kim, HW Yu, TS Lee; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.
#These authors contributed equally to this work.
Background: The changing medical education environment emphasizes the need for time efficiency, increasing the demand for competency-based medical education to improve trainees’ learning strategies. This study was performed to determine the competencies required for successful performance of robotic thyroidectomy (RT) and to determine the consensus of experts for performance of RT.
Methods: Data were collected through 12 semi-structured interviews with RT experts and 11 field observations. Cognitive task analysis was performed to determine the competencies required for experts to perform RT. A modified Delphi methodology was used to determine how 20 experts rated the importance of each item of RT performance on a Likert 7-point scale. The criteria for the Delphi consensus were set at a Cronbach’s α≥0.80 with two survey rounds.
Results: After 11 field observations and 12 semi-structured interviews, 89 items were identified within six modules. These items were grouped into sub-modules according to their theme. The modified Delphi survey, involving 21 experts, reached the consensus standard during the second round (Cronbach’s α=0.954), enabling the identification of the 64 most important items within six modules related to RT performance: midline incision to isthmectomy (MID module; n=8), lateral dissection (LAT module; n=7), preservation of inferior parathyroid glands (INF module; n=16), preservation of recurrent laryngeal nerve and dissection of the ligament of Berry (BER module; n=21), dissection of the thyroid upper pole (SUP module; n=10), and specimen removal and closure (END module; n=2).
Conclusions: This mixed-method study combining qualitative and quantitative methodology identified modules of core competencies required to perform RT. These modules can be used as a standard and objective guide to train surgeons to perform RT and evaluate outcomes.
Keywords: Thyroidectomy; clinical competence; robotics; surgical procedures; reference standards
Submitted Nov 16, 2023. Accepted for publication Feb 25, 2024. Published online Mar 22, 2024.
doi: 10.21037/gs-23-467
Highlight box
Key findings
• This study organized the key tasks and decision-making procedures for robotic thyroidectomy (RT) into modules and assessed their significance in delineating the competencies for expert performance and the cognitive strategies essential for RT.
What is known and what is new?
• While RT demands advanced surgical skills and employs various surgical instruments and approaches, including behavioral and cognitive strategies, there is a lack of research defining the competencies necessary for successful RT performance when compared to conventional thyroidectomy.
• We identified 89 items within six modules that define the competencies for RT.
What is the implication, and what should change now?
• These modules can be used as a standard and objective guide to train surgeons to perform RT and evaluate outcomes.
Introduction
Recently, the educational paradigm in surgery has changed from the traditional apprenticeship model to competency-based medical education, which emphasizes the acquisition of standard technical and cognitive proficiency for performing surgery safely and effectively (1-3). To incorporate intraoperative decision-making ability into the standard surgical training curriculum, many studies have attempted to identify cognitive components associated with the success of various types of surgery, such as conventional open thyroidectomy, management of trauma patients, laparoscopic transabdominal adrenalectomy, and flexible pharyngo-laryngoscopy (3-6). Research on the techniques required for open thyroidectomy was utilized to develop high-definition video teaching modules and interactive web-based educational platforms (7,8). Although modules and platforms have been developed for open and laparoscopic surgery, surgeons are still being trained to perform robotic surgery through apprenticeship, without a standardized educational curriculum (9).
Robotic thyroidectomy (RT) is remote-access surgery and results in higher cosmetic satisfaction than conventional surgical methods (10). These advantages have increased demands for RT worldwide, increasing the number of training programs for RT. Previously, Madani and colleagues defined a model of competencies for successful open thyroidectomy performed by expert endocrine surgeons to use it as references for trainees in the context of competency-based medical education (3). RT, however, requires advanced surgical skills and utilizes different surgical instruments and approaches including behavioral and cognitive strategies compared with open thyroidectomy (11). Thus, this study was designed to organize key tasks and decision-making procedures for RT by modules and to evaluate their importance in determining the competencies of expert performance and cognitive strategies required for RT.
Methods
Setting and participants
This study recruited surgeons who perform RT with the bilateral axillo-breast approach (BABA). We defined subject matter experts (SMEs) as individuals who had conducted 40 or more BABA RT operations (11-14). Moreover, we included individuals with a minimum of 5 years of clinical experience to guarantee proficiency in BABA-RT. We used convenience sampling to enlist 12 SMEs for the qualitative cognitive task analysis (CTA) via email. Additionally, we recruited 21 surgeons through email for the modified Delphi survey. The purpose and method of the study were explained to the recruited surgeons, and they provided informed consent before participating in the study via telephone or email. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study protocol was approved by the Institutional Review Board of Seoul National University Hospital (No. H-1912-116-1090).
BABA RT
BABA RT is a surgical method in which the thyroid gland is dissected using a da Vinci robot inserted through small wounds on both sides of the axilla and breast (15). BABA RT differs from conventional open thyroidectomy, in that it starts with the lower pole of the thyroid gland and continues to the upper pole. BABA RT allows thyroid lobectomy, total thyroidectomy, and total thyroidectomy with modified lateral neck dissection (16,17). This study, however, was restricted to surgery involving a midline incision to remove the dissected specimen and midline closure. The RT procedures were divided into six modules, as described (15).
Qualitative study: CTA
This study was of mixed-method design, combining qualitative and quantitative methods. To collect qualitative data, one of the researchers observed the performance of RT by study participants, focusing mainly on communication with colleagues, emergency situations and coping methods, and changes in decisions during surgery. Each RT was video recorded and used as an adjunct in the subsequent semi-structured interview. The interviewee asked prescribed questions about the intent, cautions, and decision-making during each step of RT (Table 1) (3). The interviewer was careful not to interrupt the interviewee and to not ask close-ended questions, facilitating open and unbiased answers. Each interview was recorded, and each interview took roughly 1 hour. The recorded interviews were transcribed verbatim.
Table 1
| Structure | Questions |
|---|---|
| General questions | Describe the tasks required to perform BABA RT in key steps |
| Questions by module | What is the purpose of this task? |
| What is the sequence of actions necessary to complete this task? | |
| What conditions must be present before starting this task? | |
| What decisions have to be made during this task, including the various options and criteria to choose among options? | |
| What errors can occur and what tips/tricks can be used to avoid such errors during this task? | |
| What performance standards or quality indicators are used to ensure successful completion of this task? |
BABA RT, bilateral axillo-breast approach robotic thyroidectomy.
The qualitative data were analyzed by CTA, a robust method that systematically captures automated cognitive tasks performed by experts. CTA was based on naturalistic decision-making studies and has been used extensively to explore cognitive processes in medical settings (18). In this study, each surgical procedure performed by the surgeon was recorded and reviewed several times, and the cognitive factors of the task were analyzed by two task analysis experts (Ph.D. in cognitive psychology and medical education and two clinical health psychologists). To ensure the saturation of data, surgery textbooks that described the steps for open thyroidectomy and RT were reviewed to add any missing information (15,19-21).
The contents were subsequently reviewed and modified by two expert surgeons to develop a structured framework, completing it as modules in structured stages for the entire operation. This method defined the explicit and implicit knowledge and skills required, without bias, for surgical judgment and decision making.
Quantitative study: modified Delphi survey
The modified Delphi method, which is effective in drawing consensus from experts through iterative survey rounds and has been recommended for solving problems in clinical practice that lack scientific evidence (22), was utilized for quantitative analysis. Survey items were constructed from the data obtained from CTA. During the first round of the Delphi survey, participants were asked to anonymously rate the importance of each item, ranging from 1 (not at all important) to 7 (very important). During the second round, 1 week later, participants were asked to again rate the importance of each item. Rounds one and two included 21 and 20 RT experts, respectively, of various backgrounds and experience. The design of this study is shown in Figure 1.
Statistical analysis
To quantitatively verify the results obtained from the qualitative study, the responses to each survey round were reported as the means and variances. A consensus among experts was defined as a Cronbach’s α≥0.80 for the reliability of the participants’ responses to each item. Items evaluated by over 80% of respondents as ≥5 on the Likert scale were selected as the key component in implementing RT. The results were reported as the mean (standard deviation), median (range), and N (%). All statistical analyses were performed using Microsoft Excel® software. This mixed methodology study design has been used in various studies that define key operative competencies (2,3,6).
Results
Qualitative study: CTA
CTA analysis of the data collected by the interviews and the review of video recordings of RT identified 89 meaningful items, which were divided into six modules based on the order of the tasks: midline incision to isthmectomy (MID, n=18), lateral dissection (LAT, n=16), preservation of inferior parathyroid glands (INF, n=18), preservation of recurrent laryngeal nerve (RLN) and dissection of the ligament of Berry (BER, n=22), dissection of the thyroid upper pole (SUP, n=12), and specimen removal and closure (END, n=3). The items within each module were categorized based on structural similarities and arranged as submodules (Table 2; for the full list, see Table S1).
Table 2
| MID module (n=18) | LAT module (n=16) | INF module (n=18) | BER module (n=22) | SUP module (n=12) | END module (n=3) |
|---|---|---|---|---|---|
| Identification of the midline | Dissection of the surgical plane between the thyroid and the strap muscle | Identification of the RLN | Dissection between the medial thyroid and trachea | Dissection in the upward direction | Specimen out |
| • Find the correct midline location on the strap muscles | • Separate the strap muscles and thyroid gland from cranial to caudal | • Identify the RLN between the central lymph nodes | • Separate the trachea and cricothyroid muscle from the thyroid gland | • To prevent EBSLN injury, attach as much to the thyroid as possible and simultaneously ligate the upper thyroid artery well. | • If the thyroid is too large to remove, expand the trocar tunnel site sufficiently |
| Retract the midline bilaterally with both graspers | • Dissect the surgical plane as close as possible to the surface of the thyroid gland | • Ensure safe distances based on the range of heat conduction to prevent thermal injury | Dissection between the thyroid gland and fascia | • Avoid EBSLN injury | • Use a surgical lap bag to safely remove the specimen from the operative field to prevent metastasis to other sites |
| • When retracting the midline with graspers on both sides, provide proper symmetrical tension and cautions to prevent muscle tearing | • Be careful when dissecting between the thyroid gland and strap muscle | • Avoid retracting the thyroid gland excessively, as it may cause mechanical injury to the RLN | • Finish the lateral dissection on the lateral side of the thyroid gland | • Map the course of the EBSLN using nerve monitoring | Use of hemostatic dressing and anti-adhesion adjuvant |
| Midline incision | • Determine whether the strap muscle is injured | • Identify the course and the location of the RLN | Thyroid retraction—Zuckerkandl | Identification and preservation of the superior parathyroid glands | • Sew the strap muscles with running sutures during midline closure (cranial to caudal) |
| • Follow the surgical plane well and dissect it | Lateral retraction of the strap muscle | Identification of the inferior parathyroid gland | • Retract the thyroid to enable entering the harmonic | • Avoid bleeding during the ligation of the superior thyroid artery because the operative field is narrow | Drain insertion and midline closure |
| • Identify the sternothyroid and sternohyoid muscles | • Sufficiently separate the thyroid from the strap muscle | • Recognize the typical location and shape of the parathyroid gland | • Avoid RLN injury caused by traction | • Determine which blood vessels to leave | |
| • Ensure sufficient incision to Delphian lymph node | • Be careful not to pull the strap muscle excessively, as it can tear and bleed | • Identify the color of the parathyroid gland | Preservation of the RLN | • Avoid upper parathyroid injury | |
| • Avoid muscle injury when making the midline incision | Identification of the common carotid artery | • Identify the anatomical variations in the location of the parathyroid gland | • Consider the various shapes of the RLN | Identification and preservation of EBSLN | |
| • Incise from the thyroid cartridge to the suprasternal notch (or the location at which central node dissection is possible) | • Identify the correct depth and course of the common carotid artery | • Identify the blood stream distribution and blood vessel travel of the parathyroid gland | • Continue to check the course of the RLN from view to view | • Determine whether the EBSLN is functional | |
| Identification of the trachea | • Avoid the blood vessels around the common carotid artery | • Determine whether to leave the parathyroid or perform auto-transplantation after removal | • Distinguish the artery from the RLN | • Determine whether the signals come from the EBSLN using nerve monitoring | |
| • Avoid injury to the trachea | • Determine whether the common carotid artery moves well according to the heartbeat | • If it is difficult to distinguish between the lymph nodes and parathyroid, determine whether to leave some or remove all depending on the cancer stage | • Predict RLN location and angle | • Determine whether the cricothyroid muscle twitches | |
| • Expose the trachea as much as possible | • Determine whether the common carotid artery is well exposed along the thyroid gland | • Avoid damage to the parathyroid gland and blood vessels leading to the parathyroid | • Determine the intensity of pulling when the Berry ligament and RLN are adjacent | Ligation of the superior thyroid artery and vein | |
| Identification of the isthmus | Thyroid retraction—lower | Preservation of the blood stream of the parathyroid | • Dissect the RLN while protecting it by covering it with a gauze ball to prevent thermal or mechanical injury | • Adjust robotic arms for better visibility | |
| • Avoid injury to the trachea | • Accurately locate the parathyroid gland and RLN | • Preserve blood vessels that affect the parathyroid | • Strong retraction of the thyroid may damage the RLN | • Rapidly expose and ligate the superior thyroid artery | |
| • Confirm the location of the isthmus carefully | • Avoid bleeding in the thyroid capsule | • Ensure safe distances considering the range of heat conduction to prevent thermal injury | • Avoid thermal injury | Other items related to dissection of the thyroid upper pole | |
| • Determine whether the isthmus was visible as soon as the midline was opened from the sternothyroid muscle | • Expose the lower pole and part of the upper part of the thyroid gland | • Identify the inferior and middle thyroidal veins | • Occasionally, a non-recurrent laryngeal nerve is present that drives directly into the vagus nerve from the upper part of the subclavian artery and enters the larynx | • Use nerve monitoring to identify the vagus nerve (located close to the carotid artery) | |
| Isthmectomy | • Use a switching motion to support and lift the thyroid gland to check the tissue around the common carotid artery | • Preserve the parathyroid as much as possible | • Nerve monitoring determining the amplitude of the nerves when initially stimulated (whether the signal has been reduced by more than 50%) | ||
| • Preserve the inferior thyroid vein on the non-operative side | Other items related to lateral dissection | • Avoid injury to the parathyroid and parathyroid feeding vessels | Dissection of the ligament of Berry | ||
| • Consider the location of the isthmus | • Determine whether the central lymph nodes are removed cleanly along the thyroid gland | Other items associated with the preservation of the inferior parathyroid glands | • Expose Berry ligament sufficiently | ||
| • Avoid vessel injury (such as thyroid ima) | • Determine whether the middle thyroid vein is exposed and properly ligated | • Avoid retracting the parathyroid directly to prevent damage to the parathyroid | • Determine the intensity of pulling when the Berry ligament and RLN are adjacent | ||
| • Avoid injury to the cricoid cartilage | • If inevitable, retract tissues around the parathyroid or grab the blood vessels going to the parathyroid | • Dissect the thyroid gland below the Berry ligament | |||
| • Determine whether the left and right sides of the thyroid are separated | • Check the cricothyroid muscle in the upper area | ||||
| Other items related to midline incision and isthmectomy | • Minimize residual thyroid tissue, as microscopic amounts of thyroid tissue may remain when the thyroid and the RLN are attached, or when the thyroid tissue covers the RLN, similar to the ears | ||||
| • Determine preoperatively whether the isthmectomy is possible (if there is cancer on the isthmus itself, the isthmectomy position might have to be changed) | • Hemostasis is difficult if bleeding occurs in the Berry ligament | ||||
| • Determine whether the Berry ligament is well removed while protecting the RLN | |||||
| Other items related to preservation of the RLN, dissection of the ligament of Berry | |||||
| • Use a compression method with energy or a gauze ball for hemostasis |
MID, midline incision to isthmectomy; LAT, lateral dissection; INF, preservation of inferior parathyroid glands; BER, preservation of RLN and dissection of the ligament of Berry; SUP, dissection of the thyroid upper pole; END, specimen removal and closure; RLN, recurrent laryngeal nerve; EBSLN, external branch of superior laryngeal nerve.
Quantitative study: modified Delphi survey
In the quantitative study, the participants rated the 89 identified Delphi items by their importance. Twenty-seven RT experts, including participants in the first phase, were asked to participate in the Delphi survey, with 21 SMEs participating in the first round and 20 in the second round. The concordance of the responses among experts met the criteria for consensus set in the study, with the Cronbach’s α values of 0.91 in round 1 and 0.954 in round 2.
The most important of the 89 items was determined by selecting the items (n=64) that received scores ≥5 from >80% of respondents, with these items regarded as the critical behavioral and cognitive competencies for RT. Figure 2 shows the results of this study.
Table 3 shows the 10 most important items required for safe and efficient RT, as determined by experts. The five top-ranked items of each of the six modules are graphically shown in Figure 3. The order of these modules is not necessarily linear and can be altered at the discretion of the surgeon (for the full list, see Table S2).
Table 3
| Final rank | Module | Items | Round 1 | Round 2 | ||||
|---|---|---|---|---|---|---|---|---|
| Mean (SD) | % rating over 5 | Rank | Mean (SD) | % rating over 5 | ||||
| 1 | BER | Continue to check the course of the RLN from view to view | 6.67 (0.64) | 100 | 3 | 6.85 (0.65) | 95 | |
| 2 | LAT | Accurately locate the parathyroid gland and RLN | 6.76 (0.61) | 100 | 2 | 6.8 (0.68) | 95 | |
| 2 | INF | Identify the course and location of the RLN | 6.81 (0.50) | 100 | 1 | 6.8 (0.51) | 100 | |
| 4 | INF | Ensure safe distances based on the range of heat conduction to prevent thermal injury | 6.67 (0.56) | 100 | 3 | 6.7 (0.56) | 100 | |
| 5 | SUP | To prevent EBSLN injury, dissect as close to the thyroid as possible, and at the same time, ligate the upper thyroid artery well | 6.52 (0.59) | 100 | 6 | 6.55 (0.50) | 100 | |
| 6 | MID | Determine whether isthmectomy is possible preoperatively (if there is a cancer on the isthmus itself, the isthmectomy position might be changed) | 6.38 (0.79) | 100 | 12 | 6.5 (0.81) | 95 | |
| 6 | BER | Avoid thermal injury | 6.52 (0.66) | 100 | 6 | 6.5 (0.50) | 100 | |
| 8 | MID | Avoid injury to the trachea when identifying the isthmus | 6.43 (0.85) | 100 | 11 | 6.45 (0.67) | 100 | |
| 8 | INF | Excess retraction of the thyroid gland may cause mechanical injury to the RLN | 6.19 (0.91) | 90 | 20 | 6.45 (0.86) | 95 | |
| 8 | BER | Avoid RLN injury caused by traction | 6.48 (0.73) | 100 | 10 | 6.45 (0.74) | 100 | |
SD, standard deviation; BER, preservation of RLN and dissection of the ligament of Berry; RLN, recurrent laryngeal nerve; LAT, lateral dissection; INF, preservation of inferior parathyroid glands; SUP, dissection of the thyroid upper pole; EBSLN, external branch of superior laryngeal nerve; MID, midline incision to isthmectomy.
Additional comments regarding the MID module included: (I) the need for preoperative evaluation of the possibility of isthmectomy, as cancer on the isthmus itself may alter the isthmectomy position; and (II) that care should be taken not to injure the trachea. An additional comment regarding the LAT module consisted of accurate determination of the locations of the parathyroid gland and RLN, whereas additional comments regarding the INF module included (I) identifying the course and location of the RLN, (II) ensuring safe distances from the heat source based on the range of heat conduction thereby preventing thermal injury, (III) not retracting the thyroid gland excessively as it may cause mechanical injury to the RLN. Additional comments regarding the BER module included (I) checking the course of the RLN from view to view, (II) avoiding thermal injury, and (III) avoiding RLN injury caused by traction, whereas an additional comment regarding the SUP module noted that to prevent external branch of superior laryngeal nerve (EBSLN) injury, dissection should be made as close to the thyroid gland as possible while ligating the upper thyroid artery well (Table S3).
Discussion
The present study utilized CTA to identify 89 items within six modules for RT, with the modified Delphi survey identifying the 64 items finally in these six modules most important for RT performance, including 8, 7, 16, 21, 10, and 2 items in the MID, LAT, INF, BER, SUP, and END modules, respectively. The core performance skills and cognitive strategies required to perform BABA RT were determined using a mixed research method.
The list of procedural tasks and non-technical skills defined throughout this study may function as the basis for developing a standardized and valid training program, which can enhance surgeon proficiency in surgical skills and ultimately ensure patient safety. In particular, we attempted to evaluate the factors associated with competency-based medical education, including knowledge, skills, values, and attitudes, which could be reflected in both the behavioral tasks and cognitive schema of RT (4,23). The skills learned in the operative field are based on both situational and practice-based learning, both of which could affect the development of professional identity throughout the proper training and evaluation as a thyroid surgeon.
Previous studies have attempted to define the competencies required for each targeted medical treatment, including open thyroidectomy, radiation protection, advanced care planning, endoscopic submucosal dissection, and laparoscopic transabdominal adrenalectomy (2-4,24-26). In some of these studies, when the researchers developed the items for the Delphi consensus, they only used literature review or task analyses of their own performances (24-26). In addition, two of these studies analyzed the performance of multiple SMEs, but did not apply modified Delphi methodology for validation (2,4). This study adopted a mixed-method approach, differentiating this study from previous research, enabling the collection of as much unbiased information as possible and determining the most important competencies for RT, as assessed by various experts, thereby improving its validity (3).
This study, however, did not assess pre-operative steps, such as adjusting the settings on the robot. Although intraoperative patient care includes pre-operative preparation and post-operative management, this study assessed operative skills and techniques with the surgical robot. Pre-operative preparation for RT may include steps associated with the ease of tool usage, as settings for the surgical robot before the operative procedure can be important factors. Another consideration is that our study is demographically homogeneous. Since BABA RT is a surgical method that originated in Korea, this initial Delphi study had to focus on expert surgeons in Korea. However, presently, the same surgery is being conducted on individuals of different races in various countries worldwide. With the insights gained from this study, it will be possible to develop surgical guidelines from a global perspective in the future. Nevertheless, this study focused on the surgical process and analyzed the surgeon’s cognitive behavior when performing RT, resulting in meaningful results, regarded as offsetting the preoperative process. The unique features of RT that is distinct from open thyroidectomy identified in the present study were the proficient use of robotic instruments and the method of securing the operative field visually. These characteristics may be regarded as results associated with differences in surgical tools.
Surgeons should be mindful that, despite their proficiency in the surgical skills we provide, complications may arise in rare cases. A recent analysis of a large-scale robotic surgery study revealed rare complications in 60 out of 5,011 patients. These rare complications comprised hematoma in 4 cases (0.44%), chyle leakage in 15 cases (0.3%), flap injury in 4 cases (0.08%), RLN injury in 7 cases (0.14%), open conversion in 8 cases (0.16%), and pneumothorax in 4 cases (0.08%) (27).
Conclusions
Because of the lack, to date, of step-by-step surgical procedure guidelines for BABA-RT, the results of this study may be used to develop standardized educational criteria for training novice surgeons sufficiently to perform operations independently. These systematic and evidence-based procedures for RT could reduce the quality gap in accordance with training settings, and may contribute to the long-term narrowing of medical gaps among communities and surgeons. It might also help RT trainees to form accurate mental representations of successful performance and improve their surgical skills based on more detailed feedback.
Acknowledgments
The authors thank all expert panel members participating in the Delphi process.
Funding: This research was supported by
Footnote
Data Sharing Statement: Available at https://gs.amegroups.com/article/view/10.21037/gs-23-467/dss
Peer Review File: Available at https://gs.amegroups.com/article/view/10.21037/gs-23-467/prf
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://gs.amegroups.com/article/view/10.21037/gs-23-467/coif). The authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study protocol was approved by the Institutional Review Board of Seoul National University Hospital (No. H-1912-116-1090) and informed consent was taken from all the participants.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- Ryan MS, Lomis KD, Deiorio NM, et al. Competency-Based Medical Education in a Norm-Referenced World: A Root Cause Analysis of Challenges to the Competency-Based Paradigm in Medical School. Acad Med 2023;98:1251-60. [Crossref] [PubMed]
- Madani A, Grover K, Kuo JH, et al. Defining the competencies for laparoscopic transabdominal adrenalectomy: An investigation of intraoperative behaviors and decisions of experts. Surgery 2020;167:241-9. [Crossref] [PubMed]
- Madani A, Watanabe Y, Vassiliou M, et al. Defining competencies for safe thyroidectomy: An international Delphi consensus. Surgery 2016;159:86-94, 96-101. [Crossref] [PubMed]
- Madani A, Vassiliou MC, Watanabe Y, et al. What Are the Principles That Guide Behaviors in the Operating Room?: Creating a Framework to Define and Measure Performance. Ann Surg 2017;265:255-67. [Crossref] [PubMed]
- Melchiors J, Henriksen MJV, Dikkers FG, et al. Diagnostic flexible pharyngo-laryngoscopy: development of a procedure specific assessment tool using a Delphi methodology. Eur Arch Otorhinolaryngol 2018;275:1319-25. [Crossref] [PubMed]
- Madani A, Gips A, Razek T, et al. Defining and Measuring Decision-Making for the Management of Trauma Patients. J Surg Educ 2018;75:358-69. [Crossref] [PubMed]
- Madani A, Gornitsky J, Watanabe Y, et al. Measuring Decision-Making During Thyroidectomy: Validity Evidence for a Web-Based Assessment Tool. World J Surg 2018;42:376-83. [Crossref] [PubMed]
- Chorath KT, Luu NN, Douglas JE, et al. Assessment of YouTube as an educational tool in teaching thyroidectomy and parathyroidectomy. J Laryngol Otol 2022;136:952-60. [Crossref] [PubMed]
- Larkins K, Khan M, Mohan H, et al. A systematic review of video-based educational interventions in robotic surgical training. J Robot Surg 2023;17:1329-39. [Crossref] [PubMed]
- Li C, Gao Y, Zhou P, et al. Comparison of the Robotic Bilateral Axillo-Breast Approach and Conventional Open Thyroidectomy in Pediatric Patients: A Retrospective Cohort Study. Thyroid 2022;32:1211-9. [Crossref] [PubMed]
- Fassari A, Gurrado A, Iossa A, et al. Definition of learning curve for thyroidectomy: systematic review on the different approaches. Gland Surg 2023;12:989-1006. [Crossref] [PubMed]
- Liu SY, Kim JS. Bilateral axillo-breast approach robotic thyroidectomy: review of evidences. Gland Surg 2017;6:250-7. [Crossref] [PubMed]
- Lee J, Yun JH, Choi UJ, et al. Robotic versus Endoscopic Thyroidectomy for Thyroid Cancers: A Multi-Institutional Analysis of Early Postoperative Outcomes and Surgical Learning Curves. J Oncol 2012;2012:734541. [Crossref] [PubMed]
- Kim WW, Jung JH, Park HY. The Learning Curve for Robotic Thyroidectomy Using a Bilateral Axillo-Breast Approach From the 100 Cases. Surg Laparosc Endosc Percutan Tech 2015;25:412-6. [Crossref] [PubMed]
- Youn YK, Lee KE, Choi JY. Robotic Thyroidectomy: Bilateral Axillo-Breast Approach (BABA). In: Youn YK, Lee KE, Choi JY, editors. Color Atlas of Thyroid Surgery: Open, Endoscopic and Robotic Procedures. Berlin, Heidelberg: Springer; 2014:83-125.
- You JY, Kim HK, Kim HY, et al. Bilateral axillo-breast approach robotic thyroidectomy: review of a single surgeon's consecutive 317 cases. Gland Surg 2021;10:1962-70. [Crossref] [PubMed]
- Kang YJ, Stybayeva G, Hwang SH. Surgical safety and effectiveness of bilateral axillo-breast approach robotic thyroidectomy: a systematic review and meta-analysis. Braz J Otorhinolaryngol 2023; Epub ahead of print. [Crossref]
- Swaby L, Shu P, Hind D, et al. The use of cognitive task analysis in clinical and health services research - a systematic review. Pilot Feasibility Stud 2022;8:57. [Crossref] [PubMed]
- Korean Association of Thyroid and Endocrine Surgeons. Textbook of Endocrine Surgery. Seoul, Republic of Korea: Koonja Publisher; 2018.
- Zollinger R, Ellison E, Pawlik T, et al. Zollinger’s Atlas of Surgical Operations. 11th edition. McGraw Hill; 2021.
- Pertsemlidis D, Inabnet WB 3rd, Gagner M. Endocrine Surgery. 2nd edition. Boca Raton: CRC Press; 2016.
- Nasa P, Jain R, Juneja D. Delphi methodology in healthcare research: How to decide its appropriateness. World J Methodol 2021;11:116-29. [Crossref] [PubMed]
- The Accreditation Council for Graduate Medical Education. Surgery Milestones. 2019.
- Doyen B, Maurel B, Cole J, et al. Defining the Key Competencies in Radiation Protection for Endovascular Procedures: A Multispecialty Delphi Consensus Study. Eur J Vasc Endovasc Surg 2018;55:281-7. [Crossref] [PubMed]
- Sudore RL, Heyland DK, Lum HD, et al. Outcomes That Define Successful Advance Care Planning: A Delphi Panel Consensus. J Pain Symptom Manage 2018;55:245-255.e8. [Crossref] [PubMed]
- Takao M, Bilgic E, Waschke K, et al. Defining competencies for endoscopic submucosal dissection (ESD) for gastric neoplasms. Surg Endosc 2019;33:1206-15. [Crossref] [PubMed]
- Kwak J, Yu HW, Ahn JH, et al. A Time Trend Analysis of 5,000 Robotic Thyroidectomies via Bilateral Axillo-Breast Approach. World J Surg 2023;47:403-11. [Crossref] [PubMed]
