Development and validation of web-based nomograms for predicting lateral lymph node metastasis in patients with papillary thyroid carcinoma
Original Article

Development and validation of web-based nomograms for predicting lateral lymph node metastasis in patients with papillary thyroid carcinoma

Yi Dou, Yingji Chen, Daixing Hu, Wei Xiong, Qi Xiao, Xinliang Su

Department of Endocrine and Breast Surgery, the First Affiliated Hospital of Chongqing Medical University, Chongqing 400016, China

Contributions: (I) Conception and design: Y Dou, X Su; (II) Administrative support: X Su, Y Chen; (III) Provision of study materials or patients: Y Dou, D Hu, W Xiong; (IV) Collection and assembly of data: Y Dou, Y Chen, W Xiong, Q Xiao; (V) Data analysis and interpretation: Y Chen, W Xiong, Q Xiao, X Su; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

Correspondence to: Xinliang Su, MD. Department of Endocrine and Breast Surgery, the First Affiliated Hospital of Chongqing Medical University, Chongqing 400016, China. Email: suxinliang@21cn.com.

Background: The purpose of this study was to evaluate the factors associated with lateral lymph node metastasis (LLNM) in patients with papillary thyroid carcinoma (PTC), and to develop two web-based nomograms that predict the probability of level-II and level-III/IV LLNM in these patients.

Methods: The records of 653 patients with PTC were retrospectively reviewed. Univariate and multivariate analyses were performed to identify risk factors associated with LLNM in 460 patients (“derivation group”). Two models [including and excluding the subregions of central lymph node metastasis (CLNM)] were used to predict the probability of level-II LLNM; the same two models were also used for level-III/IV LLNM. Model performance was assessed using receiver operating characteristic (ROC) analysis and decision curve analysis (DCA) in 193 patients (“validation group”). Two web-based nomograms were established.

Results: Increased tumor size, a tumor in the upper lobe, and prelaryngeal and ipsilateral paratracheal lymph node metastasis (LNM) were significantly associated with level-II LNM (P<0.05). Increased tumor size, a tumor in the upper lobe, and certain subregions of CLNM were associated with level-III/IV LNM (P<0.05). Use of ROC analysis of each model indicated that including subgroups of CLNM led to better model performance than excluding these subgroups. We quantified the benefit of each model by using DCA analysis in the validation group.

Conclusions: Our web-based nomograms provide quantification of risk for LLNM in patients with PTC before and during surgery.

Keywords: Papillary thyroid carcinoma (PTC); lymph node dissections; lymphatic metastasis; nomogram


Submitted Aug 24, 2019. Accepted for publication Jan 04, 2020.

doi: 10.21037/gs.2020.01.11


Introduction

Papillary thyroid carcinoma (PTC) is the most common type of endocrine malignant tumor, and epidemiological studies indicate that its incidence has increased significantly in recent years (1). PTC is expected to be the third-most common female malignant tumor in 2019 (2). Previous studies have reported that 20% to 90% of PTC patients had cervical lymph node metastases at the time of diagnosis (3,4). Other studies reported that lateral lymph node metastases (LLNM) occurred in 17.3% to 60% of all cases (5,6), and metastasis in this region was associated with an increased rate of recurrence (7) and poor outcome (8,9).

Ultrasonography (US) and US-guided fine needle aspiration biopsy (FNAB) are the main methods used to evaluate lymph node status. However, the diagnostic sensitivity of US for detection of LLNM is only 27.4% to 84% (10,11), and the false negative rate is as high as 44.6% (12). Lymph node status can also be assessed by frozen section examination (FSE) of the tissue, and this method has a sensitivity of 80.7% and a specificity of 100% (13).

Lymph node metastasis (LNM) of PTC typically occurs first in the central compartment (level-VI), before spreading to the ipsilateral and lateral compartments. Central lymph node metastasis (CLNM) is usually a strong predictor of LLNM (14). Moreover, a meta-analysis showed there was a 7.64-fold increased risk of LLNM when CLNM was present (15). However, the association between positivity of subregions of the central lymph nodes (CLNs) and lateral lymph nodes (LLNs) is still uncertain. Thus, it is necessary to identify risk factors for CLNM to reduce unnecessary lymph node dissection and to perform a more precise operation, especially when CLNs are involved.

The purpose of this study is to establish two web-based nomograms that consider subregions of CLN status to predict the preoperative risk of LLNM in patients with PTC.


Methods

Patients

The study was approved by the local institutional ethics committee. The records of 1,089 consecutive patients with PTC who underwent surgery between January 2016 and December 2017 at the Department of Endocrine and Breast Surgery of the First Affiliated Hospital of Chongqing Medical University were retrospectively reviewed. Patients were excluded if they had a history of head or neck irradiation, other types of thyroid malignancies, or a previous thyroid operation. Based on these criteria, 653 patients with PTC were included.

Treatment strategy

Physical examination, ultrasonography, and fine-needle aspiration biopsy (FNAB) were routinely performed on each patient before surgery. All 653 patients underwent total thyroidectomy with prophylactic or therapeutic ipsilateral CLN and LLN dissection. Frozen biopsies of the CLNs with subregions (including prelaryngeal, pretracheal, and ipsilateral paratracheal lymph nodes) were routinely performed. Because the posterior lymph node of the recurrent laryngeal nerve is only on the right side, it was classified with the paratracheal lymph nodes. The LLNs were removed by formal modified radical neck dissection, including level-II (IIa and IIb), -III, and -IV. All harvested lymph nodes were examined pathologically according to the different regions, and were diagnosed by three pathologists.

Clinicopathological variables

Clinicopathological information, including age, sex, tumor size (maximal diameter), tumor location (upper/middle/lower/whole thyroid), tumor bilaterality, tumor multifocality, Hashimoto thyroiditis (HT), extrathyroidal extension (ETE), CLNM, LLNM, and number of positive metastatic prelaryngeal, pretracheal, and ipsilateral paratracheal lymph nodes were recorded. A tumor occupying the entire thyroid was considered localized to more than two subareas. Tumor bilaterality, multifocality, HT, and ETE were confirmed by pathological examination. Bilaterality was defined as at least one tumor in both lobes.

Statistical analysis

Patients were randomly divided to a “derivation group” and “validation group”. A t-test, chi-square test, or Fisher’s exact test was used to compare the baseline characteristics of these two groups, and the differences between patients with or without level-II or level-III/IV metastases in the derivation group.

All variables with significant differences in univariate analysis were entered into a multivariate logistic regression analysis. Odds ratios (ORs), adjusted odds ratios (aORs), and 95% confidence intervals (CIs) were calculated to determine the significance of all potential predictors. Two different models were independently used to compare the predictive ability of CLN for level-II and level-III/IV LLNM.

Receiver operating characteristic (ROC) curves were used to determine the discriminability of the models. Decision curve analysis (DCA) was used to calibrate and evaluate the models for the validation group, which allowed comparison of predicted probabilities with actual probabilities. Two web-based nomograms were designed using the better model (based on above analyses) for prediction of LLNM.

A two-tailed P value below 0.05 was considered statistically significant. Univariate and multivariate data were analyzed using SPSS version 23.0 (SPSS Inc., Chicago, IL, United States). ROC curves, DCA curves, and nomograms were generated using R software. The web applications were built using the DynNom package and the shinyapps within R software.


Results

Patient characteristics

We examined 472 women (72.3%) and 181 men (27.7%) with PTC (Table 1). The average age at diagnosis was 42.4±12.1 years, the average tumor size was 13.5±9.0 mm, Hashimoto’s thyroiditis (HT) was present in 120 patients (18.4%), and histological ETE was present in 142 patients (21.7%). Analysis of metastases indicated that 423 patients (64.8%) had CLNM and 275 patients (42.1%) had LLNM. The derivation and validation groups had no significant differences in any baseline characteristics (P>0.05 for all comparisons).

Table 1
Table 1 Baseline clinicopathological characteristics of patients with PTC (n=653)
Full table

Clinicopathological characteristics of patients in the derivation group

A total of 72 patients (15.6%) had level-II LNM, and a univariate analysis (Table 2) indicated that this condition was significantly associated with tumor size (P<0.001), tumor location (P<0.001), ETE (P=0.026), tumor bilaterality (P=0.022), and the number of subregions with CLNM (P<0.001). A total of 186 patients (40.4%) had level-III/IV LNM, and a univariate analysis (Table 2) indicated that this condition was significantly associated with tumor size (P<0.001), tumor location (P<0.001), ETE (P<0.001), tumor bilaterality (P=0.005), and number of subregions with CLNM (P<0.001).

Table 2
Table 2 Univariate analysis and multivariate analysis of factors associated with level-II, and level-III/IV lymph node metastasis in the derivation group (n=460)
Full table

Multivariate analysis (Table 2) indicated that large tumor size (10 to 20 mm vs. more than 20 mm) was associated with a higher risk of level-II LLNM (P=0.001) and level-III/IV LLNM (P=0.001). In addition, the risk of level-II LNM was nearly 6-fold higher when the tumor was in the upper lobe rather than elsewhere in the thyroid (OR =5.95; 95% CI, 2.79 to 12.65; P=0.001). Subregions of the CLNM were also significant independent predictors for level-II and level-III/IV LNM.

Prediction models

Based on a multivariate logistic regression analysis of the derivation group, we developed two models that incorporated significant risk factors for each lymph node group area, one with and one without subregions of CLNM.

Level-II lymph nodes

Model A1: size group + location + CLNM.

Model A2: size group + location + prelaryngeal LNM + paratracheal LNM.

Level-III/IV lymph nodes

Model B1: age group + size group + location + CLNM.

Model B2: age group + size group + location + prelaryngeal LNM + pretracheal LNM + paratracheal LNM.

We then performed ROC analysis for the derivation and validation groups using each model (Figure 1). The derivation cohort had an area under the curve (AUC) of 0.795 (95% CI, 0.78 to 0.83) for Model A1, 0.835 (95% CI, 0.79 to 0.86) for Model A2, 0.835 (95% CI, 0.78 to 0.85) for Model B1, and 0.855 (95% CI, 0.82 to 0.87) for Model B2. The AUC of model A2 was greater than that of model A1 (P=0.01), and the AUC of model B2 was greater than that of model B1 (P=0.01). The results were similar for the validation cohort.

Figure 1 Receiver operating characteristic curves for the four models. (A) Derivation group (AUCA1 =0.795, AUCA2 =0.835); (B) derivation group (AUCB1 =0.835, AUCB2 =0.855; (C) validation group (AUCA1 =0.739, AUCA2 =0.786); (D) validation group (AUCB1 =0.773, AUCB2 =0.820).

Decision curve analysis

The AUC from the ROC analysis only considers the accuracy of a model, but it is difficult to use these results to determine the optimal combination of sensitivity and specificity. DCA can examine different diagnostic and prognostic approaches, and identify the net benefit of a prediction model; a greater net benefit determined from DCA indicates that a model is better. Thus, we used DCA for Models A1, A2, B1, and B2 to predict the correct diagnosis of LLNM in PTC patients. The results (Figure 2) indicated that all models were useful for threshold probabilities of 40% to 60%. The net benefit of Model A2 was better than that of Model A1 for threshold probabilities of 40% to 60%, and the net benefit of Model B2 was better than that of Model B1 between threshold probabilities of 40% to 80%.

Figure 2 Decision curves for the four models. (A) Net benefit of Model A1 and A2 in making a correct diagnosis of LLNM; (B) net benefit of Model B1 and B2 in making a correct diagnosis of LLNM. “None” indicates that all samples were negative without intervention and the net benefit was 0. “All” indicates that all samples were positive with intervention.

Nomograms

Based on the results of the DCA, we developed nomograms for level-II and level-III/IV LNM (Figure 3). Analysis of level-II lymph nodes indicated that prelaryngeal LNM had the largest effect, followed by tumor size, tumor location, and paratracheal LNM. Pretracheal LNM was not an independent factor for level-II metastasis. Analysis of level-III/IV lymph nodes indicated that prelaryngeal, pretracheal, and paratracheal LNMs had significant and independent effects. The nomograms show the score of each measured variable on each scale. Thus, the probability of level-II or level-III/IV metastasis for each individual is determined by the total score of all variables. A clinician can estimate the individual probability of a level-II or level-III/IV metastasis by simply entering the necessary clinicopathological data, which are available online (https://thyroidcarcinomacqmu.shinyapps.io/level2/ and https://thyroidcarcinomacqmu.shinyapps.io/level34/). Figure 4 shows a screenshot of an example from the web-based nomogram for level-III/IV metastasis. This example shows a hypothetical patient with PTC who had an 8 mm tumor on the upper lobe with metastasis to 2 prelaryngeal lymph nodes, 0 pretracheal lymph nodes, and 1 paratracheal lymph node. The results show that this patient has a 56% probability of level-III/IV LLNM. Any computer system with R software can also launch the nomogram locally, following download of the bundled program.

Figure 3 Nomograms for (A) prediction of level-II lymph node metastasis and (B) prediction of level-III/IV lymph node metastasis.
Figure 4 Screen shot of the web-based nomogram used to predict level-III/IV lateral lymph node metastasis (https://thyroidcarcinomacqmu.shinyapps.io/level34/).

Discussion

Decisions regarding the extent of surgery needed for a patient with PTC are mainly based on the preoperative assessment of lymph node status (16,17). Even patients with clinical negative lymph node (cN0) PTC, 41.6% have CLNM (18) and 35% have LLNM (19). Intraoperative FSE is the best method of providing pathological evidence during the operation, and the results of this procedure can be used to guide more accurate lymph node dissection.

To the best of our knowledge, this is the first study of PTC patients to compare the risk of LLNM in different subgroups of patients with CLNM. We found that different subregions of the CLNs played important roles in predicting LLNM. For example, the presence of 2 or more prelaryngeal lymph node metastases (rather than no metastases) was associated with a 17.08-fold increased probability of level-II metastasis, but pretracheal lymph node metastasis had no significant effect on level-II metastasis. The relationship of the anatomical distance from the site of metastasis to the pretracheal lymph node area and level-II metastasis should also be considered. Other studies have reported that the positivity of CLNs is valuable in prediction of LLNM (20,21), however, “skip metastasis” (LLNM without CLN involvement) is also common (3.0% to 21.8%) (22,23). In agreement, 35 of our patients (5.4%) had positive LLNM and negative CLNM. Therefore, a single pathway, or sentinel lymph node, may not adequately characterize cervical lymph drainage. It is better to regard the subregions of CLNs as independent regions.

Level-II to level-IV is the main area of thyroid lymphatic drainage, and LNM is most common in this region (24). However, the American Thyroid Association (ATA) and the National Comprehensive Cancer Network (NCCN) provide no clear recommendations regarding the extent of lateral cervical lymph node dissection to be used. Considering the low rate of metastasis, the presence of “skip metastasis”, and the greater difficulty and complications associated with level-II lymph node dissection, we considered level-II lymph nodes in an independent prediction model.

Our univariate and multivariate analyses indicated that the probability of LLNM (level-II and level-III/IV) increased with primary tumor size. This is in accordance with a previous study (25), which reported that a PTC size of 1.0 cm or more was associated with a 2.49-fold increased risk of LLNM. This is also in line with our finding that clinical TNM stage is related to tumor size. Therefore, regardless of clinical lymph node positivity, clinicians should consider the LLNs in patients with larger PTCs.

Many studies have shown that the location of a PTC affects the probability of LLNM (26). In particular, a tumor in the upper third of the thyroid is associated with a greater risk of LLNM (27,28). In agreement, we found that a tumor in the upper area of the thyroid is the strongest predictor of level-II LLNM. It is possible that lymphatic drainage in the upper part of the thyroid is mainly collected by lymphatic vessels of the superior thyroid artery, which converges with the lateral lymphatic system. Thus, the first level for lymph node drainage may be level-II rather than level-VI.

Our univariate analysis showed that the risk of LLNM was not associated with patient age, in agreement with the results of Farrag et al. (29) and Merdad et al. (30). A previous study reported that males had a higher risk of LNM (31). However, we found that males and females had similar risk for LLNM (level-II and level-III/IV), in agreement with two other studies (32,33).

Our univariate analysis indicated that the risk of level-II and level-III/IV LNM was significantly greater in patients with bilateral tumors, but this association was not significant in the multivariate analysis. Girardi et al. (34) found that ETE was an independent risk factor for LLNM. In contrast, we found that ETE had no significant predictive value for LLNM, as was also the case in several other studies (29,35).

We found that tumor location (left vs. right side of the thyroid) had no impact on the risk for LLNM. This suggests that lymph nodes on each side of the thyroid use the same drainage pathway, as previously reported (32). Our multivariate analysis indicated no significant effect of multifocality or HT on LLNM. A previous study also reported no association between multifocality and LLNM (34), and another study reported no significant association of HT with cervical LNM (36). Interestingly, a meta-analysis (37) suggested that PTCs in patients with HT had a marginally significant negative association with LLNM [odds ratio (OR) =1.3, P=0.041].

There were several limitations in this study. First, the validation of the nomograms might be biased by institutional diagnostic patterns, so further evaluations using external datasets and long-term follow-up are necessary. Second, we did not have comprehensive data on the preoperative clinicopathological characteristics of the patients. Data on microRNAs, PTC subtypes, and BRAF v600e mutations should be considered for subsequent predictive models.

In conclusion, our study of patients with PTC indicated that level-II LLNM was significantly associated with tumor size, presence of a tumor in the upper lobe, and prelaryngeal and paratracheal CLNM. Level-III/IV LLNM was independently associated with tumor size, tumor location, and all subregions of CLNM. Based on these findings, we established two web-based nomograms to predict LNM in patients with PTC. Clinicians can use these nomograms to evaluate patients based on preoperative characteristics and intraoperative FSE, and consider regional LLN dissection for those with high scores. Use of these nomograms may also reduce unnecessary preventive lymph node dissection.


Acknowledgments

Funding: None.


Footnote

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/gs.2020.01.11). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All patients provided written informed consent for their information to be stored in the hospital database and used for research, and this study was approved by the Medical Ethics Committee of The First Affiliated Hospital of Chongqing Medical University (ID: 20197901).

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.


References

  1. Haugen BR, Alexander EK, Bible KC, et al. 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 2016;26:1-133. [Crossref] [PubMed]
  2. Aschebrook-Kilfoy B, Schechter RB, Shih YC, et al. The clinical and economic burden of a sustained increase in thyroid cancer incidence. Cancer Epidemiol Biomarkers Prev 2013;22:1252-9. [Crossref] [PubMed]
  3. Roh JL, Kim JM, Park CI. Central lymph node metastasis of unilateral papillary thyroid carcinoma: patterns and factors predictive of nodal metastasis, morbidity, and recurrence. Ann Surg Oncol 2011;18:2245-50. [Crossref] [PubMed]
  4. Lee YM, Sung TY, Kim WB, et al. Risk factors for recurrence in patients with papillary thyroid carcinoma undergoing modified radical neck dissection. Br J Surg 2016;103:1020-5. [Crossref] [PubMed]
  5. Ducoudray R, Tresallet C, Godiris-Petit G, et al. Prophylactic lymph node dissection in papillary thyroid carcinoma: is there a place for lateral neck dissection? World J Surg 2013;37:1584-91. [Crossref] [PubMed]
  6. Mulla MG, Knoefel WT, Gilbert J, et al. Lateral cervical lymph node metastases in papillary thyroid cancer: a systematic review of imaging-guided and prophylactic removal of the lateral compartment. Clin Endocrinol (Oxf) 2012;77:126-31. [Crossref] [PubMed]
  7. Aydin Buyruk B, Kebapci N, Yorulmaz G, et al. An Evaluation of Clinicopathological Factors Effective in the Development of Central and Lateral Lymph Node Metastasis in Papillary Thyroid Cancer. J Natl Med Assoc 2018;110:384-90. [Crossref] [PubMed]
  8. Nixon IJ, Wang LY, Palmer FL, et al. The impact of nodal status on outcome in older patients with papillary thyroid cancer. Surgery 2014;156:137-46. [Crossref] [PubMed]
  9. Podnos YD, Smith D, Wagman LD, et al. The implication of lymph node metastasis on survival in patients with well-differentiated thyroid cancer. Am Surg 2005;71:731-4. [PubMed]
  10. Ito Y, Tomoda C, Uruno T, et al. Ultrasonographically and anatomopathologically detectable node metastases in the lateral compartment as indicators of worse relapse-free survival in patients with papillary thyroid carcinoma. World J Surg 2005;29:917-20. [Crossref] [PubMed]
  11. Kim E, Park JS, Son KR, et al. Preoperative diagnosis of cervical metastatic lymph nodes in papillary thyroid carcinoma: comparison of ultrasound, computed tomography, and combined ultrasound with computed tomography. Thyroid 2008;18:411-8. [Crossref] [PubMed]
  12. Jun HH, Kim SM, Kim BW, et al. Overcoming the limitations of fine needle aspiration biopsy: detection of lateral neck node metastasis in papillary thyroid carcinoma. Yonsei Med J 2015;56:182-8. [Crossref] [PubMed]
  13. Raffaelli M, De Crea C, Sessa L, et al. Can intraoperative frozen section influence the extension of central neck dissection in cN0 papillary thyroid carcinoma? Langenbecks Arch Surg 2013;398:383-8. [Crossref] [PubMed]
  14. Zeng RC, Zhang W, Gao EL, et al. Number of central lymph node metastasis for predicting lateral lymph node metastasis in papillary thyroid microcarcinoma. Head Neck 2014;36:101-6. [Crossref] [PubMed]
  15. Lan X, Sun W, Zhang H, et al. A Meta-analysis of Central Lymph Node Metastasis for Predicting Lateral Involvement in Papillary Thyroid Carcinoma. Otolaryngol Head Neck Surg 2015;153:731-8. [Crossref] [PubMed]
  16. Ito Y, Tsushima Y, Masuoka H, et al. Significance of prophylactic modified radical neck dissection for patients with low-risk papillary thyroid carcinoma measuring 1.1-3.0 cm: first report of a trial at Kuma Hospital. Surg Today 2011;41:1486-91. [Crossref] [PubMed]
  17. Hartl DM, Leboulleux S, Al Ghuzlan A, et al. Optimization of staging of the neck with prophylactic central and lateral neck dissection for papillary thyroid carcinoma. Ann Surg 2012;255:777-83. [Crossref] [PubMed]
  18. Nixon IJ, Ganly I, Patel SG, et al. Observation of clinically negative central compartment lymph nodes in papillary thyroid carcinoma. Surgery 2013;154:1166-72; discussion 1172-3. [Crossref] [PubMed]
  19. Noda S, Onoda N, Morisaki T, et al. The significance and the predictive factors of microscopic lymph node metastasis in patients with clinically node negative papillary thyroid cancer: A retrospective cohort study. Int J Surg 2015;20:52-7. [Crossref] [PubMed]
  20. Zhu Y, Zheng K, Zhang H, et al. The clinicopathologic differences of central lymph node metastasis in predicting lateral lymph node metastasis and prognosis in papillary thyroid cancer associated with or without Hashimoto's thyroiditis. Tumour Biol 2016;37:8037-45. [Crossref] [PubMed]
  21. Lei J, Li G, Li Z, et al. CUNR scoring system for the prediction of lateral lymph node metastasis in papillary thyroid carcinoma. Oncotarget 2017;9:167-77. [PubMed]
  22. Lei J, Zhong J, Jiang K, et al. Skip lateral lymph node metastasis leaping over the central neck compartment in papillary thyroid carcinoma. Oncotarget 2017;8:27022-33. [Crossref] [PubMed]
  23. Chung YS, Kim JY, Bae JS, et al. Lateral lymph node metastasis in papillary thyroid carcinoma: results of therapeutic lymph node dissection. Thyroid 2009;19:241-6. [Crossref] [PubMed]
  24. Kupferman ME, Patterson M, Mandel SJ, et al. Patterns of lateral neck metastasis in papillary thyroid carcinoma. Arch Otolaryngol Head Neck Surg 2004;130:857-60. [Crossref] [PubMed]
  25. So YK, Kim MJ, Kim S, et al. Lateral lymph node metastasis in papillary thyroid carcinoma: A systematic review and meta-analysis for prevalence, risk factors, and location. Int J Surg 2018;50:94-103. [Crossref] [PubMed]
  26. Hunt JP, Buchmann LO, Wang L, et al. An analysis of factors predicting lateral cervical nodal metastases in papillary carcinoma of the thyroid. Arch Otolaryngol Head Neck Surg 2011;137:1141-5. [Crossref] [PubMed]
  27. Kwak JY, Kim EK, Kim MJ, et al. Papillary microcarcinoma of the thyroid: predicting factors of lateral neck node metastasis. Ann Surg Oncol 2009;16:1348-55. [Crossref] [PubMed]
  28. Hu D, Zhou J, He W, et al. Risk factors of lateral lymph node metastasis in cN0 papillary thyroid carcinoma. World J Surg Oncol 2018;16:30. [Crossref] [PubMed]
  29. Farrag T, Lin F, Brownlee N, et al. Is routine dissection of level II-B and V-A necessary in patients with papillary thyroid cancer undergoing lateral neck dissection for FNA-confirmed metastases in other levels. World J Surg 2009;33:1680-3. [Crossref] [PubMed]
  30. Merdad M, Eskander A, Kroeker T, et al. Predictors of level II and Vb neck disease in metastatic papillary thyroid cancer. Arch Otolaryngol Head Neck Surg 2012;138:1030-3. [Crossref] [PubMed]
  31. Ito Y, Tomoda C, Uruno T, et al. Clinical significance of metastasis to the central compartment from papillary microcarcinoma of the thyroid. World J Surg 2006;30:91-9. [Crossref] [PubMed]
  32. Jin S, Bao W, Yang YT, et al. Establishing a prediction model for lateral neck lymph node metastasis in patients with papillary thyroid carcinoma. Sci Rep 2018;8:17355. [Crossref] [PubMed]
  33. Xu Y, Xu L, Wang J. Clinical predictors of lymph node metastasis and survival rate in papillary thyroid microcarcinoma: analysis of 3607 patients at a single institution. J Surg Res 2018;221:128-34. [Crossref] [PubMed]
  34. Girardi FM, Barra MB, Zettler CG. Predictive factors for lymph node metastasis in solitary papillary thyroid carcinomas: a retrospective study. Pathol Oncol Res 2015;21:59-64. [Crossref] [PubMed]
  35. Koo BS, Yoon YH, Kim JM, et al. Predictive factors of level IIb lymph node metastasis in patients with papillary thyroid carcinoma. Ann Surg Oncol 2009;16:1344-7. [Crossref] [PubMed]
  36. Yoon YH, Kim HJ, Lee JW, et al. The clinicopathologic differences in papillary thyroid carcinoma with or without co-existing chronic lymphocytic thyroiditis. Eur Arch Otorhinolaryngol 2012;269:1013-7. [Crossref] [PubMed]
  37. Lee JH, Kim Y, Choi JW, et al. The association between papillary thyroid carcinoma and histologically proven Hashimoto's thyroiditis: a meta-analysis. Eur J Endocrinol 2013;168:343-9. [Crossref] [PubMed]
Cite this article as: Dou Y, Chen Y, Hu D, Xiong W, Xiao Q, Su X. Development and validation of web-based nomograms for predicting lateral lymph node metastasis in patients with papillary thyroid carcinoma. Gland Surg 2020;9(2):172-182. doi: 10.21037/gs.2020.01.11