Clinical case data of patients who underwent thyroid surgery and were diagnosed with PTMC postoperatively from January 2015 to December 2022 in Traditional Chinese Medicine Hospital of Xinjiang Medical University were retrospectively selected. The variables included patient age, sex, BMI, maximum tumor size, history at diagnosis, thyroid function, USG parameters, pathological results and other indicators. The exclusion criteria were as follows: (1) patients who underwent secondary or further thyroid surgery; (2) patients who did not undergo radical surgery or CLND; (3) previously diagnosed other concomitant malignancies, such as breast cancer, lung cancer, colon cancer, and other cancers; (4) patients who also had other types of thyroid carcinoma, such as medullary, follicular and anaplastic thyroid carcinomas; (5) clinically and/or pathologically detected distant metastasis, such as secondary malignant tumor of bone, liver, and other body parts; (6) patients lost to follow-up or with incomplete clinical information.

The possible risk factors of PTMC with CLNM analyzed in this study can be divided into 4 categories including 40 variables.

The general clinical conditions include: gender, age, ethnicity, marital status, body mass index (BMI), diabetes, heart disease, smoking, drinking, family history of cancer, family history of thyroid disease. All of these factors can be obtained from hospitalized records.

Biochemical variables recorded include thyroid function [free triiodothyronine (FT3), free thyroxine (FT4), thyroid stimulating hormone (TSH), thyroglobulin (Tg)], thyroid antibody [thyrotropin receptor antibody (TRAb), Anti-thyroglobulin antibodies (TgAb), thyroid peroxidase antibodies (TPOAb)], blood lipid [triglyceride (TG), cholesterol (CHOL)], tumor markers [alpha fetoprotein (AFP), carcinoembryonic antigen (CEA), calcitonin (Ct)]. The results of these biochemical indicators are exported from the hospital information system.

A GE-E11 ultrasound system (GE Medical System, USA) with a linear array probe was used to acquire ultrasound images in the frequency range of 9-11MHz. All patients were in the supine position with the lower shoulder pillow neck extended to better expose the inferior thyroid rim. Both thyroid lobes and isthmus of the thyroid were scanned in the transverse and longitudinal planes. Longitudinal and transverse images of the thyroid were obtained according to American College of Radiology accreditation standards. The thyroid nodule ultrasound chart was prejudged by two experienced sonographers.

The following USG parameters of the nodules were recorded as follows: (1) margin, (2) shape, (3) echogenicity, (4) calcification, (5) diffuse, (6) color doppler flow imaging (CDFI), (7) tumor position: location_Upper/Middle/Lower, location_Left/Right/Isthmus, (8) ECLN.

The clinicopathological test results included: (1) diameter, (2) hashimoto thyroditis, (3) nodular goiter, (4) ETE, (5) multifocality, (6) bilateral, (7) lateral lymph node metastasis (LLNM). Postoperative histopathologic evaluations were performed by pathologists experienced in thyroid pathology.

This judgement criteria of the study were as follows: (1) malignant tumors were considered to be multifocal when more than one malignant tumor foci were found in the same or different lobes of the gland; (2) if the same patient had multiple lesions, the malignant tumor with the largest diameter was selected as the research object. When the diameter was the same, the higher Thyroid Imaging Reporting and Data System (TI-RADS) level would be selected. The maximum tumor diameter was categorized in two groups: the small group (< 0.5 cm) and the large group (0.5-1.0 cm); (3) ETE was confirmed when there was evidence of a malignant tumor invading adipose tissue, banded muscle, larynx, trachea, esophagus, recurrent laryngeal nerve, prevertebral fascia or blood vessels; (4) lateral lymph node dissection (LLND), including levels II-V, was performed only in cases with clinically evident lateral neck lymph node metastasis. Assessed by the experienced physician, the patients without LLND were classified as non-occurrence LLNM; (5) the range of TGAb and TPOAb actual detection results is wide, being 0-4000 IU/ml and 0-600 IU/ml respectively. In this study, these were set as dichotomous variables: TGAb was divided into Normal (≤ 115 IU/ml) and Abnormal (> 115 IU/ml) groups; TPOAb was also divided into Normal (≤ 34 IU/ml) and Abnormal (> 34 IU/ml) groups.

In order to clarify the trend of thyroid function in PTMC patients at different postoperative periods and to compare the differences between the CLNM (+) and the CLNM (-) groups, thyroid function indicators were retrospectively collected in this study. FT3, FT4, TSH, Tg, TgAb, and TPOAb in PTMC patients during the following consecutive periods: preoperatively, within 3 days, 1 month, 3 months, 6 months, 1 year, 2 years, 3 years, and 4 years postoperatively. By the end of 2022, those eligible for follow-up 4 years thyroid function indicator data were PTMC patients who underwent surgery during 2015-2018. Data were obtained from thyroid function markers at the time the patients underwent regular follow-up of patients during outpatient or hospitalization. Patients with omissions or missing data are excluded, and those with high compliance, completeness, and continuity of thyroid function indicators were retained for analysis. In accordance with guidelines for the diagnosis and management of subclinical thyroid disorders and the standard practices in hospital laboratories, the normal ranges of thyroid function indicators were determined as follows: 3.1 ≤ FT3 ≤ 6.8 pmol/L, 12 ≤ FT4 ≤ 22 pmol/L, 0.27 ≤ TSH ≤ 4.2 uIU/ml, 3.5 ≤ Tg ≤ 77 ng/ml, 0 ≤ TGAb ≤ 115 IU/ml, 0 ≤ TPOAb ≤ 34 IU/ml. Subclinical hypothyroidism is defined as an elevation in serum TSH level above the upper limit of the reference range and a normal FT4 level. Overt hypothyroidism is defined as an increase in serum TSH level and a decrease in FT4 level.

Between January 2015 and December 2022, there were 7005 thyroid surgery in-patients based on the medical record data of Traditional Chinese Medicine Hospital Affiliated to Xinjiang Medical University. According to strict exclusion criteria, 3023 PTMC cases were included in this study in the end, 621 had CLNM, as shown in Figure 1 . To better verify the predictive power of the model and to analyze the possibility of CLNM in PTMC, 3023 cases were randomly divided into two groups by a ratio of 7 to 3, 2116 were in the training group and 907 in the validation group.

A total of 789 patients with complete serum thyroid function indicators at 4-year postoperative follow-up among the patients with PTMC, including 158 with CLNM (+) and 631 with CLNM (-), as shown in Figure 1 . There were 1424 eligible patients for PTMC surgery from 2015-2018, with a complete follow-up rate of 55.41%.

The study was approved by the Medical Ethics Committee of Traditional Chinese Medicine Hospital Affiliated to Xinjiang Medical University (IRB No. 2022XE0171). All participants gave written informed consent for their clinical records to be used in this study.

During surgery, the tumors were sent for frozen sections and confirmed as PTMC. After rediagnosis, these patients underwent thyroidectomy and cervical lymph node dissection, and the lymph nodes were sent for frozen sections to confirm CLNM and LLNM. After surgery, the remaining specimens were sent for paraffin sections for final confirmation of PTMC, CLNM and LLNM. Two or more experienced pathologists microscopically reviewed and cross-checked all pathology specimens. Principles of thyroidectomy: total thyroidectomy was performed in patients with extraglandular invasion, bilateral or multiple lesions, and bilateral cervical lymph node metastases, and ipsilateral and isthmus resection were performed in patients with unilateral intrathyroidal tumors. Principles of cervical lymph node dissection: central lymph nodes (level VI) were routinely dissection, and lateral compartment lymph nodes (levels II-V) with evidence of lymph node metastasis should be dissection. Upper mediastinal lymph nodes (level VII) should be also cleared if there are enlarged lymph nodes based on neck imaging. The recurrent laryngeal nerves should be directly visualized throughout the node clearance process to avoid injury. The parathyroid gland and their blood supply should be retained.

A total of 3023 patients with PTMC who underwent surgical treatment were included in this study, of which 1360 patients underwent total thyroidectomy, 1341 patients underwent unilateral lobectomy + isthmus resection, 314 patients underwent unilateral lobectomy + isthmus + partial resection of contralateral lobectomy, and 8 patients underwent bilateral partial lobectomy + isthmus resection. Lymph node dissection was performed in zone VI only in 2798 patients (ipsilateral 1164 patients and bilateral 1154 patients) and in zones II, III, IV and VI in 224 patients (ipsilateral 5 patients and bilateral 219 patients).

The open source R software, Version 4.3.3 (https://www.r-project.org) and the Statistical Package for the Social Sciences (SPSS) for Windows, Version 21.0 (SPSS Inc., Chicago, IL, United States) were used for data analysis and modeling in this study. When contrasting the baseline information and clinical features between the modeling group and the control group, the chi-square test or Fisher’s exact test were used for categorical variables, expressed by actual cases and percentages (%); the student’s t-test or Wilcoxon rank sum test was used for continuous variables, shown by mean ± standard deviation (SD). After assessing risk factors by the least absolute shrinkage and selection operator (LASSO) regression model, the non-zero variate was considered significant. A multivariate logistic regression model for prediction was then established, and the coefficient, odds ratio (OR) with 95% confidence interval (CI) and p-values of variables were established at the same time; the statistical significance variables (p-values< 0.05) were the influence factors. All statistical tests were two sided, and p-values< 0.05 were considered statistically significant.

A nomogram, based on influencing factors determined by the multivariate logistic regression model, was established by rms/pROC pack in R software, which was used to evaluate the risk of CLNM preoperatively and gave every variable to a score between 0 and 100. The goodness of fit test of the model between modeling and validation groups was assessed by receiver operating characteristic (ROC) curve and calibration curve. The ROC curve was plotted to quantify and contrast the risk of PTMC with CLNM between forecasting and observing cases. The reasonable range of area under ROC curve (AUC) is 0.5 to 1, the closer the value is to 1, the stronger the model prediction ability. The calibration curve, as a very important criteria, was used to assess how well the model fits the real data. Our conclusion from the contained 1000 bootstrap samples: the smaller the mean absolute error, the higher the accuracy in forecasting was. Vice versa, the lower accuracy showed the model may over- or underestimate the risk of illness. A Hosmer-Lemeshow goodness-of-fit test was also performed to assess predictive ability; the larger the p-value, the better the predictive ability. The DCA was drawn to make the clinical decision benefit in predicting the risk of PTMC with CLNM.

In our study, a total of 3023 PTMC patients were recruited according to the inclusion and exclusion criteria; the average age was 47.33 ± 10.39 years, there were 2381 (78.76%) females and 642 (21.24%) males (ratio 3.71:1). CLNM and non-CLNM were observed in 621 (20.54%) and 2402 (79.46%) cases respectively.

A total of 2116 patients with PTMC were included in the training group consisting of 1665 (78.69%) females and 451 (21.31%) males (ratio 3.69:1), and the average age was 47.27 ± 10.44 years. CLNM and non-CLNM were observed in 433 (20.46%) and 1683 (79.54%) cases respectively. Meanwhile, 907 PTMC patients were assigned to the validation group, there were 716 (78.94%) females and 191 (21.06%) males (ratio 3.75:1), and the average age was 47.46 ± 10.27 years. CLNM and non-CLNM were observed in 188 (20.73%) and 719 (79.27%) cases respectively.

We collected 43 indexes to identify the risk factors for CLNM in PTMC. There were no statistically significant differences (P > 0.05) in general clinical data, biochemical indicators, ultrasonic characteristics and clinicopathological results between the training and the validation group, except for the Ct, as shown in Table 1 .

For the study, the LASSO regression was used for feature selection in the training group. With the change of the penalty coefficient λ, the variables included in the model were gradually reduced. The λ in the LASSO model was selected via tenfold-validation based on minimum criteria (lambda.min = 0.0099). At the same time, 18 non-zero coefficient characteristics ( Figures 2A, B ) were selected to build the multivariate logistic regression analysis, including age, gender, BMI, marital status, hashimoto thyroditis, location_UML, calcification, diameter, multifocality, ETE, ECLN, LLNM, TG, FT3, TSH, Tg, TPOAb, CEA.

The multivariate logistic regression analysis concluded that younger age (OR = 1.02, 95% CI, 1.01-1.03, P< 0.001), Lower BMI (OR = 1.03, 95% CI, 1.00-1.07, P = 0.049), being male (OR = 1.56, 95% CI, 1.16-2.09, P = 0.003), location in lower pole (OR = 1.43, 95% CI, 1.04-1.96, P = 0.028), calcification (OR = 1.34, 95% CI, 1.06-1.69, P = 0.015), 1 ≥ diameter ≥ 0.5 cm (OR = 1.78, 95% CI, 1.38-2.31, P< 0.001), multifocality lesions (OR = 1.64, 95% CI, 1.28-2.10, P< 0.001), ETE (OR = 1.47, 95% CI, 1.02-2.09, P = 0.035), ECLN (OR = 1.65, 95% CI, 1.18-2.29, P = 0.003), LLNM (OR = 5.47, 95% CI, 3.55-8.51, P< 0.001) and CEA (OR = 1.18, 95% CI, 1.05-1.32, P = 0.003) were the independent risk factors of PTMC patients with CLNM. It should be emphasized that age (-) and BMI (-) were correlated with CLNM negatively, the higher the value, the lower the risk of CLNM. We calculated the risk value of Age (+) and BMI (+) by conversion of the dependent variables. The detailed results are shown in Figure 3 . Meanwhile, we list the distribution characteristics of the 11 risk factors in CLNM (+) group and CLNM (-) group respectively in Table 2 . The differences were statistically significant (P< 0.05).

Based on LASSO and multivariate logistic regression analysis, we produced a nomogram model consisting of 11 factors, which were used to estimate the risk of CLNM in PTMC patients. According to the scored ruler as shown in Figure 4 , each variety has a corresponding point value on each variable axis: all the points were added up to obtain the overall points. The sum of these numbers is located on the total points axis to express the incidence of the risk for CLNM. The higher the total points, the greater the possibility of CLNM risk. In this study, the risk of PTMC combined with CLNM ranged from 0% to 90%, as assessed by 11 risk factors.

The ROC curve was drawn to predict the nomogram effect ( Figures 5A, B ). The area under the ROC curve (AUC) of the training group was 0.73 (95% CI, 0.70-0.76), the diagnostic cut-off point was 0.21 (sensitivity, 0.61, specificity, 0.74). The AUC of the validation group was 0.75 (95% CI, 0.71-0.79), the diagnostic cut-off point was 0.24 (sensitivity, 0.56, specificity, 0.81). In addition, the AUC value of the validation group was only 0.02 higher than that of the training group, which indicates that the model has good prediction discrimination in both groups.

The calibration plots present a good agreement between the actual and predicted metastasis probability with an additional 1000 bootstraps, the mean absolute error of the training and validation groups were 0.015 and 0.012, respectively ( Figures 6A, B ). Meanwhile, the calibration of the prediction model was evaluated by Hosmer-Lemeshow goodness of fit test: when P > 0.05, the calibration ability of the model is good. The P values of the training and validation groups calibration curves were 0.95 and 0.60, respectively, which indicated that the nomogram model was well consistent between prediction and actual values.

Results of DCA in training and validation groups were shown in Figures 7A, B . The DCA showed that using the nomogram model to predict the occurrence of CLNM risk would be better than using another two conditions (all patients and none of the patients treated). This means that the model is clinically useful when intervention is decided on in the range of the threshold probability between 0.09-0.82 and 0.07-0.86 in the training and validation groups respectively.

Each variable contained in the nomogram has its corresponding risk point, and the total points calculated for all patients can quantitatively predict their respective CLNM risk. The distribution characteristic of the total CLNM risk points are shown in Figure 8 , and the risk value is shown in Figure 4 . We therefore determined three cut-off values (150, 200 and 240) by using recursive partition analysis, and classified three subgroups as follows: (1) extreme low-risk group (total points ≤ 150, risk ≤ 18%), low-risk group (151 ≤ total points ≤ 200, risk 19-36%), (2) moderate-risk group (201 ≤ total points ≤ 240, risk 37-58%), (3) high-risk group (total points > 250, risk > 58%). In the training and validation groups, the rates of CLNM for extreme low-, low-, moderate-, and high-risk groups were 10.57%, 24.64%, 45.89%, 69.88% and 10.65%, 22.92%, 53.57%, 83.33%, respectively, showing a gradual upward trend. We further studied whether the relative risk of CLNM in low-, moderate-, and high-risk groups was significantly different from one another. The Chi-square test showed that there were significant differences between all groups (P< 0.05). The results of the training and validation groups confirmed that there were no significant differences in the same category group (P > 0.05). As shown in Table 3 .

We retrospectively collected complete follow-up data on serum thyroid function indicators for 4 years after surgery in 789 patients with PTMC, of whom 158 (20.03%) developed CLNM and 631 (79.97%) did not, which was not statistically significant difference when compared with the overall distribution of CLNM occurrence ( χ 2 = 0.068 , P = 0.794). The differences in FT3, TSH, and TPOAb in the preoperative CLNM (+) group were statistically significant compared with the CLNM (-) group, which were also previously screened by LASSO regression.

The follow-up data of serum thyroid function indicators in PTMC patients during 4 years after surgery and the results of comparison between CLNM (+) and CLNM (-) groups are shown in Table 4 , Figure 9 . FT3 decreased to the lowest value 3.28 ± 0.77 pmol/L at 3 days postoperatively, then gradually increased to a relatively stable state of 4.97 ± 0.97 pmol/L at 3 months postoperatively, and was significantly higher in the CLNM (+) group than in the CLNM (-) group (P< 0.05). FT4 remained elevated for 3 months postoperatively and then stabilized at 20.09 ± 4.44 pmol/L, which was higher than the preoperative level of 15.90 ± 2.52 pmol/L, the CLNM (+) group was higher than the CLNM (-) group (P< 0.05). The mean TSH level reached the highest value 6.84 ± 13.19 uIU/ml at 1 month postoperatively, then gradually decreased and was maintained at a stable level (2.36 ± 8.52 uIU/ml) at 3 months postoperatively, there was no statistically significant difference between the CLNM (+) and CLNM (-) groups (P > 0.05). The mean Tg level reached a maximum value of 60.72 ± 93.59 ng/ml at 3 days postoperatively, then gradually decreased and remained at a stable level 4.00 ± 13.95 ng/ml at 1 month postoperatively, which was lower than the preoperative level of 21.65 ± 50.23 ng/ml, and the difference between the CLNM (+) and CLNM (-) groups was not statistically significant (P > 0.05). TGAb showed a gradual decrease, with no statistical difference between the CLNM (+) and CLNM (-) groups (P > 0.05). TPOAb also showed a gradual decline, with CLNM (+) group being lower than the CLNM (-) group, with a statistically significant difference within 1 year postoperatively (P< 0.05).

Thyroid function indicators FT3, FT4, and Tg varied within the normal range, TSH exceeded the maximum normal level at 1 month postoperatively, the CLNM (+) group with TGAb and the CLNM (-) group with TPOAb exceeded the highest normal level within 6 months and then gradually decreased to the normal range.

The composition ratio of normal and abnormal reference ranges of serum thyroid function indicators during the 4 years postoperative follow-up in the CLNM (+) and CLNM (-) groups are shown in Figure 10 . The proportions of FT3< 3.1 pmol/L and FT4< 12 pmol/L reached its maximum at 3 days postoperatively and were higher in the CLNM (+) group than in the CLNM (-) group, then gradually decreased to 0. TSH > 4.2 uIU/ml reached highest value at 1 month postoperatively, with 33.49% and 28.42% in the CLNM (+) and CLNM (-) groups, respectively, and then gradually declined. The proportion with Tg< 3.5 ng/ml increases dramatically at 1 month postoperatively. TGAb and TPOAb varied steadily, with an overall gradual decline in the portion above the normal range.

The incidences of subclinical hypothyroidism and hypothyroidism tended to increase and then decrease in both the CLNM (+) and CLNM (-) groups, as shown in Figures 11A, B . Subclinical hypothyroidism occurred predominantly within 6 months postoperatively and reached a maximum at 1 month postoperatively, with a higher incidence in the CLNM (+) group than in the CLNM (-) group, at 24.68% and 19.65%, respectively. The overall incidence were 42.41% (67 of 158 patients) in the CLNM (+) group and 42.31% (267 of 631 patients) in the CLNM (-) group. Hypothyroidism appeared mainly within 3 months postoperatively and peaked at 1 month postoperatively, with a incidence of 8.86% in the CLNM (+) group higher than that of 5.71% in the CLNM (-) group. The overall incidence were 16.46% (26 of 158 patients) in the CLNM (+) group and 12.04% (76 of 631 patients) in the CLNM (-) group.