All MRI studies were performed at 3T (Siemens, Magnetom Skyra, Siemens Healthineers, Erlangen, Germany) using a commercially available 20 channel head-neck coil. The tumor MRI protocol consisted of: magnetization-prepared rapid gradient-echo sequence (MPRAGE), DWI, susceptibility weighted imaging (SWI), and after contrast media application, DSC-perfusion imaging, T2 TSE-weighted images, fluid-attenuated inversion recovery (FLAIR), and T1 MPRAGE post contrast images. Details concerning sequence parameters: Precontrast: MPRAGE: TR/ms 2,300, TE/ms 2.27, TI/ms 900, FOV/mm 250, slice thickness/mm 1, matrix 256 mm × 256 mm, voxel size 1 mm × 1 mm × 1 mm, orientation sagittal, acquisition time/min 04:44, images evaluated: source, MPR: axial, coronal. DWI TR/ms 9,600, TE/ms 98, FOV/mm 220, slice thickness/mm 3, matrix/mm 162, voxel size 1.4 mm × 1.4 mm × 3.0 mm, orientation axial, acquisition time/min 1:38, images evaluated: axial. SWI TR/ms 27, TE/ms 20, FOV/mm 220, slice thickness/mm 3, matrix 256 mm × 256 mm, voxel size 0.9 mm × 0.9 mm × 3.0 mm, orientation axial, acquisition time/min 02:15, images evaluated: axial. Post contrast: DSC-Perfusion TR/ms 1,600, TE/ms 30, FOV/mm 230, slice thickness/mm 6, matrix 128 mm × 128 mm, voxel size 1.8 mm × 1.8 mm × 6.0 mm, orientation axial, acquisition time/min 01:34, images evaluated: axial.T2 TSE TR/ms 4,070, TE/ms 89, FOV/mm 200, slice thickness/mm 3, matrix 384 mm × 384 mm, voxel size 0.5 mm × 0.5 mm × 3.0 mm, orientation coronal, acquisition time/min 03:01, images evaluated: coronal. FLAIR TR/ms 9,000, TE/ms 81, TI/ms 2,500, FOV/mm 220, slice thickness/mm 3, matrix 320 mm × 320 mm, voxel size 0.7 mm × 0.7 mm × 3.0 mm, orientation axial, acquisition time/min 02:44, images evaluated: axial. MPRAGE parameters: see above precontrast MPRAGE.

An overview of the evaluation of items is given in Table 2.

In order to describe which items achieved the best accordance we ordered the items from highest to lowest congruence rates. In case of equal congruence percentages, the items were ranked according to the highest percentage of partially congruent findings.

For each structured report the reporting time was measured. Reporting time was defined from the start of reading the first diagnostic sequence until the finalization of filling out the template form. An example of a screen during reading is given in Figure 1.

Identical mounting of all MR-studies was performed prior to reading on the Syngovia platform and procedural remarks were documented, when relevant. The time needed for these tasks was not included in the measurements.

Table 3 summarizes the results of the comparison between structured and free-style reported findings for tumor-related items. Congruent findings were found from 39 to 98% in the 17 items. Partially congruent findings were found in 17 of 17 items in up to 50%. Incongruent findings were seen in 7 of the 17 items in up to 5%. In 12 items free text reports did not mention the finding (range, 7–46%).

The results of the comparison between free-style and structured reports’ findings concerning the non-tumor-related items can be seen in Table 4.

Congruent findings were found between 15 and 62% in the four items. Partially congruent findings were found in 3 of 4 items ranging between 3 and 20%. Incongruent findings were seen in 1 of the 4 items in 5% of reports. In all four items free-text reports did not mention the addressed topic (range, 18–85%).

Figure 3 shows the results of the ranking of items.

The mean time for structured reporting was 7 min and 49 s (median, 07:09 min; range, 3:12–17:06 min).

The structured reporting missed a vestibular schwannoma, an aneurysm of the anterior communicating artery and an embolic small infarction in the white matter.

The texture of tumor was not explicitly addressed in the structured report and in general it was clearer by reading the free-style reports as necrosis, cysts, and solid areas were described.

Numbers of tumors and their locations were the items with the highest percentages of congruent findings between free-style and structured reports (range, 82–98%) and these were the only five items without missing information in free style. Between 70 and <80% congruent findings were seen in the item “perfusion,” “SWI signal changes indicating bleeding,”, “tumor diameter 1,” and “space-occupying effect.” Interestingly, “perfusion” (79%) reached a higher percentage of congruent findings than DWI (63%) and a second tumor diameter was found to be congruent only in 68%. Generally speaking, a basic tumor report can be performed by addressing all of the above mentioned items. But some remarks need to be made: Free-style reports did not address all these items. In 12% (“space-occupying effect”), 13% (“perfusion” and “SWI signal changes indicating bleeding”), 20% (“tumor diameter 1”), and 22% (“tumor diameter 2” and “DWI”) of free-style reports remarks were missing. Although in our patient collective no acutely dangerous findings were overlooked, the completeness and information provided by free-style reporting was substantially lower. Taking into account that free-style reports did not address the topic “eloquence” in 30%, the quality of reports becomes, yet again, an issue. With structured reporting important tumor information can not be as easily “overlooked,” as the items need to be answered on the template. On the other hand, if the template is not designed to cover all relevant imaging features, such as intratumoral necrosis, cysts, or solid areas, this information will not be provided at all. This underlines the importance of performing a feasibility-check for newly designed templates (like in this study) for structured radiological reporting. We will add these missing but relevant items to our template. In addition, the information provided to our referring physicians will be enhanced by implementing representative images into the written neuroradiological reports.

The ranking of the items as documented in Figure 3 shows the method of traditional reading quite well as the first 13 items seem to represent “normal” standard reporting which necessarily does not include measurement of edema or measurement of a third diameter of lesions.

Measurement of tumors is established for follow-up assessment of high-grade gliomas by the RANO criteria (10). In this study, we assessed images with first tumor diagnosis for diverse tumor entities. We looked at different methods of measurement. We considered one diameter measurement in three planes as represented by “tumor diameter 1–3” (here the angle between the diameters could be random) as well as tumor measurement by two diameters in one axial plane where the diameters were perpendicular to each other. We evaluated the 13 patients with high-grade gliomas separately. The measurement items had in general the highest rates of omissions in free-style reports, increasing from “tumor diameter 1” (20%) up to “2D size of high-grade gliomas on T1w post contrast images” (46%). As two diameters are regularly measured a third diameter is performed in less than 2/3 of the cases. A possible explanation could be—while RANO is referring to measurements in the axial plane in the follow-up situation—that no primary preoperatively established measurement system exists.

Except for the 2D size measurement of high-grade gliomas T2w/FLAIR signal changes achieved the lowest congruent results in tumor-related items, i.e., for “2D size of tumor-induced T2w/FLAIR signal changes” 40% and “size of maximum perifocal edema” 43%. Congruent results were found in “2D size of tumor-induced T2w/FLAIR signal changes” when the tumor was identified by its signal changes on T2w/FLAIR and showed no enhancement on T1w post contrast images and was, therefore, measured on T2w or FLAIR images or in the case when the tumor had, incidentally, the same size on T1w post contrast as on T2w/FLAIR images. The perifocal maximum edema size was the item which had the most partially congruent reports (50%) which was mainly due to the strict definition accepting only similar measurement or classification in either minor (≤1 cm), moderate (≤3 cm), or extensive (≥3 cm) and evaluation of all lesions (up to 10) as congruent.

In the free-style tumor reports vessels, white matter (microangiopathy), brain volume, and viscerocranium received a minor priority shown by the percentages of omissions (“vessels” 60%, “white matter” 35%, “brain volume” 85%, and “viscerocranium” 18%). Interestingly, findings of the viscerocranium were least omitted. “Microangiopathy” and “viscerocranium,” the latter consisting mainly of reports on the paranasal sinuses, had the highest congruence, 62 vs. 57%, whereas “brain volume” was mostly neglected in free-style reports of intracranial tumors. The reason for this might be primarily due to the different focus while reporting an intracranial tumor setting. Furthermore, it has to be considered that due to space-occupying effects a proper report on brain volume might not be possible in some cases. In the nine patients with reports on brain volume, there was a 100% congruence between free-style and structured reports. In summary, the four items showed that, if a tumor was present, it was the main focus of the report. This is also underlined by the missing report of an anterior communicating artery aneurysm missed in the structured report showing that even if asked for alternative pathology the reader-bias can be misleading. On the other hand, it has been shown that structured reporting can help to decrease missed findings (4).

A manual documentation of findings was performed in the template as technical issues could not be solved during the time of our study. For implementation in the clinical routine a digital integration into the reporting system by speech recognition voice control reporting is necessary. To cite the results of a focus group meeting “Structured reporting will fail if it compromises accuracy, completeness, workflows, or cost-benefit balance” (11). In the future, a reduction of reporting time may be achieved using template integration in the electronic reporting system taking into account that the mean reporting time was already clinically feasible in the presented setting. A comparison of structured reporting times to free-style reports was not possible as completed, existing free-style reports were used in this study. This is a limitation due to the retrospective nature of this study and, therefore, we cannot provide the exact free-style reporting times for the reports of the 60 included patients. But we would like to add that the mean reporting time of the expert reader in general intracranial tumor reporting is about 10 min (measured by the reader as mean of 3 months). The free-style reports performed by radiologist in training and senior neuroradiologist take between 15 and 30 min (measured by the readers as range during 3 months).

A further limitation of this pilot study was that free-style reports would have probably performed better in mentioning more of the features of the structured reporting list in a prospective setting. Additionally, it needs to be mentioned that not all items were covered by our preliminary template. Tumor texture items like, e.g., necrosis, cysts, or solid areas were not explicitly addressed as well as spectroscopic and fMRI items. In a future template, these items will be included, e.g., for spectroscopy and fMRI in addional advanced tumor templates. Due to important diagnosis missed by structured reporting the template should be improved by adding a checklist highlighting relevant items, like, e.g., questions for infarcts or aneurysms comparable to an “autopilot” in aviation.

Within this retrospective work, it was not possible to measure consistency of 2D size measurements. It is known that inter-observer agreement on tumor boundaries in high-grade gliomas is problematic on MRI images (12). In the recent years, automatic segmentation tools have improved reproducibility of measurements of tumor metrics (13, 14). In future prospective tumor studies automatic measurements should be part of the applied evaluation tools.

Structured reporting and comparison of structured reports and free style was performed by one senior neuroradiologist who took part in the template generation. This design was chosen on purpose as we wanted to avoid possible effects, e.g., changes in time consumption, due to learning curves from non-primarily involved readers. We refrained from a multireader setting or focus on reproducibility on purpose as this study was planned as a first test-phase to discover major drawbacks of the template from an expert reader point of view. In the future, the template will be revised on the basis of this expert reader evaluation, implemented in routine reporting and will be tested by less experienced readers.

This study focused on the quality of structured and free-style reporting and did not address the impression or degree of satisfaction of the clinicians. It is known that referring clinicians judge structured reports better than free-style reports (3). This study was an initial internal quality check for structured reporting of intracranial tumors. In a second step, the template will be discussed with and revised according to the needs of our clinicians as has been done successfully previously (15). The revised templates’ reports and free-style reports will be used as a survey to explore satisfaction of the clinicians with each structured report. Furthermore, by implementing structured reports in our radiological daily routine we aim to improve report quality and simplify extraction of data for major analysis like big data evaluation (1) and through this enhance the scientific value of daily routine work.