Early literacy encompasses a range of skills related to oral language, phonological awareness, word awareness, print awareness, concepts of print, alphabet knowledge, and narrative abilities that develop before children formally learn to read and write. Phonological awareness refers to the ability to detect the smallest sound units within words. When children develop this skill, they understand that language can be analyzed and manipulated, a key milestone in their literacy journey toward understanding the concept of words (Gee, 2012; Snow and Dickinson, 1991). Word awareness involves recognizing that words, as elements of language, have properties independent of their meaning. For example, children learn to connect printed words in their oral vocabulary while learning to read. This awareness also includes understanding word boundaries and what constitutes a word (Justice and Ezell, 2001). The term print awareness refers to the knowledge that print carries meaning, and differs structurally from other sign systems (e.g., numbers). To develop print awareness, it is crucial for young children to be exposed to letters and written text in their environment. Familiarity with books and print culture also involves understanding the characteristics of books and how they are read, which relates to concepts of print (Meindl and Jungmann, 2019; Piasta et al., 2012). Alphabet knowledge entails recognizing the characteristics of different graphemes and associating them with their corresponding phonemes. Early reading skills include understanding that words are made up of graphemes (Elimelech and Aram, 2020; Morrow, 2007). Narrative skills reflect children’s ability to produce a fictional or real account of a temporally sequenced experience or event (Engel, 1995). All these aspects represent crucial milestones in the successful development of reading and writing (Justice and Ezell, 2001).

Early literacy tasks such as phonological awareness, print awareness, and word awareness reflect how children process and integrate the relationships between various linguistic rule systems on the metacognitive level. Nagy and Anderson (1995) documented that metalinguistic awareness helps young children to become aware of the structure of their writing system and its relationship to their spoken language. For instance, children with higher metalinguistic awareness perform better on tasks related to concepts of print and phonological awareness, both of which are strong predictors of reading success (Chaney, 1994). Without proper assessments, children in need of additional support may be overlooked, leading to long-term consequences such as ongoing academic difficulties, low self-esteem, and limited future opportunities (Brassard and Boehm, 2007; Justice et al., 2002; OECD, 2013). Screening early literacy skills with standardized assessments has gained significant attention in early childhood for various reasons: (a) to evaluate a child’s strengths and weaknesses in specific areas, (b) to identify key target skills and provide tailored support, (c) to structure educational programs, (d) to monitor children’s progress over time, and (e) to improve educational outcomes by facilitating a smoother transition to school. Assessment tools also offer significant benefits at different stages of assessment, from testing to linking appropriate interventions, by collaborating with children’s parents and other stakeholders to help overcome disadvantages. Despite the benefits of standardized testing, research shows that data-driven decision-making remains significantly underutilized in early education. Children enter kindergarten with a wide range of literacy and language abilities (Catts et al., 2002), making individualized feedback essential (Brassard and Boehm, 2007; McLachlan et al., 2018; Snow and van Hemel, 2008). To address these needs, computerized adaptive testing (CAT) provides an innovative solution for assessing children’s literacy skills. By dynamically adjusting to each child’s ability level, CAT improves measurement precision while reducing the required test items. As a cutting-edge tool, EuLeApp© leverages CAT to streamline the assessment process, ensuring both accuracy and efficiency. This adaptability allows practitioners to deliver individualized feedback, enabling more targeted interventions and improving educational outcomes.

Given the ongoing expansion of digital media in educational settings (OECD, 2015) and the demand for more effective assessment methods, researchers have increasingly focused on the potential of digital tools for enhancing educational processes. These tools offer greater efficiency (Marsh et al., 2020) and provide visually engaging reports for tracking learning progress (Neumann et al., 2019). The use of digital assessment tools is increasing for numerous reasons, such as technological portability and ease, the touchscreen’s tremendous potential to reach young children, and the need for mobile, adaptable, and accessible assessment tools (Neumann et al., 2019). Furthermore, online assessment tools have become widespread and are actively used. For example, Ho et al. (2024) developed a short-term online test to assess word reading. This test, which covers a wide age range, is primarily designed for individuals who can read to some degree at a basic level (ages 7 and above). Tablet-based assessments, on the other hand, allow for greater control over the evaluation process for younger children and may be a more suitable alternative. Additionally, app-based assessments offer the advantage of functioning offline, making them more accessible in diverse settings such as homes, preschools, and clinics, where stable internet access is not always available.

App-based assessments can be administered anywhere with a suitable device, making it easier for educators and researchers to assess children in various settings, including homes, classrooms, and clinics. Another benefit of mobile media devices is that they can help advance the goal of reaching many children for educational opportunities and equity because of their low costs and good accessibility (Hirsh-Pasek et al., 2015). Additionally, traditional paper-pencil assessments usually require considerable time, effort, and expertise, such as organizing, rewriting, and preparing children for the test in person (Schildkamp and Kuiper, 2010). Also, some assessment procedures rely on contextual factors such as observation, making it challenging to maintain strict objectivity (Hindman et al., 2020). Moreover, children may not be able to show their best performance when assessed by someone they do not know (Halliday et al., 2018). In contrast, computerized assessments offer the potential to gain insights into children’s responses, such as disengagement, rapid guessing, or unexpected answers (Bulut and Cormier, 2018; Lee and Jia, 2014; Wise and Kong, 2005). Thus, app-based assessments enable teachers to receive prompt feedback, facilitating more effective support and remediation. However, technology-based assessments also raise concerns regarding developmental appropriateness, item development, psychometric validity, and teacher training (Bulut and Cormier, 2018; Neumann et al., 2019). Tablet-based assessments are common in many schools across the United States, where they are used to assess and monitor students’ performance in mathematics, reading, and science throughout the school year (Davey, 2005; Sharkey and Murnane, 2006). Despite the increasing integration of technology in education, the development of digital tools specifically for early literacy assessment remains limited, underscoring the need for further innovation. Emerging research shows that only a few app-based tools are available to assess children’s language and literacy skills in the early years. One such tool is Logometro^®, a reliable app-based test that evaluates children’s phonological awareness, listening comprehension, vocabulary, narrative skills, speech, morphological awareness, and pragmatic skills (Antoniou et al., 2022). Administered through a specially developed Android app, Logometro^® allows for accurate directional vocalization and easy capture of children’s responses via touchscreens and direct recordings (Antoniou et al., 2022). Another innovative app, QUILS (Golinkoff et al., 2017) focuses on assessing children’s language-learning processes, offering insights into how they acquire new words and grammatical structures. Golinkoff et al. (2017) demonstrated that a technology-based assessment tool measuring children’s phonological awareness and letter knowledge was efficient in terms of time usage and effectively differentiated these skills. One app was also developed by Neumann (2018) to assess children’s letter knowledge and vocabulary using both expressive and receptive response formats. The valuable insights gained from such innovative assessment tools can make them more attractive, accurate, and accessible for educators and parents.

Computerized Adaptive Tests (CAT) dynamically adjust to a child’s ability by selecting test items based on their responses. Unlike traditional fixed-form tests, CAT tailors the difficulty of each item to match the child’s performance, selecting more challenging questions after correct answers and easier ones after incorrect responses (Meijer and Nering, 1999; Weiss and Kingsbury, 1984). This individualized approach helps maintain engagement and ensure a more accurate assessment of abilities (Tomasik et al., 2018; Wise, 2014). CAT relies on an Item Response Theory (IRT) calibrated item bank, selecting items sequentially to estimate a child’s ability (θ) more precisely with fewer items than conventional tests (Chalmers, 2015; Keuning and Verhoeven, 2008). The efficiency of CAT is enhanced by large item banks, allowing the test to adapt to individual performance effectively (Nelson et al., 2017). IRT provides a robust framework for interpreting test scores, as it allows for accurate item selection and predicts the likelihood that a child will respond correctly based on their skill level (Bjorner et al., 2007; Baker, 2001). One key advantage of IRT is that item parameters remain stable across different samples, meaning they are not dependent on the specific group being tested (Magis and Barrada, 2017). Additionally, IRT offers a precision measure or standard error for each skill estimate (He and Min, 2017), providing insight into the accuracy of the assessment across varying levels of ability (Weiss, 1982).

The quality of assessment tools is important for educators to understand what children are expected to learn before formal school readiness. A primary aim of the current study is to adapt a digital assessment tool from the paper-pencil-based EuLe 4–5 assessment (Meindl and Jungmann, 2019), a standardized tool designed to assess narrative and early literacy skills in German children aged 4;0 to 5;11 years. Based on this goal, the study also aims to validate the EuLeApp© as a digital, adaptive assessment tool for children aged between 4;0 and 7;11 years. For this, a Multidimensional Computerized Adaptive Test (MCAT) was used based on Item Response Theory (IRT), allowing for individualized and precise measurement of children’s early literacy skills across multiple dimensions. An item pool was constructed through calibration based on the content of the items, and model fit was estimated and established using an IRT model.

These research questions will be addressed as follows:

Does the EuLeApp© screening tool accurately assess early literacy skills in children aged 4 to 7 years?

The sample consisted of N = 307 kindergarten children (M_age = 64 months, range = 45–91) before entering formal schooling in Mecklenburg-West Pomerania and Lower Saxony in Germany. The sample distribution of boys (n = 170, 55.4%) and girls (n = 137, 44.6%) was approximately equal. In terms of the distribution of children’s ages, 31.3% were 4 years old, 47.2% were 5 years old, 13.0% were 6 years old, and 8.5% were 7 years old. Data were primarily collected from kindergartens in middle- and high-socioeconomic regions. All parents were informed about the study, and written consent was obtained. Table 1 provides an overview of the participant demographics, including age distribution, gender, and other key characteristics.

Prior to the start of the study, university ethics approval was received, and permission to assess children was obtained from the head educators of a total of 15 kindergartens. Before administering the EuLeApp©, children’s language skills were evaluated to ensure that they possessed sufficient language comprehension. This step was essential in preventing the misinterpretation of literacy test results due to underlying language receptive problems and ensuring that literacy performance was accurately measured. Then, we assessed early literacy skills in the daycare centers using the prototype of the EuLeApp© on a tablet. The test practitioners were master’s students, PhD candidates, and postdoctoral fellows, all of whom completed two training sessions: one on understanding the assessment tool and its usage, and another on practical test implementation. All assessments were conducted individually. Before the test began, practitioners informed the children about the goal of the test and how it would help them, reassuring them that the test would not show everything they knew and could do and that they had plenty of time to answer the questions. Sitting beside the child, the test practitioners asked the child to practice tapping on the screen before beginning the assessment. Once the evaluation began, the practitioners did not answer the children’s questions or provide any tips to ensure standardized administration. During the EuleApp© assessment procedure, standard administration and scoring procedures were followed.

Figure 2 outlines the structured process used for the CAT analysis in EuLeApp©: (a) Developing a calibrated item bank: Relevant items from the EuLe 4–5 paper-based assessment tool were selected, categorized, and visualized to ensure consistency between the paper and digital formats. (b) Selection of starting items: A prototype was developed, and data were collected for item calibration, with model fit tested for accuracy. (c) Continuous estimation of a child’s ability: A child’s ability was continuously estimated during CAT simulation studies, applying a stop rule based on predefined precision criteria. (d) A final item pool was established based on simulations, integrated into EuLeApp©, and validated through reliable retest processes.

The R package mirtCAT was used for psychometric development of the app’s multidimensional computerized adaptive test (MCAT) based on Item Response Theory (IRT). Initially, items were adapted from the EuLe 4–5 test, whose content and construct validity had been previously established (Meindl and Jungmann, 2019). A digital platform was then created to deliver the assessment, incorporating multimedia features (vocal instructions, touch interactions, and graphics) to enhance children’s engagement and navigation. In the next step, data collection marked the first calibration phase of item pool development. A series of confirmatory IRT models were used to estimate item parameters and assess the effectiveness of the MCAT in providing individualized early literacy assessments. Both exploratory and confirmatory Item Factor Analysis (IFA) were conducted to validate the item structure, removing items misaligned with the identified factors to improve accuracy and reliability. Fit indices, including the Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMR), were applied to evaluate the calibration data against the proposed six-factor model (Brown, 2015).

To address potential estimation challenges and improve model convergence, the model was divided to reduce the number of estimated parameters: the first submodel included scales 1–4, while the second submodel included scales 5–6. Using these submodels, exploratory and confirmatory IFA models were developed. Structural analysis indicated that items 9–40 exhibited a bifactor structure (Chen et al., 2006; Dunn and McCray, 2020). This meant that the “Concepts of Print” scale could only be derived if the model incorporated two additional dimensions related to the item-specific use of numbers and images, which did not correlate with other dimensions. These were defined as “Numerical Writing Awareness” and “Iconic Writing Awareness.” Items 1–8 were also deemed usable when assigned to new dimensions, further strengthening the bifactor structure of this scale. Based on these findings, the structural analysis was refined using a model with six dimensions for the EuLeApp© scales and two item-specific dimensions for numbers and images. Six items were removed due to poor model fit, as they could not be assigned to any specific dimension.

As shown in Figure 3, the IRT curve of the 4-Parameter Logistic (4PL) model is compressed in the y-direction, ensuring that it remains within the probability range defined by the lower bound χ j and the upper bound ϒ j (Yen et al., 2012). This structure sets minimum ( χ j ) and maximum ϒ j probabilities for the correct response to each item j , independent of the test-taker’s ability level. This allows the model to account for guessing (lower asymptote) and disengagement (upper asymptote), providing a more accurate representation of performance.

While the lower asymptote captures the probability of randomly guessing an item correctly, it is also crucial to consider factors that might cause young children to respond incorrectly to easy items despite possessing the required knowledge. Issues such as inattention, attention deficits, socially desirable responding, external distractions, or a lack of motivation to engage with simple tasks may contribute to this variability (Liao et al., 2012). To better accommodate these influences, the 4PL model incorporates an upper asymptote, which accounts for the possibility that even highly skilled individuals may not always respond correctly due to carelessness, disengagement, or momentary lapses in concentration (Antoniou et al., 2022; Barton and Lord, 1981).

From a developmental perspective, the 4PL model provides a more precise estimate of young children’s abilities. By accounting for cognitive and behavioral fluctuations common in early development, this model offers improved sensitivity to response patterns that may be influenced by variability in attention, motivation, and task engagement (Anderson et al., 2002). These features make the 4PL model particularly useful in assessing young learners, where performance is not solely determined by ability but also by contextual and developmental factors.

Specific item selection criteria and stopping rules were defined for the item bank used in the Computerized Adaptive Test (CAT) analysis process to enhance testing efficiency and precision (Ebenbeck and Gebhardt, 2022). Standard techniques, including the selection of starting items, regression analysis, and stopping rules, were implemented to optimize these goals (Roberts et al., 2000). In EuLeApp©, item difficulty is adjusted based on children’s responses. To identify age-appropriate starting items, 24 of the 183 available items were selected based on content considerations, ensuring that the MCAT process began with items that were neither too easy nor too difficult for each age group. A regression analysis was then conducted to analyze the influence of these starting items on the standard error of measurement for determining personal abilities. The stopping rule was set based on test precision, where the CAT algorithm assesses whether the confidence interval falls within specified limits. When this criterion is met, the algorithm concludes the assessment for that construct. SEM (standard error of measurement) was chosen for the stopping rule because it offers a flexible framework for modeling relationships between observed data (test items) and latent traits (ability levels), evaluating measurement precision, and enhancing testing efficiency (Sideridis et al., 2018). During the assessment process, the stopping rule can be applied to end the test when SEM falls below a specified threshold, indicating sufficient precision:

The formula 𝑆𝐸𝑀 = 1 − R e l = ⇔ 𝑅𝑒𝑙 = 1 − 𝑆𝐸𝑀² allows for calculating standard measurement errors corresponding to reliabilities of 0.75, 0.80, 0.85, and 0.90, resulting in SEM values of 0.500, 0.447, 0.387, and 0.316, respectively. When the stopping rule SEM < 0.316 is applied, the procedure maintains a minimum reliability of 0.90 for the estimates of personal abilities, with the flexibility to specify different minimum reliabilities for each dimension (Bjorner et al., 2007; Chalmers, 2016). Using this approach, SEM ensures that the adaptive assessment is accurate and efficient, balancing the number of test items with the need for reliable measurement of a child’s ability. Simulation studies were conducted to evaluate the performance of the CAT algorithm, generating response patterns from simulated test subjects with fixed parameters (Magis and Raîche, 2012). The ability of IRT models to fit depends on the match between the items and the sample (skewed items require larger sample sizes, such as 500–1,000), with larger sample sizes providing better results. Given our smaller sample, we repeated parameter estimates multiple times to enhance their stability. We also conducted simulation studies to evaluate item functionality, establishing factor models based on the intended content during the data generation. The EuLeApp© was built on measuring information on the interrelationships among various early literacy dimensions. MIRT models can estimate skills with the categorical factor structure of early literacy components (Ackerman et al., 2003). Thus, the analysis process was carried out with multidimensional CAT, which is based on Multidimensional IRT (MIRT) models and allows the simultaneous measurement of more than one dimension (Segall, 2009).

A confirmatory IRT model was developed by assigning the items to six latent dimensions based on intended measurement purposes: concepts of print (Items 1–40), print awareness (Items 41–59), word awareness (Items 60–71), phonological awareness (Items 71–100), alphabet knowledge (Items 101–146), and early reading (Items 147–183).

Next, a covariance matrix was defined for a model with correlated dimensions. The number of parameters was 408 for the M2PL model, 585 for the M3PL model, and 762 for the M4PL model (Table 3). While statistical model fit is important, it should not be the only criterion for model selection; theoretical assumptions about the underlying model should also be considered (Robitzsch, 2022). Since we can theoretically substantiate that the IRT model has four parameters, we compared M2PL, M3PL and M4PL models. However, given the relatively small sample size (n = 307), it was anticipated that the model estimates of the M4PL models might lack stability (Wolf et al., 2013). When parameter estimates are unstable, this suggests the possibility of alternative models with improved parameter estimations (Robitzsch, 2022). Therefore, the M2PL, M3PL, and M4PL models were estimated multiple times using the same item allocations to further address stability concerns (e.g., the M4PL model was estimated 17 times).

Table 3 presents the model fit indices for the M2PL, M3PL, and M4PL models, with all estimates using the final item assignments.

The results consistently indicated that M4PL models provided a superior data fit compared to M2PL and M3PL models. Specifically, M4PL models No. 1 and 2 demonstrated superior RMSEA values, while models No. 1 and 10 stood out in terms of their SRMSR values. Based on the literature by Maydeu-Olivares (2013) to prioritize SRMSR as a model fit criterion, M4PL model No. 10 was ultimately selected. Additionally, the performance of model No. 10 in the multidimensional CAT framework played a decisive role in its selection. The favored M4PL model is superior to the M2PL and M3PL models not only in terms of fit indices, but also in terms of the number of answered items required to achieve a reliability of 0.80 on the MCAT. This iterative estimation process was essential for ensuring parameter stability and validating the reliability of the adaptive testing framework, reinforcing the robustness of the final model.

Table 3 shows that M4PL-Model No. 10, based on the M4PL model, demonstrates a very strong model fit, as indicated by the following fit indices: RMSEA = 0.0182 [0.0163; 0.0200], which suggests a close fit to the data; SRMR = 0.0563, indicating a small standardized residual; TLI = 0.9904, and CFI = 0.9908, both of which suggest an excellent fit to the model.

These values collectively confirm that the model fits the observed data well, providing reliable estimates.

An optimally calibrated item pool for CAT should include a broad distribution of items covering the difficulty parameter range Δ j , from −2 to +2. This range ensures that the item pool can assess abilities across a wide spectrum of test-takers. In addition to a well-distributed difficulty range, the discrimination parameter A j , plays a critical role in item selection. Higher values of A j indicate items that are more effective in distinguishing between individuals of different ability levels, thus contributing to the overall reliability of the test. Figure 4 presents the distribution of the difficulty parameters Δ j (MDIFF on the x-axis) and the discrimination parameters A j (MDISC on the y-axis) for the 177 items in the item bank.

Regarding discrimination A j , items in the calibrated bank exhibit varying levels of effectiveness in distinguishing between children with different abilities. The majority of the items falls within the range of high discriminatory power ( A j > 1.5, dashed line), indicating that they are very effective in distinguishing between different levels of ability. Only a small subset of 16 items shows an acceptable discriminatory power of 0.5 < A j < 1.5 (area between the dotted and dashed line). In the area of average abilities (−1 < Δj < 1), there is a subset of 15 items that stand out with an excellent selectivity of A j >15. These items are especially valuable in increasing the precision of the ability estimates for individuals whose abilities lie near the population mean. The calibrated item pool reveals that items with the highest discriminatory power are concentrated in the range of average abilities, specifically between −1 and +1 on the ability scale. The item bank is concentrated with highly discriminating items from below-average to average abilities (−2 to +1), ensuring accurate assessment in this range. A dozen items adequately cover the above-average range (+1 to +3). This distribution ensures the test maintains precision and reliability across a broad spectrum of abilities.

The stability of all measures over time was examined through a correlation analysis. T1 took place in Fall 2023, followed by T2 in Spring 2024. As shown in Table 7, which presents both within-time correlations and T1-T2 test–retest correlations, each measure demonstrated good reliability across the two time points. The strong correlations (p < 0.01) among literacy components suggest that these skills are highly interrelated, with each influencing or aligning closely with the others.