2.1. Step 1: Sample selection and demographic characterization

The sample is taken from students who decided to participate in the critical reading institutional test. The data is gathered from the Admissions and Academic Registration Department and the Department of Virtual and Distance Education of the Universidad Católica Luis Amigó. The test is applied online on the virtual campus (Campus Virtual, 2023). All the students under analysis are in Medellín and are enrolled in different undergraduate programs. The students' added value analysis will be done in 2022. However, the model was developed in 2018, and its predictive capabilities were evaluated before (2019 to 2022). These past years have been evaluated during and after the COVID-19 pandemic.

2.2. Step 2: Data Collection

Data is collected from students who presented the institutional test that evaluates different skills, particularly critical reading. It is important to note that this test is carried out in the middle of the degree (between the fifth and seventh academic semester) and is carried out by expert teachers and members of the Universidad Católica Luis Amigó, together with the Department of Basic Sciences (Universidad Católica Luis Amigó, 2023). The student's consent to being evaluated for reading competence is given when they complete the form and take the institutional test. For these students, data from the Saber 11 tests is collected (Presidencia de la República, 2010). Additionally, demographic information and other qualitative and quantitative variables are gathered. The Saber 11 test assesses the skill development of eleventh-grade students and provides critical information on competencies such as reading comprehension. This data is essential for universities to design leveling courses and dropout prevention programs.

This test evaluates the educational quality of schools based on national standards and supports the creation of added value and educational quality indicators. These insights are instrumental for institutions' self-evaluation and the formulation of educational policies at various levels. Reading competence is assessed by examining students' ability to identify and understand the local content of a text, comprehend how its parts interconnect to convey a global meaning, and reflect on and evaluate its content. These skills are measured through questions that test students' interpretation of literary and informational texts. Additionally, the test assesses their ability to contextualize information and establish relationships between different statements and texts. This comprehensive evaluation provides a detailed understanding of students' abilities, highlighting their strengths and areas for improvement.

2.3. Step 3: Statistical variables analysis and selection

Critical reading competence is the dependent variable analyzed in association with sociodemographic factors and institutional test scores, which are treated as covariates. Table 1 provides a detailed description of the variables under study, categorized into dependent and covariate groups. The dependent variable, critical reading, represents the score obtained on the institution-administered test, measured on a scale from 1 to 100.

The covariates include the language score on the Saber 11 test (coded as saber11_language), the student's academic program, the type of school (public or private), socioeconomic level, living arrangements, monthly family income, whether the student has a source of income from work, the parents' years of education, whether the parents are separated, and the student's cumulative career average. According to ^{Psyridou et al. (2024}), these factors significantly influence the development of critical reading skills in children and adolescents. Their research demonstrates that high socioeconomic status, access to resources, engagement in literacy activities, parental involvement in schooling, and a structured home environment are associated with stronger critical reading skills. The study emphasizes the importance of the family environment and calls for further research to deepen understanding of these factors. Identifying the qualitative and quantitative variables is essential for appropriately applying measurement instruments, conducting value-added analysis, and building robust mathematical models.

2.4. Step 4: Development of a model for estimating critical reading

The model for estimating the Critical Reading score was developed using the multiple linear regression method. The dependent variable is critical reading, and the independent variables are those that showed an association in the correlation matrix. Multilevel or hierarchical linear models (MLM) are not used because the sample consists only of the Medellín Campus of Universidad Católica Luis Amigó. Instead, the 'forward' multiple linear regression method is applied to develop three proposed models, each incorporating independent variables associated with the Critical Reading test. The resulting regression model, described in Equation 1, offers significant insights into the factors influencing Critical Reading scores.

In Equation (1), Y is the dependent variable, Critical Reading score, β₀ is the value of the score in the absence of independent variables, β_i indicates how much the score is modified due to the independent variable, and X_i corresponds to the independent variables.

2.5. Step 5: Statistical analysis to quantify the added value of the institution

The Quintiles statistician is responsible for analyzing added value. These statistical analyses use positional measures that divide the data set into equal portions. Specifically, Quintile 1 includes the lowest 20% of results, while Quintile 5 represents the highest 20% (^{Montgomery, 2020}; ^{Posada-Hernández et al., 2023}). The comparison is conducted by graphing the results on a Cartesian plane, where the x-axis corresponds to the Saber 11 test results, and the y-axis corresponds to the institutional test results. Additionally, bar graphs are used to display positive and negative values, enabling a more comprehensive analysis. The importance of added value estimation lies in the upper Quintiles, as they reflect the institution's significant contributions. All statistical analyses are conducted using SPSS software.

3.1. Student participation in the test

The value-added analysis for 2022 focused on the population of students enrolled in the fifth semester during periods 01 and 02 of the year. Program directors at the Medellín campuses extended broad invitations to students to participate. Table 2 illustrates student participation trends over the years. Although student response in 2022 was notable, participation decreased compared to 2021. It is important to note that students typically take the test during the fifth semester of their undergraduate program, which consists of four months of academic study. The analyzed fifth-semester student population exhibits the following characteristics:

3.2. Institutional added value

The institutional test results were evaluated on a scale of 0 to 5, with an average score of 2.2-below the acceptable level of 3.0. Similarly, in the Saber 11 test, scored on a scale from 0 to 100, average results across locations did not exceed 56 points, falling short of the acceptable threshold of 60. Table 3 summarizes the averages for the Critical Reading and Saber 11 tests, offering an overview of student performance in these assessments. The distribution across quintiles reveals limited improvement in added value in critical reading. The highest percentage of students with added value is found in quintile Q2, encompassing 40% of the lowest scores. Conversely, quintiles Q3 and Q4 show negative added value. Notably, a minimal percentage (1%) of students in quintile Q5 achieve added value, indicating that even small progress is possible. While these results serve as a baseline for future improvement, significant potential for growth remains.

The comparison results obtained from 2018 to 2022 in the Critical Reading competence show no statistically significant changes in the added value in the quintiles; only an increase of 1% appears in quintile 5 when moving from 2018 to 2022. Figure 1 shows the results of the temporal analysis.

Figure 1 Percentage of students with added value by quintiles: Critical reading 2018-2022, Medellín campus. 

The distribution of deviations in test scores across the quadrants effectively visualizes the percentage of students demonstrating added value in Critical Reading competency. Figure 2 illustrates this distribution by quintiles. The institution's students are categorized into four quadrants based on their results in the Saber 11 Language tests and the Critical Reading Simulation. In Quadrant I, 35% of students score above the average on both tests, reflecting minimal institutional contribution. Quadrant II, encompassing 23% of students, demonstrates significant improvement in Critical Reading, with scores rising from below average in Saber 11 to above average in the Simulation. While this progress is encouraging, it underscores the need for continued improvement. Quadrant III, comprising 25.2% of students, includes those with low scores on both tests, indicating a lack of added value from the institution. Lastly, with 16.8% of students, Quadrant IV features those who scored above average in Saber 11 but below average in the institutional test.

Figure 2 Distribution of the Percentage of students by quadrants in the critical reading added value in 2022. 

Figure 3 compares the percentages of students in the quadrants from 2018 to 2022. A slight increase is observed in Quadrant I, which has minimal impact on added value. Quadrant II shows a small increase, which remains significant for contributing to the added value for students. Conversely, Quadrant III reflects a slight rise in the percentage of students, which is unfavorable as it indicates an increase in the number of students with low scores in both the Saber 11 Language test and the Critical Reading Simulation. Lastly, Quadrant IV shows a slight decline in the percentage of students who enter the institution with above-average Saber 11 Language scores yet achieve below-average scores in the Critical Reading simulation.

Figure 3 Added value of Critical Reading by Quadrants from 2018 to 2022. 

Figure 4 illustrates the percentage and temporal behavior of program performance in terms of added value. High percentages in previous years have represented the Critical Reading competency: in 2018, 90% of the programs demonstrated added value, increasing to 95% in 2019 before dropping back to 90% in 2020. However, a significant decline was observed in 2021 and 2022, with percentages falling to 24% and 5%, respectively.

Figure 4 Percentage distribution of the added value on the Critical Reading from 2018 to 2022. 

3.3. Results of the development of a model that evaluates critical reading competence

The linear regression model was used to estimate the Critical Reading competency score at Universidad Católica Luis Amigó. First, the correlation matrix was used to assess the relationship between each competency and various sociodemographic and academic variables (^{Montgomery, 2020}). Table 3 displays the correlations between the Critical Reading score and demographic variables, the Saber 11 score, the time taken for the Saber 11 test, and the institutional quantitative reasoning test score. A significant positive correlation is observed with the quantitative reasoning score (0.250), cumulative average (0.234), and Saber 11 scores in mathematics (0.208) and language (0.260). There is also a significant negative correlation between the time taken to complete the Saber 11 test (-0.114) and whether the student has a source of income from work (-0.160). Variables such as type of school, socioeconomic level, family income, and parents' education do not significantly correlate with the Critical Reading score.

The results of the Critical Reading test are associated with the cumulative average, the Saber 11 Language and Mathematics tests, the time taken to complete the Saber 11 test, and the presence of employment income. Based on this analysis, a model is proposed. The multiple regression process was performed, and the resulting model is shown in Equation 2. This model was obtained using SPSS software.

In Equation (2), Y is the Critical Reading score, x₁ is the cumulative average, x₂ is the Saber 11 Language test score, and x₃ is the Quantitative Reasoning test score. The Cumulative Average of the students represents the most significant contribution, around 0.358 units points of the change in the Critical Reading score. The results show that the 0.162 points contributed to the Quantitative Reasoning test score. In comparison, the Saber11 Language test score contributes only 0.011 points. As can be noted, the score obtained in Critical Reading is influenced, for the most part, by the Cumulative Average of the students.

3.4. Verification of assumptions of the multiple linear regression model for the critical reading test

Linearity

When associated with the independent variables, the dependent variable must follow a linear relationship. This assumption is validated by observing whether the data trend follows a linear pattern in the scatterplot (^{Montgomery, 2020}). The dependent variable, Critical Reading, shows a linear trend with the cumulative average, the Saber 11 Language and Mathematics tests, the time taken to complete the Saber 11 test, and the presence of employment income. Figure 5 presents the results of this test.

Figure 5 Linearity of the Critical Reading variable with the independent variables. 

Normality

Residual analysis or model errors must follow a normal distribution (^{Montgomery, 2020}). This ensures that both large and small residuals occur less frequently in the model's estimates. The normality of residuals is assessed using Q-Q or P-P plots, which show how well the residuals fit a linear pattern, or the Kolmogorov-Smirnov test, where the p-value should be greater than 0.05. Figure 6 presents the results of the normality test. The residuals exhibit a normal distribution, as the points closely follow the regression line. Additionally, the Kolmogorov-Smirnov test shows a significance of 0.051, and the Shapiro-Wilk test yields 0.096. Both values exceed 0.05, indicating that the residuals do not significantly deviate from a normal distribution.

Figure 6 Normality for the residuals of the Critical Reading variable. 

Homoscedasticity

Homoscedasticity refers to the equality of variances between the residuals and the predicted values. This condition is examined using a scatter plot with ZPRED (predicted values) and ZRESID (residuals). The assumption of homoscedasticity implies that the variation in residuals remains consistent across the range of predicted values, meaning the scatter plot should not show any clear pattern. Figure 7 illustrates the dispersion of the Critical Reading score. Upon analyzing the scatter plot of predicted values and standardized residuals, it is evident that the variation is consistent, with no discernible pattern in the distribution.

Figure 7 Scatter diagram for the residuals and forecasts of the Critical Reading variable. 

Figure 8 shows the William plot for the model described in Equation (2) and its predictive capabilities from 2018 to 2022. The William plot illustrates the influence of each data point and the standardized residuals in a regression model. The leverage values are represented on the X-axis, reflecting the influence of each data point on the regression coefficients; high leverage values can significantly affect the model. The standardized residuals are shown on the Y-axis, with values outside the ±2 range suggesting potential outliers. Yearly distribution analysis for 2018 and 2019 shows that most data points cluster around zero, with few high-leverage values. In 2020, some data points exhibit high leverage, though their residuals remain within acceptable limits. In 2021 and 2022, more variability is observed in both the residuals and leverage values. The analysis highlights several outliers, particularly in the last three years. Overall, only 1.66% of the data points are considered outliers, indicating that most of the data falls within the predictive capabilities of the model.

Figure 8 William's diagram for the predictive model described in Equation 2 over the years. 

3.5. Discussion

The analysis of institutional added value revealed that the university's contribution to developing critical reading competence needs improvement. Most students did not show significant improvements in critical reading scores between the Saber 11 tests and the institutional exam, indicating a need for stronger support and the creation of improvement plans. Overall, the results suggest that critical reading competence does not reach acceptable levels, with an average score of 2.2 on the institutional test (on a scale from 0 to 5). This highlights the urgent need to improve educational strategies and teaching practices to strengthen this vital competence, especially in the early stages. Further research and development of critical reading skills are recommended to ensure comprehensive student training and to emphasize the importance of this issue. The quintile analysis revealed that 35% of the students exceeded the average in both tests, while 25.2% scored poorly. Effective pedagogical strategies and targeted teacher training in critical reading are essential for fostering critical thinking and helping students become responsible citizens less susceptible to misinformation. Measuring critical competence presents challenges due to linguistic and cultural differences. ^{Jiménez-Pérez (2023}) recommended improving teaching practices, conducting more studies in this research area, and developing these skills early in students' education. This study used a qualitative methodology with seven undergraduate students.

Critical thinking and reading competence were evaluated using argumentative texts, interviews, and written responses. The results showed improvements in students with higher English proficiency, who demonstrated a better ability to draw conclusions and express personal opinions. In contrast, students with lower English proficiency faced significant challenges in identifying the author's conclusions and purposes.

The study concludes that it is crucial to teach critical reading skills, suggesting that teachers focus on students with sufficient knowledge of English and use interesting topics to encourage critical thinking and literacy. A similar conclusion is made by ^{Arifin's (2020}) work. ^{Le et al. (2024}), show the difficulties in applying analysis, synthesis, and evaluation strategies in students. These authors discuss the need for pedagogical approaches that promote improving language skills and foster higher-order thinking processes through focused modules and workshops, such as rhetorical analysis and debates.

On the other hand, ^{Untari et al. (2023}) advocate for fostering an explicit understanding of texts, integrating real-world experiences with reading materials, and critically evaluating content. They also emphasize the importance of improving critical reading skills from an early age to promote deeper understanding and critical analysis of information. Similarly, ^{Paakkari et al. (2024}) highlight the role of parental support and family environment in enhancing critical reading skills. In contrast, statistical modeling reveals that sociodemographic variables such as the type of school attended, socioeconomic status, and family income did not significantly correlate with critical reading scores. This suggests that students' socioeconomic backgrounds do not strongly influence their critical reading performance. However, the students' cumulative average and quantitative reasoning skills significantly impact their critical reading performance at the 95% confidence level, indicating a direct relationship.

Based on the above, a linear model is proposed to predict critical reading scores. This model serves as a useful tool for identifying students who may require specific educational interventions. The developed regression model will enable comparisons between projected and actual data, helping to assess whether critical reading competence has improved or declined. By analyzing these results, educational and intervention strategies can be adjusted to ensure a more effective approach to developing critical reading skills.