This research utilized a systematic multi-criteria decision-making methodology to establish and execute a monitoring framework for evaluating the attainment of PLOs in the Pharm.D program at a public pharmacy college in Saudi Arabia. The architecture reflects decision-analytic methodologies employed in engineering accreditation systems while being tailored to the pharmaceutical domain and the criteria set by the National Commission for Academic Accreditation and Assessment (NCAAA) (1).

The PLOs were formulated based on the Pharm.D program PLOs framework, which corresponds with the Saudi Arabian Qualifications Framework (SAQF) and the five areas of the NCAAA: knowledge, cognitive skills, interpersonal skills and responsibility, communication and IT skills, and psychomotor abilities. This study adopted the full set of eight PLOs as defined by the institutional Pharm.D curriculum. These PLOs align with the SAQF and NCAAA competency domains and are listed with their quantitative indicators in Table 1.

These outcomes were translated into measurable indicators across alumni employment records, postgraduate enrollment, and documented clinical roles.

The Analytical Hierarchy Process (AHP) was employed to determine the relative importance of each Program Learning Outcome (PLO). A total of 21 stakeholders participated in the AHP weighting process, including employers, program graduates, academic administrators, and faculty members. Each participant independently completed the pairwise comparisons using Saaty’s 1–9 scale. To ensure the internal consistency of judgments, a consistency ratio (CR) was calculated for each matrix. Only matrices with a CR < 0.1 were accepted, as recommended by AHP methodology standards. The final weights for each PLO were obtained by aggregating the normalized values from all valid matrices.

AHP Weight Calculation, which calculates the average normalized value for each PLO across all comparisons. Each panel member completed the AHP pairwise comparisons independently using Saaty’s 1–9 scale (Equation 1).

Where: W i : normalized weight for the ith Program Learning Outcome (PLO); n: total number of PLOs; a ij : importance score of PLOi compared to PLOj in the pairwise comparison matrix; Σ is column sum of comparisons for PLOj.

AHP Consistency Ratio (CR), calculates the Consistency Ratio (CR), a metric used in the Analytic Hierarchy Process (AHP) to evaluate how consistent the judgments are in a pairwise comparison matrix (Equation 2).

Where: CR: Consistency Ratio, used to check if judgments are logically consistent; CI: Consistency Index; λmax: Principal eigenvalue of the pairwise comparison matrix; RI: Random Index (average CI of a randomly generated matrix of order n); nn: Number of PLOs; CR value < 0.1 indicates acceptable consistency.

This study employed a convergent mixed-methods design, integrating qualitative stakeholder input with quantitative program outcome data. The qualitative component was the Analytic Hierarchy Process (AHP), which used structured pairwise comparisons to collect expert opinions on the relative significance of each PLO. These judgments were synthesized into normalized weights reflecting stakeholder priorities.

The quantitative strand consisted of objective performance indicators linked to each PLO (e.g., employment rates, exam scores, postgraduate enrollment). The Multi-Objective Optimization by Ratio Analysis (MOORA) method was used to process these indicators. Each graduating cohort’s composite achievement score was generated by applying the weights derived from the AHP during MOORA analysis.

This integration allowed for a combined assessment model in which stakeholder-defined priorities directly shaped the evaluation of program outcomes, ensuring both contextual relevance and data-driven rigor.

A solution based on Microsoft Access was created to collect alumni data, implement AHP weights, normalize numbers using MOORA, and illustrate annual trends. The database had tables for graduate profiles, career information, postgraduate study records, and clinical positions. Alumni statistics were updated each year using a synthesis of institutional records, LinkedIn, employment verification, and self-reported surveys.

The study received ethical clearance from the Research Ethical Committee at King Khalid University (approval no. KKU-14-2025-10). Alumni data were anonymized, and all participants in surveys or interviews provided informed consent.

A total of 21 stakeholders participated in the AHP pairwise weighting process. Their demographic and professional distribution is summarized in Table 2. Participants represented key roles relevant to curriculum delivery, program oversight, and graduate employment. All contributors had direct experience with Pharm.D program evaluation and at least 3 years of professional practice in their respective domains.

Stakeholders were selected based on involvement in curriculum delivery, program evaluation, or graduate employment. All had a minimum of 5 years of experience in pharmacy education or practice.

A panel of 21 stakeholders, including academics, graduates, and employers, engaged in pairwise comparisons of PLOs utilizing the Analytic Hierarchy Process (AHP). Their contributions determined the significance of each PLO in alignment with curricular aims and the requirements of the Saudi workforce. PLO2 (clinical application) was assigned the most weight (0.28), followed by PLO5 (ethics and professionalism, 0.19) and PLO1 (pharmaceutical/biomedical knowledge, 0.17). The outcome with the lowest weight was PLO8 (lifelong learning, 0.04), indicating its more prolonged, rather than immediate, effect on early-career performance (Table 3).

The stakeholder priorities guided the weighting of performance data across cohorts, facilitating an outcome-oriented and relevance-based comparative framework. Figure 1 illustrates a summary of PLO performance patterns throughout cohorts, indicating that PLO2 and PLO5 not only have the greatest weights but also consistently attained superior scores in real cohort achievement.

Normalized scores for all eight PLOs were calculated across five graduating cohorts from 2018 to 2022 using Multi-Objective Optimization by Ratio Analysis (MOORA). The scores were derived from quantitative metrics including postgraduate enrollment, job placement, and positions in patient care. The findings indicated that the cohorts of 2020 and 2021 consistently attained the best PLOs performance scores, especially in PLO2 and PLO5, demonstrating strong clinical integration and ethical training throughout those years (Table 4).

The chart highlights improvement over time in critical PLOs such as PLO3 (communication) and PLO4 (teamwork), and flags consistent underperformance in PLO8 (Figure 2).

Composite cohort performance was ranked after adding stakeholder-derived AHP weights to MOORA-normalized scores. The 2020 cohort attained the greatest score (0.883), followed by the 2021 cohort (0.871); however, the 2022 cohort recorded the lowest composite score (0.792), indicating potential instructional deficiencies or external disturbances (Table 5).

Figure 3 illustrates a stacked bar chart depicting the contributions of the PLOs to enhance comprehension of the composite cohort development. The uniformity in high-performing PLOs, such as PLO2 and PLO5, is visually apparent, but underperforming areas like PLO6 (digital health) necessitate targeted enhancement. Figure 4 presents a longitudinal summary graphic that highlights performance changes across years, distinctly illustrating peak outcome scores in 2020 and 2021, followed by a relative fall in 2022.

Figure 4 provides a comparative visualization of PLOs achievement trends across five graduating cohorts, illustrating longitudinal performance patterns by outcome.

This study introduced a dual-framework monitoring system using the Analytic Hierarchy Process (AHP) and Multi-Objective Optimization by Ratio Analysis (MOORA) to evaluate the longitudinal achievement of PLOs in a Saudi Pharm.D program. The tool was designed to meet national accreditation requirements set by the National Commission for Academic Accreditation and Assessment (NCAAA) while promoting internal quality assurance and responsiveness to stakeholder priorities.

The AHP-MOORA framework, utilized for the longitudinal assessment of Pharm.D Program Learning Outcomes (PLOs) at King Khalid University’s College of Pharmacy, provides a comprehensive, data-driven instrument for aligning educational outcomes with National Commission for Academic Accreditation and Assessment (NCAAA) standards (6, 10). By employing the Analytical Hierarchy Process (AHP) for stakeholder prioritization and Multi-Objective Optimization by Ratio Analysis (MOORA) for cohort performance evaluation, the framework exceeds conventional subjective assessments, delivering accurate, actionable insights for accreditation and quality assurance (14, 15). The Microsoft Access-based solution facilitated effective data management, indicating peak cohort performance in 2020 (0.883) and a decrease in 2022 (0.792) across five cohorts (2018–2022). This model’s conformity with NCAAA’s continuous quality improvement requirements and Saudi Arabia’s Vision 2030 healthcare objectives establishes it as a scalable solution for pharmacy education, potentially improving accreditation preparedness across Saudi and Gulf Cooperation Council (GCC) institutions (1, 2, 4).

The AHP results ranked PLO2 (clinical application, 0.28) and PLO5 (ethics and professionalism, 0.19), indicating stakeholder prioritization of patient-centered and ethical competencies, in alignment with Mutalib et al.’s findings regarding the significance of technical and professional skills in engineering education, which have been adapted to the clinical focus of pharmacy (16). These priorities correspond with the Saudi Arabian Qualifications Framework (SAQF) and the global CanMEDs framework, which underscore values-based practice, although they diverge from Aljadhey et al.’s emphasis on research skills (PLO7) in certain Saudi programs (3, 17, 18). The minimal emphasis on PLO8 (lifelong learning, 0.04) deviates from the Accreditation Council for Pharmacy Education (ACPE) criteria, indicating a regional prioritization on urgent workforce preparedness, as observed in Mokhtar’s GCC study (4, 19).

MOORA scores demonstrated exceptional performance in PLO2 and PLO5, particularly in 2020 and 2021 (e.g., PLO2: 0.94 in 2020), signifying effective clinical and ethical training, supported by Alshahrani’s study on enhancements in quality assurance within Saudi pharmacy programs (7). Nevertheless, ongoing underachievement in PLO6 (digital health integration, 0.70 in 2022) and PLO7 (compounding and dispensing, 0.60 in 2022) underscores deficiencies in technical and practical competencies, corresponding with national digital transformation obstacles (2, 4).

The 2020 cohort achieved a high performance of 0.883, whereas the 2022 cohort had a fall to 0.792, indicative of external factors, particularly the disruptions caused by COVID-19. Alrushud et al. showed diminished academic engagement among Saudi medical students during lockdowns (19). Conversely, Al Mohaimeed et al.’s investigation of Saudi medical education revealed consistent post-accreditation results, indicating that pharmacy programs encounter distinct difficulties in sustaining experiential learning throughout crises (7).

The longitudinal trend visualization of the Microsoft Access tool surpassed the manual techniques outlined by Khojah and Shousha and aligned with the data collecting system of Mutalib et al. for engineering outcomes; however, its AHP-driven stakeholder integration provides distinct customization (12, 16). In contrast to Malik et al.’s global quality assurance paradigm, the AHP-MOORA framework demonstrates superior stakeholder participation and data-driven decision-making, hence improving accountability (20). These findings highlight the necessity for focused interventions, such as virtual simulations and telehealth training, to rectify weaknesses in PLO6 and PLO7 and maintain performance during disruptions. Comparable outcome-based evaluation frameworks in pharmacy and engineering education have illustrated the effectiveness of systematic assessment instruments. Elkhalifa et al. (17) identified deficiencies in competency evaluation within pharmacy graduate programs, underscoring the necessity for a standardized Program Learning Outcomes tracking system such as AHP-MOORA. Similarly, Wu et al. (8) applied AHP in the assessment of student performance, emphasizing its significance in the oversight of higher education.

Beyond external accreditation, the AHP-MOORA model also serves as a robust mechanism for internal quality improvement. Pharmacy colleges can use the tool to support annual curriculum reviews, departmental performance audits, and stakeholder feedback integration. By tracking PLO attainment trends over time, faculty can identify underperforming domains and implement targeted revisions. Furthermore, the model supports real-time reporting to academic boards, enhances transparency during program self-studies, and facilitates continuous dialogue among faculty, administrators, and external partners. Consistently reduced MOORA results in PLO domains, including digital health and pharmaceutical compounding, necessitated internal task force assessments and course redesigns. These findings endorse focused teacher development and improved evaluation instruments aligned with performance gaps.

This study possesses a few limitations that need to be acknowledged when evaluating the results. The investigation was restricted to a singular public pharmacy institution in Saudi Arabia, potentially constraining its applicability to other institutional contexts or educational frameworks. The limited number of stakeholders participating in the AHP weighing process (n = 21) may inadequately represent various points of view of academics, alumni, and employers. Furthermore, although stakeholder involvement was integral to the AHP process, it may have generated subjective bias in the prioritization of learning objectives.

Data reliability is an additional factor to consider. Segments of the data—especially postgraduate enrollment and alumni job status—were self-reported and prone to recall bias or incompleteness. Despite the enhancement of accuracy through cross-validation using employment documents, LinkedIn profiles, and institutional data, deviations may persist, particularly among graduates employed abroad or from earlier cohorts.

Technical constraints also took place in the application of Microsoft Access for MOORA computation. The platform was selected for its institutional familiarity and ease of deployment; nevertheless, scalability may be limited for larger programs or cross-institutional applications. Nonetheless, the framework’s architecture facilitates prospective integration with automated analytics, as proposed by Alhakami et al. (21) and Mutalib et al. (16), potentially alleviating faculty effort and enhancing real-time responsiveness.

Notwithstanding these constraints, the AHP-MOORA framework exhibits robust conformity with both national (NCAAA, SAQF) and international (ABET) certification standards. This model incorporates structured prioritizing and longitudinal cohort tracking, improving its effectiveness for academic quality assurance compared to previous ABET-aligned methods. Al Mohaimeed et al. (7) asserts that teacher stress and workload continue to impede effective evaluation processes. Nonetheless, institutions implementing specialist teams, like to those delineated by Shaiban (22) in the realm of medical education, may alleviate these issues.

The extensive implementation of this approach at Saudi pharmacy universities, alongside a comparison with ACPE-accredited programs, may confirm its scalability and flexibility. The paradigm enables outcome-focused program assessment and curriculum development, particularly in digital health, resource distribution, and faculty engagement, offering a sustainable method for continuous quality advancement in pharmacy education (21, 23).

Quantitative decision-making tools like AHP and MOORA are essential for promoting evidence-based curriculum enhancement by providing organized, transparent, and replicable techniques for prioritizing educational objectives and monitoring the achievement of learning outcomes across cohorts. These approaches augment institutional capability to analyze intricate performance data, facilitate informed curricular modifications, and synchronize academic outcomes with national competency benchmarks. Their structured framework enables longitudinal assessment and enhances qualitative stakeholder feedback, thereby adhering to the principles of mixed-methods implementation research, which emphasizes the combination of quantitative precision and contextual insight to connect evidence with practice in actual educational environments (24).

Future studies should examine whether this framework can be applied to other Pharm.D programs in Saudi Arabia and abroad. Validating the model in various regulatory contexts may be aided by comparative assessments with ACPE-accredited institutions. Technological improvements such as cloud-based dashboards and automated analytics are recommended to support scalability. It will also be crucial to improve performance indicators for underdeveloped domains (such as ethics and digital competency) and encourage continued faculty participation in quality monitoring to ensure long-term adoption and sustainability.