Effectiveness or efficiency in the evaluation of professional and technological education? A proposal for an integrated model to evaluate the institutional performance of Federal Education Institutes

Abstract

In recent decades, the assessment of technical efficiency has occupied a prominent position in performance evaluations of public educational institutions. Although relevant, this approach, when adopted in isolation, proves limited considering the operational complexity, regional diversity, and the multiple functions assigned to the Federal Institutes of Education, Science and Technology (IFs). This study argues that institutional assessment should incorporate different analytical levels and performance dimensions to support more accurate and contextualised decision-making. The research aims to propose an integrated model for evaluating institutional performance using the Directional Distance Function (DDF), an extension of traditional Data Envelopment Analysis (DEA), applied to panel data covering the period from 2017 to 2023. The first analytical strand, with an intra-institutional focus, assessed 11 teaching units of a Federal Institute. Quantitative data on human, financial, and physical resources were used, along with desirable and undesirable variables related to teaching, research, outreach, and management. The analysis adopted an input-oriented model with variable returns to scale (VRS), combining internal benchmarking, institutional segmentation, and metafrontier modelling. It revealed that unit efficiency was more closely associated with the organisation of resources than with budget volume, indicating opportunities for improvement. The institutional average efficiency during the period was 80.3%. In the second strand, an inter-institutional analysis was conducted involving all 38 Federal Institutes. A composite indicator was developed using a DDF-based model with constant returns to scale (CRS) and output orientation, eliminating resource-related heterogeneity to emphasise performance outcomes. In parallel, institutional effectiveness was assessed based on the achievement of strategic goals. Although the directional vector was fixed and unitary for all variables, its limitations in interpreting β as a proportional efficiency measure were acknowledged. The integration of efficiency and effectiveness dimensions enabled a comprehensive institutional assessment. Results indicated that 42% of the institutions performed above average in both dimensions, while 34% fell below. The proposed methodological framework enhances the diagnosis of institutional performance and offers strategic support for public education management. The main limitations relate to the absence of external benchmarking in the inter-institutional analysis and the reliance on standardised secondary data.