https://doi.org/10.22267/rtend.262701.290
Research Article
Educational Administration
Proposal and validation of a system of indicators to measure the Third University Mission in Colombian higher education institutions
Propuesta y validación de un sistema de indicadores para medir la Tercera Misión Universitaria en instituciones de educación superior colombianas
Proposta e validação de um sistema de indicadores para medir a Terceira Missão Universitária em instituições de ensino superior colombianas
By: Alix Johana Gaffaro Garcia
1; Claudia Milena Pico Bonilla2; María de Lourdes Reyes Vergara3
1 Master's Degree in Management Engineering, Universidad Metropolitana de Caracas. Professor at Politécnico Grancolombiano. ORCID: 0000-0002-5132-4940. E-mail: agaffaro88@unisalle.edu.co. Bogotá, Colombia
2PhD in Psychology, Full Professor at Universidad Militar Nueva Granada. ORCID: 0000-0001-8839-5462. E-mail: claudia.pico@unimilitar.edu.co, Bogotá - Colombia.
3PhD in Education, Professor at Universidad Popular Autónoma del Estado de Puebla (UPAEP). ORCID: 0000-0002-7746-428X. E-mail: mariadelourdes.reyes@upaep.mx, Puebla - México.
Received: June 1, 2025 Accepted: November 4, 2025
DOI: https://doi.org/10.22267/rtend.262701.290
How to cite this article: Gaffaro, A., Pico, C. & Reyes, M. (2026). Proposal and validation of a system of indicators to measure the Third University Mission in Colombian higher education institutions. Tendencias, 27(1), 148-177. https://doi.org/10.22267/rtend.262701.290
Abstract
Introduction: The Third Mission of Universities (TMU) refers to the active engagement of universities with society, aiming to promote sustainable development and generate social value. However, in Latin America and Colombia, there are methodological and information limitations that make it difficult to measure systematically. Objective: This study proposes a system of indicators designed to measure the TMU in Colombian Higher Education Institutions (HEIs). Methodology: A mixed and quantitative approach was employed, which included a literature review, selection and categorization of indicators, and validation through expert judgment and statistical analyses, such as Cronbach’s Alpha Coefficient and Confirmatory Factor Analysis. Result: The result is a valid structure consisting of 31 indicators that demonstrate internal consistency and there are theoretically and empirically aligned to the Colombian context. Discussion: This study advances previous research by providing a replicable and adaptable quantitative instrument that facilitates the measurement of university-society engagement. Conclusion: The proposed system offers a useful methodological framework for evaluating institutional appropriation of TMU, identifying areas for improvement in HEIs, and contributing to the design of university engagement policies and strategies grounded in concrete results.
Keywords: factor analysis; institutional evaluation; university management; performance indicators; expert judgment.
JEL: D83; I23; L31; O32; O35; Q01
Resumen
Introducción: La Tercera Misión Universitaria (TMU) involucra la conexión de las universidades con la sociedad para promover el desarrollo sostenible y generar valor social. Sin embargo, en América Latina y Colombia, existen limitaciones metodológicas y de información que dificultan su medición sistemática. Objetivo: Este estudio tiene como objetivo proponer un sistema de indicadores para medir la TMU en las Instituciones de Educación Superior (IES) colombianas. Metodología: Se utilizó un enfoque mixto y cuantitativo, que incluyó una revisión bibliográfica, selección y categorización de indicadores, y validación mediante juicio de expertos y análisis estadísticos, como el Coeficiente Alfa de Cronbach y el Análisis Factorial Confirmatorio. Resultado: El resultado es una estructura válida de 31 indicadores que demuestran consistencia interna y se ajustan tanto teórica como empíricamente al contexto colombiano. Discusión: Este estudio va más allá de investigaciones previas, proporcionando un instrumento cuantitativo replicable y adaptable que facilita la medición del vínculo universidad-sociedad. Conclusión: El sistema propuesto proporciona una base metodológica útil para evaluar la apropiación institucional de la TMU, identificar áreas de mejora en las IES y contribuir al diseño de políticas y estrategias de vinculación universitaria basadas en resultados concretos.
Palabras clave: análisis factorial; evaluación institucional; gestión universitaria; indicadores de desempeño; juicio de expertos.
JEL: D83; I23; L31; O32; O35; Q01
Resumo
Introdução: A Terceira Missão Universitária (TMU) envolve a conexão das universidades com a sociedade para promover o desenvolvimento sustentável e gerar valor social. No entanto, na América Latina e na Colômbia, existem limitações metodológicas e de informação que dificultam a sua medição sistemática. Objetivo: Este estudo tem como objetivo propor um sistema de indicadores para medir a TMU nas Instituições de Ensino Superior (IES) colombianas. Metodologia: Foi utilizada uma abordagem mista e quantitativa, que incluiu uma revisão bibliográfica, seleção e categorização de indicadores e validação por meio de juízo de especialistas e análises estatísticas, como o Coeficiente Alfa de Cronbach e a Análise Fatorial Confirmatória. Resultado: O resultado é uma estrutura válida de 31 indicadores que demonstram consistência interna e se ajustam tanto teoricamente quanto empiricamente ao contexto colombiano. Discussão: Este estudo vai além de pesquisas anteriores, fornecendo um instrumento quantitativo replicável e adaptável que facilita a medição do vínculo universidade-sociedade. Conclusão: O sistema proposto fornece uma base metodológica útil para avaliar a incorporação institucional da TMU, identificar áreas de melhoria nas IES e contribuir para a concepção de políticas e estratégias de ligação universitária baseadas em resultados concretos.
Palavras-chave: análise fatorial; avaliação institucional; gestão universitária; indicadores de desempenho; juízo de especialistas.
JEL: D83; I23; L31; O32; O35; Q01
Introduction
The Third Mission of Universities (TMU) has become one of the essential components of the architecture of Higher Education Institutions (HEIs), complementing the traditional functions of teaching and research (Etzkowitz & Leydesdorff, 2000). In contemporary universities, the TMU emphasizes the need for active engagement with the environment, so that academic production addresses on specific social, economic, environmental, and cultural challenges. This involves more than the simple transfer of knowledge; it also encompasses processes of innovation, entrepreneurship, and the formation of intersectoral alliances. Based on these practices, HEIs can become strategic actors in territorial transformation and the affirmation of sustainable development (González & Vázquez, 2020).
The pioneering works of Clark (1998), Gibbons et al. (1994), Sheen (1992), and Slaugther and Leslie (1997), cited by Bueno and Fernández (2007), introduced the first ideas of what was called TMU, understood as the active engagement of the university with its social, economic, and territorial environment and as substantive function that complements teaching and research. These ideas were soon adopted by the European Commission in 1995 and 2000; since then, TMU has been considered a key element in redefining the role of higher education in the 21st century (Benneworth, 2013), as it highlights the need for active collaboration among universities, industry, the state, and society to foster innovation, entrepreneurship, knowledge transfer, and regional development (Benavides et al., 2022).
TMU encompasses a set of dimensions and activities that connect HEIs with their business, cultural, social, and economic environments (Alonso et al., 2022). Examples include university-business-government linkages, the triple helix model (Benavides et al., 2022), the transfer of technology and knowledge to the productive sector (Touriñan, 2020), the promotion of entrepreneurial universities and university social responsibility (González & Vázquez, 2020), as well as contributions to sustainable development (Mulder et al., 2012). However, due to its complexity, there is no consensual, precise, and explicit conceptualization of the dimensions and indicators for measuring the TMU, which makes its evaluation and appropriation difficult, particularly in Latin America (Dassoler et al., 2023). Several authors have highlighted this lack of conceptual consensus, arguing that TMU remains a phenomenon that requires further elaboration and discussion. In this sense, TMU can be enriched with specific regional perspectives.
In the case of Colombia, actors involved in TMU can play a strategic role in strengthening the social fabric, revitalizing the knowledge economy, and contributing to the achievement of the Sustainable Development Goals (SDGs). The development of the TMU is also a tool for acting in accordance with the guidelines of the National Accreditation Council (Consejo Nacional de Acreditación [CAN], 2021) of the Republic of Colombia, by incorporating coordination with the environment as a criterion for institutional evaluation and the demands of the Ministry of Science, Technology, and Innovation (Minciencias, 2024), which promotes the measurement of science, technology, and innovation products with social impact. In this way, the TMU contributes to reaffirming the social nature of higher education in the construction of more just and equitable societies.
In this context, this paper proposes a set of indicators to measure the TMU of Colombian HEIs. These indicators are structured around five dimensions identified in previous studies: entrepreneurship, innovation, knowledge transfer, social commitment, and sustainable development (Gaffaro & Naranjo, 2025). The proposal seeks to adapt to the particularities of the national context, provide clear evaluation criteria, and offer tools that strengthen the active role of universities in territorial development and sustainability processes.
Conceptual framework of TMU
TMU encompasses “all activities aimed at engaging with the environment, through which HEIs transfer knowledge, drive innovation, promote entrepreneurship, and strengthen sustainable development in their regional and national contexts” (Compagnucci & Spigarelli, 2020).
In this regard, different perspectives on TMU and its evolution can be identified. Molas and Castro (2007) describe it as “activities related to the generation, use, application, and exploitation of knowledge developed by the institution outside academic environments”. For their part, Ramos et al. (2010) emphasize the complex interactions between universities and companies in their geographical environments and the need to generate common objectives, while Carrión et al. (2012) point out that the “third mission” includes activities that involve the interaction of the university with society and industry beyond teaching and research.
Along the same lines, Piva and Rossi (2013) argue that collaboration between universities and industry is essential, as it is through this interaction that benefits for both industry and society are generated. Kalemis (2014) equates the “third mission” with relational capital, emphasizing that interactions between universities and stakeholders, including non-academic partners such as companies, non-profit organizations, and government entities, are fundamental.
A different perspective can be found in the work of Secundo and Elia (2014), for whom academic entrepreneurship or technological entrepreneurship and its measurement constitutes a structural component of the “third mission”. For his part, Vargiu (2014) refers to the direct and indirect contributions of universities to society as the core of TMU highlighting the need to generate indicators to assess this level of contribution.
In this regard, and as a result of the conceptual review of the literature, Table 1 presents the theoretical basis for the development of the elements that comprise TMU, offering a summary of the dimensions, their conceptualization, and the authors who support them.
Table 1
Dimensions for approaching TMU and its main authors
Dimension |
Definition |
Authors referenced |
Knowledge transfer |
Process through which universities mobilize knowledge, technologies, and expertise toward the productive, social or governmental environment through agreements, licenses, patents, or consulting services. |
Etzkowitz and Leydesdorff (2000), Touriñán (2020), Valbuena and Sánchez (2024)
|
Innovation |
Institutional capacity to generate new solutions, processes, or products with social, economic, on environmental impact, in coordination with the external sector. |
Moulaert et al. (2005), Murphy and Dyrenfurth (2012) |
Entrepreneurship |
University activity that promotes the creation of companies, spin-offs, or social initiatives led by students, teachers, or graduates, from a perspective of transforming the environment. |
Forliano et al. (2021), Herrera et al. (2010) |
Social commitment |
University participation in community and cultural development through outreach programs, volunteering, non-formal education, and engagement with vulnerable populations. |
Benneworth (2013), Brauner et al. (2020) |
Sustainable development |
Integration of sustainability principles into university actions, seeking to balance economic, social, and environmental impacts in the long term. |
Beynaghi et al. (2016), Leal et al. (2021), Mulder et al. (2012)
|
Source: Own elaboration.
Review of literature on previous measurement instruments.
Globally, the most notable advances in measuring the TMU have been led by the European Indicators and Ranking Methodology for University Third Mission (E3M) project, an initiative funded by the European Commission under the Lifelong Learning Program, aimed at developing indicators and a ranking methodology to measure the TMU in Europe (Carrión & Carot, 2012). This proposal was built using Delphi methodologies and applied through case at universities across various countries.
In Latin America, González and Vázquez (2020) made a specific proposal to measure TMU in public state universities in Mexico (UPES) adapting international frameworks such as the Science and Technology Policy Research (SPRU) project outlined by Molas et al. (2002) on how to evaluate the TMU of universities and E3M model, to local conditions, managing to assess 31 Mexican universities using the data envelopment analysis technique.
In the Colombian context, as González et al. (2014) point out, although teaching has historically guided research objectives in HEIs, critical reflection on the role and impact of TMU has not yet been sufficiently developed or debated. In 2020, an impact assessment of TMU was conducted using case studies and lessons learned at the University of Antioquia with the aim of analyzing how TMU is implemented within the institutional context (Cárdenas et al., 2023). Through case studies, interviews, and systematizations, this study presented experiences related to the measurement of these aspects.
Although many of the contributions made through various avenues have led to a better understanding of TMU, there is a clear need to move towards more structured quantitative evaluation systems that are comparable with each other and allow for longitudinal studies to be conducted. In this sense, this paper contributes by providing initial guidelines for quantifying the appropriation of the TMU by HEIs through the collection and processing of information contained in institutional databases.
Table 2 below presents a summary of studies on TMU measurement organized chronologically to observe the evolution of both methodological approaches and conceptual dimensions addressed.
Table 2
Methodological and conceptual evolution of studies on the TMU
Author(s) |
Country |
Type of study |
Methodology |
Key findings |
Molas et al. (2002) |
United Kingdom |
Technical report |
Conceptual model and activity analysis. |
Proposes evaluating activities rather than impacts; distinguishes capabilities vs. actions. |
Carrión and Carot (2012) |
Europe |
Multinational applied research |
Process mapping, Delphi method, case studies. |
Proposes three dimensions (continuing education, technology transfer and innovation, and social commitment) with 51 indicators. |
González et al. (2014) |
Colombia |
Qualitative study |
Discourse analysis and institutional review. |
Discourse on TMU has developed, but without consolidation of measurable indicators or validated psychometric scales. |
Cárdenas et al. (2023) |
Colombia |
Applied institutional study |
Case studies, interviews, systematization |
Experiences and lessons learned are identified, but not the implementation of comparable indicators or quantitative models. |
González and Vázquez (2020) |
México |
Applied quantitative study |
Enveloping Data Analysis |
Entrepreneurship, innovation, and social commitment are quantified, with a recommendation to expand the study. |
Dassoler et al. (2023) |
Brazil |
Systematic review |
Bibliometric analysis |
There is a high conceptual dispersion and Little empirical validation; challenges in measuring social impacts are identified. |
Source: Own elaboration.
Integration of university social functions and development of TMU indicators
In the literature on TMU, University Social Responsibility (Responsabilidad social universitaria, RSU), outreach, and community engagement constitute dimensions that are poorly articulated within regional measurement systems. Luque et al. (2024) point out that, although the three categories refer to TMU, it is essential to specify their common and divergent features to avoid ambiguity regarding institutional purposes. From this perspective, RSU emphasizes the ethical and social commitment of the university, outreach focuses on the transfer of knowledge and the provision of educational and cultural services, and community engagement seeks two-way relationships and strategic cooperation with social and productive actors. For their part, Benavides et al. (2022) highlight that in Colombia, the RSU and outreach functions historically been developed through continuing education programs, consulting, and student internships do not translate into consolidated indicators that allow for a systematic evaluation of the relationship between knowledge generation and social impact. When addressing the university-business-government relationship as well as models such as the triple helix, the authors cited show that, despite references to innovation and technology transfer, the social dimension and territorial development have received little attention in the literature. This demonstrates the need to move toward metrics that combine research output indicators with those of social engagement and social impact, given the value that these measures generate for society and their capacity to strengthen innovation ecosystems where local and regional research capabilities operate.
Methodology
This research adopts a quantitative, non-experimental, descriptive-explanatory approach, with a cross-sectional design based on the analysis of secondary sources (Hernández et al, 2014). This approach was complemented by a qualitative resource, the individual aggregates technique, used to collect and synthesize expert judgments (Escobar and Cuervo, 2008) during the content validation of the indicators. This approach is appropriate for analyzing complex phenomena in institutional contexts without experimental intervention, such as this research, whose purpose was to design and validate a set of indicators to measure TMU based on available and empirically observable data.
Data
A database was created to identify and statistically quantify the products used to measure TMU, integrating information from three officially validated secondary public sources in Colombia: (1) science, technology, and innovation (STl) products registered on the Minciencias platform in 2021, the most recent available data at the time of this research; (2) the updated list of all research groups currently recognized by Minciencias, corresponding to the year 2021; and (3) the list of active HEIs according to the National Higher Education Information System (SNIES) platform until the end of 2025. This data was used to perform a deterministic cross-reference using the unique code assigned to each research group and the official name of the institution to which it is linked as matching keys.
The sample used in the statistical analyses was selected intentionally, based on convenience criteria. Only products from institutions officially recognized as Colombian HEIs, active in the SNIES system, and that reported the results of their STI functions to Minciencias during the official 2021 period, were included. Companies, institutions not officially recognized as HEIs, and organizations that, despite having products registered with Minciencias, are not part of the university system were excluded.
Design phases, content validation, and indicator construct
The process of developing the instrument to validate the indicators needed to measure TMU was organized into macro-activities structured according to the phases of the PDCA cycle (Plan–Do–Check–Act), ensuring an iterative, rigorous, and continuous improvement-oriented approach (Chen & Li, 2019; Hernández & Villamil, 2021). This structure facilitated the sequential integration of conceptual design, expert judgment, statistical validation, and final feedback.
Figure 1 summarizes the methodological stages implemented and their correspondence with the quantitative methods used in each phase.
Figure 1
Methodological steps for developing and validating TMU indicators
Source: Own elaboration.
For the initial design of the instrument, consisting of a conceptual model and an assessment form, the Individual Aggregates Technique was selected, an evaluation methodology that focuses on gathering expert opinion in a single round. This technique was preferred over the Delphi model, which seeks to find consensus through consecutive group discussions, as it allows the perspectives of experts to be collected individually and anonymously. This reduces the existence of biases and facilitates the development of a collective ideal without external interventions. In addition, the technique is valued for its efficiency and applicability in contexts where rapid conceptual validation is required (Balderas et al., 2022; South et al., 2022).
A panel of five experts was formed, which is within the recommended range for the expert judgment technique, that estimates between four and twenty participants (Bojke et al., 2021). The selection criteria for the experts required at least five years of experience and publications in TMU, a track record in university management, research, entrepreneurship, and/or outreach, as well as the the knowledge and critical judgment necessary to evaluate the relevance of the proposed dimensions. Each expert was provided with the indicator system through a specific instrument and asked to evaluate the relevance, clarity, internal sufficiency, and applicability on a scale of 1 to 5; at the same time, they had the opportunity to make open comments.
Once the responses were collected, they were subjected to a quantitative analysis aimed at identifying consensus patterns and suggestions for optimizing item formulation. This allowed the estimation of the degree to which the items adequately represented the theoretical construct and provided guidelines on which items should be revised in the future.
Content validity was assessed by calculating Aiken's V coefficient, which statistically estimates the degree of consensus among expert judges on the relevance and adequacy of each item in the instrument (Merino & Livia, 2009). The procedure was based on an ordinal assessment, whose scores were synthesized into an interpretable coefficient, thereby facilitating decisions about the inclusion, modification, or elimination of items.
In addition to the content validity analysis, an internal consistency assessment was conducted using Cronbach's alpha, a key criterion for ensuring that the selected indicators function cohesively and for estimating the reliability of composite scales to be estimated based on the correlation between items (George & Mallery, 2003). This technique is appropriate in expert-judgment validation processes, especially when the aim is to verify that items grouped within the same dimension measure a common construct consistently (Oviedo & Campo, 2005).
Statistical Validation of the Construct
To verify the structural validity of the indicators, two complementary factorial techniques were applied to the consolidated data, which according to Ruiz (2022), “constitute a highly useful heuristic resource in the study of validity when developing measurement instruments” (p. 307). First, a Principal Component Analysis (PCA) was conducted for exploratory purposes, using SPSS and RStudio to identify correlation patterns between items and evaluate the existence of a general factor (Hair et al., 2009). Subsequently, a Confirmatory Factor Analysis (CFA) was conducted, evaluating each dimension separately as a first-order model (Pérez et al., 2000).
In the PCA, following the recommendations of Sanchez (2019) and Hair et al. (2009), factors that explained at least 60% of the total variance and items that shared sufficient common variance to justify the formation of latent constructs were considered adequate. Then, to visualize the internal structure of the indicators, PCA was performed again using Spearman's correlation matrix, given that the variables are ordinal and do not meet the assumption of full normality (Watanabe et al., 2024). The correlation analysis, conducted in RStudio, produced the correlation circle, in which each variable is represented by a vector whose direction and magnitude indicate its relative contribution to the principal components and its correlation with the other variables respectively. This representation allowed us to identify possible initial groupings and relationships between items, which was a fundamental exploratory stage for refining and adjusting the measurement model. Additionally, the CFA by dimensions was performed in RStudio following to the methodological instructions of Xia and Yang (2019).
Robustness tests were performed, including the χ²/df ratio to measure the ratio between model fit and complexity. In addition, the Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), and Root Mean Square Error of Approximation (RMSEA) were calculated. The values CFI = 0.998, TLI = 0.997, and RMSEA = 0.058 were used to measure the degree of model fit:
Finally, to complement the study, a correlation analysis among the dimensions was conducted to examine their internal consistency, as recommended by Chen et al. (2024) for instrument validation.
Results
The indicators for evaluating TMU in HEIs were constructed from an initial bank of products and variables previously identified through a systematic review of the literature conducted by Gaffaro and Naranjo (2025), supplemented with official secondary information derived from Minciencias and SNIES, which enables the characterization of the products documented by Colombian higher education institutions.
Tables 3, 4, 5, 6, and 7 below summarize the dimensions, their analytical categories, the associated conceptualization, and the corresponding item code for the indicators:
Table 3
Items related to social commitment dimension
Item code |
Category |
Conceptualization |
V1 |
Number of outreach projects |
Activities or programs conducted by the group in context to engage with the community, institutions, or social sectors, with the aim of transferring knowledge and generating concrete social impact. |
V2 |
Number of non-specialized editorial publications |
Publications aimed at audiences without specialized training, such as magazines, newsletters, columns, and others, dedicated to disseminating research results or addressing topics of social interest. |
V3 |
Number of digital content productions |
Creation and dissemination of digital products such as podcasts, videos, blogs, and similar resources for the purpose of disseminating the application of scientific knowledge. |
V4 |
Number of transmedia strategy and content productions. |
Creation of interactive narratives that cross various mass media, audiovisual, web, social networks, etc. In other words, it uses specific and different languages; its task is to address science and the development of critical thinking. |
V5 |
Number of consultations for the Ondas program |
Technical and training support processes in contexts involving children and adolescents served by the Ondas program of Minciencias. |
V6 |
Number of creative workshops. |
Training or practical events organized by the group with a social or community focus, aimed at developing skills, creativity, or knowledge acquisition. |
Source: Own elaboration.
The items included in this dimension reflects practices associated with the social function of the university in its relationship with the environment. Their incorporation into the instrument enables the structured capture of outreach, dissemination, and knowledge appropriation activities that form part of the social commitment of university.
The sustainable development dimension encompasses institutional efforts aimed at integrating knowledge with sustainable economic, social, and environmental transformation processes. The items selected in this category correspond to processes of social appropriation of knowledge developed by research groups recognized by Minciencias, and are presented below.
Table 4
Sustainable development dimension items
Item code |
Category |
Conceptualization |
V7 |
Number of processes for the social appropriation of knowledge to strengthen production changes. |
Systematic actions through which technical or scientific knowledge is transferred to strengthen sustainable production processes in specific communities or sectors. |
V8 |
Number of processes of social appropriation of knowledge for strengthening or resolving issues. |
Initiatives aimed at solving social problems or improving community conditions through active participation in collaborative knowledge-generation processes. |
V9 |
Number of social appropriations of knowledge processes for the generation of inputs. |
Activities aimed at producing tolos, materials, methodologies, or applicable knowledge that support local or regional sustainability projects. |
Source: Own elaboration.
The Entrepreneurship dimension covers initiatives that promote the creation of economic, social, or cultural value by HEIs. The selected items reflect institutional efforts to Foster the creation of companies and the protection of intangible assets as part of the innovation and technology transfer ecosystem (Table 5).
Table 5
Entrepreneurship dimension items
Item code |
Category |
Conceptualization |
V10 |
Number of technology-based companies (Startup and Spin-Offs) |
Creation of business initiatives promoted by the university, based on research results, innovation, or technological knowledge. |
V11 |
Number of creative and cultural enterprises |
Companies founded in university settings that develop products or services based on the orange economy: art, culture, design, digital media, among others. |
V12 |
Number of registered trade secrets |
Formal protection of strategic knowledge generated by research groups or academic units with potential for business application. |
Source: Own elaboration.
The innovation dimension within the TMU framework considers institutional capacities to generate and apply technological, organizational, or product solutions that have an impact on the environment. The items considered are detailed below.
Table 6
Innovation dimension items
Item code |
Category |
Conceptualization |
V13 |
Number of web developments |
Application, platforms, or websites developed by the research group for functional, academic, social, or commercial purposes. |
V14 |
Number of business management innovations |
Significant improvements in administrative or organizational processes applied to real-world contexts. |
V15 |
Number of innovations in procedures and services |
Development of new methodologies, forms of care, or operational mechanisms that improve existing services. |
V16 |
Number of researches, development, and innovation projects. |
Projects that integrate research, development, and innovation, aimed at generating innovative products, processes, or services. |
V17 |
Number of industrial designs |
Figures that are protected by intellectual property rights registered with the Superintendence of Industry and Commerce. |
V18 |
Number of software developments |
Computer applications created for searching, learning, or production. |
V19 |
Number of industrial prototypes |
Preliminary and functional versions of devices, tools, or technological solutions validated in a test environment. |
V20 |
Number of invention patents |
Official records of original inventions with industrial application and technical verification. |
V21 |
Number of utility model patents |
Protections granted to technical or functional improvements to existing inventions. |
Source: Own elaboration.
The knowledge transfer dimension is understood as the capacity of HEIs to share and apply the knowledge resulting from their research in society, through scientific and technological activities, specialized consulting, and dissemination strategies within the framework of university outreach. This dimension plays a key role in connecting research results to productive sectors, social organizations, and government entities, promoting effective knowledge circulation that transcends the university sphere. Table 7 presents the items that make up this dimension within the instrument.
Table 7
Knowledge transfer dimension items
Item code |
Category |
Conceptualization |
V22 |
Number of research and creation projects |
Projects that combine scientific inquiry and creative development processes with investigative and artistic purposes. |
V23 |
Number of research and development projects. |
Formal initiatives aimed at developing technological solutions or knowledge applicable to environmental needs. |
V24 |
Number of specialized knowledge networks |
Interinstitutional collaborative environments for the exchange of technical or scientific knowledge on specific topics. |
V25 |
Number of scientific events |
Academic conferences, congresses, and seminars organized by research groups that serve as spaces for dissemination and the creation of collaborative networks. |
V26 |
Number of working documents generated |
This refers to reports, progress reports, or preliminary texts that are produced as part of the research work and are open to external feedback. |
V27 |
Number of final research reports |
Documents that systematize and present the consolidated results of research projects carried out, in order to offer a structured and clear overview of the findings. |
V28 |
Number of final technical reports |
These are technical reports generated from applied research, containing significant findings and recommendations for decision-making aimed at a specific sector. |
V29 |
Number of scientific and technical consultations |
These are conceptual or technical contributions made by the researcher to projects involving aesthetic creation, functional design, or cultural heritage preservation. |
V30 |
Number of consultations in arts, architecture, and design. |
These are conceptual or technical contributions made by the researcher to projects involving aesthetic creation, functional design, or cultural heritage preservation. |
V31 |
Number of technical concepts |
Formal opinions or analyses on problematic areas, technology, or policies, carried out by an expert based on research evidence. |
Source: Own elaboration.
Results of validation by expert judgment, internal consistency, and statistic of the indicators
As part of the validation process for the proposed system of indicators to measure TMU in Colombian HEIs, two complementary statistical procedures were applied. First, Aiken’s V coefficient was used to determine the content validity of each indicator, based on the assessment of a panel of experts, as shown in Table 8. Next, Cronbach’s alpha was calculated to establish the internal consistency of the set, based on the assessments of five experts on 31 indicators. This dual methodology combined qualitative content review with statistical measures of reliability, strengthening the methodological validity of the system.
Table 8
Results of the Aiken V coefficient based on expert judgment
Item |
Relevance |
Level |
Clarity |
Level |
Sufficiency |
Level |
Applicability |
Level |
V1 |
0.95 |
Very high |
0.85 |
High |
0,8 |
High |
0.85 |
High |
V2 |
0.7 |
Acceptable |
0.9 |
Very high |
0,85 |
High |
0.65 |
Low |
V3 |
0.8 |
High |
0.75 |
Acceptable |
0,85 |
High |
0.8 |
High |
V4 |
0.75 |
Acceptable |
0.8 |
High |
0,6 |
Low |
0.8 |
High |
V5 |
0.8 |
High |
0.85 |
High |
0,75 |
Acceptable |
0.8 |
High |
V6 |
0.7 |
Acceptable |
0.7 |
Acceptable |
0,75 |
Acceptable |
0.85 |
High |
V7 |
0.85 |
High |
0.75 |
Acceptable |
0.7 |
Acceptable |
0.85 |
High |
V8 |
0.7 |
Acceptable |
0.65 |
Low |
0.75 |
Acceptable |
0.8 |
High |
V9 |
0.9 |
Very High |
0.8 |
High |
0.8 |
High |
0.55 |
Low |
V10 |
0.8 |
High |
0.7 |
Acceptable |
0.8 |
High |
0.65 |
Low |
V11 |
0.85 |
High |
0.9 |
Very high |
0.6 |
Low |
0.65 |
Low |
V12 |
0.95 |
Very high |
0.8 |
High |
0.8 |
High |
0.65 |
Low |
V13 |
0.7 |
Acceptable |
0.65 |
Low |
0.7 |
Acceptable |
0.85 |
High |
V14 |
0.75 |
Acceptable |
0.75 |
Acceptable |
0.8 |
High |
0.55 |
Low |
V15 |
0.9 |
Very high |
0.65 |
Low |
0.95 |
Very high |
0.95 |
Very high |
V16 |
0.75 |
Acceptable |
0.8 |
High |
0.7 |
Acceptable |
0.65 |
Low |
V17 |
0.8 |
High |
0.7 |
Acceptable |
0.65 |
Low |
0.7 |
Acceptable |
V18 |
0.85 |
High |
0.75 |
Acceptable |
0.7 |
Acceptable |
0.7 |
Acceptable |
V19 |
0.75 |
Acceptable |
0.8 |
High |
0.6 |
Low |
0.7 |
Acceptable |
V20 |
0.75 |
Acceptable |
0.9 |
Very high |
0.8 |
High |
0.7 |
Acceptable |
V21 |
0.75 |
Acceptable |
0.7 |
Acceptable |
0.6 |
Low |
0.75 |
Acceptable |
V22 |
0.65 |
Low |
0.9 |
Very high |
0.65 |
Low |
0.8 |
High |
V23 |
0.7 |
Acceptable |
0.65 |
Low |
0.7 |
Acceptable |
0.65 |
Low |
V24 |
0.75 |
Acceptable |
0.55 |
Low |
0.85 |
High |
0.8 |
High |
V25 |
0.8 |
High |
0.5 |
Low |
0.75 |
Acceptable |
0.7 |
Acceptable |
V26 |
0.7 |
Acceptable |
0.65 |
Low |
0.7 |
Acceptable |
0.65 |
Low |
V27 |
0.65 |
Low |
0.8 |
High |
0.8 |
High |
0.6 |
Low |
V28 |
0.7 |
Acceptable |
0.55 |
Low |
0.8 |
High |
0.7 |
Acceptable |
V29 |
0.9 |
Very high |
0.85 |
High |
0.75 |
Acceptable |
0.85 |
High |
V30 |
0.8 |
High |
0.85 |
High |
0.65 |
Low |
0.6 |
Low |
V31 |
0.65 |
Low |
0.65 |
Low |
0.85 |
High |
0.85 |
High |
Source: Own elaboration.
In terms of relevance, the content validation analysis showed that most items achieved high scores, greater than or equal to 0.70; seven of them obtained very high scores. In terms of clarity, the assessment was mostly positive, but there were some items that were difficult to interpret due to the use of technical terminology or differences in the evaluator’s conceptual frameworks. In terms of adequacy, most items obtained acceptable values, with the exception of working documents and industrial designs. Applicability is the dimension with the greatest dispersion of results, with then items at a low level; the observations made suggest that this is due to a variety of elements that may vary in each university ecosystem.
Internal consistency (Cronbach’s alpha)
When analyzing the internal consistency of the ratings issued by the experts in relation to the dimension, the reliability obtained was α = 0.724, which represents an acceptable level (Oviedo & Campo, 2005) and allows us to affirm that the ratings were consistent among the items that comprise it. This combination of methods confirms that the instrument has solid evidence of both content validity and internal consistency in the expert judgment validation phase. Consequently, it is considered methodologically appropriate to move on to the next phases of the research process, including expanded empirical application.
Principal Component Analysis (PCA)
The results obtained allow us to affirm that the instrument has a solid factorial structure in terms of overall internal correlation. The KMO simple adequacy index = 0.979, shown in table 9, and the statistical significance (p < 0.001) of Bartlett’s sphericity test confirm that the data meet the criteria for applying factor analysis, in accordance with the standards established by Hair et al. (2009).
Table 9
KMO and Bartlett Test Results
Kaiser-Meyer-Olkin measure of sampling adequacy |
0.979 |
|
Barlett sphericity test |
Approximate chi-square |
24662.456 |
df |
465 |
|
Sig. |
<.001 |
|
Source: Own elaboration.
The analysis of total variance explained performed in SPSS (Table 10) showed that the model automatically extracted a single principal component, responsible for 96.18% of the cumulative variance. This high concentration indicates that the items of the instrument share a highly homogeneous structure and solid internal consistency, suggesting that the set measures the overall TMU construct in an integrated manner without requiring strict segmentation into separate dimensions. However, although five dimensions were proposed from a theoretical perspective (knowledge transfer, innovation, entrepreneurship, social commitment, and sustainable development), empirical behavior suggests that these share sufficient elements to form a cohesive conceptual framework. To verify this finding, a Principal Component Analysis (PCA) based on Spearman coefficients was applied, a technique that transforms data into ranges and evaluates monotonic relationships, which is more appropriate when variables do not follow normal distributions. This analysis was performed in the RStudio statistical environment, using specific functions for calculating Spearman correlations and extracting principal components. The results are illustrated in Figure 2.
Table 10
Total variance explained for the first six components
Component |
Initial self-values |
||
Total |
% variance |
% cumulative |
|
1 |
29.817 |
96.185 |
96.185% |
2 |
.281 |
.908 |
97.092% |
3 |
.125 |
.403 |
97.495% |
4 |
.116 |
.373 |
97.868% |
5 |
.103 |
.333 |
98.201% |
6 |
.069 |
.223 |
98.424% |
Source: Own elaboration.
Figure 2
PCA correlation circle with Spearman executed in RStudio on the 31 initial indicators
Source: Own elaboration with RStudio.
Figure 2 shows the PCA correlation circle with Spearman coefficients applied to the 31 indicators. The proximity and alignment of most vectors indicates that the variables are strongly correlated, reflecting common patterns and internal consistency, while some items (V12, V15, V18, V29) provide additional variability useful for differentiating dimensions.
Confirmatory Factor Analysis (CFA)
Below is a summary table of the main fit indices from the confirmatory factor analyses (CFA) performed independently for each of the five dimensions of the TMU measurement instrument. The analysis considered a unidimensional model, seeking to confirm the factor structure of the proposed subscales. The values shown in Table 11 indicate the extent to which the proposed theoretical model fits the collected data empirically and serve as a reference point for analyzing the strengths and possible weaknesses of the dimensions in terms of internal consistency and latent structure.
Table 11
Results of Confirmatory Factor Analysis
Dimension / Model |
Items included |
χ² / df |
CFI |
TLI |
RMSEA (90% CI) |
SRMR |
Conclusion |
Social Commitment |
V1–V6 |
14.869 / 9 |
0.998 |
0.997 |
0.058 (0.000–0.108) |
0.035 |
Very good fit; validity confirmed. |
Sustainable development |
V7–V9 |
0 / 0 |
1.000 |
1.000 |
0.000 |
0.000 |
Saturated model; perfect fit. |
Entrepreneurship |
V10–V12 |
0 / 0 |
1.000 |
1.000 |
0.000 |
0.000 |
Saturated model; good fit, though with moderate loadings. |
Innovation |
V13–V21 |
53.425 / 27 |
0.995 |
0.993 |
0.071 (0.042–0.099) |
0.066 |
Acceptable fit; requires refinement of items with low loadings. |
Knowledge Transfer |
V22–V31 |
101.772 / 35 |
0.998 |
0.998 |
0.099 (0.077–0.122) |
0.050 |
Acceptable fit; validity confirmed. |
Overall Model (Five correlated factors) |
V1–V31 |
1015.830 / 424 |
0.994 |
0.994 |
0.085 (0.078 –0.091) |
0.070 |
Acceptable global fit; the five factors function coherently together. |
Source: Own elaboration.
Social commitment: Factor loadings were high and uniform with no need for additional adjustments. The overall indices (CFI = 0.998; RMSEA = 0.061) show a good fit of the theoretical model to the empirical data, supporting its validity as a subscale. However, it is advisable to examine its stability in different institutional settings and with larger samples to strengthen its reliability. Sustainable development and entrepreneurship: The two dimensions were fitted to saturated models because they consisted of only three items. As is usual in this type of structure, a perfect fit was achieved (CFI = 1.000), which was to be expected given the small number of parameters to be estimated. While these results reaffirm their internal consistency, they should be treated with caution because saturated models are overfitted. Future lines of research could include new items that reinforce structural validity and expand their measurement capacity. Innovation: The values of variables V13-V21 show adequate statistical fit, although there is room for improvement. Knowledge transfer: The values of variables V22-V31 show an acceptable fit. Although the absolute χ2 fit is not satisfactory, the fit in comparative indices is excellent and supports the existence of a congruent structure. Finally, the overall model of the five dimensions (V1-V31) showed an acceptable overall fit. These findings confirm that the five dimensions operate together as a single measurement framework for TMU.
Lastly, a correlation analysis was performed to observe the pattern of differentiated associations between the five dimensions of the indicators, shown in Table 12.
Table 12
Correlations between TMU dimensions
Correlation Between Dimensions |
Sustainable Development |
Social Commitment |
Knowledge Transfer |
Entrepreneurship |
Innovation |
Sustainable development |
1 |
0.58 |
0.53 |
0.04 |
0.46 |
Social commitment |
0.58 |
1 |
0.88 |
0.01 |
0.50 |
Knowledge transfer |
0.53 |
0.88 |
1 |
-0.01 |
0.50 |
Entrepreneurship |
0.04 |
0.01 |
-0.01 |
1 |
0.48 |
Innovation |
0.46 |
0.50 |
0.50 |
0.48 |
1 |
Source: Own elaboration.
Table 12 shows the correlation matrix between the five theoretical dimensions of MT, calculated using Spearman coefficients. In general, moderate to high positive correlations are evident, indicating that the dimensions tend to vary in the same direction and share a common conceptual basis.
Discussion
The results of the validation carried out in this article point to the theoretical and empirical soundness of the systems of indicators proposed for measuring TMU in Colombian HEIs, given that the statistics analyzed yielded high factor loadings and the adjustment indices were adequate, confirming that the conceptual structure of five dimensions (knowledge transfer, innovation, entrepreneurship, social commitment and sustainable development) remains stable when tested empirically, validating the internal consistency of the model and its ability to adequately represent the TMU construct
The combined use of qualitative and quantitative methods has made it possible to validate the content, consistency, and factorial structure of the indicators. This methodological approach brings greater rigor to the measurement process and helps to eliminate one of the limitations identified in the literature review, namely the lack of validated instruments for measuring TMU. In addition, the study facilitates progress from initial conceptual or exploratory approaches toward a verifiable operationalization of the phenomenon, strengthening the empirical basis of the field. Compared to the existing literature, the study has fewer limitations in relation to the descriptive approach, which has been the dominant form of work on TMU in Latin America.
From a systemic perspective, the results validate that TMU is measured comprehensively, or all its dimensions together at the same time. In addition, this research contributes not only to academic debate, but also to international university policies, by providing a methodological model that can be adopted by institutions interested in evaluating and improving their performance in terms of outreach, innovation, and sustainability. Therefore, beyond addressing the methodological gap in the region, it also helps to establish a path for the development of more coherent evaluation frameworks, reducing bias and increasing comparative capacity and evidence-based decision-making.
Conclusions
The study reinforces that TMU can be evaluated. The conceptual and methodological development presented, as well as the subsequent validation of a system of comparable, structured, and systematic quantitative indicators to measure the key dimensions of this function in Colombian HEIs, demonstrate this. This set of indicators, anchored in variables such as knowledge transfer, innovation, entrepreneurship, social commitment, and sustainable development, provides a framework for measuring the level of appropriation of the construct by the products developed within HEIs, leading to greater visibility of the degree of maturity of institutions in society.
The results obtained demonstrate the consistency and reliability of the proposed indicator system, in that the five dimensions have a positive and complementary relationship that validates their articulation in a single construct. The combination of quantitative and qualitative techniques for validating the instruments confirms, from an empirical and verifiable approach, the descriptive casuistry that prevails in the regional literature.
In terms of contributions to the international academic community, it is important to highlight that a validated a contextualized model was provided for Colombia, but it serves as a basis for Latin America. This indicator model can not only be a reference for future comparative studies, but also for necessary adaptations within the institutions themselves. Thus, science is moving towards the standardization of university metrics, while consolidating management policies linked to the transfer and socialization of knowledge in the field of higher education.
The research did not involve or require human or animal subjects, as it was based solely on the analysis of institutional data obtained as freely accessible open data. Therefore, no threat to ethical standards was identified. Confidentially and ethical treatment of data were applied in accordance with the principles of academic integrity and transparency developed by Universidad de la Salle, in line with internationally accepted ethical standards for this type of study.
All authors made substantial contributions to this document and declare that there is no conflict of interest associated with this article.
Author contribution statement
Source of funding
This article has not received funding from any source. The research was conducted as part of the lead author’s doctoral research at Universidad de la Salle, with academic collaboration from the co-authors.
References