Skip to main navigation menu Skip to main content Skip to site footer

Conference paper

Vol. 14 (2023): XIV Congreso Internacional Ideice

Evaluation of programs designed under the competency approach. Measurement validation

DOI
https://doi.org/10.47554/cii.vol14.2023.pp389-395
Submitted
September 30, 2023
Published
2024-08-08

Abstract

An evaluative study is presented about the design of the subject programs in the study plans. Of the three phases of evaluating plans and programs, the work is located in the first. Thus, the purpose was to present a program design evaluation framework valid for the context of higher education. An instrument of 38 items was developed, with a response scale of 4 points (or 3) and five dimensions, taking the model of Pérez Juste (2000, 2017): adaptation to the context, content, technical quality, evaluability and feasibility. Expert judgment involved assessing the appropriateness, relevance, clarity of wording, sufficiency, and coherence in each item. A content validity analysis was carried out using Lawshe's (1975) model. Content validity answers the question of whether the elements that make up a test measure the construct being evaluated. With the content validity index achieved and some improvement modifications in five items, the final version of the instrument was obtained for the evaluation of the design. The usefulness of this type of evaluation has effects on the quality of the programs, because it promotes reflection, the identification of difficulties and strengths, as well as the compilation of proposals for improvement. It is a type of evaluation for the improvement of institutional quality, complementary to the evaluation of implementation and the evaluation of results.

References

Ayre, C., & Scally, A. J. (2014). Critical values for Lawshe’s content validity ratio: revisiting the original methods of calculation. Measurement and evaluation in counseling and development, 47(1), 79-86. https://doi.org/10.1177/0748175613513808

Escobar-Pérez, J., & Cuervo-Martínez, Á. (2008). Validez de contenido y juicio de expertos: una aproximación a su utilización. Avances en medición, 6(1), 27-36.

Lawshe C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563-575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x

Matas, A. (2018). Diseño del formato de escalas tipo Likert: un estado de la cuestión. Revista electrónica de investigación educativa, 20(1), 38-47. https://doi.org/10.24320/redie.2018.20.1.1347

Ministerio de Educación Superior Ciencia y Tecnología. (2019). Plantilla para la evaluación de los programas de asignaturas de educación.

Ministerio de Educación Superior Ciencia y Tecnología. (2020). Guías, Protocolos y Manual operativo para el diseño y evaluación de planes de estudio.

Pérez Juste, R. (2000). La evaluación de programas educativos: conceptos básicos, planteamientos generales y problemática. Revista de investigación educativa, 18(2), 261-287.

Pérez Juste, R. (2017). Evaluación de programas educativos (3.a Ed). La Muralla.

Scriven, M. (2003). Evaluation in the new millennium: The transdisciplinary vision. In S. I. Donaldson y M. Scriven (Eds.). Evaluating social programs and problems: Visions for the new millennium (pp. 19-42). Mahwah, NJ:Lawrence Erlaum Associates, Inc.