Vol. 1 No. 30 (2025)
Professional Pedagogics

Professional Pedagogics

MODEL OF EXPERT ASSESSMENT OF PEDAGOGICAL RESEARCH RESULTS

Published 05.06.2025

  • Vitalina Puhach
    • ,
  • Oleksandr Kobylianskyi
    • ,
  • Sofia Dembitska
    • ,
  • Iryna Kobylyanska

Vitalina Puhach
Вінницький навчально-науковий інститут економіки, Західноукраїнського національного університету , Vinnytsia Educational and Scientific Institute of Economics, Western Ukrainian National University

Oleksandr Kobylianskyi
Vinnytsia National Technical University image/svg+xml

Sofia Dembitska
Vinnytsia National Technical University image/svg+xml

Iryna Kobylyanska
Vinnytsia National Technical University image/svg+xml

Abstract

Relevance: in the conditions of rapid development of scientific and technological progress, there is an urgent need to use expert evaluation in pedagogical research, which is an effective tool for establishing the possible impact of developed innovations on higher education applicants and the educational process as a whole.

Purpose: experimental verification of the developed model of the expert evaluation process in pedagogical research.

Methods: theoretical; empirical: praxemetric, survey-diagnostic, self-assessment and expert evaluation; mathematical and statistical.

Results: based on theoretical analysis, a number of factors that complicate the process of scientific research have been identified, and the need to improve the quality of pedagogical research has been substantiated. The need for the use of expert evaluation in pedagogical research is confirmed as one of the effective tools for determining the possible impact of the developed innovations on higher education applicants and the educational process as a whole. The developed model reproduces the general sequential process of conducting the research, but allows the researcher to independently choose the options for its application, depending on the unique features of his specific research. The effectiveness of the model is demonstrated on the example of determining the significance of the proposed innovations in improving the training of specialists in the specialty "Environmental Protection Technologies". 54 experts were involved in the experimental study, of which: 22 doctors and 25 candidates of sciences. Thanks to the use of a number of statistical criteria during the expert evaluation of the proposed innovations aimed at improving the training of specialists in the field of "Environmental Protection Technologies", compliance of the proposed innovations with didactic, informational and scientific and technical requirements, as well as their readiness for testing in real conditions of the educational process, which are also prospects for further scientific research, was established.

Conclusions: currently, expert assessment has unequivocally emerged as a vital component of contemporary pedagogical research, significantly contributing to the quality and validity of pedagogical practices and decision-making. Our study included a quantitative assessment of expert competence, yielding an average competence coefficient (K) of 0.79. This value surpasses the normative threshold of 0.67, indicating a high level of qualification among the experts involved in evaluating the innovations. All participating experts provided positive assessments of the updated educational program for "Environmental Protection Technologies." Moreover, with a calculated coefficient of variation of 8.45% (which is < 10%), this assessment can be considered highly reliable, signifying a strong consensus among the experts' opinions.

Keywords

pedagogical research, innovations, expert assessment, model, educational environment

pdf

Author Biography

Vitalina Puhach

Candidate of Pedagogical Sciences, Associate Professor, Associate Professor of the Department of Law and Humanities, Vinnytsia Educational and Scientific Institute of Economics, Western Ukrainian National University,  https://orcid.org/0000-0002-1653-7473, e-mail: pugach.vitalina@gmail.com 

Oleksandr Kobylianskyi

DSc in Education, Professor, Head of the Department of Life Safety and Safety Pedagogy of the Faculty Information Technologies and Computer Engineering, Vinnytsia National Technical University, http://orcid.org/0000-0001-5351-0002, e-mail: akobilanskiy@gmail.com 

Sofia Dembitska

DSc in Education, Professor, Professor of the Department of Life Safety and Safety Pedagogy of the Faculty Information Technologies and Computer Engineering, Vinnytsia National Technical University, http://orcid.org/0000-0002-2005-6744, e-mail: sofiyadem13@gmail.com

Iryna Kobylyanska

Candidate of Pedagogical Sciences, Associate Professor, Associate Professor of the Department of Life Safety and Safety Pedagogy Vinnytsia National Technical University, https://orcid.org/0000-0002-3430-5879, e-mail: irishakobilanska@gmail.com 

References

  1. Грабовецький, Б. Є. (2010). Методи експертних оцінок: теорія, методологія, напрямки використання: монографія. Вінниця: ВНТУ.
  2. Корченко, О. Г., Горніцька, Д. А., & Захарчук, Т. Р. (2010). Дослідження методів апріорної оцінки якості експерта для реалізації експертиз у сфері інформаційної безпеки. Захист інформації, 4, 53–60.
  3. Леонов, В. В., & Ворович, Б. О. (2014). Методика оцінки компетентності експертів у процесі розробки пропозицій до програмних документів. Збірник наукових праць Центру воєнно-стратегічних досліджень Національного університету оборони України імені Івана Черняховського, 3, 13–18.
  4. Подолянчук, С. В. (2014). Визначення компетентності експертів з оцінювання наукової діяльності у вищому педагогічному навчальному закладі. Теорія і практика управління соціальними системами: філософія, психологія, педагогіка, соціологія, 4, 112–122.
  5. Яшкіна, О. І. (2013). Статистичні інструменти визначення узгодженості думок експертів в маркетингових дослідженнях. Економічний вісник Національного технічного університету України «Київський політехнічний інститут», 10, 442–449.
  6. Cleaver, E., Lintern, M., & McLinden, M. (2014). Teaching and Learning in Higher Education: Disciplinary Approaches to Educational Enquiry. London: Sage.
  7. Cotton, D. R. E., Miller, W., & Kneale, P. (2018). The Cinderella of Academia: Is Higher Education Pedagogic Research Undervalued in UK Research Assessment? Studies in Higher Education, 43 (9), 1625–1636.
  8. Dembitska, S., & Kobylianskyi, O. (2022). Formation of work safety culture of the technical specialists. Professional Pedagogics, 2(25), 138-146. https://doi.org/10.32835/2707-3092.2022.25.138-146.
  9. Dembitska, S., Kobylianskyi, O., & Puhach S. (2022). Improvement of professional training of technical specialists according to requirements of integration methodological approach. Нова педагогічна думка, 3 (111), 14-23.
  10. Evans, C., Howson, C. K., & Forsythe, A. (2018). Making Sense of Learning Gain in Higher Education. Higher Education Pedagogies, 3 (1), 1–45.
  11. Evans, C., Zhu, X., Winstone, N., Balloo, K., Hughes, A., & Bright, C. (2019). Maximising Student Success through the Development of Self-Regulation. Addressing Barriers to Student Success, Office for Students’ Final Report. Southampton: University of Southampton with Office for Students, UK.
  12. Ferretti, F., Pereira, Â. G., Vértesy, D., & Hardeman, S. (2018). Research Excellence Indicators; Time to Reimagine the Making Of’? Science and Public Policy, 45 (5), 731–741.
  13. Kaynardağ, A. Y. (2019). Pedagogy in HE: Does It Matter? Studies in Higher Education, 44 (1), 111–119.
  14. Kuzmenko, О., Dembitska, S., Miastkovska, M., Savchenko, I., & Demianenko, V. (2023). Onto-oriented Information Systems for Teaching Physics and Technical Disciplines by STEM-environment. International Journal of Engineering Pedagogy (iJEP), 13(2), 139–146. https://doi.org/10.3991/ijep.v13i2.36245.
  15. Wood, P., & O’Leary, M. (2019). Moving beyond Teaching Excellence: Developing a Different Narrative for Britain’s Higher Education Sector. International Journal of Comparative Education and Development, 21 (2), 112-26.
  16. Переклад і транслітерація
  17. Hrabovetskyi, B. Ye. (2010). Metody ekspertnykh otsinok: teoriia, metodolohiia, napriamky vykorystannia [Methods of expert assessments: theory, methodology, areas of use]: monohrafiia. Vinnytsia: VNTU, [in Ukrainian].
  18. Korchenko, O. H., Hornitska, D. A., & Zakharchuk, T. R. (2010). Doslidzhennia metodiv apriornoi otsinky yakosti eksperta dlia realizatsii ekspertyz u sferi informatsiinoi bezpeky [Research on methods of a priori assessment of expert quality for the implementation of expertise in the field of information security]. Zakhyst informatsii [Information Protection], 4, 53–60, [in Ukrainian].
  19. Leonov, V. V., & Vorovych, B. O. (2014). Metodyka otsinky kompetentnosti ekspertiv u protsesi rozrobky propozytsii do prohramnykh dokumentiv [Methodology for assessing the competence of experts in the process of developing proposals for program documents]. Zbirnyk naukovykh prats Tsentru voienno-stratehichnykh doslidzhen Natsionalnoho universytetu oborony Ukrainy imeni Ivana Cherniakhovskoho [Collection of scientific works of the Center for Military and Strategic Studies of the Ivan Chernyakhovsky National Defense University of Ukraine], 3, 13–18, [in Ukrainian].
  20. Podolianchuk, S. V. (2014). Vyznachennia kompetentnosti ekspertiv z otsiniuvannia naukovoi diialnosti u vyshchomu pedahohichnomu navchalnomu zakladi [Determining the competence of experts in evaluating scientific activity in a higher pedagogical educational institution]. Teoriia i praktyka upravlinnia sotsialnymy systemamy: filosofiia, psykholohiia, pedahohika, sotsiolohiia [Theory and Practice of Social Systems Management: Philosophy, Psychology, Pedagogy, Sociology], 4, 112–122, [in Ukrainian].
  21. Yashkina, O. I. (2013). Statystychni instrumenty vyznachennia uzghodzhenosti dumok ekspertiv v marketynhovykh doslidzhenniakh [Statistical Tools for Determining the Consistency of Experts’ Opinions in Marketing Research]. Ekonomichnyi visnyk Natsionalnoho tekhnichnoho universytetu Ukrainy «Kyivskyi politekhnichnyi instytut» [Economic Bulletin of the National Technical University of Ukraine “Kyiv Polytechnic Institute”], 10, 442–449, [in Ukrainian].
  22. Cleaver, E., Lintern, M., & McLinden, M. (2014). Teaching and Learning in Higher Education: Disciplinary Approaches to Educational Enquiry. London: Sage, [in English].
  23. Cotton, D. R. E., Miller, W., & Kneale, P. (2018). The Cinderella of Academia: Is Higher Education Pedagogic Research Undervalued in UK Research Assessment? Studies in Higher Education, 43 (9), 1625–1636, [in English].
  24. Dembitska, S., & Kobylianskyi, O. (2022). Formation of work safety culture of the technical specialists. Professional Pedagogics, 2(25), 138-146. https://doi.org/10.32835/2707-3092.2022.25.138-146, [in English].
  25. Dembitska, S., Kobylianskyi, O., & Puhach S. (2022). Improvement of professional training of technical specialists according to requirements of integration methodological approach. Нова педагогічна думка [New pedagogical thought], 3 (111), 14-23, [in English].
  26. Evans, C., Howson, C. K., & Forsythe, A. (2018). Making Sense of Learning Gain in Higher Education. Higher Education Pedagogies, 3 (1), 1–45, [in English].
  27. Evans, C., Zhu, X., Winstone, N., Balloo, K., Hughes, A., & Bright, C. (2019). Maximising Student Success through the Development of Self-Regulation. Addressing Barriers to Student Success, Office for Students’ Final Report. Southampton: University of Southampton with Office for Students, UK, [in English].
  28. Ferretti, F., Pereira, Â. G., Vértesy, D., & Hardeman, S. (2018). Research Excellence Indicators; Time to Reimagine the Making Of’? Science and Public Policy, 45 (5), 731–741, [in English].
  29. Kaynardağ, A. Y. (2019). Pedagogy in HE: Does It Matter? Studies in Higher Education, 44 (1), 111–119, [in English].
  30. Kuzmenko, О., Dembitska, S., Miastkovska, M., Savchenko, I., & Demianenko, V. (2023). Onto-oriented Information Systems for Teaching Physics and Technical Disciplines by STEM-environment. International Journal of Engineering Pedagogy (iJEP), 13(2), 139–146. https://doi.org/10.3991/ijep.v13i2.36245, [in English].
  31. Wood, P., & O’Leary, M. (2019). Moving beyond Teaching Excellence: Developing a Different Narrative for Britain’s Higher Education Sector. International Journal of Comparative Education and Development, 21 (2), 112-26, [in English].

Most read articles by the same author(s)