Meta-Evaluations: Quality Assurance for Evaluation Programs - Research Paper

Introduction

More often than not, meta-evaluation exercises are carried out to ascertain the quality of the data collection processes and findings of an evaluation program. They are very important since they are able to pinpoint the strengths and shortcomings of evaluation programs (Stufflebeam, 2001). They may be conducted for formative or summative purposes if the main aim of the meta-evaluation is to scrutinize a single study. Formative meta-evaluation is centred on improving the evaluation process while it is still underway. If it is carried out at the end of the initial evaluation program, the purpose of the meta-evaluation exercise is considered to be summative.

Is your time best spent reading someone else’s essay? Get a 100% original essay FROM A CERTIFIED WRITER!

Order now

In the case of the RAMS project, carrying out a meta-evaluation exercise would serve to provide the evaluators with more information on what may have gone wrong in the internal evaluation exercise. Internal or Self-evaluation exercises tend to provide results that are biased and unreliable. Therefore, in order to ascertain that the initial evaluation exercise was up to par, the stakeholders involved in the RAMS project could request for a metaevaluation exercise to be carried out.

Impact of Standard and Alternative Forms of Measurement

In the course of carrying out the meta-evaluation exercise, the evaluators are required to collect information from all the stakeholders associated with the RAMs Project. The evaluators could make use of a comprehensive checklist in the meta-evaluation process so that they ensure that they have covered all the key areas that they ought to scrutinize. Therefore, they need to select suitable methods that would ensure that the final information of the meta-evaluation exercise is valid and reliable. Failure to use accurate methods would jeopardize the entire meta-evaluation exercise, rendering it an exercise in futility. With regards to the RAMS project, the evaluators may need to ascertain the findings of the student and teacher evaluation reports. In order to verify if the information that was collected is true, they may have to get in touch with the students and teachers using communication channels such as phone interviews.

In the course of carrying out meta-evaluation exercises, there are a number of standard principles of measurements that evaluators are required to adhere to. These principles include; feasibility, utility, proprietary, and accuracy. The principle of feasibility requires the evaluators to carry out the meta-evaluation in the context of the RAMS project. The evaluators are required to develop meta-evaluation checklists and questions that will provide them with the requisite information from the stakeholders such as teachers and students in line with the principle of utility (Thomas, 2017). The principle of accuracy behoves the evaluators to discern every single piece of information they collect from the stakeholders so that their final findings are accurate and reliable. Last but not least, the evaluators are required to carry out the meta-evaluation exercise in an ethical manner in line with the principle of proprietary. Failure to abide by any of the aforementioned standard principles of measurement could greatly undermine the efficacy of the meta-evaluation exercise.

Reasons for Metaevaluation

Metaevaluation exercises are carried out for a number of reasons depending on the objectives of the evaluators. They could be carried out at the end of an evaluation exercise or during the course of the evaluation program.

The meta-evaluation of the RAMS project's evaluation exercise may be carried out by the evaluators for the purpose of sharing information with the beneficiaries of the project so as to garner their feedback.

It may also be carried out so as to identify the key messages in the findings of the initial evaluation exercise. The key message of the initial evaluation report ought to be consistent with the conclusion of the final reports,

The meta-evaluation exercise could also be carried out to determine the shortcomings of the initial evaluation exercise so that future school-based programs may be evaluated using a properly designed rubric.

Roles and Responsibilities of People Involved in Metaevaluation Programs

Evaluators can assume multifarious responsibilities during the process of conducting a meta-evaluation exercise. For instance, they may assume the role of a methodologist in the process of acquiring when collecting data from the stakeholders (Luo, 2010). Furthermore, they may assume the role of educators during the process of disseminating information to the stakeholders and other beneficiaries of the program that was under evaluation. In addition, they may also assume the role of a judge when selecting the best approach towards collecting, analysing, and disseminating information.

Evaluators who are involved in any meta-evaluation exercises are required to be honest in as far as analysing data is concerned. They should not falsify any findings for purposes of hoodwinking the stakeholders into believing that a particular program fully met its objectives. In the process of collecting information from the stakeholders, the evaluators should ensure that the welfare of every informant is taken into consideration.

Conclusion

In order for the RAMS project to be considered for adoption in other schools across the state, a meta-evaluation exercise should be carried out to ensure that the preliminary evaluation exercise was thorough and comprehensive. It would also enable the evaluators to provide their feedback to the parents, teachers, students, and school administrators so that they can finally understand the relevance of such a program that is geared towards helping students excel academically.

References

Luo, H. (2010). The Role for an Evaluator: A Fundamental Issue for Evaluation of Education and Social Programs. International Education Studies, 3(2), 42-50. Retrieved from (https://eric.ed.gov/?id=EJ1065989)

Stufflebeam, D. L. (2001). The meta-evaluation imperative. American journal of evaluation, 22(2), 183-209. Retrieved from (https://journals.sagepub.com/doi/abs/10.1177/109821400102200204)

Thomas, D. R. (2017). Feedback from research participants: are member checks useful in qualitative research?. Qualitative Research in Psychology, 14(1), 23-41.Retrieved from (https://www.tandfonline.com/doi/abs/10.1080/14780887.2016.1219435)