The Advantages and Disadvantages of Using an External and Internal Evaluator in an Evaluation

This essay is an attempt to describe the advantages and disadvantages of using an external and internal evaluator in an evaluation. The author will propose an internal evaluator for a functional literacy program. The structure of the essay will start by defining programme evaluation and the background of evaluation, and then give advantages and disadvantages of using external or internal evaluator in an evaluation. This essay will then propose one evaluator for a functional literacy program and give reasons for choosing such an evaluator. The essay will conclude by giving the author’s personal opinion on the matter above.
Evaluation is the process of collecting and/or using information for the purposes of determining the value and worth of the subject of the evaluation process (Birley & Morel 1998). Australian Development Cooperation (2009) has expanded the definition by stating that programme evaluation is the systematic and objective assessment of an on-going or completed project, programme or policy its design, implementation and results. Other authors like Mbozi, (2007) defined evaluation as a systematic assessment of the worth or merit of some project. The aim is to determine the relevance and fulfilment of objectives, development efficiency, effectiveness, impact and sustainability. In general, the purpose of your evaluation should be to establish the outcomes or value of the program you are providing and to find ways to improve the program (House, 1993).
The process of evaluation is considered to be a relatively recent phenomenon. However, planned evaluation has been documented as dating as far back as 2200BC, Shadish, Cook & Letish, (1991). Evaluation became particularly relevant in the U.S. in the 1960s during the period of the Great Society social programs associated with the Kennedy and Johnson administrations.
It is argued in Letish, (1991) that for programme evaluation to take place there must be an evaluator. A programme evaluator is said in Letish (1991) as a person who systematically assesses the process and outcomes of a program with the intent of furthering its development and improvement on a programme. It’s clearly stated in international standards of evaluators number U2 about the Evaluator’s Credibility that the person conducting an evaluation should be both trustworthy and competent to perform the evaluation, so that the evaluation findings achieve maximum credibility and acceptance. In this case there are two types of evaluators, internal and external.
An internal evaluator is a program staff or worker who conducts program evaluation on behalf of the program. The whole evaluation is an intervention conducted by a unit/and or individuals reporting to the management of the donor, partner, or implementing organization. An external evaluator is a person or persons who conduct an evaluation of a program from outside the program and they are not part of the program being evaluated. Other writers like Kushner, (2000) have added that external evaluation is an intervention conducted by entities and/or individuals outside the donor and implementing agencies.
The advantages of internal evaluators according to Newman, & Brown, (1996) are immense which includes having better overall knowledge of the program and also possess more informal knowledge of the program because they operate within. Farther more internal evaluators have less threatening from management of fellow staff because they are already familiar with every staff. Allowances for evaluators are proved less costly compared to external evaluators.
The disadvantages for hiring an internal evaluator include being less objective in conducting an evaluation and may be more preoccupied with other activities of the program. It’s also known fact that internal evaluators do not give complete attention to evaluation because their income will not be affected. Other writers like Dolley, (1994) have included that internal evaluators do not have adequately training as an evaluator.
On the other hand the advantages of the external evaluator in Newman, & Brown, (1996) include being more objective to the process because they need to meet those objectives and offers new perspectives of approaching the evaluation process. The external evaluators have different angles to observe the evaluation process and they criticize management if necessary. External evaluator is dedicated to greater amount of time to the evaluation process and attention compared to internal evaluators. In most cases external evaluators have greater expertise and evaluation brain to do the evaluation properly compared to internal evaluators who are sometimes accountants or office managers.
The disadvantage of external evaluators includes high costs to pay and known fact that they may require more time for the contract to be signed or negotiations. It’s been proved that external evaluators are unfamiliar with program staff, the program culture and they sometimes create anxiety among program staff about being evaluated by others. It’s also possible that external evaluators may be unfamiliar with organization policies, such that certain constraints affecting the program might be overlooked.
But when it comes to functional illiteracy program, functional literacy is imprecisely defined differently from nation to nation, and study to study. Functionally illiterate is whereby persons can read and possibly write simple sentences with a limited vocabulary, but cannot read or writes well enough to deal with the everyday requirements of life in their own society.
An organisation wishes to evaluate functional literacy programs can request the service of a staff member or hire someone outside the organisation. A wide literature focuses on evaluation as a means of improving organisational performance in the business, government programs and the community projects. In this case therefore evaluation is a tool or means of improving functional literacy program’s performance in providing literacy and show impact in the community where the service is being provided. The aim is to confirm a place for evaluation as an integral aspect or essential part of performance improvement (Love, 1991).
The best method of sourcing a suitable evaluator for literacy program will need to be assessed in each case Knowledge of program and literacy operations. Most evaluators stress the importance of gathering detailed knowledge about the functional literacy program being evaluated in order to ensure that the evaluation reflects the program and its context. Posavac and Carey (1997) note that stakeholders where literacy programs are conducted are far more likely to give recommendations a respectful hearing if they feel the evaluation team understood the population served, the school housing the program, the literacy program itself, and the staff. Such understanding cannot be gained by brief meetings. This means that internal evaluators have an advantage over external evaluators since they work in the environment in which the program operates and may have been personally involved in some part of the program planning (Weiss 1972). The following are the other reasons of choosing internal evaluator for an evaluation of a functional literacy program.
The internal evaluators are very familiar with the work of functional literacy, it’s culture, the aims and objectives which someone from outside the functional literacy may not understand. Generally these works assume that evaluation for program improvement is conducted by internal evaluators, more specifically, by managers. A good example is said in Globerson, et al. (1991). Defining middle management as an ‘endangered species’, Globerson et al.(1991) offer a new role for middle managers as internal planners and evaluators who can exercise organisational control and improve performance of schools. These managers should be able to evaluate performance by using specific performance criteria, measuring actual results, and comparing the results with expectations’ (Globerson et al. 1991). It’s the best away of self-correcting and much less threatening than external evaluation. This may make it easier for those involved to accept findings and criticisms and may misunderstand what you want from the evaluation and may not give what you need. It also helps in planning, setting teaching standards, measurement of student results and comparison with other programs which rests on the unstated assumption that only management is involved. Similar works include Love (1991), Sluyter (1998) and Czarnecki (1999) who generally assume that managers rely on internal evaluators for measurement. Therefore internal evaluation is very clearly a management activity. Sluyter (1998) presumes that internal evaluators should measure ‘indicators of quality and stability’ for performance improvement (Sluyter, 1998) while Goldberg and Sifonis (1994) presume that monitoring and evaluation processes such as surveys, class activity-based, overall pass trend and benchmarking will be carried out by internal evaluators.
As part of school organisation, internal evaluators have first-hand knowledge of the organisation’s philosophy, policies, procedures, personnel and management. This allows selection of evaluation methods ‘tailored to the reality of the organization’ (Love 1991).
As well as detailed knowledge about the program being evaluated, it may also be useful in some cases for the evaluator to have an understanding of cultural and political factors that affect the operating environment of the organisation being evaluated (Braskamp et al. 1987). In the case of a university, this might be sensitivity to university politics (Shapiro & Blackwell 1987) while in the case of a religious institution it might be the need to be ‘attuned to the uniqueness’ of the particular group (Faase & Pujdak 1987).
Evaluation does not record, but rather creates ‘reality’. Because of the subjectivity of the values involved, there is no reason to suppose that internal or external evaluators would be any more or less likely to share the values or ‘reality’ of the people involved in the evaluation. The individuals employed by an organisation are substantially more likely to share ‘value systems’ with each other than outsiders. Value systems are complex and made up of factors including culture, upbringing, experience and education. ‘Objectivity’ should be a overtime (Cummings et al. 1988) and prepare the ground for acceptance and utilisation of factor in selecting internal.
Utilisation of evaluation is a key issue for many practitioners (Posavac & Carey 1997; Patton 1997). Some argue that an internal evaluator is better placed to understand the environment and prepare findings in the style most likely to be used (Shapiro & Blackwell 1987). An internal evaluator can also build credibility evaluation results (Gunn 1987. Love (1991). Because internal evaluators know the all corners of the program, they are able to see ways that the evaluation can make a difference and promote the use of evaluation findings over the longer term (Weiss 1972).
Dissemination of results is also an important issue for evaluators (Rossi & Freeman 1982). Cronbach et al. (1981) point out that a successful evaluation must do more than just ‘amass good data’, timely communications should distribute information to the persons rightfully concerned, and those hearers should take the information into their thinking. All evaluators should be aware of communication issues. This factor should not favour internal evaluators or external ones.
Finally, the availability of an internal evaluator could be seen as an investment that an organisation makes in ‘an enduring corporate resource’ that is useful in a number of contexts (Love 1991). Whether this is a reasonable investment will depend on the size of the organisation and its likely future evaluation needs. The argument for building staff capacity may be a strong factor in some cases but may make no sense for an organisation that is only going to conduct one evaluation per decade.
Ultimately, the ‘Great House’ of evaluation should have room for both internal and external evaluators if it wishes to continue to grow.
The essay concludes therefore that the evaluation remains critical regardless which evaluator undertakes the process of evaluation advantages and disadvantages that both play independent roles in program evaluation. In schools especially functional literacy programs mentioned above, it’s arguably that internal evaluators play more part in sharping not only the picture of a functional literacy program, but also helps to develop a continuous improvement process of the program.

References

Australian Development Cooperation, (2009). Guidelines for Project and Programme Evaluations. Vienna: Austrian Development Agency.
Birley, M. and Morel, N. 1998. A practical guide to academic research. London: Kogan Page
Braskamp, LA, Brandenburg, DC & Ory, JC. (1987). ‘Lessons about clients’ expectations, in J Nowakowski (ed.), The client perspective on evaluation: New directions for program evaluation, no. 36, Jossey-Bass, San Francisco.
Cronbach, LJ Ambron, SR, Dornbusch, SM, Hess, RD, Hornik, RC, Phillips, DC, Walker, DF & Weiner, SS (1981). Towards reform of program evaluation: aims, methods, and institutional arrangements . San Francisco: Jossey-Bass.
Dolley, J. 1994. Planning, monitoring and evaluating learning programmes. Buckingham: Open University Press
Faase, TP & Pujdak, S (1987). Shared understanding of organizational culture’, in J Nowakowski (ed.), The client perspective on evaluation: New directions for program evaluation San Francisco: Jossey-Bass.
Globerson, A, Globerson, S & Frampton, J. (1991), You can’t manage what you don’t measure: control and evaluation in organizations. Aldershot: Gower Publications.
Gunn, W.J. (1987), ‘Client concerns and strategies in evaluation studies’, in J Nowakowski (ed.),The client perspective on evaluation: New directions for program evaluation. San Francisco: Jossey-Bass,
House, E.R. (1993). Professional evaluation: social impact and political consequences. California: Sage Publications,
Kushner, S. (2000), Personalizing evaluation. California: Sage Publications.
Love, A.J.(1991). internal evaluation: building organizations from within. California: Sage Publications.
Mbozi, E.H. (2007). Programme Evaluation in Adult Education. Lusaka: UNZA
Newman, D.L. & Brown, R.D. (1996). Applied ethics for program evaluation, San Francisco: Sage Publications
Posavac, E.J. & Carey, R.G. (1997). Program evaluation: methods and case studies , 5th edn, New Jersey: Prentice Hall.
Sluyter, GV 1998, Improving organizational performance: a practical guidebook for the human services field , California: Sage Publications,Thousand Oaks,

References: Australian Development Cooperation, (2009). Guidelines for Project and Programme Evaluations. Vienna: Austrian Development Agency. Birley, M. and Morel, N. 1998. A practical guide to academic research. London: Kogan Page Braskamp, LA, Brandenburg, DC & Ory, JC Cronbach, LJ Ambron, SR, Dornbusch, SM, Hess, RD, Hornik, RC, Phillips, DC, Walker, DF & Weiner, SS (1981). Towards reform of program evaluation: aims, methods, and institutional arrangements . San Francisco: Jossey-Bass. Dolley, J. 1994. Planning, monitoring and evaluating learning programmes. Buckingham: Open University Press Faase, TP & Pujdak, S (1987) Globerson, A, Globerson, S & Frampton, J. (1991), You can’t manage what you don’t measure: control and evaluation in organizations. Aldershot: Gower Publications. Gunn, W.J. (1987), ‘Client concerns and strategies in evaluation studies’, in J Nowakowski (ed.),The client perspective on evaluation: New directions for program evaluation. San Francisco: Jossey-Bass, House, E.R Kushner, S. (2000), Personalizing evaluation. California: Sage Publications. Love, A.J.(1991). internal evaluation: building organizations from within. California: Sage Publications. Mbozi, E.H. (2007). Programme Evaluation in Adult Education. Lusaka: UNZA Newman, D.L Posavac, E.J. & Carey, R.G. (1997). Program evaluation: methods and case studies , 5th edn, New Jersey: Prentice Hall.