Occupational Psychologists

“Occupational psychologists stress the need for validity and reliability throughout the selection process. Explain how this can be achieved and critically examine the practical case for and against such rigour in selection, in practice, in an English context. In your essay, examine other perspectives dictating features of the selection process and if and how these can be reconciled with the need for validity and reliability”

Word Count: 2,637 words
INTRODUCTION:

The essay explains about Validity and Reliability and the other aspects of evaluative standards. It also describes about the various stages in a selection process and an example which shows the role of these evaluative standards in each of the process with a conclusion.
Validity refers to “the accuracy of measurement and it must measure what it purports to measure” [Cooper, Robertson and Tinline 2003]

It is told that there are different types of validity where, Content Validity, Construct Validity and Criterion related Validity is of more importance to the process of recruitment [Cooper, Robertson and Tinline 2003]. The other types include face validity, faith validity, rational validity, factorial validity and synthetic validity.[Cook, M., 1998]

Reliability is “Dependability of a measurement device or test; the underlying principle is consistency of measurement” [Cooper, Robertson and Tinline 2003] Different types reliability is explained in the example illustrated.

IS VALIDITY OFTEN MORE IMPORTANT THAN RELIABILITY?

“Psychological research shows that references and interviews are inaccurate selection methods. Accuracy divides into reliability and validity.” [Cook, M., 1998]

“Reliability and validity, although related, are not the same thing. It is possible for an instrument to be reliable, but not valid. For example, a watch that is consistently five minutes fast is reliable; it will indicate 12.05 pm every day at midday. But the watch is not valid; it is five minutes out. While it is possible for an instrument to be consistent (reliable) but not accurate (valid), it is not possible for an instrument to be valid if it is not also reliable.” [IRS Health and Safety Bulletin, Dyer, C., 2001]

It could be told from the given example that are validity and reliability are both important to any stage of the recruitment process, but validity is often more important.
There are two more complementary aspects of evaluating standards which could be considered for more accurate selection process. “Systematic selection requires that they should meet certain standards concerning reliability, validity, interpretability and practicality.” [Cooper, Robertson and Tinline 2003]

Interpretability is “the extent to which the scores on a test are interpretable and meaningful” and practicality of a method is assessed from two further perspectives: “its perceived usefulness and fairness and the extent to which it devours organizational resources.” [Cooper, Robertson and Tinline 2003]

It can be told that for any test or selection method to be evaluated accurately, it is not sufficient for the method to be valid and reliable, it should be interpreted well and it should be practical. In other terms the consequences of the process should be generally acceptable. For example, if the most reliable and valid test discriminates a set of people, then it is not considered to be practical.

STAGES IN A RECRUITMENT
PROCESS:

Although there are different approaches to list out the stages in a recruitment process, Taylor suggests two distinct approaches to the process, traditional job based approach and competency-based approach, that should be carried out the before in hand. Fig 1 illustrates the combined approach or stages in good practice recruitment and selection [Taylor, S., 2002]

[pic]

Job analysis is considered to be the start of the selection process and foundation of various other issues after selection of the candidate like “pay differentials determining, identification of training needs, setting of performance targets and the drawing-up of new organisational structures.”[Taylor, S., 2002:92] Also, for retaining or making the job redundant, evaluating performance and so on. Job analysis systems divide into: “job-oriented techniques, content/worker-oriented and attribute-oriented techniques” these need both inter-observer and intra-observer reliability. Analysis comparisons or inventory ratings with similar jobs is a good way to achieve reliability [Cook, M., 1998] uses of job analysis extends to form job descriptions and person specification. In selection process job analysis doesn’t stop with analysing the tasks to be achieved, it should also focus on the skills necessary, the equipment used and the environment where the tasks are carried out. [Taylor, S., 2002:93] Validity of Job analysis is shown through the results and its relevance that leads to more accurate selection decisions [Cook, M., 1998:36]

It could be said that a valid Job analysis would result in a proper Job description and person specification. Job descriptions can be used as a “tool in recruitment, selection decision, basis of employment contracts, part of an employer’s defence in unfair discrimination cases and statement that clarify what the employee is expected to achieve and how they will be rewarded for so doing”[Taylor, S., 2002:98,99]

Person specification emphasises information under skills, knowledge, personality attributes, education, qualifications and experience and is commonly divided into desirable and essential [Taylor, S., 2002:101].

The competency based approach which is more often the alternate to the job-based one is not much different from a person specification, as Taylor says “a competency approach is person-based rather than job-based” using this approach alone would result in the organisation having like-minded people causing lack of ideas and innovation and it is sometimes considered reliable to “select for attitude – equip with skill approach”, unless for highly skilled professionals, whose qualification and experience speaks more than their attitude[Taylor, S., 2002:105-107]

Let us take an example to discuss the different types of evaluating standards and its importance and influence in each stage of the recruitment process or various selection methods.

‘Bookswiz Inc.’ is an American based publishing company who decided to expand by opening a new factory in UK in order to increase their market value. The American company nominated a UK based recruitment specialist to analyse jobs for managerial level. Using the Job Analysis Checklist (Lewis, 1985), the job descriptions and person specifications were identified with the help of US based sister concern. It was found out that managers from each department needed varied set of skills and attributes, which was tailored to whom they manage. After the detailed discussion with the other human resource specialists, decisions on which selection method should be used for each level were made. These were achieved by assessing the four requisite evaluative standards of practicality, interpretability, reliability and validity. It included various psychological tests, biodata and ‘in-tray’ exercises. Potential applicants were then to be interviewed. The initial screening of applicants was by job-related biodata questionnaire which is a reliable and valid predictor of future job performance and turnover. To identify the suitable applicants for different layers of management, a series of psychological tests were conducted. A Job Choice exercise (Stahl, 1983) was conducted to distinguish between managers performing poor and well as it was sexually and racially neutral. A personality questionnaire, the Management Potential Scale of the California Psychology Inventory (Gough, 1984) was chosen to differentiate between productive, goal-oriented managers and moody, erratic and self defeating individuals. In-tray exercises evaluated applicant’s supervision, planning and decision-making skills.

Forty potential applicants were invited to attend the selection process after sifting out unsuitable managers through biodata questionnaire.
Only twenty-five made it to the series of structured interviews with four separate interviewers. In order to establish the interviewers’ consensus, Inter-interviewer reliability checks were conducted. Apparently, twenty candidates were hired as these resulted in 88 percent of agreement between the different interviewers. After a year in the follow-up assessments, it was found that there was 25 percent increase in productivity when compared to the US plants and that the workforce were highly motivated and satisfied with their job. In two and five year assessments, year on year, there was an increase in productivity of 15 percent and the growth through expansion resulted in increased market share by 3 percent. [Fictitious example, idea extracted from Cooper, Robertson and Tinline
2003]

In the above example, ‘Bookswiz Inc.’ was able to achieve what they wanted especially because they allocated time for recruitment specialists to evaluate the selection methods based on the four evaluative standards. Taylor’s approach of ‘good practice’ was achieved in terms of considering the competencies involved in the managerial levels for different departments also with the job description and person specification. Validity, reliability, interpretability and practicality do not stop with the selection process itself but should be extended to any validation process including the follow-up assessment or the subsequent job performance. If not, the validation of the selection procedure becomes meaningless. [Cooper, Robertson and Tinline 2003]

According to Cooper, Robertson and Tinline, the advantage of ‘Bookswiz Inc.’ was the earlier established selection methods for the US plants, the specialists could see what went wrong in terms of the selection process and subsequent job performance and later could give an enhanced system which overcomes the past issues. This also gives a chance for the company to understand and alter their way of future selection for better performance. The reliability of the ‘in-tray’ exercises conducted in the selection method of ‘Bookswiz Inc.’ might have been checked by test-retest reliability where the exercises were tested on an existing group of job holders prior to operational use. This gives a correlation between the two tests which often determines the reliability of the exercise. “The higher the correlation is the better the match between each set of applicant’s scores and reliable the method will prove to be.” [Cooper, Robertson and Tinline 2003]

It can be argued that the test-retest reliability is often expensive and time consuming; it could be said that when a test is retaken by the same group of people, there will certainly be an improved performance which might result in low correlation scores. There is another, yet expensive type of reliability parallel form reliability for evaluating the ‘in-tray’ exercises and tests. This is when a reliability coefficient is obtained by estimating the results of two different versions of the same test or selection procedure. This is considered to be time-consuming and impractical as it takes a lot of time to design a different version of the same test. Internal consistency, based on parallel form, is widely used where the same test is divided into two halves, which are then correlated. Validation is more concerned with the interpretation of scores, arising from a particular selection test or method, rather than the tests or procedures themselves. [Cooper, Robertson and Tinline 2003] This means that the importance need to be given not only in the selection of a method, also to the extent of validating it after the process. Content validity and face validity has to be considered in validating these exercises to see how these tests are perceived. “Content validity depends on job analysis.” [Cook, M., 1998:211] “A sampling issue related to the measure’s development.” [Cooper, Robertson and Tinline 2003] “Construct validity can focus on a test or on a trait or ability.” [Cook, M., 1998:217] As we can see from the given example that the managerial motivation is not a physical entity, it is more of a behaviour that includes Need for Power (NPow) and Need for Achievement (NAch). Stahl(1983) in his Job Choice Exercise, has proved that high scorers on NPow and NAch had higher promotion rate and they were more likely to be leaders and managers. This helped ‘Bookswiz Inc.’ to select motivated managers as this test was validated to yield better results. [Cooper, Robertson and Tinline 2003]

The inter-rater reliability was measured by the correlation obtained between different interviewers [Cook, M., 1988:44]. In ‘Bookswiz Inc.’ the amount of acceptance or agreement between the four interviews who interviewed the potential managers resulted in an effective selection decision-making as they used structured or standardised interviews with the same set of questions rather than a traditional unstructured interview involving questions considered appropriate to each interviewer. Structured interviews were more reliable as they consistently led to higher agreement between the selectors [Cooper, Robertson and Tinline 2003].

“A typical validation study collects two sets of data: predictor through interviewer ratings and criterion through some index of productivity” [Cook, M., 1998] in ‘Bookswiz Inc.’ example, “criterion-related validity is concerned with the links between the test used and job performance on some criteria (e.g. meeting production targets). How closely the test is related to the subsequent performance is defined as test’s predictive validity.” [Cooper, Robertson and Tinline 2003]

It has been told that there yet another way to assess criterion-related validity. “Concurrent validity is administering the selection measure to existing employees and measuring their current performance in the job.” [Cooper, Robertson and Tinline 2003]

The reason to use more evaluative standards is probably because a considerably valid and reliable selection method might not differentiate suitable applicants from the others. That leads to the use of two main ways of interpretability: criterion-based and norm-based scoring system [Cooper, Robertson and Tinline 2003]. Criterion-based scoring should exactly specify what is expected out of selection method and how well it attributes the work that is to be done and Norm-based scoring is “interpreting one applicant’s performance scores in relations to others” [Cooper, Robertson and Tinline 2003] It involves certain norms based on which grading of score is dependant on. For example, the norms can be of people who have already taken the test. Ranking all the applicants is not considered to be a good idea. It is better that these decisions be made before the recruitment process.

Practicality of a method is often assessed by “its perceived usefulness and fairness” [Cooper, Robertson and Tinline 2003] in ‘Bookswiz Inc.’ example, the in-tray exercise would’ve resulted differently if conducted in a quiet room rather in a busy office environment. The very practical reasons of how acceptable the test is and if it applies to everyone in terms of the legislation, fairness has to be taken into account of.
CONCLUSION:

To have an optimised result from a selection process and selection methods being used, the four evaluative standards, Validity, Reliability, Interpretability and Practicality has to be satisfied and more time and research has to be devoted to finalise and evaluate the process to be used, rather than the process itself. “Time should be taken to develop fully the methods and tests to ensure that they are reliable, accurate and able to be properly interpreted, while also avoiding unfair discrimination against ethnic minorities and other social groups.” [Cooper, Robertson and Tinline 2003] It could be told that most of the factors mentioned above are completely relevant and related to each other to obtain a successful selection process and thereby adding value to any organisation which solves the purpose. But, considerations should be made according to the situations in terms of the cost, time, importance and certain other factors involved. Note that the most inappropriate method for someone might turn out to be the most appropriate method for someone else. And the decision-making for each level of the process might involve different stakeholders who have different views to different issues. It is considered better to rely on specialists who have more experience in facing different scenarios, rather than to have taken the wrong decision.
REFERENCES:

❖ Cook, M., Personnel Selection: Adding Value through People, West Sussex: Wiley, 1998

❖ Cooper, D., Robertson, Ivan T., and Tinline, G., Recruitment and Selection, A Framework for Success, London: Thomson, 2003

❖ Gough, H. G. ‘A managerial potential scale for the California Psychological Inventory’. Journal of Applied Psychology, 1984

❖ IRS Health and Safety Bulletin, Health and safety management employee health, Dyer, C., www.xperthr.co.uk Issue: 302 Date: 01/10/2001, (Accessed online November 2008)

❖ Lewis, C., Employee Selection London: Hutchinson, 1985

❖ Stahl, M., Achievement, power and managerial motivation: Selecting managerial talent with the job choice exercise’. Personnel Psychology, 1983

❖ Taylor, S., People Resourcing, Second edition London: CIPD, 2002
-----------------------
JOB ANALYSIS

JOB DESCRIPTION

PERSON SPECIFICATION

COMPETENCY ANALYSIS

COMPETENCY FRAMEWORK

RECRUITMENT

SELECTION

APPOINTMENT

INDUCTION

FIGURE 1

Taylor, S., 2002:92