Analysis Of Cronbach's Alpha

The reliability of the PKLA test in piloting calculated through Cronbach’s Alpha was .71, which is considered acceptable. Out of the initial 35 items each having 1 point, the minimum score was 6 and the maximum score was 28, with the mean and standard deviation of 15.78 and 5.26 respectively. The preliminary investigation showed that contrary to teachers’ claims in the first phase of the study (Table 4), the majority of them had low levels of PKLA; the mean score of the participants was also low (15.78 out of 35). Item analysis was run on all the items, that is, item facility, item discrimination, and choice distribution, were calculated and the non-functioning and mal-functioning items were edited. In addition, two of the items were deleted
This shows that the different parts of the test had acceptable correlations with each other. On the other hand, all the different parts of the test had significant correlations of .55 to .70 with the total test at .01 probability level. Since a correlation of .30 to .70 is desirable between different parts of an instrument (Ref?), these correlations are all considered acceptable.

The next important step was to check the validity of the PKLA test. Content validity of the test was observed when developing the test as it was based on the information from the needs assessment phase which was also cross-checked with the topics in the available language assessment textbooks. Further, construct validity of the test was checked through running factor analysis on the collected data, the results of which are reported in Table
Two of the items loaded on one factor whereas three of them loaded on several factors. This shows that in spite of their seeming similarity in terms of checking knowledge of test types, the items do not load under similar factors. On the other hand, there is less variation among the items comprising Part B (stages of language test design). Two of the items loaded on only one factor, and three of them on another factor. The items in Part C (statistical techniques), show a similar pattern as the ones in Part B, three loaded on one factor and one loaded on another factor. Part D (key terms/concepts), like Part A, showed a lot of variation in terms of the loading of the items comprising it. This was expected and is quite natural as the items checked a variety of terms and concepts. The best pattern of loadings was seen for items in Part E (assessment techniques) as all loaded on only one factor. Even though two items loaded on two factors, they all commonly loaded on one factor too. This shows that new assessment techniques have the same underlying factor. Finally, the items in Part F (assessing language components/skills) loaded on two factors. The items checking assessing vocabulary, listening, and speaking loaded on one common factor; items checking assessing grammar, pronunciation, and reading loaded on another common