The Reliability of the Questionnaires: Teacher Confidence With ICT Survey and Satisfaction Scales

Whilst Chapter 3 focused on the research methods adopted for this study, this chapter provides evidence to support the reliability and validity of the questionnaires administered to address the first research question.

4.2 Validity of the Teacher Confidence with ICT (TCICT) Survey

Factor loadings and internal consistency reliability measures were used to identify the integrity and validity of the Teacher Confidence with ICT Survey. As a first step, the multivariate normality and sampling adequacy of the data were tested. Bartletts test of sphericity indicated that 2 = 1754.04 and this value was statistically significant (p<0.001). The Kaiser-Maiyer-Olkin measure of adequacy was high (0.92), confirming the appropriateness of the data for further analysis. Once this was confirmed, analysis was conducted to examine the factor structure and internal consistency reliability. Section 4.2 reports on the validity and reliability of the Teacher Confidence Scale (TCS); section of the TCICT survey and the surveys Reliability is discussed in Section 4.4. The results of the data analyses are discussed in Section 4.5. The chapter concludes with a Summary 4.6.

4.2.1 Factor Structure

Principal axis factoring with oblique rotation was carried out to extract salient factors. Factor loadings indicate how strongly each item is related to a particular factor and eigenvalues show the relative importance of each factor (Field 2009). Principal components factor analysis with oblique rotation was used to examine the factor structure of the three variable Teacher Confidence Scale. At this stage, one item, (Item 11 for the Organisational Knowledge in ICT Knowledge scale), did not meet the criteria and was removed from all further analysis.

The factor loadings for the remaining 14 items, reported in Table 4.1, were all above 0.40 on their a priori scale and less than 0.40 on all other scales. In all cases, the factor loadings were higher than 0.50. The percentage of variance, reported at the bottom of Table 4.1, was 7.19% for Teacher Knowledge, 55.56% for Organisational Knowledge and 9.13% for Wider Community Knowledge, with a total variance of 71.88%. The eigenvalues for the three scales were all greater than 1, the widely-accepted cut off recommended by Kaiser (1970).

Table 4.1 Factor Loadings, Eigenvalues and % Variance for the Teacher Confidence Scales.

Factor Loading

Item Teacher Knowledge Organisational Knowledge Wider Community Knowledge.

1 0.50 2 0.68 3 0.83 4 0.71 5 0.72 6 0.76 7 0.62 8 0.85 9 0.76 10 0.79 12 0.84

13 0.70

14 0.80

15 0.77

% Variance 7.19 55.56 9.13

Eigenvalue 1.01 7.78 1.28

N= 163 teachers in 14 schools.

The Eigen values for the teachers knowledge, wider community knowledge and organization knowledge were above 1. This is an indication that all the constituent factors were to be retained. This also shows that there are a reasonable proportion of the above variances as well as the substantive sense. All the factors are a linear combination of all the observed variables (Buss & Perry, 1992). This shows that the variables are stable, with the organizational knowledge with the Eigen value 7.78 being the most stable. This can also be seen in the % variance where the organization knowledge has the largest percentage variance.

4.2.2 Internal Consistency Reliability

Cronbachs Alpha Coefficient, with both the individual and school mean as the unit of analysis, was calculated to provide a measure of internal Consistency Reliability for the three variables. The Cronbach alpha coefficient, reported in Table 4.2, ranged from 0.87 to 0.91 for the individual as the unit of analysis and from 0.84 to 0.96 for the school mean as the unit of analysis. Given that Cronbach alpha coefficient for all factors were above 0.80, with three results above 0.90, these were considered to be highly satisfactory.

Table 4.2 Internal Consistency Reliability (Cronbach Alpha Coefficient) for the Confidence Scales

Scale No of Items Unit of analysis Alpha Reliability

Teacher Knowledge 5 Individual 0.87

School Mean 0.95

Organisational Knowledge 5 Individual 0.91

School Mean 0.84

Wider Community Knowledge 4 Individual 0.87

School Mean 0.96

N= 163 teachers in 14 schools.

From the above table, there is a clear illustration of the internal consistency reliability for the confidence scale. The data entries show the measure of internal consistency. This shows how they are closely related to the set of items that they are a group. They are therefore regarded as the measure of the scale reliability. The high value for either the individual mean or the school mean does not show that the measure is undimensional. It is because the items which are the unit being utilized have high covariances and that explains why the variability of both the individual and the school mean for the scales used are approaching one. The number of items in the scale was approaching infinity. In the teacher knowledge, the Alpha reliability of school mean is higher than the school mean. This is an indication that the higher alpha coefficient means that more items have shared covariance and possibly the measure for the same underlying concept. Both the alpha coefficient was above 0.8 which makes them good.

4.2.3 Validity of the Teacher Self-belief and Satisfaction scales

The Teacher Self-belief and Satisfaction survey was designed to measure teachers confidence to influence decision making, teacher confidence when using ICT, teachers work satisfaction and teachers output. The development of this instrument is described in Chapter 3 above.

Principle axis factor analysis with oblique rotation was used to extract salient factors such as decision making, instructional self-belief and the teachers work satisfaction. These are indicated in Table 4.3 provides a factor analysis for the Teacher Self Belief and Satisfaction Scales along with the Burnout scales of 4 Factors as outlined in Chapter 3.3.3. The analysis is well evidenced to determine the integrity of the survey with 24 of the original 26 items being included in the analysis. For each factor, there are at least 3 items each scoring greater than 0.50.

Table 4.3 Factor Loadings, Eigenvalues, and % Variances for the Outcomes Scales

Item Factor Loading

Decision Making Instructional Self-Belief Teacher Work Satisfaction Burnout

1 0.89 2 0.84 3 0.70 4 0.65 5 0.57 6 0.92 7 0.72 8 0.76 9 0.77 10 0.71 11 0.84 12 0.85 13 0.86 14 0.81 15 0.87 17 0.85

18 0.85

19 0.89

20 0.82

22 0.59

23 0.76

24 0.70

% Variance 6.36 8.07 37.71 16.96

Eigenvalue 1.40 1.78 8.30 3.73

N = 163 Teachers in 14 schools.

Cronbachs Alpha Coefficient was used to establish the internal Consistency Reliability for the Outcomes Scales. Table 4.4 shows the calculation for four factors of the outcome scale. Decision making has three items, Teacher work satisfaction has five items whilst instruction self-belief and teacher burnout both have seven items. The unit of analysis provided a score for both the individual and a school mean. An alpha reliability scores above 0.6 is considered satisfactory, above 0.7 considered good and above 0.8 considered high. The results recorded in Table 4-4 indicate the Cronbach alpha coefficient for all factors were above 0.7 with three results above 0.8 and three above 0.9. This analysis suggests the constructs are reliable and valid.

Table 4 - 4 Internal Consistency Reliability (Cronbach Alpha Coefficient) for the Outcomes ScalesScale No of Items Unit of Analysis Alpha Reliability

Decision Making 3 Individual 0.80

School Mean 0.84

Instruction Self-Belief 7 Individual 0.89

School Mean 0.78

Teacher Work Satisfaction 5 Individual 0.95

School Mean 0.92

Teacher Burnout 7 Individual 0.90

School Mean 0.94

N = 163 Teachers in 14 schools.

According to the table 4-4, the alpha reliability is an indication that there is a strong relationship between the decision making and the teachers belief and their satisfaction. Factor structure and internal consistency in the evaluation of the data entries have played a role to explain this (Kagan, 1990). Factor structure refers to a statistical method that is applied when describing the variability among the correlated variables and observed variables in terms of the possibility of the factors. The data set in table 4-4 have been split and what has been produced is a model from the first set with the exploratory factor evaluation. A confirmatory factor evaluation has been done with the evaluation of the derived model for the second half. The rotation method and the number of factors to extract were some of the crucial points that were considered. Internal consistency has also been used in assessment of how the reliably survey that have been designed to measure the ability of the teachers to make decision in their comfort. In deed the graphs have shown how the items are correlated and in deed they have predicted each other.

4.4 Validity and Objective 2

Table 4 - 5 Simple Correlation and Multiple Regression Analysis for Associations between teachers confidence in their knowledge of IT and the four outcomes.

Teacher Confidence Scale Confidence-Outcome Associations

Influencing Decision Making Instructional Self-Belief Teacher Work Satisfaction Teacher Burnout

r r r r

Teaching Knowledge 0.46** 0.16 0.51** 0.26** 0.63** 0.41** -0.02 0.03

Organisational Knowledge 0.51** 0.33** 0.47** 0.12 0.59** 0.25** -0.05 0.12

Wider Community Knowledge 0.42** 0.11 0.51** 0.27** 0.48** 0.07 0.05 0.15

Multiple Regression (R) 0.54** 0.57** 0.66** 0.12

*p<0.05 **p<0.01

N=163 teachers in 14 schools

Table 4 5 has been further understood in section 4.4.1 by developing a scatterplot graph for each set of correlations. As described by Pallant (2013), Scatterplot graphs have been used to show visual representation to provide a general indication of the strength of the relationship between two variables (p.77).

Internal Consistency Reliability

Essentially, internal consistency is an approach of reliability where how the items on a test that are suggested to estimate the same construct accrue similar results (Henson, 2001). Multiple regression analysis and simple correlation analysis were embraced to determine the relationship between teachers confidences in their knowledge of information technology based influencing decision making, teacher work satisfaction, instructional self-belief and teacher burnout. These factors were extracted by the use of principle axis factor analysis, as recommended by Bentler, (2009). The confidence-outcome relations as shown in table 4-5 for the three variables. The consistency reliability ranged for the three variables ranged from -0.02 to 0.63 for teaching knowledge, -0.05 to 0.51 for organizational knowledge and 0.11 to 0.51 for wider community knowledge. Cronbachs alpha coefficient indicates that the consistency reliability for teacher burnout was the least among the three variables. There was a relatively higher relationship between for instrumental self-belief. From the results it is therefore, clear that have a greater instructional self-belief when they have a good command when they have adequate teaching knowledge. Similarly, with adequate teaching knowledge there is least teacher burnout. It can also be deduced that there is a higher confidence-outcome relationship between teaching knowledge and influencing decision making. Similar results are congruent for organizational knowledge and wider community knowledge. For the multiple regression it has been determined that there is a higher relationship between for all the variables expect for teacher burnout which is relatively low.

4.4.1 Relationship between Influencing Decision Making and Wider Community Confidence.

Graph 4 1 provides a visualisation of the correlation between Influencing Decision Making and Wider Community Confidence. The graph demonstrates t...