OUTPUT INTERPRETATION

OUTPUT INTERPRETATION

A BRIEF CIEQ INTERPRETATION GUIDE

The following outline is provided as an aid to the rapid interpretation of the CIEQ results. This guied can be used as a checklist when examining the computerized analysis output of the CIEQ. CIEQ interpretation is discussed in complete detail in the Manual.

Step 1 Adequacy of Results

A. Refer to the top of the first page of CIEQ output. Check the "Sample Size". If the sample size or number of students responding is less than one half of the course enrollment, results may be biased and should be interpreted with caution.

B. At the bottom of the first page is the section entitled "Subscale Results" that contains a column of figures labeled "Rel.". This column contains the obtained reliabilities for the six subscales of the CIEQ. Any subscale with a "Rel." below .65 should be interpreted with caution. Consult he Manual for further details.

Step 2 Comparative Information

A. In all cases, comparative information is provided by decile rank (DEC). The decile rank describes the current course "MEAN" in relation to other courses that have administered the CIEQ. Decile ranks are always interpreted as follows:

1 - 3 Substantial improvement needed

4 - 7 Some improvement needed

8 - 10 No improvement needed

Differences between adjacent pairs of decile ranks within each interval (e.g. 1 vs. 2, 4 vs. 5) are not considered to be significantly different.

B. First refer to the "Subscale Analysis" listing at the bottom of the output page 1. Each subscale represents a different aspect of the course as indicated by its title. Decile ranks for the current course/instructor are listed for each subscale in comparison to six normative groups:

1. "IR", all instructors of the same faculty rank;

2. "CL", all courses at the same grade level (e.g. freshman, sophomore, etc)

3. "D", all courses within the same department;

4. "C", all courses within the same college;

5. "UA", all courses at the University of Arizona;

6. "N", all courses that have used the CIEQ in the United States.

C. On the following two pages under the heading "Individual Item Results" are listed each of the 21 individual items of the CIEQ along with the proportion (%), frequency (#), mean, and standard deviation (S.D.) of responses to each individual item of the CIEQ. Also listed is the text of each item and the most favorable response or "BEST" answer, for each item. All means have been scaled such that 4.00 is the most favorable response and 1.00 is the least favorable response regardless o the initial wording of the item. To the far right of each individual item are listed decile ranks that compare each item mean to the item means obtained in all courses within the same college.

D. In interpreting results, refer first to he decile ranks for subscales. Low deciles for a subscale identify potential problem areas. individual items can then be examined for more specific information. The subscales are composed f the following individual items:

"Attitude" Items 1, 6, 15, 21; "Interest" Items 5, 10, 17, 19;

"Method" Items 2, 7, 12, 18; "Instructor" Items 3, 9, 11, 14, 20;

"Content" Items 4, 8, 13, 16; "Total" Items 1 - 21.

Step3 Descriptive Information

A. Refer to the top of the first page of CIEQ output. Following the initial titles, information is listed on the composition of the responding sample under the heading "Class Description Results". Both the proportion and frequency of responses is listed for each alternative of the following items: "Class Information", "Gender", "Course Option", "Pass-Fail option", "Major-Minor", and "Expected Grade".

B. The next portion of the output lists the proportion (%), Frequency (#), "Mean", and standard deviation (S.D.) of responses to three global ratings: "Content Rating", "Instructor Rating", and the "Course Rating". A mean value of 6.00 is the most favorable rating. These three items have Not been validated and should, therefore, be used only for the purpose of feedback to the instructor.

Step 4 Optional Item Information

A. If Optional Items Section I was used, there will be two pages of output entitled "Optional Items Analysis" for items 22-42. Each of these items will have the same information that was presented for items 1-21 except for the decile information. NOTE: the best response is always AS for Optional Items Section I. If the best response for an item should be DS, then subtract the MEAN of that item from 5.00 to obtain the correct MEAN value. The standard deviation (S.D.) is not affected by the positive or negative direction of the item statement.

B. If Optional Items Section II was used, there will be two pages of output for items 43-63. Each of these items will have the same information that was presented for items 22-42 except that the best response is always A. If the best response for an item should be E, then subtract the MEAN of that item from 6.00 to obtain the correct MEAN value.