Language Teaching Methodology

                    Using Student Evaluations to Improve Teaching

1  Introduction

         Most colleges and universities require instructional staff to assess student feedback on the quality of teaching. Take our university, Wuhan Institute of Chemical Technology for an example, has since 1990, required its faculty to collect student evaluations of instruction and course at the middle and end of each semester. This policy, developed by a campus-wide faculty task force and adopted by many educational sections, states that all academic divisions shall develop written policies that identify multiple methods for evaluation. One method is based on information provided by students using end-of-semester student evaluation forms either developed by the Program for Excellence in Teaching (PET) or by the division.

This policy requires that this evaluation process go beyond simple collection of data and must include components designed for self-improvement (formative) and merit-performance (summative) purposes. For the latter, the policy indicates that all individuals (administrators and faculty) with responsibilities for evaluating teachers for merit-promotions must receive instructions regarding the use of teaching evaluation methods. These individuals should also, on an annual basis, receive information on reliability, validity and interpretation of relevant data.  

 

2  The Benefits of Students Evaluation

 

Probably the most important benefit of student evaluation is the feedback that these data provide to instructors, so that courses and teaching procedures can be refined to enhance student learning. By highlighting instructional methods, teaching activities and student input, this evaluation can have a positive impact in improving the climate of teaching and learning across campus.

Although student evaluations provide much information about student perceptions of what instructors are doing and offer suggestions for course improvement, instructors are often not aware of how to use these data to make meaningful changes. Instructors express difficulty translating student comments into action, especially when these comments are often contradictory. Following are some suggestions for instructors to help resolve problem and facilitate the use of student evaluations to improve teaching and learning.

By many accounts, the best first step in using student comments to improve teaching is to consult with a colleague or teaching specialist regarding the meaning of these data. Contact your teaching center to set up a private consultation to review these forms, discuss them with a senior member of your department or set up a group of colleagues to review each other's student comments. It would be very helpful if departments or academic units develop mentoring mechanisms to facilitate this collegial support. In addition, it is suggested that academic units develop and publicize a list of instructors who receive exceptional ratings on each of the individual items of the evaluation forms. In this way, an instructor who wants assistance with change in response to comments on a specific dimension of teaching (i.e., use of examples, enthusiasm, course organization) can easily identify a colleague to consult.

        For student evaluations to lead to change, it is important for you to review and reflect on course objectives, teaching philosophy, and student responsibility, learning outcomes and teaching methods. Ask yourself questions such as "What did I want to accomplish in this course? How well were these objective met? How do student's comments correspond to my assessment?"

Review your syllabus, or share your syllabus with a colleague for review to get another perspective, to see if modifications can directly address student comments. Often, comments about course organization, workload and desired outcomes can be resolved by improved specificity and clarity in one's syllabus.

It is important for you to consider your strengths as well as weaknesses. Look at what your students say you are doing well before you work on the suggested changes. Identify the teaching behaviors or course variables that contribute to high ratings in these areas. Modification of your teaching or course should be seen in this context and future changes should not jeopardize what is currently working right.

It is recommended that you focus on a few areas for change instead of wholesale transformation of your course or teaching methods. Identify which student comments have the highest priority to you in terms of their importance in meeting your teaching and learning objectives and focus on them first. Change should be incremental and gradual. Get comfortable with student feedback and become informed about departmental procedures and policies before making additional course or instruction modifications.

 

3  How to use students evaluation

 

     Generally, when collecting the students’ evaluation, we always ask the students to give different ratings for teachers in some topics related to the teachers’ teaching performance. For example, for the topic: the teacher always prepares for the lecture carefully, students can give different ratings. Then, as for how to use these ratings, we’ll discuss in the following. 

    3.1  What do student ratings mean?

How did you do on the true-false quiz about student evaluations? A great deal of research has been conducted over the last three decades on student ratings of teaching/courses. The empirical findings from this research do not always fit faculty members' beliefs. Much of this research focuses on correlates of student ratings of courses/instructors. In addition, studies have assessed factors that account for the most variance in student ratings. The literature also contains discussion of bias in evaluations (what is bias and when should we control for it). Finally, writers in this area discuss the practical implications of their research.

Some of the factors significantly and positively associated with student ratings include the following: measures of student achievement; alumni, peer and administrative ratings; qualitative student comments; workload/difficulty level; energy and enthusiasm of the teacher; being a regular faculty member (as opposed to a graduate assistant); faculty research productivity; student motivation; student expected grade; and course level. The size and practical significance of these relationships vary. For example, most agree that there is little practical significance to the small positive correlation between expected grade and student ratings. Similarly, research shows a small and negative, but practically insignificant, relationship between class size and student ratings.

Factors generally found to be unrelated to student ratings include faculty age and teaching experience, instructor's gender, most faculty personality traits, student's age, class level of student, student's GPA, student's personality, and student's gender (with the exception of a slight preference for same-sex instructors).

    A variety of studies using several different methodologies have assessed which factors account for the most variance in student ratings. This research shows that it is the instructor or instructor variables, rather than the course variables, which account for the most variance in student ratings. Researchers conclude that most student evaluations should be used to evaluate instructors, not courses.

    Finally, what is bias in student evaluations? When should we be concerned with bias? Experts in this area do not agree. Some have suggested that anything not under the control of the teacher constitutes bias. Others have posited that all variables unrelated to teaching effectiveness be considered bias. Still others argue that bias involves variables that are not a function of the instructor's teaching effectiveness. Finally, in some writings the focus is on variables that correlate with student ratings but are unrelated to teaching effectiveness and student learning. For example, one might argue that faculty enthusiasm is a form of bias because it is part of his or her personality. Yet one could also argue that this variable does affect teaching effectiveness and is, in part, under the faculty member's control, so it doesn't need to be controlled. The issue of bias is still under debate.

    3.2  About the adopting of the students evaluation

Many instructors prize student written comments more highly than numerical evaluation scores. However, while these comments are often rich with insights and suggestions, it is often difficult to make sense of them in ways that lead to identifiable change. This is compounded by the often-contradictory nature of written comments and the disorganized way that they are presented and read. It is suggested that you try to impose some structure on these comments by sorting them into categories. The most basic categorization would be by strengths and weaknesses, with student comments listed under the appropriate heading. Another method for organizing student comments is to group them by overall course rating; one group containing comments from students who rated the course high, another with comments from less satisfied students. This puts the comments into a context that may be helpful for interpretation and future change.

        Constructing a graph can also facilitate organization of written comments. First, make a chart listing the four or five characteristics you believe lead to effective college teaching. You can then place student comments under the characteristic with which it most closely relates. Place a minus sign (-) next to negative comments and a plus sign (+) next to the positive comments. Tally up the plusses and minuses at the bottom as a way to highlight and summarize the comments and provide direction for change.

  

4 Alternative methods of evaluating teaching

 

The below methods of evaluating teaching are some alternatives that may used in addition to student evaluations.

Peers can review many aspects of your teaching, including your syllabus and course content, your grading and testing-writing skills, as well as such things as your control of the class, rapport with students, and communication styles and skills.

4.1 Course and syllabus review

        Colleagues within your discipline are best at evaluating such things as the content of your courses, the textbooks and articles you select, the learning goals your have for your students, and your organization of the course materials. As well as asking your peers within your department to review these aspects of your teaching, you might consider sending your syllabus off to colleagues teaching similar courses in comparative institutions and asking them to assess the content and syllabus - much like you'd ask a colleague to review an article you wrote. You might include a review form and request that your colleagues write a letter reviewing the course content and syllabus. The completed review form and the letter could assist with documentation of your teaching effectiveness for annual review, tenure and promotion, etc.

 4.2 Review of your teaching actions in the classroom or laboratory

Peers outside of your discipline are better at observing and assessing your teaching actions within your class or lab. Since they are less informed about the content of the material and skills that you are teaching, they are better able to focus on your teaching actions that are colleagues within your discipline. If you do ask a peer to evaluate your teaching actions, you should provide that person with observation guidelines. Sample observation forms are available at the PET Library in the Conley House. It's best to take a look at several examples and then develop a your own form that will guide your observer to look for the kind of information that you will find most useful in developing your teaching or measuring your teaching effectiveness.

To ensure that you receive a fair and valid review of your teaching actions, you need to be aware of issues such as observing more than one day of your teaching and having more than one person view your teaching, as well as the right to refuse to have a specific assigned observer if you know that person is biased against you or your style of teaching.

5 Conclusion

        Instructors may believe that student course/teacher evaluations are unreliable, invalid, biased, and affected by many inappropriate factors. In general, the research does not support these beliefs. It is still the case, however, that student ratings, and their interpretation, have limitations. Student evaluations should be only one source of information used for the evaluation of teaching, and the interpretation and proper uses of student ratings are at least as important as the evaluations themselves.

 

References

 

[1]  Cashin, W. E. 1990. Student ratings of teaching: Recommendations for use. IDEA paper no. 28. Center for Faculty Education and Development, Kansas State University.

[2]  Centra, J. 1993. Using student evaluations: Guidelines and benefits. In reflective faculty evaluation. San Francisco: Jossey Bass.

[3]  Flash, P., Tzenis, C., & Waller, A. 1995. Using student evaluations to increase classroom effectiveness, (2nd ed.). Minneapolis: University of Minnesota.

[4]  Lewis, K. 1991-1992. Making sense (and use) of written student comments. Teaching Excellence, 3:8.

[5]  Morrison, T. 1995. Analyzing qualitative responses on student evaluations: An efficient and effective method. Rockingham: Higher Education Research and Development Society of Australasia.

[6]  Groccia, J. Stanford University Center for Teaching and Learning. 1997. Using student evaluations to improve teaching. Speaking of Teaching, 9: 1.