Approaches to evaluation

Evaluation of teaching and learning is increasingly important in our competitive, management focused, higher education sector but the structural imperative to focus on outcome evaluation should not obscure the need for evaluative practices which improve teaching and learning.

Evaluation and the possibilities of change

My approach to evaluation is to try to integrate it into the teaching and learning process as an integral part of reflective practice. Thus at every point in curriculum development and delivery I am asking: "How can this be be done better?"

While I believe that careful curriculum planning is critical, the evaluation imperative should mean that change is a possibility at any point, if it can be successfully negotiated and if such change might improve the learning environment.

There are of course limits to such flexibility and the established learning contracts already established with students, particularly arrangements regarding assessment, must remain relatively stable in order to provide a "safe" learning environment.

Evaluation as process

Evaluation is a critical part of the overall process of learning design, delivery and improvement. This is expressed clearly in the "wheel" model of Southern Cross University

 

 

The authors describe the model as providing a "holistic view of the curriculum process that includes all phases of design, development, teaching, delivery and review". They continue:

As we know in practice, however, things may not be as neat as this: phases overlap, some things occur concurrently, and so forth.

The metaphor of the wheel is employed to represent curriculum as an ongoing, cyclical process, commencing with design, moving through to resource development, teaching, assessment, student support and then evaluation of how it all went. The information gleaned from evaluation leads back into re-designing the unit, and the whole process starts over again.

Evaluation appears as a separate phase in the cycle, but it also underpins the whole process of the wheel. Evaluation needs to occur in all segments or phases in the wheel if we are to achieve best practice in our various activities.

Mechanisms of evaluation

There are a range of formal and less formal mechanisms that I use to evaluate my teaching and course design:

• ongoing discussion with teaching peers about what is working and what is not
• monitoring student reaction in class
• monitoring unsolicited reaction such as positive or negative email and personal comments from students
• monitoring quality of student work both formal assignments and class discussion
• end of semester quality assessment questionnaire
• end of semester class discussion of the course
• in classes where portfolios are appropriate these provide a fantastic record on the learning process
• conceptual evaluation of course content such as concept mapping

This style of evaluative practice demands building a relationship of trust with your students. Actively seeking feedback from students is of course one mechanism that helps build this type of trust relationship.

Example of evaluation in practice

In the course review process of Professional Practice and Culture/Journalism Studies we engaged in a semi-formal focus group style discussion of the course at the end of the semester which gave us a range of good feedback that we incorporated into course design for the next semester. We also looked at the antonymous questionnaire results for the subject which confirmed results from the focus group that, although students liked the course and found it stimulating they felt overloaded by the demands of the assessment procedures.

One unexpected piece of information about my own teaching also arose from an analysis of the questionnaire results. Although I scored high on perceived knowledge of the subject, quality of feedback and general satisfaction, I was rated low by the students on use of varied teaching techniques. This led to reflection with the course coordinator on the structure of the course. We felt that because the tutorials had been highly structured through a debate format this had led to a perception of my teaching contribution as unvaried even though I had attempted to bring different perspectives and techniques to these occasions.

This was an important lesson about the close relationship between the structural elements of course design and the interpersonal elements of teaching delivery. This led us to build in a more varied structure for the next semester's tutorials. This involved eliminating the debate even though this had been rated well by many students. This allowed us to both open up the tutorial space for discussion and interaction as well as a way of answering concerns about the assessment work load.