Student Evaluation of Teaching , Units and Learning
Wednesday August 27, 2008. How can we improve the SET and SEU?
One of the responsibilities in my new role at UB is to assume resposibility for our Student Evaluation of Teaching (SET) and Student Evaluation of Units (SEU) processes. Currently for most people @ UB this process involves a paper-based summative assessment occuring late in the Semester.
Recently there has been suggestions about a mid-semester assessment activity. This would be for me formative in nature representing assement as learning (click here to learn more about this ) involving learners setting learning goals and staff recieving feedback on the success/otherwise of their learning intentions through self and peer assessment.
If you were or are working @ a university what information do you think learners and staff would need to collect and how could this information be used to improve learning?
I look forward to reading your comments
Wednesday, September 3 2008. How do we assess the Global Aspects of Learners and Learning ?
I am sitting here in Antalya (Turkey) at a the 13th World Conference in Education. The conference theme is ’ Global peace: Strategies for curriculum development and implmentation’. As the Chariperson of the Gloabal Education special interest group’ I was introduced to the idea of replacing the term global education with the education with a global perspective. Given our focus this month I would like to hear your thoughts on how the current evaluation practices at your isntitution reflect a global perspective? You may also wish to comment on any suggestoins you would have for implementing an intiative in tis area for example what question would you inlcude on a SET or SEU to recognise the global aspect of learning or learners in your class?
[…] The focus topic for this month is Student Evaluation of Teaching , Units and Learning […]
Pingback by James O’Meara’s Blog » Purpose of this Blog — August 27, 2008 @ 3:10 pm
Dear James,
Nice to receive mail after a long gap. I think for learner it is very important to know about the instructional objectives and his/her progress of learning for each topic. And for teacher to get feedback for each learning aspect is important to conduct formative evaluation either by peers or by instructors.Sometimes written through worksheets and sometimes oral evaluation should be given.
Thanks,
Comment by Dr.Ummed Singh — August 27, 2008 @ 11:18 pm
Dear Ummed,
Great to hear from you and thnak you for you comment. I am aware of where you work , it may be helpful to others to learn about the numbers of students you work with and how you receive feedback from them. Given your background in technology do you have any suggestions about how it could be used to help academic teaching staff get feedback about their teaching and the course they have designed?
Loking forward to hearing from you my friend
Comment by jameso — August 28, 2008 @ 8:42 am
Dear Majed,
This may look wierd on the website as my reply wil appear before your message is posted.
This is due to a moderation button, which I have asked to be switched off , requires me to approve your comment before it is posted. Such moderation appears to me to defeat the purpose of inviting comment , especially when you and others have chosen your words so carefully. Hopefully this will be rectified soon
You wrote that you believe that students should have the opportunity and be able to say what they think concerning the evaluation of faculty members, strategies and methods of teaching and learning, and about the curricula. Could I ask you to ask your students in Jordan to share their stories about the evaluation of faculty members ( please ask them not to identifgy individual faculty in their stories)
I look forward to hearing their stories
Comment by jameso — August 31, 2008 @ 9:12 pm
First comment of the Spring!
For me student feedback has two functions:
1) It is (potentially) to teachers as peer review is to researchers. The world is crying out for better teachers but the means to evaluate, reward and promote excellence in teaching is sadly lacking
2) To refine the approach in specific teaching situations. The current SET/SEU mechanism makes it quite hard to focus on specific issues - it is cumbersome to ask specific questions - I used to try but now I accept the defaults
Dave
Comment by David Stratton — September 1, 2008 @ 12:50 pm
Thanks Steve P for the great advice on how to turn off the moderation button. Great advice, and a good example of how IPOL and LEWS are complementing each other.
Comment by jameso — September 2, 2008 @ 2:28 pm
Great thoughts Dave, for me some of the questions are, how can we tailor this process to meet the needs of individuals seeking specific feedback without the whole proces becoming un-sustainable?
I would also like to hear your thoughts, as well as the thoughts of others, on the idea that staff and students at different stages of their time/career at UB would need different sets of questions to satisfy their current needs. In other words does the fact that the questions do change assume that we(staff and students) do not change over out time at UB?
Comment by jameso — September 2, 2008 @ 2:36 pm
The following comment is from a staff member who wished to provide feedback without identifying themselves publicly. Given the quality of the insights I asked and was given permission from them to post the comment on their behalf
The University’s SEU has for a long time now had one very “ominous” question on it (Question 8). “Ominous” in terms what it is assuming, and that is that students will fail a unit at least once. How else can one determine whether improvements have been made in a unit? If the question was asked in relation to the up-to-date content of the unit then it has been poorly formulated. Although the option is there for students to say “don’t know”, most students still try and answer it in the “agree/don’t agree” vain, which influences the outcome of the SEUs.
Questions 2 & 6, and 5 & 9 are almost identical and have drawn comments from a number of students. Question 3 belongs in the SET not in the SEU.
That leaves 6 questions that tell us very little.
We now see a lot of students not attending classes, either lectures, tutes, labs, all three or a combination of the three. This in turn has resulted in a fractured understanding of the content of a unit. There ought to be questions related to the attendance before we are being summoned to explain why the SEUs are not of a high enough standard.
I don’t think we have ever interrogated a student as to why they have only achieved 75% of the total mark for a unit, rather we give them a Distinction and a pat on the back. Teaching Staff on the other hand are being interrogated in a way that suggests there is something wrong with the teaching of a unit when they receive the same result via the SEUs. We are seeing less and less involvement of students due to their outside commitments to family, work, and social life. Education is more and more seen as having to be undertaken in order to get a better job prospect. As you well know, there is a vast difference between students who want to learn but do not always find it easy, and students who do not want to learn and who will blame everyone else for their lack of success.
The only students who are truly capable of answering any questions with regard to the value of a unit, are those students who have involved themselves with it. WE are supposed to teach students in the best way we can, that is why WE develop units/courses based on our understanding of pedagogy. Our assessment methods are there to help us evaluate how much learning has taken place NOT to mainly please some administrative body, in order that they can look good on paper.
I find it rather sad that in all the years I have been teaching we are still tied to a number on a graph that tells people how good or bad we are. Thank goodness that there is still oral feedback otherwise the good teachers among us would have left a long time ago.
Comment by jameso — September 2, 2008 @ 3:30 pm
Hello James and everyone else who has contributed - well done.
I do not use set/seu very much since I have always disliked its “input” structured measures.
I often receive emails of thanks from students for making their learning exciting and interesting. Some times I have been given a standing ovation. I like these reactions to learning.
I always teach in a student centred way and often into actual or potential roles in organisations since that is where I think students will have their learning from us and in their future moderated by the values, attitudes and views of a working organisation.
So I use my own feedback survey about outcomes. It is structured like a market research survey that has objective scaled closed questions about how the respondent sees things Now and Would Like plus an open short answer comment space.
It is short (under 1 page) and the topics are Satisfaction. Learning and Change, example I ask how much did you learn, the topics you learnt most about and the topics you learnt least about least plus suggestions for change.
For me this is superior without trying to change my delivery and teaching style using measures of the less effective input factors that seem to me to be removed from the main result - learning.
I hope this helps.
David Flynn
Comment by David Flynn — September 3, 2008 @ 4:27 pm
Many students do not attend class so it is very hard even to get a representative evaluation of a Unit. The reason for non attendance are numerous such as distance to campus, work and family commitments, access to Podcasts.
There are many variables that influence students learning yet these are not addressed in the SET and SUE. For example students who enter with an ENTER score of over 80 are often self motivated and do not need as much input as students who have ENTER score of 50 and sometimes below.
Comment by Maureen — September 3, 2008 @ 4:58 pm
Hey David,
Thanks for the acknowledgment, we hope that this site will become a hub for HE staff and academics friom around the world.
If possible couldm you share some more information about your one page document. Given your business background , you may know of Kirkpatrick’e evaluation model. If you do, which level of feedback do you characteristcally recieve from students who complete your form?
Comment by jameso — September 4, 2008 @ 5:35 pm
Hi Maureen,
On the issue of non-attendance, do you think the return rate could be higher if an on-line option was avaliable? If so what changes would need to be made to the current forms to suit this new medium?
Comment by jameso — September 4, 2008 @ 5:36 pm
Hello James,
Great meeting you in Turkey!It is relly good that you have established this forum to facilitate the “voices” of all participants in the educational sphere.The theme this month on teacher and student evaluation not only gives the primary participants an avenue to air our perspectives,but also re-enforces the fact that we all have a personal and collective reponsibility to foster real change in education. Thanks for your interest!Great blog! Will definitely share it with my friends.
On the issue of an online option-I believe that personal motivation must also be present or the results may be the same as face to face.What do my fellow colleagues think?
Comment by Lisanne Edwards — September 5, 2008 @ 12:57 am
Hi James et al,
It has always been my belief and backed up by some 40 years of practice that the essential critical factor is the nature of the social relationship between teachers and learner. Unfortunately in most of the world the size of teaching groups and the modularisation of courses has to large extent destroyed this especially for undergraduates. I have also relaised that no one type of teaching or created learning situation fits all. Consequently both teachers and institutions need to ensure that a range is offered and encourage students to seek out that suit them.
I have just completed an evaluation of e learning and assessment. Like TV, radio, OHPs etc. clearly not about to revolutionise learning and teaching but in right palce and used correctly a useful addition to the enterprise
Comment by Prof. Ivan Reid — September 5, 2008 @ 9:54 pm
I have also relaised that no one type of teaching or created learning situation fits all. Consequently both teachers and institutions need to ensure that a range is offered and encourage students to seek out that suit them.
Comment by download — September 7, 2008 @ 8:27 am
Reflecting on my literature class where we are exploring “The Pearl” by Steinbeck, it really came home to me that the traditional teacher-centered classroom even in terms of the physical arrangement of classes need to be less evident as students are in a global culture and can learn so much from each other.I wanted to set up learning corners where the different learning stles could be facilitated but it was not feasible.We therefore had whole class discussion and paired work but my classroom did not look like I wanted it to - a classroom where students would be facilitated at differenrt learning corners.Sometimes I wonder if the focus of our school is on certification or education for life.I am faced with 45 students and a classroom that is so packed that the traditional style is rthat most viable option.The problem is that I need to have my students attain a different view of education but so many inherent structures are still so traditional.While the worl cries out for better techers,how serious are we really? practically?I realise that students need to express themselves through the different learning styles but there are structural classroom and space constraints.Do you think that I will achieve creative students who have the ability to thinkfor themselves even when the structures around them remin so fixed at each level of their education? I wonder how much of a global perspective students can really attain when educational facilities remain so entrenched?
Perhaps some of the information universities need is how students feel in such a situation and these views be presented to government bodies responsible for education!just a thought!Anyway,I am thinking of taking 45 third form students under a tree outside and set up my leanring centres there…dont know how that will work but wish me luck!
Comment by Lisanne Edwards — September 20, 2008 @ 4:32 am
Www On-line Universities…
This is similar to comment spam but avoids some of the safeguards designed to stop the latter practice. Six Apart started a working…
Trackback by Www On-line Universities — October 7, 2008 @ 9:15 pm
We’ve been collecting material on this topic at: http://www.tltgroup.org/studentcourseeval/Main.htm. A few summary observations. First, any such process ought to illustrate to students what responsible, collaborative, interdisciplinary, and effective research is all about. The more an institution prides itself on its research, the better it should be at this process. (Of course, that’s almost never the case. Academic staff often ignore what they know about research when they design these processes.) Second, these processes should be seen as part of the students’ education, and academic staff ought to ask students what students are learning from them. What, for example, do students assume is done with the reports? Third, my working assumption is that any mid-term evaluative activity will be more important and influential than any end-of-course activity, since its findings will be applied in full view of the students. Fourth, I’m very excited about the potential of a new kind of survey called a matrix survey for this kind of inquiry, during the term and at the end of term, inquiry by faculty and by administrators. If you’d like to talk more, please get in touch with me at ehrmann at tltgroup.org.
Comment by Steve Ehrmann — October 19, 2008 @ 9:56 am
Perhaps you need to ask for the Moderation Button to be turned on once again. It seems we have been invaded by ‘work at home’ spammers… On one hand, it’s merely an irritation but, on the other, it does detract from the blog.
Comment by Chuck — October 31, 2008 @ 1:27 pm
This matrix survey idea could potentially be a very useful tool. I’m eager to see how it works and what kind of results it will produce. Very interesting idea.
Comment by email templates — November 1, 2008 @ 12:07 am
Technology can indeed have and impact and be used effectively in the process of feedback. It allows students to provide feedback anonymously without fear of retribution by the instructor. Surveys can be set up for free with sites like surveymonkey.com. But a survey is only as valuable as the thought that goes into it’s creation. Choose the questions and working carefully. Once it is determined there is a problem, a different set of questions is needed to determine why and how to fix it.
Comment by Rochester MN — November 1, 2008 @ 2:55 am
That was a great post. I will have to bookmark this site so I can read more later.
Comment by CLEP Testing — November 11, 2008 @ 5:09 am
Very interesting idea about the matrix survey. I think this could be used in an array of applications to better results. Thanks for the post, very intriguing to say the least. Can we expect more articles in the near future?
Comment by concert tickets — November 22, 2008 @ 1:46 am