Course Evaluations and Their Uses- Initial Thoughts

It surprises me that we haven’t yet made a post about course evaluations. Perhaps because this blog focuses more on preparing to teach and then actually teaching a course, we have paid relatively little attention to date to what happens at the end of a course besides handing out grades.

It is important, however, to pay attention to course evaluations, because they play such a significant role in tools that teachers can use to self-evaluate, but also because administrations use them to evaluate teaching effectiveness. This latter point is incredibly important for those who are in a tenure-track position, but also important for all others whose performance will be measured in part through course evaluations.

For starters, please read the article “How to Use Student Evaluations Wisely” by Professor David D. Perlmutter,, Dean of the College of Media & Communication at Texas Tech University.

In it, he discusses how such evaluations can be and are used both by administrators as well as by faculty. In future posts, I will discuss my take on course evaluations and address some of the points Professor Perlmutter makes.

Advertisements

Pedagogy Now! Creating Effective Evaluation Techniques

Thanks to everyone who attended our CAA workshop, Creating Effective Evaluation Techniques. We had a great time presenting and got a lot of feedback through all the fabulous questions and comments.
As promised, here is our powerpoint presentation PDF. Please feel free to give us any feedback, to ask us questions via the blog or email, or to ask us to address particular issues in the blog. We are also happy to come to your school to do a workshop tailored to your program’s needs. Contact us here:

PedNow_CAA2014_final

Technology in the Classroom- Use of Laptops, Notebooks & Smartphones

The issue of whether students should be allowed to use their laptops, smartphones, notebook computers, etc. while class is going on has been a hot topic for many professors for a long time now. This is a topic for which there is no one perfect solution, as use of these devices can make total sense for some courses but not for others. While more and more professors seem to be finding ways to integrate the use of these devices into their classes in a way that is acceptable to both them and to the students themselves, it still remains a challenge.

David Gooblar recently published an article in his Pedagogy Unbound blog in which he cites a method used by two professors at the University of British Columbia. At the start of the semester, they have the students themselves collaborate on a usage policy that everyone in the class will be expected to follow throughout the term. This approach has the advantage of making the students be part of the solution to the problem, thus increasing the probability of their buying into whatever policies are decided upon. (The solution that seems to be least-successful is the one in which the professor dictates the terms of usage to the students. That seems to light the fire of student ingenuity as they then try to find ways around whatever you’ve dictated.)

In a lecture class of roughly 70 students, I prefer a more hybrid solution. I would address this issue on the first day of class. A colleague of mine who is a psychology professor had polled her students about what they found to be the top five most annoying classroom behaviors exhibited by fellow students during class. Number one on their list was “side conversations”, closely followed by “laptop and cell phone use”. For each point on the list, I asked my students to articulate why they thought that item was on the list. What made that behavior particularly annoying?

They determined that, in the case of laptops and cell phones, it was the distraction factor, I then gave them a few minutes to discuss amongst themselves possible solutions. They spoke up and in the end agreed to try to minimize disruptive use of technology.

I went on to tell them that they would have plenty of opportunities to use both laptops and smartphones during class time, as there were times that I would expect them to conduct in-class research or to participate in online polls, take notes, etc. But I also told them that there would be times when it would not only be inappropriate but unnecessary to be on their devices and that, at those times, I would require that they be put to sleep or turned off.

I am convinced that this kind of open conversation on the first day was the reason why  the use of technological devices never became a problem in that class, despite its size. It also helped that I asked them to use their devices at minimum in every second class, even if only for a few minutes. This seemed to diffuse some of the pent up need to power up a screen and allowed them to focus better for the remainder of the class.

You’ve Made a Mistake- Now What??!!

Every person who has ever taught a class has made an error of some sort. Sometimes you know you’ve made one immediately, while sometimes the mistake becomes evident only after a period of time.

I was recently contacted by an adjunct professor who had been provided by her university with a syllabus, lesson plans, assignment instructions, grading rubrics, etc. for a very technical class. One of the assignments had a set of relatively complex instructions. Five weeks into the project, she discovered to her horror that she had interpreted the instructions for the assignment in a certain way, and while half of the class had followed her lead, the other half of the class had interpreted them completely differently. Because of this, this second group was generating work that was different than that of their classmates. How should a professor handle something like this and still be fair to all?

While there were a number of solutions to this specific problem, the key to resolving any problem that arises from a teacher error is to admit to the class that you made a mistake. Give them a brief outline of what happened- they don’t need to know all the details, just enough to give them a picture of what led to the confusion/misunderstanding/error. This shows them that you are on top of it now, and that you understand the necessity of working towards a fair solution. Then present them with that solution, which is just what this professor did.

In this case, the best solution had to strike a balance between what was done by both groups prior to discovery of the error, what the assignment originally intended (learning outcomes), how well the work already done by both groups would achieve the learning outcomes of the assignment, and importantly how the professor wanted to grade the assignment from this point forward.

Learning Outcomes vs. Goals/Objectives

Is there a difference between the terms “learning outcomes”, “goals”, and “objectives”?

Well, yes.

In reality, these terms are often used interchangeably by educators. We all know what we are talking about when we use these terms, whether we are talking about them in terms of an individual class or an entire departmental curriculum. Although the differences between “goals” and “objectives” are relatively slight, there is in fact an important difference between those terms and “learning outcomes”:

A “goal” is the general result or achievement toward which effort is directed. It implies that you might not get there, as these are only the results you intend to get. (ex. “To gain an understanding of basic composition principals.”)

An “objective” is the specific result or achievement toward which effort is directed. Like “goals”, it also implies that you might not get there, as these are only the results you intend to get. (ex. “To  become familiar with the principals of line, shape, form, contrast, and color in artwork.”)

An “outcome”, on the other hand, is something that follows from an action. It states that you will get there and that there will be evidence of learning that took place. It also outlines the means by which students will achieve the learning. (ex. “Students will demonstrate effective use of the compositional principals of artwork, both graphic and aesthetic, in the creation of a studio project and/or a written research project.”

An “outcome” will also make evident what exactly will be assessed. This is why creating a learning outcome can be so helpful when figuring out what to grade and how to grade it. A well-written learning outcome for a course or a project will initiate the seamless “threading” process that leads a teacher from the outcome to the assignment to the critique to the grading of the work produced.

Please note that both “goals” and “objectives” are usually written with the passive voice. “Outcomes”, on the other hand, are written in the active voice. This difference is clear in the above examples. Verbs like “interpret”, “identify”, “present”, “design”, “analyze”, “apply”, “illustrate”, “explain”, “demonstrate”, and “create” are all very useful when creating a learning outcome statement.

For more information on this subject, the University of Connecticut has published an excellent Assessment Primer that covers these differences in depth.

Self-Assessment = Getting them to Think about their work

There is a lot of chatter in the education community about Self-Assessment. In feeling somewhat dissatisfied that my rubrics are often more useful to me than to my students, I decided this fall to institute more consistent self-assessment into assignments. This assessment has taken two forms – the most obvious is actually giving them the rubric and asking them to complete it and turn it in with the project (the hope here is that this will provoke them to actually read the rubric!). The second is providing them with a series of questions about their work. The goal for me with self-assessment is that they think through what they are doing, rather than waiting for a longed for, but often mysterious grade from me. In the past I have asked them to write a self-evaluation with their projects, building up to an overall statement by the end of the semester. My new approach is giving them self-assessment questions tailored to the assignments.

It’s nearing the end of the semester and what has come of this new approach? Some students have been diligent in their responses, some slackers, just as I imagined. The real bonus has been getting a fix on whether they actually understand the assignments – they may be doing it right, but are they connecting the dots? Ultimately, their self-assessments have been more valuable all along the way than the end of semester course evaluations that at my school at least, are an agony of filling in bubbles and navigating obtuse data. If I ask a student “Which aperture gave you consistently shallow depth of field?” not only am I getting them to think about the result (rather than just look at it), I am also getting a sense as to whether they understand what they have done. It’s helping me to see on an assignment-to-assignment basis whether I am an effective teacher. How carefully or thoughtfully they respond, helps me to also understand whether my students are diligent learners. An obvious lesson is that those who skip the assessment are often the most problematic learners. A corollary bonus is that I get a sense of my student’s expectations. By offering them the opportunity to score their own rubric then match it to mine, room opens up for a conversation about those expectations.

Self-assessment takes time and for me is always a work in progress. I highly recommend incorporating it into assignment delivery and course expectations.

New links

I’ve been working on putting up some new links, especially on teaching and learning. Note especially The University Center for the Advancement of Teaching (Ohio State) which has some excellent resources: http://ucat.osu.edu/selected_links/teaching_portfolio/PortfolioLinks.html

A fabulous new link is from the Innovation League is called Getting Results: http://www.league.org/gettingresults/web/
This is a crash course in course creation targeted towards community college instructors, but the information is valuable for anyone, especially if you are interested in learning more about those buzz topics: learning outcomes, active learning and assessment.

I’ve created a new Rubrics category for links, and will add more in the future.