Pedagogy Now! Creating Effective Evaluation Techniques

Thanks to everyone who attended our CAA workshop, Creating Effective Evaluation Techniques. We had a great time presenting and got a lot of feedback through all the fabulous questions and comments.
As promised, here is our powerpoint presentation PDF. Please feel free to give us any feedback, to ask us questions via the blog or email, or to ask us to address particular issues in the blog. We are also happy to come to your school to do a workshop tailored to your program’s needs. Contact us here:

PedNow_CAA2014_final

Advertisements

Learning Outcomes vs. Goals/Objectives

Is there a difference between the terms “learning outcomes”, “goals”, and “objectives”?

Well, yes.

In reality, these terms are often used interchangeably by educators. We all know what we are talking about when we use these terms, whether we are talking about them in terms of an individual class or an entire departmental curriculum. Although the differences between “goals” and “objectives” are relatively slight, there is in fact an important difference between those terms and “learning outcomes”:

A “goal” is the general result or achievement toward which effort is directed. It implies that you might not get there, as these are only the results you intend to get. (ex. “To gain an understanding of basic composition principals.”)

An “objective” is the specific result or achievement toward which effort is directed. Like “goals”, it also implies that you might not get there, as these are only the results you intend to get. (ex. “To  become familiar with the principals of line, shape, form, contrast, and color in artwork.”)

An “outcome”, on the other hand, is something that follows from an action. It states that you will get there and that there will be evidence of learning that took place. It also outlines the means by which students will achieve the learning. (ex. “Students will demonstrate effective use of the compositional principals of artwork, both graphic and aesthetic, in the creation of a studio project and/or a written research project.”

An “outcome” will also make evident what exactly will be assessed. This is why creating a learning outcome can be so helpful when figuring out what to grade and how to grade it. A well-written learning outcome for a course or a project will initiate the seamless “threading” process that leads a teacher from the outcome to the assignment to the critique to the grading of the work produced.

Please note that both “goals” and “objectives” are usually written with the passive voice. “Outcomes”, on the other hand, are written in the active voice. This difference is clear in the above examples. Verbs like “interpret”, “identify”, “present”, “design”, “analyze”, “apply”, “illustrate”, “explain”, “demonstrate”, and “create” are all very useful when creating a learning outcome statement.

For more information on this subject, the University of Connecticut has published an excellent Assessment Primer that covers these differences in depth.

Self-Assessment = Getting them to Think about their work

There is a lot of chatter in the education community about Self-Assessment. In feeling somewhat dissatisfied that my rubrics are often more useful to me than to my students, I decided this fall to institute more consistent self-assessment into assignments. This assessment has taken two forms – the most obvious is actually giving them the rubric and asking them to complete it and turn it in with the project (the hope here is that this will provoke them to actually read the rubric!). The second is providing them with a series of questions about their work. The goal for me with self-assessment is that they think through what they are doing, rather than waiting for a longed for, but often mysterious grade from me. In the past I have asked them to write a self-evaluation with their projects, building up to an overall statement by the end of the semester. My new approach is giving them self-assessment questions tailored to the assignments.

It’s nearing the end of the semester and what has come of this new approach? Some students have been diligent in their responses, some slackers, just as I imagined. The real bonus has been getting a fix on whether they actually understand the assignments – they may be doing it right, but are they connecting the dots? Ultimately, their self-assessments have been more valuable all along the way than the end of semester course evaluations that at my school at least, are an agony of filling in bubbles and navigating obtuse data. If I ask a student “Which aperture gave you consistently shallow depth of field?” not only am I getting them to think about the result (rather than just look at it), I am also getting a sense as to whether they understand what they have done. It’s helping me to see on an assignment-to-assignment basis whether I am an effective teacher. How carefully or thoughtfully they respond, helps me to also understand whether my students are diligent learners. An obvious lesson is that those who skip the assessment are often the most problematic learners. A corollary bonus is that I get a sense of my student’s expectations. By offering them the opportunity to score their own rubric then match it to mine, room opens up for a conversation about those expectations.

Self-assessment takes time and for me is always a work in progress. I highly recommend incorporating it into assignment delivery and course expectations.

Examples of Strong and Weak Learning Outcomes

Less Effective:

Students will learn the methods of visual analysis of artwork (in terms of form, color, line, etc.)

Better:

Students will demonstrate their ability to conduct visual analysis (in terms of form, color, line, etc.) through oral and written exercises.

 

Less Effective:

Students will gain a greater understanding of the human body through observation and modeling three-dimensional form in space.

Better:

Students will demonstrate a greater understanding of the human body by observing and creating three-dimensional figurative forms.

 

Less Effective:

Students will learn to draw in perspective, use correct proportions, and how to use value and contrast.

Better:

Students will evaluate and draw objects using correct proportion, perspective, and lighting (value/contrast).

 

Less Effective:

Students will learn the principles of good composition.

Better:

Students will create effective compositions in their paintings utilizing design (or compositional) principles such as line, shape, form, hue, and perspective.

 

 Less Effective:

Students will gain a facility in handling the equipment, materials & processes of DSLR photography.

Better:

Students will demonstrate a facility with digital SLR cameras and printing processes through the production of a portfolio of photographic prints.

Formulating a Learning Outcome

In order to figure out what a learning outcome for a course or assignment might be, start by asking yourself: “What kind of measurable and observable knowledge, skills, abilities, or attitudes should students be competent in by the end of this experience?” AND “Through what means will they achieve those outcomes?”

Your learning outcomes should answer those two questions.

The key to writing a strong learning outcome is using verbs that indicate higher order thinking skills. Here are a few examples:

Derive, Design, Formulate, Frame, Predict, Interpret, Evaluate, Demonstrate, Analyze, Synthsize, Create, Identify, Compare, Explain

Less Effective Example of a Learning Outcome:

“Students will learn the art of critical thinking.”

A Better Example of the Same Thing:

“Students will be able to think critically by effectively analyzing assigned readings and evaluating the views of fellow students  both verbally and in writing.”

The better example contains both observable abilities (thinking critically) as well as a description of the means by which they will prove their competency (verbally and in writing). Including the latter is important because that is a product that you can grade, which is the “measurable” part of the learning outcome.

What Exactly is a Learning Outcome?

The term “learning outcome” is a phrase you see a lot of nowadays. It has replaced “goals” and “objectives” as the term to use when discussing what you want a student to become competent in at the end of a course or assignment.

A learning outcome states what students can DO with what they have become competent in by the end of the course or assignment, which results in a product that can be evaluated.

Please go to the “Formulating a Learning Outcome” post to see how to write a learning outcome.

Things to Consider When Planning an Assignment

Is the assignment /project consistent with the learning outcomes for the class? Have you adequately communicated to the students how it fits into the large context of the class?

What skills must be demonstrated in the completion of the assignment?  How much (if at all) does it engage student creativity?

How realistic is the level of stringency of the assignment?  Are the students equipped with the expertise to handle it? (Don’t give freshmen a senior level project and expect success.)

How clear have you made your expectations/criteria for grading? Do you have a specific ‘vision’ for how the project should turn out?  Have you communicated this to your students?

How realistic is the time-demand of the assignment? Remember that they are taking other classes, too!