"Measuring Learning: The Ultimate Teaching Evaluation."
Linda B. Nilson, Ph.D, Director of Office of Teaching Effectiveness and Innovation, Clemson University
August 25, 2011

Dr. Nilson began her presentation by drawing attention to widespread facutly dissatisfaction with the use of student evaluations as a measure of faculty teaching quality. She also reported research indicating (1) that there is only a mild correlation between student learning and intructor's rating on student evaluations and (2) that instructors who better prepare students for upper-level work receive lower ratings (Glenn, 2011).

As alternatives, she presented a series of measurements intended to evaluate student learning. These include end-of-course measures as well as pre- and post-test combinations.

Students can be asked, for example, to write a targeted essay about what they learned in a course. Other strategies get at perceived student learning gains by asking students to rate their ability to answer a particular question or type of question. Examples include the SALG survey instrument (see below) and the Transparency in Learning & Teaching survey instrument from the University of Illinois (see below). Another possibility is to ask students in a survey: "How much has this course improved your skills or abilities in each of the following?"--and then to select appropriate student learning outcome from a list of possible learning outcomes. (Nilson provided her own list of 32 outcomes, divided into categories such as "communication" and "research skills"--it is available on the supplemental handout (PDF) for this course.) 

Nilson also described pre- and post-test possibilities, including a very straightforward one: give the final exam twice--once in the first week of class (ungraded) and again at the end of the course (for a grade.) Shocking? What's to lose, she suggests? At the worst, students (who should not be allowed to take home the initial exam) will pay closer attention over the course of the semester to the same things that you have decided are the most important for them to learn.  

For more ideas, and for her suggestions for possible ways that faculty can use some of these strategies to supplement traditional student evaluations, see her PowerPoint slides and additional handouts. It should also be possible to view the recorded session for a week after the 8/25 session. For her bibliography, see the handouts or the list below.

http://www.magnapubs.com/handouts/?cid=222


Examples of Perceived Learning Gains model surveys
:

1. Student Assessment of Learning Gains   http://www.salgsite.org

This site (requires free registration) facilitates the creation of surveys to evaluate "learning gains" by adapting a well-studied set of model questions that evaluate student perceptions of their own development of concepts, skills and attitudes as a result of their work in any particular class. (See excerpt of model below)

2. Transparency in Learning & Teaching survey instrument from the University of Illinois

http://teachingandlearning.illinois.edu/transparency.html

 

References and Additional Resources (from Nilson's handout)

  • Atlas, J. L. (2007). The end of the course: Another perspective. The Teaching Professor (June/July), 3.
  • Bender, A. (n. d.). Resources for assessment of student learning. Available at
    http://sites.bio.indiana.edu/~bender/resources/Assessment/ResourcesAssessmentOfStudentLearning.html
  • Glenn, D. (2011, January 9). One measure of a professor: Students? grades in later courses. Chronicle of Higher Education. Available at http://chronicle.com/article/One-Measure-of-a- Professor-/125867/
  • Griffiths, E. (2010). Clearing the misty landscape: Teaching students what they didn?t know then, but know now. College Teaching, 58, 32-37.
  • Nuhfer, E. B. (1996). The place of formative evaluations in assessment and ways to reap their benefits. Journal of Geoscience Education, 44, 385-394.
  • Nuhfer, E. B., & Knipp, D. (2003) The knowledge survey: A tool for all reasons. To Improve the Academy, 21, 59-78. Available at http://www.isu.edu/ctl/facultydev/KnowS_files/KnowS.htm
  • Understanding what our geoscience students are learning: Observing and assessing. On the Cutting Edge - Professional Development for Geoscience Faculty. Available at http://serc.carleton.edu/NAGTWorkshops/assess/index.html http://serc.carleton.edu/NAGTWorkshops/assess/knowledgesurvey.html
  • University of North Dakota. (2005). Welcome to knowledge surveyor (project home pages). Available at http://www.ks.und.edu/
  • Weimer, M. (2007). Helping students take stock of learning. The Teaching Professor (February), 4.
  • Wirth, K. R., & Perkins, D. (2005). Knowledge surveys: The ultimate course design and assessment tool for faculty and students. Proceedings of the Innovations in the Scholarship of Teaching and Learning Conference, St. Olaf College/Carleton College, Northfield, MN, April 1-3. Available at http://www.macalester.edu/geology/wirth/CourseMaterials.html
  • Wirth, K. R., & Perkins, D. (2008). Knowledge surveys. Understanding what our geoscience students are learning: Observing and assessing. On the Cutting Edge - Professional Development for Geoscience Faculty. Available at http://serc.carleton.edu/NAGTWorkshops/assess/knowledgesurvey/index.html