More than a decade ago, Thomas Tobin, coauthor of the new book, Evaluating Online Teaching: Implementing Best Practices, was hired to teach a business English and communications class in a hybrid format. When the time came for evaluation, he received a very thorough evaluation based on the chair’s observation of the face-to-face portion of his class, but the section of the evaluation instrument meant for the online component was left completely blank. “The department chair eventually confessed that because he had not himself taught using the institution’s LMS, he didn’t feel qualified to rate Tom’s use of its tools,” the book explains. Evaluation of the online component of the class was not something the administrator was equipped to do.
The problems inherent in evaluating online teaching arise understandably. “Deans, department chairs, faculty members, and students rate and evaluate teaching at their institutions mostly through home-grown processes and forms,” write the authors. “Although these are often constructed to help observers and raters to provide meaningful information, it is often the case that even now [years after Tobin’s experience], little training is provided for those using the evaluation instruments.” Many institutions find that one size cannot fit all.
Take the experience shared by author Jean Mandernach. She relates a conversation she had with a colleague at another university: “Though well-established, the university was relatively new to the world of online education, and, despite her confidence that they were providing online students with a quality educational experience, they lacked a system to document teaching effectiveness and student learning.”
Mandernach helpfully shared all of the resources she could from her current university. However, “the evaluation system [she] had shared addressed many factors that were not relevant for my colleague’s university. Implementation of the process was reliant on an operational structure that didn’t exist at their institution. Furthermore, . . . [the colleague] became increasingly concerned about the likelihood of gaining faculty and administration support for this type of evaluation. Despite a shared goal to evaluate the quality of online instruction, the contextual differences between our institutions made it impossible to simply adopt the tools and processes of on institution for use at the other.”
This is one of the key strengths of Evaluating Online Teaching. Divided into sections including planning, formative evaluation, summative evaluation, and sustaining a culture of evaluation, the book draws on research to give the reader important concepts about how to develop and execute evaluation programs, and it clearly explains questions that institutions should address when building the system that will work best on their campus. Sometimes these systems take the form of complicated forms and checklists, but sometimes the perfect solution is easily planned and executed.
Another story by Mandernach illustrates the point. “During a recent chat, a fellow online educator mentioned her constant struggle to find the right balance of instructor postings in the asynchronous discussion. As she explained the challenge, ‘If I post too much, the students become passive and rely on me to carry the discussion; if I post too little, the conversation becomes shallow and fails to explore key issues. I’m just not sure where the tipping point is.’” Mandernach replied with the very simple question, “Have you asked your students?”
For example, the author describes an instructor who has worked very hard to develop a supplemental resources web page for his students, populating it with the latest information from his field. He wondered if the students were using it; a simple, four-question poll answered his question. The instructor was pleased to discover that the students either were using the web page, or intended to, now that it had been called to their attention. This small example of formative assessment let the instructor know, with a minimum of fuss, if his efforts were misplaced.
Another element of successful evaluation of online teaching is understanding who is qualified to review what portions of a course. For example, student evaluations often cause a great deal of concern among faculty members and are often given a great deal of weight in overall instructor evaluation. Indeed, the book points out that, regardless of course delivery medium, students are typically asked to assess things like course organization and structure, instructor communication skills, teacher-student interactions, course difficulty and student workload, assessments and grading, and student learning. Other items might better be left to administrators.
Of course, administrators may find that they have difficulties assessing as well. Tobin points out that he once received a question from an administrator preparing to do his first online “observation.” The administrator asked, “Our observation form has an item on it: ‘Instructor demonstrates enthusiasm.’ How can instructors demonstrate enthusiasm in an online course? After all, the students can’t see the professor or hear his voice.” A lengthy discussion ensued about how to deal with some of the nuances posed by a survey instrument and a context that were both designed to evaluate a traditional classroom.
Dealing with these issues is a challenge facing administrators as they prepare to evaluate online teaching. Evaluating Online Teaching: Implementing Best Practices provides the tools readers need to maximize their impact and evaluate with an eye to continuous improvement.
As author Ann Taylor notes, “Finding the right tool to support an institution’s formal administrative review of online teaching can be challenging . . . Even when suitable tools are identified, rarely can they be used as-is. Adaptations need to be made so that the tools can be used for institution-specific context and needs.” This book will help administrators identify the tools that can be adapted, develop new ones, and employ them all effectively.
Thomas J. Tobin, Jean B. Mandernach, and Ann H. Taylor, Evaluating Online Teaching: Implementing Best Practices (San Francisco: Jossey-Bass, 2015).
Review by Jennifer Patterson Lorenzetti, MS
Jennifer Patterson Lorenzetti, MS, is managing editor of Academic Leader and chair of the 2016 Leadership in Higher Education Conference. She owns the writing, speaking, and consulting firm Hilltop Communications.
Reprinted from Academic Leader, 32.9 (2016): 8. © Magna Publications. All rights reserved.
