Elwood F. “Ed” Holton III, former director of the School of Human Resource Education & Workforce Development at Louisiana State University, recognized as early as 1996 that the Kirkpatrick Model of Training Assessment, although so widely adopted that it has become virtually an industry standard, had several serious drawbacks (Holton, 1996). To begin with, he noted that the Kirkpatrick model is essentially a taxonomy, or classification scheme, and he stated, “One shortcoming of taxonomies is that they do not fully identify all constructs underlying the phenomena of interest, thus making validation impossible” (Holton, 1996, p. 6).

With all the investments that colleges and universities make in trying to develop their academic leaders—sending them to conferences and workshops, creating their own in-house professional development programs, assigning new leaders to mentors, and so on—institutions want to know whether they’re getting any return on their investment. In short, does the leadership development that current and prospective academic leaders participate in make any real difference? If so, what difference does it make? And in either case, how do we know?