Creating a Data-Informed Culture 

Data. Like a thread woven through every discussion, that four letter word appears in everything from the news to our board meetings to our strategic plans to our accreditor’s reports. In fact, four letters do not adequately describe the depth and breadth of the era we live in, so we need two more consonants and a vowel: BIG data. Couple that with words like “data-driven,” “dashboards,” and “predictive analytics,” and you have a representative sample of the current buzzwords that are swarming our emails, committees, and even our hallway conversations. In this era of “doing more with less,” institutions are searching for operational efficiencies and predictors to increase enrollment and retention, reduce cost, ensure students graduate, and improve the learning experience. Finding those efficiencies, evaluating their effectiveness, and then implementing those changes requires data.

Furthermore, in this time of increased public scrutiny (examining the value of a degree versus its cost and accreditation ensuring the quality of a degree), leaders need to be able to tell their story of student success and lifelong learning in the face of legislative mandates such as outcome-based funding and accrediting bodies that demand evidence that you are “doing what you say you’re doing.” The accrediting body that my institution belongs to, the New England Association of Schools and Colleges (NEASC), calls their supplementary documents the “Data First” forms, named intentionally so that institutions understand what to put at the head of the line before they write the first word of that story. Used effectively, data can assist in framing the discussion, educating those decision makers, and providing proof that the assessment loop is truly being closed.

But when leaders begin putting their story into action, they tend to talk data but walk anecdote. While stories of individual students’ adventures, tragedies, successes, and failures can be compelling, they often become the de facto version of the truth, due in large part to the absence of reliable, accessible information at the various levels of the university and a dearth of resources with data expertise. According to a 2016 Survey of Chief Academic Officers, only 23 percent said their institutions were very effective at identifying and assessing student outcomes, and only 24 percent said they were very effective at using data to inform campus decision making. According to another recent report surveying senior higher education leaders, 36 percent of colleges have sufficient data but outsource analytics because they lack that skill internally. Fewer than a third have both sufficient data and the resources to analyze them for strategic and operating decisions. And 22 percent say they have sufficient data but fail to incorporate them effectively in decision making. Clearly we are drowning in data, but we are starving for information.

So perhaps the first step in creating a culture that is more than just “data rich, information poor” is to shift the emphasis from the typical buzzword of “data-driven” to a more advanced state of understanding: “data-informed.” Data elements by themselves are merely discrete symbols or signs; for example, there are five students registered for an upper-level mathematics course. A data-driven institution might consider cancelling that course due to low enrollment. Being data-informed balances the numbers with a person’s expertise and understanding of the data. A data-informed institution might let that course run because those five students need the course to graduate. Or maybe the course is a late-night or Saturday section, and because the Mathematics Department has been tracking their course enrollments over time, they are willing to not make the cancellation decision immediately, because they observe student registration patterns and students have historically enrolled later in courses offered in these day/time slots. This higher-order thinking gives data meaning and purpose, revealing relationships that enable institutions to ask better questions, find deeper meaning in their answers, and ultimately make smarter decisions.

Becoming a data-informed culture begins at the top of the organizational chart; leaders need to start with a strategic question before consulting or collecting data, not the other way around. The process of questioning, analyzing, and decision making should not be influenced by preconceived ideas; instead, it should rely more on the data and less on intuition or anecdotes. Communication and collaboration allow data and information to be spread throughout the organizational chart so that everyone understands that decisions are made through a thoughtful methodology instead of a random conversation in the parking lot at the end of the day. As this develops, the institution begins the enterprise of knowledge management: documenting what it knows and what it does not know. The “to-do” list of this enterprise can be lengthy, requiring a commitment of resources, mostly internal but some external as well. Building cross-disciplinary teams so that as many relevant people as possible understand the goals and process are key.  Centralizing data allows for constant updates, keeping data up-to-date and accessible. Creating metrics that are well-defined, clearly understood by the university community, and consistently measured enables an institution to tell a story that is faithful to its mission and vision. Developing formal data policies that control user access, establishing a traceable chain of custody to mitigate risk, specifying levels of access, and determining how and by whom data may be altered eliminates the “Wild West” of data control. Regardless of what a salesperson says, there is no software or service that will connect these dots. Yes, there are firms that can help guide the process, but you cannot buy a product off the shelf, bring it home and plug it in, and expect that data magic will occur. Institutions tend to jump right to the slick presentation of the moment, but it is an investment in people over tools that will provide the best bang for the buck, both short and long term.

Ultimately, data do not make change . . . people do. Building relationships across departments, educating colleagues about the value of data, and reporting on outcomes are important parts of our job. Data are not going to give you a decision. It is your experience and wisdom that lead you to make decisions. It’s not what the data say that matters, it’s what you say and do about the data that matters.

References

2016 Survey of Chief Academic Officers (http://www.insidehighered.com/news/survey/2016-inside-higher-ed-survey-chief-academic-officers)
KPMG’s 2015-2016 Higher Education Industry Outlook Survey
(http://www.edtechmagazine.com/higher/article/2016/04/survey-data-and-analytics-higher-ed-can-be-one-two-punch)

Richard L. Riccardi is associate vice president for institutional effectiveness at Southern Connecticut State University. Contact him at riccardir1@southernct.edu.