Wouter Neef
5 min read
December 19, 2024
Data literacy is a container concept for a lot of different skills. To understand if your data literacy program is creating business value, you should look at data literacy program indicators. Two concrete examples of these indicators are the number of course completions by your employees or the average assessment scores for the completed data literacy training. These program indicators give you high-level insights on whether your data literacy program is being completed by learners.
While it is difficult to directly assign a monetary value to completing data literacy courses, it is an early indicator of how your data literacy training program could enable value from data-driven working. For example, by measuring the number of completed data literacy courses or the average assessment scores over time, you can link them to other important business metrics such as revenue growth, customer retention, or revenue per employee. If you measure this before and after rolling out your data literacy program, you can get a quantifiable estimate of the value created.
Measuring the number of course completions is rather easy and can be done for almost all data literacy programs. However, there are also more sophisticated KPIs that may be harder to measure but may be more suited for your business to quantify the value created.
To measure if learners not only complete the data literacy training but also change their way of working, you should measure tool adoption rates. By measuring tool adoption rates of e.g. Tableau, Power BI, Google BigQuery, or Thoughtspot, you have a tangible way to quantify the effectiveness of your data literacy program in changing work behavior.
Ideally, you would like to measure the adoption rates, pre-training, and post-training so that you can accurately tell if, and by how much, the tool adoption rates have changed. KPIs that work well as a measure of tool adoption are:
The weekly or monthly average number of users of tools will show you what percentage of people use certain tools, and how often. This will give you better insights into what tools are used most often. And if the tools are fully utilized. Also, the number of queries or quantity of data queried will give you more insights into how much data your employees are using for their work. And the number of people with access to dashboards will give you more insights into how many employees are using descriptive analytics. For example, to track certain business objectives or to make more data-driven decisions.
In the end, you can relate all these metrics back to the business performance metrics of the entire organization or specific teams. For example, is there a noticeable increase in revenue when sales managers are using dashboards more (frequently)? Or is there an increase in marketing ROI, when marketers use data tools more frequently? These associations can give you a better understanding of how your data literacy program is influencing business metrics through tool adoption rates.
Not every business case is about the value you create; some are about the cost you avoid. If your people cannot read, question and use data, that shows up as real work: reports that get rebuilt because nobody trusts the first version, decisions that wait a week for an analyst, and dashboards that quietly duplicate each other.
You can put a number on that. Ask a sample of teams how much time per week they spend finding, checking or reworking data, multiply it by the loaded cost per hour, and you have a defensible estimate of what the gap costs today. Two other numbers are usually easy to get from your data team: the size of the request backlog, and how many of those requests are questions people could answer themselves after training.
Use these figures as your baseline. Measured again six to twelve months into the program, a shrinking backlog and less rework are among the most convincing signals a leadership team will accept.
A data literacy program should make answers arrive faster. Time-to-insight, the time between someone asking a question and having an answer they trust, is a metric business stakeholders recognize immediately.
Measure it on a fixed set of recurring questions before the program starts, then again afterwards. Track it alongside two supporting numbers: the share of questions answered without help from the data team (your self-service ratio), and the average time a request sits in the queue. When literacy improves, the first number goes up and the second goes down.
Hard metrics tell you what moved; people tell you why. Send learners a short survey before and after the program about how confident they are working with data, and how often they use it in their week. Do not stop there: ask their managers what they see differently in the work, because self-reported confidence and observed behavior are not the same thing.
Keep it to a handful of questions, ask the same ones every cohort, and report the results next to your business metrics. Together they answer the question your leadership team actually has: is this changing how we work, and is it worth continuing?
On 24 September we are hosting the 10th Data & AI Literacy Round Table in Utrecht: HR and data leaders in one room, on who owns AI literacy and how you prove it worked. Invite-only, 30 to 40 seats.
Member of
Nominated
