Organizations are investing in AI training, tools, and new ways of working, but deciding where to focus next isn't always straightforward. Should the priority be building employee skills, redesigning workflows, or scaling what teams have already built? And how can leaders tell whether those investments are paying off?
AI usage data has been the early metric for companies trying to answer some of these questions, but it doesn't tell the whole story. A team might use AI every day without improving the quality or efficiency of its work. Another might be making progress on a high-value workflow that hasn't yet produced a measurable financial return.
Measuring AI's business impact means choosing metrics that help leaders make three decisions: where to invest, whether the benefits justify the cost, and whether the organization is making meaningful progress.
The right metrics will depend on the business outcomes you're pursuing and how far your teams have progressed with AI. In this article, we explore how to put this decision-centered measurement approach into practice.
What metrics help guide AI investment decisions?
When it comes to investing in AI, the options are seemingly endless. You could change or upgrade your tech stack, publish guidelines for employee use, train employees, assess their AI fluency, and more.
Ultimately, you’re making a bet as to what investment will get you closer to your desired outcomes. In order to make the most informed bet, many companies now are looking for benchmarks to understand how they’re doing in the context of their peers.
At Clarinet, we’ve created an AI Maturity Model to help organizations understand where they are now versus where they want to be and where they believe their competitors to be.
The right AI metrics change as an organization matures. Early on, progress might mean giving employees access to approved tools, increasing weekly AI use, and building the skills to use those tools effectively. As teams become more fluent, measurement should shift toward improvements in individual productivity, workflow quality, and the ability to connect AI to relevant business data.
At more advanced stages, organizations can measure whether important workflows have been redesigned, whether teams have enough people capable of building with AI, and whether those changes are improving cycle times, output, and business performance.
This progression isn't necessarily uniform across an organization. Engineering may be redesigning entire workflows while another function is still establishing baseline AI fluency. Understanding where each team stands helps company leadership identify what needs investment next.
If you're working with an AI consulting firm, they should be able to help you establish these measures, connect their work to your business priorities, and demonstrate progress throughout the engagement. The following questions can help you evaluate whether a potential partner is equipped to do that.
- What baseline will you establish, which records will you use, and who will own measurement after the engagement?
- How will training change a specific workflow, and how will you check the quality of the work?
- What support will employees and managers receive after training to make the new workflow stick?
- Which customer results can you substantiate, including the group measured, the time period, and other changes that may have contributed?
- What enabling costs and internal commitments are included in the proposal, and which are assumptions?
Board-level reporting on AI investments
Getting clear on your metrics is also how senior leaders can make the case to board and executive teams on continued AI investment. Boards need a clear picture of where resources are going, what progress looks like, and which priorities deserve funding.
Clarinet Co-founder Dhruv Singh uses the rocks, pebbles, sand, and water analogy to explain how to communicate that progress. The rocks are the few organization-wide priorities that warrant board-level attention, such as building AI fluency across the workforce. Pebbles, sand, and water represent smaller improvements across teams and everyday tasks.
“Organizations that are taking purely a top-down approach in this are going to get blown out of the water in the long term,” Dhruv said.
His point is that while boards need to focus on the biggest priorities, the cumulative value of smaller improvements can be substantial. Leaders need metrics that demonstrate progress on foundational investments, such as workforce AI fluency, alongside evidence of how those capabilities are improving work across the organization. That gives them a stronger case for continued funding without requiring the board to track every individual initiative.
How do you calculate the ROI of AI investments?
“What is so hard about calculating the ROI of AI at this moment in time,” says Diane Sadowki-Joseph, Clarinet’s Co-founder and Head of Product, “is that we are still so early in figuring out what this technology is going to mean for companies.” This leads many organizations to want to skip foundational stages like fluency, but they’ll often find they won’t make it far without it.
AI transformation aligns to a J-curve, where the investments that matter most won’t necessarily pay off right away. Based on our work with companies on AI strategy and market reports, we predict that companies are past their early wins and on the cusp of the dip that comes after early wins.
So the question becomes, how can you be ahead of the curve?
It comes down to how to decide where it makes sense to invest, and in doing that, avoiding the biggest mistakes we see people make:
- Running before you crawl or walk. You can accelerate the time between stages, but you can’t skip the foundations: investing in training people and data readiness.
- Setting AI as a budget line independent of people. AI is not operating independently yet, the value comes from how people use it. Measuring ROI means evaluating whether the combination of people and technology is improving the speed and quality of work.
Avoiding these mistakes starts with understanding what you're investing in and what a reasonable return looks like. Some AI initiatives have a relatively direct connection to business outcomes, making their impact easier to measure. Others require foundational investments in employee skills, data infrastructure, and tooling that will support improvements across multiple workflows over time. Expecting the return from one application to justify all those costs can make a worthwhile investment appear unsuccessful.
Consider a sales team that wants to use AI to spend less time on administrative work and more time building customer relationships. A business case should account for the training, data access, and tools required, along with the expected impact on proposal turnaround time, sales conversion rates, or other existing performance measures. Leaders can compare results before and after implementation, just as they would when evaluating sales training or a new CRM. The key is to establish a realistic timeline for improvement and identify what progress should look like along the way, rather than expecting an immediate financial return.
Use AI to build and pressure-test an investment case
AI can help you estimate the costs, returns, and timelines of a proposed investment, but it becomes more useful when you incorporate the judgment of the people responsible for approving it.
Diane used AI to develop a hypothetical business case for improving a sales team's performance. She then asked Dhruv to evaluate the proposal as he would an actual investment. His questions went beyond the projected ROI: What alternatives had been considered? What were the risks? Were the timelines realistic? And what made this approach a better bet than the alternatives?
Diane used his feedback as an input into AI to create a reusable prompt for evaluating future business cases and identifying progress measures at 30 days, 90 days, six months, and beyond. It's an approach any organization can replicate: involve decision-makers in defining what a convincing business case looks like, then use AI to apply those criteria consistently.
Try the prompt below, then discuss the results with your leadership team or AI steering committee.
How Clarinet helps organizations measure AI impact
Measuring the impact of AI starts with understanding where your organization is today, what you're trying to achieve, and which investments will help you get there. At Clarinet, we work with leaders at midsize companies to answer those questions and identify the measures that will help them evaluate progress.
Our AI Fluency Assessment establishes a baseline of employees' applied AI skills, while our strategy and advisory work helps leaders prioritize investments based on their organization's goals and readiness. Through hands-on training and workflow development, we help teams put those capabilities to work and connect improvements to business outcomes.
We also recognize that meaningful returns take time. Some investments can be measured through improvements to specific workflows, while others build the skills, data readiness, and organizational capabilities that make future improvements possible. We help organizations understand what progress should look like at each stage so they can make informed decisions about where to invest next.
Frequently asked questions about measuring AI impact
What metrics show whether AI training is improving business outcomes?
Measure whether employees can apply what they've learned, whether that changes how they work, and whether those changes improve outcomes such as quality, cycle time, or productivity. Establish a baseline before training and use existing business measures wherever possible.
How do you distinguish AI output quality from business impact?
Output quality measures whether AI-assisted work meets standards for accuracy, completeness, and usefulness. Business impact measures whether that work improves an operational or financial outcome. For example, producing a better sales proposal doesn't necessarily mean the team is closing more deals.
How can a company measure AI ROI six months after rollout?
Compare results against a baseline using operational records, accounting for the full cost of training, tools, data infrastructure, and employee time. Distinguish between time saved and financial value realized. If you don't have reliable baseline data, start tracking now rather than relying on AI usage as a proxy for ROI.
What should companies measure before AI investments produce a financial return?
Look for evidence that the capabilities needed to generate value are developing. This might include improvements in employee AI fluency, progress on high-value workflows, more employees capable of building with AI, and fewer barriers to accessing the data AI needs. These indicators help leaders evaluate progress while longer-term returns are still developing.
