Abstract
Many published figures for the value of AI in business are estimates: an assumed number of minutes saved per task, multiplied by a volume and a wage. The inputs are rarely measured in the business concerned, and the reader cannot check them. This paper sets out a stricter method built on a business's own records. It separates three levels of claim: an outcome that followed an AI action within a stated window; an outcome that differs from the business's own history by more than normal variation; and an outcome shown to be caused by the AI action through a comparison group assigned at random. For each level it gives the test, the sample needed and the main threats to the conclusion. It measures time saved from timings taken in the business, counting review and rework. It requires each figure to state its level, the records it comes from and what it does not show. The paper then compares MatruOS's own figures with the method.
Section 1Where common AI value figures go wrong
A typical claim reads: "Your AI team saved 120 hours this month." Behind it sits a calculation: the AI completed 1,800 tasks, each would have taken a person four minutes, and 1,800 × 4 minutes is 120 hours. Each step can be challenged.
- The minutes per task are assumed
- Four minutes may be right for one business and wrong by a factor of three for another.
- The tasks may not have needed doing
- If the AI drafts 1,800 reminders where a person would have sent 300, the saving is not 1,800 reminders' worth of time.
- Review and rework are left out
- A person still reads each draft, edits some and redoes the ones they reject.
- Freed hours may not be used
- Time saved is worth money only if it reduces cost or goes to other work of value.
Revenue claims have a further problem. "Your AI collections officer collected ₹42 lakh this month" usually means that ₹42 lakh was paid on invoices that received an AI reminder. Some of those customers would have paid on the same day without one. The figure counts payments that followed a reminder; it does not show how many of those customers would have paid anyway.
Section 2Related work
Causal inference
The potential outcomes framework, which goes back to Neyman (1923) and which Rubin (1974) extended to non-randomised studies, defines the effect of an action on a case as the difference between what happens with the action and what would have happened without it. Holland (1986) set out why only one of the two can ever be observed for a single case, so effects must be estimated across groups. Kohavi, Tang and Xu (2020) describe the practice of controlled experiments in online systems.
Quasi-experimental designs
Where random assignment is not possible, interrupted time series and difference-in-differences designs compare a measure's path before and after a change, against its own trend or against a group that did not change (Shadish, Cook and Campbell, 2002; Bernal, Cummins and Gasparrini, 2017). They are the basis of level 2 in this paper.
Measured effects of generative AI at work
Brynjolfsson, Li and Raymond (2025) studied the staggered introduction of an AI assistant to about 5,000 customer support agents at one software firm, using the firm's own records. Issues resolved per hour rose by about 15 percent on average (14 percent in the 2023 working paper), with large gains for less experienced agents and little gain for the most experienced. Noy and Zhang (2023), in a randomised experiment with 453 professionals writing business documents, found that time taken fell by 40 percent and rated quality rose by 18 percent. Dell'Acqua and colleagues (2023), in a field experiment with 758 consultants, found large gains on tasks within the AI's capability and worse results on a task just outside it. These studies measure from records or experiments, and they show that effects vary widely by task and by person. A figure from one setting should not be carried to another.
Time study
Section 5 follows the basic practice of work measurement described by the International Labour Office in Introduction to Work Study (fourth edition, 1992).
Section 3Three levels of claim
Every figure in a value report states which level it is.
| Level | Claim | What it requires | Example (illustrative) |
|---|---|---|---|
| 1. Recorded | An outcome followed an AI action within a stated window | Records of the AI action, the outcome and their order in time | ₹18 lakh paid in October on invoices within 14 days of an AI reminder |
| 2. Compared with baseline | The measure moved outside its normal range after the AI started | 12 months of the business's own history, a test against normal variation, and other changes noted | Days sales outstanding below the baseline's lower limit for three months running |
| 3. Caused | The AI action changed the outcome | A comparison group assigned at random, large enough to detect the effect | Customers who received reminders paid 9 days sooner on average than a held-out group |
Every level gives a usable figure when the report states which level it is. The error is presenting a level 1 figure as a level 3 claim.
3.1 Level 1: recorded
A level 1 figure needs four facts from the records: the AI action happened, the outcome happened, the action came first, and the gap between them was within a window stated in advance. For collections: a reminder about a specific invoice was delivered, a payment against that invoice was recorded, and the payment came within, say, 14 days of the most recent reminder.
Without a window, a reminder sent ten months before a payment still qualifies, and the figure grows with every month of history. Report the window with the figure. Showing the figure at 7, 14 and 30 days lets the reader see how much depends on the choice.
Level 1 figures are cheap and easy to verify: anyone can open the invoice, see the reminder and see the payment. They say nothing about what would have happened without the AI.
3.2 Level 2: compared with the business's own baseline
A level 2 figure compares a measure after the AI started with the same measure before, in the same business. Days sales outstanding, the share of invoices paid by the due date, time to post a supplier bill and stock-out days are all suitable.
Test against normal variation. Take at least 12 monthly values from before the AI started and set a normal range, for example the mean plus or minus two standard deviations, or the limits of a control chart. Choose one headline measure in advance. Report a change only when it stays beyond the limit for three consecutive months, or on one side of the mean for eight consecutive months; a single month outside two standard deviations happens about one time in 22 by chance. Monthly measures such as DSO are often correlated from one month to the next, which makes limits from 12 points too narrow; where enough history exists, an interrupted time series that allows for trend and autocorrelation is stronger (Bernal, Cummins and Gasparrini, 2017).
Illustration. A business's monthly DSO over the 12 months before reminders averaged 74 days with a standard deviation of 4 days, giving a normal range of 66 to 82 days. A fall to 70 days in the first month after reminders started is inside the range and is not a level 2 finding. Three consecutive months at 64 to 65 days would be.
Threats to name in the report.
- Seasonality
- Compare a month with the same month a year earlier. A textile trader's collections in the festive season are not comparable with the monsoon.
- Regression to the mean
- Businesses often adopt a collections tool after a bad quarter. Measures tend to drift back towards normal after an extreme period whatever is done, so a baseline that ends in a bad patch flatters the period after.
- Novelty
- Customers may respond to a new kind of reminder for a few months and then less.
- Other changes
- If the business also tightened credit terms, hired a collections clerk or lost a large slow-paying customer, the report must say so.
Level 2 figures measure what the owner cares about in the business's own terms. They do not prove cause.
3.3 Level 3: caused, shown by a comparison group
The most direct way to show that an AI action changed an outcome is to compare similar cases, assigned at random, that did and did not receive it.
- Take the customers with invoices falling due in a period.
- Assign a share at random to a held-out group that receives the business's usual follow-up but not the AI reminders.
- Run until both groups reach the sample size below.
- Analyse at customer level, using each customer's mean days to payment, since a customer with many invoices is one unit. Invoices still unpaid when the test ends must not be dropped: compare the share paid by a fixed day, such as day 30, or use a survival curve.
Sample size. To detect a difference δ in average days to payment, with standard deviation σ, 80 percent power and a two-sided test at 5 percent, each group needs about n = 2 × (1.96 + 0.84)² × σ² ÷ δ², which is about 15.7 × σ² ÷ δ².
| Difference to detect | Customers per group, σ = 15 days | Customers per group, σ = 20 days |
|---|---|---|
| 5 days | 142 | 252 |
| 7 days | 73 | 129 |
| 10 days | 36 | 63 |
These assume equal groups. A smaller held-out group needs more customers in total: with one customer in five held out, the total rises by about 56 percent, so detecting 5 days at σ = 20 needs about 790 customers, 158 of them held out. Use the business's own standard deviation of days to payment.
A level 3 test has a cost: the held-out customers do not receive reminders that might have helped. Keep the held-out share as small as the sample allows, limit the test in time, and run it only where the business accepts that cost. Most small and medium businesses will not have enough customers, and level 1 and level 2 figures, labelled as such, are their ceiling.
Section 4Counting money
4.1 Money collected after a reminder (level 1)
Definition. The amount recorded in a period against invoices where a reminder prepared by the AI team was delivered within a stated window before the payment.
Rules.
- Count each payment once, however many reminders were sent.
- Count only reminders confirmed as delivered, and only those the AI team prepared.
- Count only the amount allocated to the reminded invoice, up to its outstanding balance. A payment that covers several invoices counts only for the reminded ones.
- Report it as "collected after a reminder", with the window.
Read alongside the total collected in the period, it shows how much of the business's collection work the AI team is involved in.
4.2 Days sales outstanding (level 2)
Definition
Money owed by customers ÷ credit sales in the period × days in the period. Where sales are growing or seasonal, use average receivables for the period, or the count-back method, which works back through recent months' sales until it accounts for the closing balance. State which method is used.
Cash value
A fall in DSO releases about one day of credit sales for each day of the fall. Cash released × annual borrowing rate gives interest saved per year. Illustration: a fall of 8 days on ₹30 crore of annual credit sales releases about ₹66 lakh (₹30 crore ÷ 365 × 8); at a cash-credit rate of 10.5 percent that saves about ₹6.9 lakh a year. Use the business's own rate.
4.3 Orders and sales (level 1)
Orders taken by an AI member from a customer message and billed can be counted as "orders billed from AI-handled conversations". Whether the customer would have ordered anyway is unknown without a comparison.
4.4 Amounts at stake
Some value comes from errors caught: a duplicate supplier bill, input tax credit at risk, a stock-out prevented. Count these only when the record shows the AI flagged the case and a person acted on it. For input tax credit, count only where a flagged GSTR-2B difference led to credit being claimed or protected in the return. Report the total as "amount at stake"; call it a saving only when the business can show the error would otherwise have gone through.
Section 5Counting time
5.1 Measure in the business
Do not use minutes per task from a vendor, a study or another business. Measure them.
- Choose the tasks the AI team takes on, such as posting supplier bills or matching bank lines.
- Before the AI starts, time people doing each task on ordinary days. Time the same people before and after where possible, and discard the first days of observation, when people work differently because they are watched. Thirty cases per task is a practical minimum; report the interval as well as the average.
- After the AI starts, time the review of AI-prepared work, and record the share of drafts that are edited and the share rejected.
5.2 Include review and rework
For a rejected draft the person pays the review time and then does the task by hand. For an edited draft they pay the review time and the editing time. So:
Time per case with AI = review time + (share rejected × manual time) + (share edited × editing time)Use the mean for totals: a total is a count multiplied by a mean. Report the median separately as the typical case, and give an interval for the mean, for example by bootstrap.
Illustration. Posting a supplier bill by hand takes a mean of 6.2 minutes. Review of an AI draft takes 1.4 minutes; 8 percent of drafts are rejected and 15 percent edited, with editing taking 2 minutes. Time with AI is 1.4 + (0.08 × 6.2) + (0.15 × 2) = 2.2 minutes, a saving of 4.0 minutes a bill. At 1,500 bills a month that a person would have posted anyway, the saving is about 100 hours a month.
5.3 Count only work that would have been done
Multiply by the cases a person would actually have handled. If the AI drafts reminders for every overdue invoice but the team used to remind only the larger ones, report the extra cases separately as additional coverage.
5.4 Report time as time
Report hours freed as hours. Convert to money only when the business has reduced cost or moved people to work that earns revenue, and say which.
Section 6Reporting
Each figure carries four things: the figure with its unit and period; its level; its basis (records and definition, including any window); and what it does not show.
₹18.4 lakh collected after a reminder in October (level 1, 14-day window). Amounts allocated to invoices where a delivered AI reminder preceded payment by 14 days or less. Some customers would have paid without a reminder.
A line that does not meet the method:
Your AI team made you ₹18.4 lakh in October.
Section 7How MatruOS's figures compare with the method
MatruOS is a business system in which AI members work alongside the people of a business. Its Home screen shows what the AI team did and the money recorded after its work, counted from the business's records. At the time of writing these are level 1 figures.
7.1 What the product shows
- Money collected after a reminder
- payments this month on invoices where an automatic due-soon or overdue message about that invoice went out before the payment.
- Orders billed from conversations
- the total of chat orders created this month and since billed.
- Work done
- tasks the AI team finished, AI changes that took effect or that a person approved, and messages sent.
- Activity trail
- a list of everything the AI team did or asked to do, newest first, with each member's reason and whether a person approved or rejected it.
- Auto mode Home
- a business that chooses Auto mode sees a "Worth this month" total of the loops' recorded outcomes.
- Performance per member
- for each AI member, over the last 30 days by default, work finished and failed, changes approved and rejected, ratings people gave and AI used.
7.2 Where the product does not yet follow the method
- No window
- A payment counts if any automatic reminder about the invoice went out at any time before it.
- Counts sent messages
- Messages marked sent count as well as those confirmed delivered or read.
- Counts scheduled reminders
- Scheduled reminder templates count alongside messages an AI member wrote. Messages a person wrote do not count.
- Orders and messages include work no AI did
- The orders figure counts every billed chat order, including orders whose lines MatruOS read by rule and that a person billed. The messages count includes scheduled reminders.
- No drill-down
- The Home figure does not yet open into the list of payments and messages behind it; those records are in the business's books and conversations.
- The Auto mode total adds unlike things. It sums money paid after reminders, the value of invoices raised and quotes won, and the value of purchase orders placed, and it credits the loop when a person approved the step. Only the paid-after-reminder part is a level 1 money figure, counted within 14 days of the step. Invoices raised and quotes won are sales, and purchase orders are spending; none of them is money the AI team earned.
We have recorded these as product work. Level 2 comparisons and level 3 comparison groups are not available in the product and are not claimed here.
AI can make mistakes. The Home figures count records; they do not show that the AI team caused the outcomes.
Section 8Limits of the method
- Level 1 can be misheard
- A level 1 figure is accurate as stated, but readers may hear "because of". Labelling and a window reduce this; they do not remove it.
- Baselines can mislead
- A business that adopted AI during fast growth, a slump or a change of customer mix may see changes unrelated to AI. The normal-range test catches part of this.
- Small businesses have small samples
- With 40 credit customers, 20 in each group, the smallest difference a level 3 test could detect is about 13 days at σ = 15 and 18 days at σ = 20, and monthly DSO moves for many reasons. Level 1 figures and a long-run level 2 view are the ceiling.
- Time studies take effort
- A business that cannot time 30 cases per task should report the AI team's work as counts of tasks.
- Some value resists counting
- Faster replies to customers and less overtime at month end are real and hard to price. The method reports them as counts and leaves the judgement to the owner.
Section 9A tool for the method
A value-line builder that applies the four reporting fields in Section 6 and the time formula in Section 5.2, and carries the result into the Home screen at os.matruos.com, is planned for the resources library.
Section 10Conclusion
These figures are smaller than most published AI value claims. Each states its source records, its definition and its level, so an owner or auditor can recompute it.
Sources
Checked 31 October 2026.
- Neyman, J. (1923). On the application of probability theory to agricultural experiments. Translated in Statistical Science, 5(4), 465–472 (1990).
- Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701.
- Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960.
- Shadish, W. R., Cook, T. D. and Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin.
- Bernal, J. L., Cummins, S. and Gasparrini, A. (2017). Interrupted time series regression for the evaluation of public health interventions: a tutorial. International Journal of Epidemiology, 46(1), 348–355.
- Kohavi, R., Tang, D. and Xu, Y. (2020). Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press.
- Brynjolfsson, E., Li, D. and Raymond, L. (2025). Generative AI at work. Quarterly Journal of Economics, 140(2), 889–942.
- Noy, S. and Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192.
- Dell'Acqua, F. et al. (2023). Navigating the jagged technological frontier: field experimental evidence of the effects of AI on knowledge worker productivity and quality. Harvard Business School Working Paper 24-013.
- International Labour Office (1992). Introduction to Work Study, fourth edition. Geneva.
Change log
- 1.2 (31 October 2026): product gaps corrected (orders and messages; allocation already per invoice); level 2 rule unified; censoring and clustering added to level 3.
- 1.1 (31 October 2026): attribution window, delivery and allocation rules added; normal-range test and threats for level 2; sample sizes for level 3; time formula corrected to means and rework; related work added; product section rewritten to state gaps.
- 1.0 (October 2026): first version.