Process Mining as an Epistemic Prerequisite for Automation

Most automation projects that fail were built well enough, and the trouble lies in what the team knew about the process. They build a system that faithfully runs a process nobody actually follows, and then spend the rest of the budget finding out how the real one differs. Our argument is that process mining, meaning evidence from event logs about how the work really runs, has to come before automation and cannot be left as a later optimisation. Without conformance checking, the automated system encodes the organisation's picture of the process, and that picture can be far from what people do. This article separates the designed, the described and the executed process, gives a minimum data specification for telling them apart, and offers a decision rule for when watching the work directly is cheaper than instrumenting it.

The Three-Process Gap: Designed, Described and Executed

Every process an organisation believes it runs exists in at least three versions, and the distance between them is the main risk in an automation programme.

The designed process

The designed process is the intended one: the flow modelled when the system was procured, the swimlane diagram in the requirements document, the state machine encoded in the ERP configuration. It says how things should go. It is usually internally consistent, because it was written in one sitting by people who were thinking about the process at the time, not doing it.

The described process

The described process is what people say they do when asked, in interviews, workshops, or documentation exercises driven by quality-management requirements such as ISO 9001. Nobody is lying, but people reconstruct the work from memory, and the reconstruction is biased in predictable ways. They describe the most common path, leave out exception handling, shorten the waiting time, and present the version of the work they think is officially approved. The workarounds they invented to keep the designed process working in real life are the details they are least likely to mention.

The executed process

The executed process is the sequence of state changes that occurred, case by case, with real timestamps. It includes the rework loops, the four attempts at a single approval, the case that sat untouched for eleven days, and the parallel spreadsheet nobody mentioned. Of the three, it is the only one that records what happened, while the other two are claims about it.

Automation turns a process specification into software. If the specification comes from the designed or described process and the executed process is materially different, the system you build automates an account of the work while the work itself carries on around it. Hammer's argument for redesigning processes instead of automating the existing ones (Hammer, 1990) is usually read as a call for ambition. It is also a call for evidence, because you cannot redesign a process before you know what it is. Davenport and Short (1990) made the related point that information technology and process redesign are one problem, to be worked on together and not one after the other.

An example makes the shape of the gap easier to see. This one is constructed, and its numbers are assumptions chosen to keep the arithmetic simple. Suppose an operations lead describes a handoff step as a fifteen-minute task, and timing the step on the floor over repeated runs puts it closer to forty-five minutes. A gap like that is rarely exaggeration. Say the explanation is three spreadsheets sitting between the two systems named in the documentation. A different person maintains each one, each exists because the designed process has no way to represent a condition that comes up often, and none of them appears in any diagram. Automating the documented step would then cover roughly a third of the actual work and break the rest, which nobody wrote down. That is the failure this article is about, and the reason to think carefully about where an automation programme starts.

Event Log Requirements and the Minimum Viable Event Log

Process mining needs only a modest data structure. An event log is a set of cases, and a case is an ordered sequence of events. At minimum, an event says that a named activity happened for a named case at a known time. So the minimum viable log has three mandatory columns:

case_id, activity, timestamp
ORD-10231, order_received,  2026-01-14T09:12:03+03:00
ORD-10231, credit_checked,  2026-01-14T11:40:55+03:00
ORD-10231, order_received,  2026-01-15T08:03:12+03:00

Look at the third row. The same order was received a second time, which no diagram will ever show and a log shows immediately. Optional columns add analytical power without changing the model. A resource identifier lets you analyse handovers, a start and complete pair separates time spent queueing from time spent working, and case attributes let you segment. Van der Aalst (2016) is the standard reference for the discipline and its algorithms.

The hardest decision is about definitions: what counts as a case. The same dataset can be analysed per order, per order line, per shipment or per customer, and each choice gives a different process from identical data. A rework loop you can see at line level disappears once lines are rolled up into orders. Pick the unit whose cycle time the business cares about as the case, and write that choice down before generating any model.

Timestamps also tie the log to classical operations theory. Little's Law (Little, 1961) states that L = λW: the mean number of cases in progress equals the arrival rate multiplied by the mean time in the system. Interview-based documentation cannot support this relation, because the durations people describe are touch times and the law needs elapsed time. An event log gives W for every case directly. Work in progress becomes something you measure, and you can locate the constraint in Goldratt's sense (Goldratt, 1984) from the data instead of guessing it from where people feel busiest.

Discovery, Conformance and Enhancement Are Different Questions

People often lump these three uses of an event log together as "we did process mining", and afterwards nobody can say which question was answered.

Discovery

Discovery asks what process the log implies, without a reference model as input. The output is a model built from observed behaviour. It is the right question when there is no credible design, or when the design is old enough that it should be treated as a hypothesis.

Conformance checking

Conformance checking asks how the log differs from an existing model. It returns deviations: activities out of order, mandatory steps skipped, forbidden transitions taken, and cases that cannot be replayed at all. For automation this is the question that matters, because it is the only one that puts a number on the three-process gap. A single fitness figure is only where the analysis starts. You work from the list of deviating behaviours, each with the number of cases it affects.

Enhancement

Enhancement asks what the log can add to an existing model: where the time goes, where rework concentrates, which handovers come before a delay. Run it on a model that was never conformance-checked and you get confident performance statistics about a process that may not exist.

Variant Explosion and the Long Tail of Process Paths

The first model you discover from real data is almost always unreadable, because real processes have far more distinct paths than any diagram shows. Each unique sequence of activities is a variant, and a process documented as a dozen boxes can easily have hundreds of them.

The next example is also constructed for illustration. Assume a log of 12,000 cases from a mid-sized order-to-cash process, with 640 distinct variants and a distribution typical of long-tailed operational data.

A distribution like this has two consequences. First, straight-through automation of the happy path covers roughly half the volume, so a business case built on the total case count is overstated by about a factor of two. This is one reason the return calculation belongs in a range rather than a point estimate. Second, you cannot simply clean the tail away. Part of it is data-quality artefacts and part is real variation that the designed process cannot express, and only looking at the cases tells you which is which. Deming's argument that variation has to be understood before anyone acts on it (Deming, 1986) applies directly here. An exception path removed by decree tends to come back as an undocumented workaround. When the same nominal process takes different paths at different sites, Conway's observation that systems mirror the communication structures that produce them (Conway, 1968) is a useful starting assumption. The process may be splitting along reporting lines instead of along what customers require.

What Process Mining Cannot See

Process mining sees digital traces, so it is blind to any work that leaves none.

Work that is not digitised does not show up. A physical inspection, a conversation at someone's desk or a phone call to a supplier produces no events. The log shows a gap with no explanation, and an analyst who reads that gap as waiting, when people were in fact working, will get the constraint wrong. Shadow systems are invisible for the same reason. Spreadsheets, local databases and messaging threads carry a substantial share of the real coordination, and none of them write to the systems being mined. In the handoff example above, most of the elapsed time is shadow work. A log of the two named systems would show a clean, fast trace that is completely misleading.

Judgement stays invisible even when its result is logged. The log records that an approval was granted at a certain moment. It does not record which criteria the approver weighed or what unwritten rule they applied. For automation this is the blind spot that matters most, because the steps most worth automating in economic terms are often the ones whose logic exists only in the approver's head. Logs also have artefacts of their own. Batch writes destroy ordering, overwritten status fields lose the intermediate states, and a "user" in the resource column is sometimes a shared service account.

A Decision Rule for Observation Versus Instrumentation

Instrumentation, meaning adding logging or event capture so the process can be mined, is the lasting answer, but it takes weeks or months to put in place. Direct observation, in the industrial-engineering sense of standing next to the work and timing it, takes days, and the sample it can produce is much smaller. Most projects end up needing both. Which one comes first depends on how much of the process is already digital and on what is at stake.

observe_first if:
    digital_coverage < ~70% of steps
 or shadow_tooling_suspected
 or case_notion_still_disputed
 or decision_needed_within_weeks

instrument_first if:
    digital_coverage >= ~70% of steps
and process_is_high_volume_and_repetitive
and conformance_must_be_monitored_after_go_live

In practice the order looks like this. Agree on the case notion and on what a case is worth before touching any data. Pull whatever log exists and check its coverage against a small observed sample, not against the documentation. Where coverage is poor, observe first and use what you see to decide what to instrument. Run conformance checking against the designed model before scoping any automation, and scope the work variant by variant. Keep conformance checking running after go-live, because an automated process drifts as its inputs and the systems around it change.

Our automated quality-control pipeline for a collectible card producer followed this order. We did the mapping before the build. The pipeline covers 90+ validation checkpoints and roughly 95% of the QC process. The manual pass that used to take eight hours now takes about fifteen minutes, and the production cycle is approximately 300% faster. A follow-on pre-press tool converts approved designs into press-ready montage files. The checkpoint count is the relevant figure here. Each checkpoint covers something that could go wrong, and we found each one by examining what actually happened, not by reading a diagram.

Limitations

We have not shown that process mining makes automation succeed. The argument is a logical one: conformance checking is the only available way to measure the gap between specification and behaviour, so skipping it means going ahead without that measurement. There is no controlled comparison here of projects with and without a mining phase, and no measurement of how often the gap is large enough to matter. Our own observations come from a small number of engagements in manufacturing and print production. They are field notes, not a sample, and the variant distribution above is constructed. We are also not claiming that discovered models are correct. A discovered model summarises one log over one period and inherits every artefact of that log. It is a better hypothesis than a diagram, and it should still be checked against how the work is done.

References

• van der Aalst, W. (2016). Process Mining: Data Science in Action.
• Hammer, M. (1990). Reengineering Work: Don't Automate, Obliterate.
• Davenport, T., & Short, J. (1990). The New Industrial Engineering: Information Technology and Business Process Redesign.
• Deming, W. E. (1986). Out of the Crisis.
• Goldratt, E. M. (1984). The Goal.
• Little, J. D. C. (1961). A Proof for the Queuing Formula L = λW.
• Conway, M. E. (1968). How Do Committees Invent?