Missing flow detection
- Last UpdatedFeb 28, 2025
- 5 minute read
Missing flows can occur for the following reasons:
-
Missing movement when an operator has forgotten to record a transfer of material from one tank to another during the course of day
-
Mis-specified movement when an operator may have inadvertently specified the wrong source or destination tank for the movement
-
Large process leaks in the unit, pipeline, tank and etc.
-
Wrong flow line connection in the data reconciliation model
These missing flows can pose a major threat to the material balance because the DR algorithm tries to force the imbalance to zero in the nodes with missing flows.
Example 1

Figure: Color-coded flows
For example, the above diagram shows the critical problem due to the missing flow in the “R_MTBE_C4RA” balance point.
There are some missing movements pertaining to the process but the data reconciliation algorithm will attempt to balance the process balance point unless we enable missing flow detection.
The reconciled values of most of the flows around this balance point show a big difference from the associated measured values. This will cause many of the measurements to be treated as faulty when, in fact, they were valid.
But if you enable the missing flow detection feature, which is performed before data reconciliation, the results will be much better, as shown in the figure below.

Figure: Results when missing flow detection enabled
The Missing Flow Detection step automatically detects the balance point with missing flows and then eliminates it from the reduced list of balance constraints to be met. That is, the process that has the missing flows is allowed to remain unbalanced.
As you can see in the figure above, if the “R_MTBE_C4RA” process is allowed to remain unbalanced because it has missing flows, the reconciled values of most of the flows around this balance point are close to the measured values.
In a real plant, it is very likely that a combination of faulty measurements and missing flows will occur concurrently. It is very difficult to detect faulty measurements prior to determining which flows are missing. Therefore, missing flow detection is done by the DR algorithm before faulty measurement detection.
Example 2

In the above example, “ME2” is a faulty measurement and the “NO5” balance point has missing flows. If we run data reconciliation without the missing flow detection step, the reconciled value of most flows are much different from the measured values. Furthermore, the “ME7” and “ME8” flows may be wrongly categorized as faulty measurements in the faulty measurement detection step even though these two flows are well matched to surrounding, redundant measurements.
But if we run data reconciliation with the missing flow detection step, the “NO5” balance point is detected as one with missing flows and therefore its balance constraints are not enforced. Next, the faulty measurement detection step detects “ME2” as a faulty measurement. Therefore the reconciled values in the rest of the flowsheet end up being calculated as close to the measured values.
How AVEVA Production Accounting handles missing flows
AVEVA Production Accounting has a proprietary algorithm to detect and eliminate the missing flows separately from faulty measurements.
A serial search scheme is used first to isolate the candidate sources of missing flows and then simultaneous identification/elimination of missing flows is accomplished. The performance and accuracy of the detection are well proven from many applications of AVEVA Production Accounting to the needs of many different, real plants.
Real-World examples of missing flow detection
The following are real examples of missing flow detection.
Example 1

Figure: Example 1 of missing flow detection
In the above figure, tank “20D415” is detected as a balance point with missing flows.
In the real-life situation modeled in the above figure, it was ultimately proven that there was a missing movement at the tank “20D415,” just as AVEVA Production Accounting indicated.
Mathematically, the missing flow detection step uses the reduced balance matrix to detect that a balance point has missing flows, rather than using the full incidence matrix. This means it detects missing flows using the reduced set of balance envelopes, not the set of individual balance points. The two tanks shown above are part of the same balance envelope, due to the presence of the unmeasured flow between the two tanks.
The missing flow detection algorithm detects that there is a missing flow in the balance envelope and then has to assign the missing flow to one of the two tanks. The algorithm chooses the tank with the larger initial imbalance, i.e. tank “ 20D415.”
Example 2

Figure: Example 2 of missing flow detection
The “2103-FB” node is detected as a balance point with missing flows.
In this real-world example, it was proven that there was a shipment from this node with 2,090,000 kg at that time.
The calculated amount of the missing flow is 2,102,556 kg, as shown by the imbalance amount for the process. It is very close to the real missing amount.
Example 3

Figure: Example 3 of missing flow detection
The “120D823” and “20D103” tanks are detected as balance points with missing flows. These tanks are widely separated from each other on the flowsheet, and they both actually have outlet streams and inlet streams. (The diagram only shows streams of relevance that had flow.)
In this real-world example, it was proven that there was an oil movement from “120D823” to “20D103” of about 350,000 kl at that time. This can be deduced from the fact that the imbalance on tank “120D823” is of roughly the same magnitude as the imbalance on tank “20D103”.
The calculated imbalance in the “120D823” tank is 347,565 kl and the calculated imbalance in “20D103” is 351,179 kl. Both amounts are close to the real missing oil movement amount.