Data reconciliation overview

Measurements in a chemical process shall have errors of type random and systematic, so that the law of conservation of mass is not obeyed across in any of the mass balance envelope of the measured data.

In order to record the performance of the process, these measurements are adjusted such that they conform to the conservation laws and any other constraints imposed upon them by eliminating the faulty measurements in the reconciliation process. The process of performing the material balance which obeys law of conservation of mass and volume is known as data reconciliation.

Errors can be of the following three types:

• Balance model error due to balance points consisting of only input flows or output flows

• Missing flow error due to missing input data and process leaks

• Faulty meters due to bias or malfunction of instruments, inadequate calibration or lack thereof, or poor sampling

In a large processing plant, fortunately, there is much redundancy of measurements, so the adjustments which bring measured values into harmony with each other result in greater overall accuracy.

Having redundancy, we can increase the accuracy of the reconciled data by identifying and ignoring the faulty measurements which conflict with related other redundant measurements so that we may safely exclude the faulty measurements before performing the data reconciliation.

While most measured values are within accepted tolerances and thus are valid candidates for use as inputs in the reconciliation algorithm, if you fail to identify and ignore gross errors, all of the reconciliation adjustments are greatly affected by those gross errors. In such a case, the reconciled values would not be reliable indicators of the state of the process. Thus, gross errors must to be identified and either corrected or eliminated before data reconciliation. In other words, since meaningful data adjustments can be obtained if and only if there is no gross error in the data, gross errors must be detected and then eliminated or corrected before valid data reconciliation can be achieved.

Furthermore, identification of measurements that are outside of accepted tolerances is one of the primary useful results of data reconciliation.

For this reason, AVEVA Production Accounting identifies the set of measurements that were found to be gross errors. The gross error detection algorithm is one of the strong points of AVEVA Production Accounting because:

• The algorithm has been rigorously tested by industry and found to be very reliable at finding all gross errors.

• AVEVA Production Accounting excludes gross errors from the reconciliation algorithm automatically, rather than forcing you to rerun reconciliation after telling you that there were errors.

AVEVA Production Accounting finds all possible types of gross errors such as balance model problems, missing flows and faulty measurements with proprietary and unique algorithms before data reconciliation.

The following figure shows the overall step in data reconciliation, including gross error detection in AVEVA Production Accounting.

The other important thing in data reconciliation is mathematically reducing the complexity of the problem to be solved, because brute force methods often result in algorithms that take impractically long to complete. The solvability analysis through exclusion of gross errors reduces the size of the reconciliation problem, while extracting and retaining the redundant flows which affect the data reconciliation.

In the data reconciliation modeling of chemical processes, measuring instruments are placed in some but not all of the streams. Fortunately, redundancy and the fact that the inputs and outputs of a balance point must equal each other often permit the algorithm to infer the values of unmeasured streams. However, such inference must take place after the reconciliation of measured quantities has taken place, because it depends on comparisons of balanced results. After reconciling measured quantities, AVEVA Production Accounting fills in all unmeasured stream values that can be inferred, which means they are solvable despite being unmeasured. Any unmeasured streams whose values cannot be inferred are clearly identified by AVEVA Production Accounting as non-solvable.

Data reconciliation requires redundancy. Reconciliation can only be done if all of the flows are measured around some part of the process, such that all input flow quantities and output flow quantities are known. The sum of the inputs must equal the outputs. Theoretically, one can infer (at least one of) the output quantities if all the input quantities are known. In this case, while AVEVA Production Accounting will perform the inference, the data is not truly reconciled because there was not adequate redundancy.

AVEVA Production Accounting will infer where necessary and reconcile where it can, exactly as a human Production Accountant would do – only with greater speed, accuracy, and repeatability. In practice, the final set of reconciled and inferred measurements may be referred to as the “reconciled” results, because they are the best set of values that mathematics can produce.

Why does redundancy produce accuracy? Any single meter measurement is guaranteed to be at least slightly biased, because a perfect meter does not exist. By measuring the same flow with multiple meters, we use the law of averages to cancel out bias. Using the data from multiple meters also allows you to use all of the information present in your plant, rather than relying only on a select few. It is the same principle that caused seamen to take three chronometers to sea rather than one.

Data reconciliation is a constrained minimization problem of constrained least squares. When balancing either mass or volume alone, the constraints are linear. However, when balancing mass and volume simultaneously, the constraints are nonlinear. The objective function is usually a quadratic form in the adjustments to the measurements, which is a general weighted sum of squares.