Scenario 8: Fill in data gaps

In a perfect world, instruments and data acquisition always capture measurements reliably, but the real world is full of interruptions. In some environments, these interruptions are so routine that they interfere with operational reporting. You can use the optimistic retrieval feature to help fill data gaps. For example, you could answer:

• If there is no data available at midnight when my report runs, how can find the most recent value before that?

• How can I fill in the gaps in my trend caused by unreliable communications?

• How do I make a "best guess" for information lost because of a communication failure between my equipment and the application server?

What is Optimistic Retrieval?

In addition to capturing data values, AVEVA Historian records the data quality (OPC quality) of a value. AVEVA Historian also marks any disconnects with NULL values and an associated QualityDetail of 10 and 24. By default, these appear in trends as gaps and in reports as NULL values .

Details about data gaps and less-than-optimal data quality are important for diagnosing communications problems and for supervisory control. However, there may be gaps in the good quality data, making it hard to create the reports you need to have. For example, it would be difficult to create a report with a total for yesterday's production or the raw material consumption at midnight if there were communications problems.

Using optimistic retrieval allows you to make a best guess without tampering with the underlying data.

Using the Optimistic quality rule lets you retrieve information that is possibly incomplete but likely to help fill in some gaps in the good quality data. This setting calculates using the last known good value before the gap (if possible). The logic for determining the percent good, value count, and OPC quality remains unchanged.

How Optimistic Retrieval Works

The following illustration shows a Counter retrieval situation where three of the four shown cycle boundaries are located in data gaps. Without using the Optimistic quality rule, Counter queries return NULL at all cycle boundaries because the mode needs valid good values at each end of the cycle calculate a precise difference.

If the query specifies Optimistic, the results include a PercentGood column indicating whether all the data was "good" (100) or included some combination of "bad" and "uncertain" data (anything less than 100).

For the illustration above, there is no previous data and no available data in the first cycle, so it is skipped. At the second cycle boundary, the value 0 is returned, because there was a gap in the data for the entire first cycle. In the second cycle, there are two points, P1 and P2. The query uses P2 as the end value of the cycle and infers a start value of the cycle from P1. At the third cycle boundary, Tc2, the query returns P2 – P1. Similarly, at the last cycle boundary, the query returns P4 – P3.

Example: Masking Gaps in Your Data

At times, the results of a query might return null values, as in the following example:

SELECT TagName, DateTime, Value, PercentGood, QualityDetail

FROM History

WHERE TagName = 'BatchPercentConc'

AND DateTime >= '2015-09-16 08:30'

AND DateTime <= '2015-09-17 08:30'

AND wwRetrievalMode = 'Delta'

The results are:

You can improve the usefulness of your data by adding this statement:

wwQualityRule = 'Optimistic'

For example:

SELECT TagName, DateTime, Value, QualityDetail

FROM History

WHERE TagName = 'BatchPercentConc'

AND DateTime >= '2015-09-16 08:30'

AND DateTime <= '2015-09-17 08:30'

AND wwRetrievalMode = 'Delta'

AND wwQualityRule = 'Optimistic'

The results are: