Compression testing

PI Snapshot Subsystem uses compression testing to determine what events need to be saved in the archive. Compression testing ensures that enough data is stored to accurately reproduce an original signal. In the following illustration, all the events fall on the same straight line. In a simple case like this, there is no need to store all the points on the line. If just two points are stored, the point value can be recreated at another time.

The same principle applies to compressing real-world data. Data Archive uses a compression algorithm to determine which events to store. Compression test eliminates events which are not necessary for reproducing the original signal. The original signal is reproduced by interpolating between stored events. This compression method is called swinging door compression.

With compression there are only three values/events involved:

• Last archive value - the last value which was stored and is necessary for reproducing the original signal.

• Current snapshot value - the most up-to-date value. It is the current instantaneous state of the process (a snapshot). However, as time goes on, it is possible that the snapshot won't be needed in the PI archive.

• New value - a value which just reached PI Snapshot Subsystem and triggered compression test.

The below image shows the most recently archived value (A), and the current or snapshot value (S).

To determine if the snapshot will not be needed, the first step is to calculate the slopes. The slopes define areas of values that can safely be discarded without losing a given precision (Compression Deviation - compdev) when rebuilding the trend, based on archive values. To calculate the maximum slope and the minimum slope, the last archived value, a snapshot value, and the compression deviation settings are used. The next step is to calculate a reference slope using archive value and the new event. The reference slope is then compared to the maximum and minimum slopes.

If the reference slope is between the max and min slopes, the snapshot event is not needed and will not be archived. The new incoming event then becomes the new snapshot value, and the old snapshot value is discarded. If the reference slope is outside of the min/max slopes, the snapshot event will be archived. You may encounter one of the two following scenarios when using compression testing.

Scenario 1: Old snapshot value is not archived

If the old snapshot value is not archived, and the old snapshot value was replaced by the new event, the slopes need to be recalculated based on the last archived event, and the new snapshot event. The slopes are calculated as the new minimum and maximum slopes.

When another event is received, the same process is repeated; the reference slope is within the calculated minimum and maximum slopes. Therefore, the new event becomes the new snapshot. The minimum and maximum slopes need to be recalculated. However, this time the new min and max slopes will be slightly different.

One of the rules of compression is that these angles must always be narrowing; the newly calculated minimum and maximum slopes need to always be between the old slopes (the narrowing angle). The newly calculated minimum slope, Smin, exceeds the old minimum slope, Smin-old. Therefore, the Smin-old slope is used as the new Smin. This is shown in the picture below:

Below are the resulting new Smin and Smax slopes.

The process of replacing snapshot values with the new values and narrowing the angle repeats until the Scenario #2 is encountered.

Scenario 2: Old snapshot value is archived

The second scenario you might encounter will resemble the image below. The next snapshot value has arrived, and it is outside of an acceptable angle. The old snapshot value will be sent to the archive, and the new event becomes the new snapshot.

The process can now be repeated, new minimum and maximum slopes are calculated using the new archived value (old snapshot) and the new snapshot value.

Tag attributes related to compression

Each point has three attributes that comprise the compression specifications: CompDev or CompDevPercent (compression deviation), CompMin (compression minimum time), and CompMax (compression maximum time.)

An event bypasses the compression process and is put in the event queue in the following instances:

• The Compressing attribute for the point is set to Off.

• The event's time stamp is older than the time stamp of the current snapshot. Such an event is considered out of order.

• The event is annotated.

Swinging-door compression is not used when the Step attribute is set to On. Instead, an exception calculation is applied using the CompDev value. If the absolute difference between the current snapshot and the last archive value is greater than CompDev, then the snapshot is sent to the archive. For more information, see Step attribute in PI points.

Note: For more information on compression testing, search YouTube for "Exception and Compression" on the AVEVA PI System Learning channel.