How published data gets updated

How published data is synchronized with PI Data Archive

PI Integrator Sync monitors assets in the PI System for asset views, scheduled streaming views, and key-value-triggered streaming views. It keeps your published data current in the following ways:

• PI Integrator Sync monitors your view shape and tracks changes to PI AF that affect the matches in your view. For example, assume you have a view of pumps and there are 10 matches. A new pump is added to PI AF that results in 11 matches of your view shape. PI Integrator Sync publishes data for the 11 matches going forward, and for asset views and scheduled streaming views, it will backfill the data for the new pump.

• PI Integrator Sync monitors changes to data in Data Archive and updates any published data. For example, when a data value in Data Archive is updated with a new value, PI Integrator Sync publishes the new value to the target.

  Note: For a definition of the different types of streaming views, see About message triggers.

PI Integrator Sync monitors out-of-order and backfilled data, and it updates previously published data once every 30 minutes. Therefore, once a change occurs in PI AF or Data Archive, it may take up to 30 minutes for this change to be reflected in your data. If data is late to arrive in PI Data Archive and PI Integrator for Business Analytics has already published data for the time range of this late-arriving data, the late-arriving data will not be automatically updated in the destination table. To ensure that all data is published, configure a delay in the End Time parameter of the view (for example, *-1h) or manually backfill the data once it has arrived in PI Data Archive. For more information, see Update data manually.

To take advantage of the PI Integrator Sync synchronization features, you must be using Data Archive 2017 or later.

The following table shows each target and the type of synchronization supported.

Note: For Data Lake and file targets, PI Integrator Sync monitors changes to the shape of asset views, and publishes the data for the matches going forward. However, PI Data Archive changes are not supported. Therefore, PI Integrator Sync does not backfill data for these targets or update changes to published data. The exception to this is Azure Data Lake Storage Gen 2 which supports both the asset view shape and changes to the data.

Note: Streaming targets offer sync support for scheduled streaming only. Key-triggered streaming views do not have sync support.

How published data is synchronized with PI AF

The following describes how published data is synchronized with changes to the PI AF hierarchy:

• Element is added to PI AF

  Note: Make sure all changes made in PI System Explorer / AF Server have been checked-in before creating a view that targets those changes. Elements that did not match your view shape upon creation, that are later modified to match your view shape, will not automatically be added to your existing view(s). Additionally, if the view shape contains attributes that are not marked as optional in the view configuration, these attributes must be present on the element upon the element’s creation for that element and the corresponding attributes to be added to your existing view(s).

  • Asset views and scheduled streaming views – If an element or combination of elements and attributes is added to PI AF that results in a new match in your view, data for the new element is published going forward and data is backfilled to the original view start time.

  • Key-value-triggered streaming views – If an element or combination of elements and attributes is added to PI AF that results in a new match in your view, the data for the new element is only added going forward. Previously published data is not backfilled.

• Element is deleted from PI AF – If an element is deleted from PI AF that changes the number of matches in your view, the data for the element is not published going forward. However, previously published data is retained.

• Element is renamed in PI AF – If an element is renamed in PI AF, then the element is automatically renamed in the view. The new name appears in published data going forward, but previously published data retains the old element name.

  Note: AF identity permissions changes on existing elements are not tracked by PI Integrator Sync. For example, if AF security permissions are added to an element currently not being published by a view, data for this element will not be automatically published going forward, nor will it be included in data backfills. If AF security permissions are removed from an element, subsequent publications will still be able to access the element data despite the revoked permissions. To reflect AF identity permissions changes on an existing element, views referencing that element will need to be republished, or the element itself must be recreated. For details on which AF identity the PI Integrator Sync uses, see Data security.

The following describes how published data is synchronized with out-of-order and backfilled Data Archive data for each type of view:

• Asset views – When there are changes to the PI Data Archive data, the target data is updated automatically. PI Integrator Sync overwrites the data in the target.

• Scheduled streaming views – If there is a change to a data value, the new value is published to the target. However, the previously published value is not deleted. The Backfill data option in the Modify View page has to be set to enable this functionality.

• Key-value-triggered streaming views – Updating data is not supported for key-value-triggered streaming views.

• Event views – Event views do not capture change data and out-of-order data automatically, but that type of data can be published manually using the update data feature (button.) The time range you specify in your manual data update will capture all event frames that have an EndTime inside of that range.

Manual updating

You can manually update data regardless of your PI Data Archive version.

If you are using PI Data Archive 2016 R2 or earlier, however, PI Integrator for Business Analytics does not support automatically updating data. Therefore, manually updating data that has changed since publication is the only option available.

You can only manually update data in asset views, event views, and scheduled streaming views for supported targets. You cannot update data in views that are published only once or published to a file or HDFS target, and you cannot update key-value-triggered streaming views.

You only need to manually update data if the data has changed since it was last published to the target data store. Any changes to the data that occur before a scheduled publish are written to the data store with the latest values.

These are some circumstances that might cause the data to change:

• Data was being buffered on an interface node at the time the view was published.

• Data is backfilled or recalculated. This often occurs with PI AF Analytics tags.

• Data is manually entered after the view was published.