Transfer event frames to CONNECT data services (preview)

PI to CONNECT now enables you to transfer event frames from a PI Asset Framework server to events in CONNECT data services. This feature is in preview, meaning that transferring event frames to CONNECT is not available as a general release to all AVEVA customers, but is offered to certain customers in a state that can be previewed for feedback and validation prior to any general release.

Transfer event frames to CONNECT data services to take advantage of the scalability, flexibility, and advanced computing that CONNECT offers. Customers may transfer event frames with PI to CONNECT to:

• Centralize events from across the corporate ecosystem in a single, easily accessible location.

• Take advantage of AVEVA Advanced Analytics to perform analysis on event data.

• Expose event data to custom or partner applications.

PI to CONNECT will transfer any event frames that are referenced by assets in an existing PI to CONNECT transfer.

Installation

Version 2.4.2520 of PI to CONNECT Agent is required to transfer event frames to CONNECT.

If your PI to CONNECT Agent is already configured to transfer AF elements for an AF server, then upgrading the agent is the only agent-side step that you need to perform.

Transfer Event Frames to CONNECT data services

Configuration

When the TransferEventFrames feature flag is enabled for your tenant, a new checkbox will be available in the Transfer Settings window labeled Transfer Event Frames. For new transfers, this checkbox defaults to selected. However, for existing transfers on upgraded instances of PI to CONNECT, the checkbox will be deselected and you will need to edit the existing transfer. Open the Transfer Settings window for your PI to CONNECT transfer and select Transfer Event Frames.

You can deselect the Transfer Event Frames checkbox at any time. Deselecting Transfer Event Frames will not delete any event frames that have already been transferred and will merely suspend the transfer of additional event frames to CONNECT.

Select event frames to transfer

PI to CONNECT Agent will transfer any event frames that are referenced by AF elements selected for transfer. One can think of these event frames as being “directly included” for transfer. To give additional context to these directly included event frames, their parent and child event frames will also be transferred, regardless of any element references they might have. Moreover, the entire parent/child hierarchy is traversed to include grandparent/grandchildren, great grandparent/great grandchildren, and so on.

Open event frames that have previously been transferred, according to the rules above, will continue to be transferred until they are closed, regardless of any element references they might have.

Transfer of events between PI AF and CONNECT data services

When the event frame transfer begins for the first time, the PI to CONNECT Agent makes a call to PISystem.GetFindChangedItemsCookie. Any event frames that are created or changed after this call is made will be transferred, if they meet the criteria discussed under Select event frames to transfer, and if they are not excluded for reasons discussed under Limitations. If the transfer is edited to include additional AF elements in the transfer, event frames that reference these elements will start to be transferred from the point-in-time that the transfer was edited.

When the agent is stopped, the cookie is saved. When the agent is restarted, the cookie is read and the transfer resumes. Event frames that were created or updated while the agent was stopped are recovered if the shutdown period does not exceed ClearChangeTable days, as described under ClearChangeTables setting.

PI to CONNECT Agent does not perform history recovery of event frames. For example, if a transfer is started with a start date of two months ago, the agent will not perform event frame history recovery for the last two months. The agent will only transfer event frames that have been created or edited after the event frame transfer has started.

To capture historical event frames with the PI to CONNECT Agent, events can be either edited or recreated on the AF server after the PI to CONNECT transfer is started. For example, once a transfer is started, a user can backfill/recalculate analyses on their AF server to transfer them to CONNECT. Similarly, PI Batch Interfaces can be run in recovery mode with a simple configuration change to transfer them to CONNECT.

Limitations

• CONNECT visualization has limited support for viewing transferred events. You can view events using API calls with the GraphQL console or API Console.

• Event frame transfer limitations:

  • History recovery is not supported for event frames. Only those event frames that are created after the transfer is started will be transferred.

  • Event frames are "selected" for transfer only by including elements in the transfer.

  • Event frame deletes are not transferred.

  • Event frames without templates are not transferred. Furthermore, only those event frame attributes that are templatized will be transferred. For pharmaceutical and life science customers, this means that not all event frame attributes may be transferred on batch event frames.

  • Event frames that have two properties that coincide to the same GraphQLName are not transferred. See Event data overview and GraphQLName notes below for details.

  • Only point-in-time (or static values) are transferred. These are analogous to captured values on the event frame in AF.

  • Attribute hierarchy on an event frame is flattened, analogous to the attribute representation in transferred AF elements. For example, if Attribute1 has a child Attribute2, then Attribute2 is transferred with the flattened name Attribute1|Attribute2.

  • Only “basic type” or “enumeration set” attributes are transferred. The basic types include Boolean, Byte, DateTime, Double, Guid, Int16, Int32, Int64, Single, and String. “Object type” or “array type” attributes cannot be transferred. While only “basic type” or “enumeration set” attributes can be transferred, the data reference can be None (a static attribute), PI Point, Formula, String Builder, URI Builder, or Table Lookup.

  • Some properties on event frames are not transferred. These include:

    • Annotations

    • Categories

    • Extended properties

    • Traits

    • “Secondary” element references

      Although “secondary” element references are not transferred, the secondary references are still considered when determining which event frames are included in the transfer. For example, if an event frame has a primary reference to element1 and a secondary reference to element2 and only element2 is included in the transfer, the event frame will be transferred even though element1 is not included in the transfer.

  • Attributes that correspond to reserved names in the events service will have "userDefined_" prepended to the name. Reserved names include:

    • assetDatabase, path, acknowledgedBy, acknowledgedDate, severity, isRoot, id, modifiedDate, createdDate, createdByUser, authorizationTags, name, description, startTime, endTime, duration, eventType, state, and asset

• Event frame template limitations:

  • Event frame template deletes are not transferred.

  • Two event template names in AF can coincide with the same GraphQLName due to sanitization. See GraphQLName notes. These event templates will not be transferred.

  • Event frame templates that have two properties that coincide to the same GraphQLName are not transferred.

  • Default values of attributes on event frame templates are not transferred because event types in CONNECT cannot store default values.

  • Attribute hierarchy is flattened, analogous to the Element Template transfer behavior.

  • Event frame template hierarchy inheritance is flattened. If an event frame template inherits from a base template, the attributes from base templates are flattened into the derived template, and the resultant flattened derived template is transferred.

  • Attribute templates that correspond to reserved names in the events service will have "userDefined_" prepended to the name.

  • Changes to the data type of attribute templates are not transferred.

GraphQLName notes

• Event frame GraphQL names will be sanitized to include only [A-Z][a-z][0-9][_].

• Leading [0-9] and leading [_] will also be sanitized.

• Sanitization of names into GraphQLNames does not affect display names. The problems/limitations introduced by sanitization are related to possible side effects, rather than display names in the cloud. An event type in the cloud can fail creation in the cloud if:

  • The sanitized event type name (GraphQLName) coincides with another existing event type with the same sanitized name.

  • The sanitized name of any two event properties collide. For example, the attribute names "1Attribute" and "2Attribute" will both be sanitized to the same property name "Attribute" and creation of the event type in the cloud will fail.

ClearChangeTables setting

ClearChangeTablesDays can be set in %ProgramData%\osisoft\pitoocs\appsettings.json under PIAssetFrameworkServerNames. This setting corresponds to the ClearChangeTables setting that can be configured in C:\Program Files\PIPC\AF\AFService.exe.config. By default, both ClearChangeTablesDays (in appsettings.json) and ClearChangeTables (in AFService.exe.config) are 7 days. If the user updates ClearChangeTables, they can manually adjust ClearChangeTablesDays to be the same. Keeping them in synch does not change agent behavior other than messaging. For example, if the user changes ClearChangeTables to 1, then the user can also change ClearChangeTablesDays to 1 so that the agent will log a warning message if the agent cannot retrieve change notifications for AF Elements for more than 1 day, indicating that some AF Element changes may be missing from the transfer.