Identification of available points

The second phase of synchronization identifies available points. This phase is skipped if the rule for handling available points is do nothing. The flowchart in the following figure shows the steps taken by the Synchronization Engine in this phase:

Search for available points

The following paragraphs contain details for the corresponding numbered steps in the preceding figure. The Synchronization Engine follows these steps in the second phase:

1. The Synchronization Engine obtains the list of key attributes by calling the PI APS Connector.

2. Obtains a list of the PI points associated with the interface instance being synchronized. The Synchronization Engine uses the same algorithm as it used in step 1 of the first phase.

3. The list of PI points includes UniInt Performance, Health, and Failover points that do not correspond to tags in the data source. With the exception of Phase 1 UniInt Failover active ID and heartbeat points, these points must be excluded from the list of existing and hidden points passed to the PI APS Connector in the first call to obtain available points in step 4.

4. For every point in the list, determine whether the point is an excluded UniInt point. If the point is an excluded UniInt point, remove it from the list. Otherwise, get the key attributes for the point from the PI Server.

5. Adds the key attributes for every hidden point in the APSPoints database to the list from step 2.

6. Calls the PI APS Connector function to obtain available points. On the first iteration, the entire list of existing and hidden points from step 3 is a parameter to the PI APS Connector; an empty list is passed for this parameter on subsequent iterations. On every call, another parameter specifies the maximum number of available points (from configuration settings) that the PI APS Connector should return.

7. The PI APS Connector function follows these steps:

   1. If this is not the first iteration, proceeds to step 4c.

   2. On the first iteration, the PI APS Connector uses the data source programming interface to enumerate the tags in the data source. Each data source tag is compared with the list of existing and hidden points parameter from the Synchronization Engine. Data source tags found in the list are ignored. The data source tags not found in the list of existing and hidden points are added to an internal list of available points.

   3. Get the next data source tag from the internal list of available points. If no more tags are in the list, set a return parameter to indicate no more available points and proceed to step 4f.

   4. Get the current data source attribute values that correspond to known attributes for the data source tag. Add the data source tag and known attribute values to the current group of available points for return to the Synchronization Engine.

   5. If the current group of available points contains fewer points than the group size parameter, returns to step 4c. If the group size is reached, sets a return parameter to indicate that more available points exist and proceeds to step 4f.

   6. Returns the current group of available points with their known attributes to the Synchronization Engine.

8. The Synchronization Engine applies the rule for available points to every point in the group returned by the PI APS Connector:

   • The store for review rule adds the known attributes for each point to the APSPoints database and optional point log.

   • The automatic rule creates each point in the PI Server. The Synchronization Engine creates an attribute collection for the point creation operation that contains all attribute values from the PI APS Connector. The values for the PointSource and InterfaceID attributes of the point are obtained from the interface instance configuration and are added to the attribute collection. If the interface instance is configured for multiple PointSource strings, the first one in the PI ICU PointSource list is used. PI APS has configurable default values for some attributes that have no corresponding attributes in most data sources, which is discussed in Security & Archive Settings tab. If any of these specific attributes are missing from the attribute collection, the Synchronization Engine adds the default value. The attribute collection is a parameter to the PI Server point creation operation. The PI Server provides default values for any attributes in the point class that are not in the attribute collection.

   • After PI APS creates a point in PI Server, it sets per-point synchronization settings for the new PI point from the default settings for available points. See Initial Sync Masks tab.

9. If the PI APS Connector call in step 4 indicates that more available points exist, the Synchronization Engine suspends operation for the time interval specified in the configuration settings. After the suspension interval elapses, the Synchronization Engine returns to step 4.

   Note: All known attributes are either stored in the APSPoints database or used for point creation. Specifically, the per-point synchronization settings do not apply to point creation.

The reason for processing groups of available points (instead of one point at a time) is to reduce the total overhead of calling the PI APS Connector to obtain available points (step 4) by making fewer calls. The overhead associated with calling this method is a result of having to structure the parameters to accommodate lists of key attributes and known attributes where both the names and order are unknown until run time. Before calling the PI APS Connector for available points, the Synchronization Engine must encode the key attributes in a way that allows the PI APS Connector to determine how to extract them. Similarly, the PI APS Connector must encode the known attributes and the Synchronization Engine must dynamically determine how to extract them.

The number of points in a group is configurable and the default is 1000. See APS Connector tab.

After processing each group of points and if the PI APS Connector indicates that more available points exist, the Synchronization Engine suspends operation for a configurable time to reduce competition with other programs for resources. The default suspension time is 10 milliseconds. See Sync Engine tab. After the suspension time elapses, the Synchronization Engine calls the PI APS Connector for the next group of available points.

An important distinction between synchronizing existing points in the first phase and the search for available points in this phase is that large numbers of available points are not expected to occur frequently. Only in unusual circumstances (like the first synchronization of a new data source or after a major data source configuration change) will the PI APS Connector find enough available points to cause the Synchronization Engine to call for available points more than once. With only one group, the configurable pause between groups is ineffective as a means of distributing resource usage over a longer period. In the normal case, practically all of the resource usage associated with searching for available points occurs during initialization in steps 2 and 3 and the first (and only) request to the PI APS Connector for available points in step 4. As can be inferred from the descriptions of the steps in this phase, each step intensely uses one specific resource. Step 2 makes a large number of calls to PI SDK. If the PI SDK cached the existing points (during the first phase), the PI SDK calls use local CPU cycles and memory. Otherwise, the PI SDK calls result in load on the PI Server. Step 3 consumes local CPU cycles and memory. Step 4 is a heavy consumer of data source resources.

If PI APS is on the same computer as either the PI Server or data source, the priority reduction feature indirectly reduces the rate of resource consumption by preventing the Synchronization Engine from running except when the computer would otherwise be idle.

PI APS presently has no means to throttle its rate of remote resource consumption during the search for available points. An enhancement request is on record to address this issue in a future PI APS release.

Note: Resource usage in this phase of synchronization cannot be throttled by the Synchronization Engine because most resource consumption occurs in the PI APS Connector. Therefore, every PI APS Connector must be modified to throttle resource usage, which is an ongoing effort.