Understand how data flows in Data Archive

This section provides an overview of the data flow in Data Archive.

1. Interface sends data to Data Archive. PI interfaces collect data from the data sources and pass them on to the server in the form of events. An event consists of a time stamp, a value, and a status. Before sending an event to Data Archive, the interface evaluates the event: it sends significant events to Data Archive and discards the others. This is called exception reporting. Interfaces complete the following steps:

   1. Get data and create an event.

   2. Perform exception testing to determine if event is significant.

   3. Send significant events to Data Archive and discard the rest.

   4. If buffering is configured, save the events on disk in case Data Archive is unavailable.

      Note: The buffering service may also perform compression testing and fans data to all members of a PI collective. This is called n-way buffering.

2. PI Snapshot Subsystem receives all data. On Data Archive, all new events are handled first by PI Snapshot Subsystem. PI Snapshot Subsystem holds a single value for each PI point. If data comes in time order, incoming events become new snapshot values.

3. When a new value comes in, PI Snapshot Subsystem evaluates the old value by performing compression testing. Based on the results of the compression testing, PI Snapshot Subsystem either discards the old value or sends it to the event queue. PI Snapshot Subsystem completes the following steps:

   5. Get a new event and make this event the new snapshot value.

   6. If the new event is out-of-order, send it directly to the event queue without performing any compression testing. Otherwise, perform compression testing between the old and new events.

   7. Send significant events to the event queue and discard the rest.

4. PI Archive Subsystem reads data from the event queue. PI Archive Subsystem continuously reads data in the event queue and stores it in a memory cache, and eventually into archive files. However, when PI Archive Subsystem is shut down or unable to receive data, the event queue holds the data, serving as a high-speed buffer. When available, PI Archive Subsystem resumes data processing. PI Archive Subsystem completes the following steps:

   8. Read events out of the event queue.

   9. Validate and store events in the write cache.

   10. Periodically move data from the write cache into archive files.

5. Users access the data through client applications. The ultimate goal of a PI System is to distribute information across the enterprise or organization to those who use it to make key decisions. To achieve this goal, we produce a number of client tools that display Data Archive data in various formats.