CONNECT to PI Agent buffering

You can configure the CONNECT to PI Agent to buffer data egressed from the agent. The default maximum buffer size is 1024 MB. If you are transferring a large amount of data, you should increase the size of this buffer to ensure that all data is transferred correctly. Buffering is configured through the buffering configuration parameters in the system configuration.

Note: We recommend that you do not modify the default buffering location unless it is necessary. Changes to the buffering configuration parameters only take effect during agent service startup.

Configure buffering

Complete the following steps to configure buffering. Use the PUT method in conjunction with the http://localhost:5590/api/v1/configuration/system/buffering REST endpoint to initialize the configuration.

1. Using a text editor, create an empty text file.

2. Copy and paste an example configuration for buffering into the file.

   For sample JSON, see the Retrieve the buffering configuration example below.

3. Update the example JSON parameters for your environment.

   For a table of all available parameters, see the Buffering parameters section.

4. Save the file. For example, ConfigureBuffering.json.

5. Open a command line session. Change directory to the location of ConfigureBuffering.json.

6. To initialize the buffering configuration, send the ConfigureBuffering.json content as request body by Postman (which uses the PUT method) to "http://localhost:5590/api/v1/configuration/System/Buffering/"

   Note: If you installed the agent to listen on a non-default port, update 5590 to the port number in use.

   For a list of other REST operations you can perform, like updating or replacing a buffering configuration, see the REST URLs section below.

Buffering parameters

The following parameters are available for configuring buffering:

1 Buffering to disk - disk is only used if required:

• Data is only written to the disk buffer if queued in the memory buffer for more than 5 seconds.

• The MaxBufferSizeMB is applied per configured endpoint except the health endpoint.

• An agent creates 20 MB buffer files that are stored in BufferLocation.

• When MaxBufferSizeMB is reached, the oldest buffer file is deleted and a new buffer file is created.

• The health endpoint is fixed at 20 MB. When the health endpoint buffer file becomes full, a new buffer file is created and the previous buffer file is deleted.

  The following rules apply in case of an error when creating a new buffer file:

  • Attempt to delete oldest buffer file and retry.

  • If unable to buffer, errors are logged to indicate data loss.

  • If a buffer file is corrupted, an attempt is made to recover individual records and any failure to recover records is logged.

2 Buffering only to memory:

• The MaxBufferSizeMB is applied per configured endpoint except the health endpoint.

• When MaxBufferSizeMB is reached, the oldest messages in the memory buffer are removed. Depending on the size of a new message, several old messages may be removed.

• The health endpoint is fixed at 20 MB. When the health endpoint buffer file becomes full, the oldest messages in the memory buffer are removed and new messages are added.

Examples

The following examples are buffering configurations made through Postman.

Retrieve the buffering configuration

GET "http://localhost:5590/api/v1/configuration/system/buffering"

Sample output:

{

"bufferLocation": "C:/ProgramData/OSIsoft/Adapters/CONNECT/Buffers",

"maxBufferSizeMB": 1024,

"enablePersistentBuffering": true

}

200 OK response indicates success.

Update MaxBufferSizeMb parameter

"PATCH http://localhost:5590/api/v1/configuration/system/buffering"

{

"MaxBufferSizeMB": 100

}

204 No Content response indicates success.

REST URLs