Optional: Configure Buffering

You can configure adapters to buffer data egressed from the adapter to endpoints. Buffering is configured through the buffering configuration parameters in the system configuration.

Note: It is recommended that you do not modify the default buffering location unless it is necessary. Changes to the buffering configuration parameters only take effect during adapter service startup.

Configure buffering

Complete the following steps to configure buffering. Use the PUT method in conjunction with the http://localhost:5590/api/v1/configuration/system/buffering REST endpoint to initialize the configuration.

1. Using a text editor, create an empty text file.

2. Copy and paste an example configuration for buffering into the file.

   For sample JSON, see the buffering examples below.

3. Update the example JSON parameters for your environment.

   For a table of all available parameters, see the Buffering parameters section below.

4. Save the file. For example, as ConfigureBuffering.json.

5. Open a command line session. Change directory to the location of ConfigureBuffering.json.

6. Enter the following cURL command (which uses the PUT method) to initialize the buffering configuration.

   curl -d "@ConfigureBuffering.json" -H "Content-Type: application/json" -X PUT "http://localhost:5590/api/v1/configuration/system/buffering"

   Note: If you installed the adapter to listen on a non-default port, update 5590 to the port number in use.

   For a list of other REST operations you can perform, like updating or replacing a buffering configuration, see the REST URLs section below.

Buffering schema

The full schema definition for the system buffering is in the System_Buffering_schema.json file located in one of the following folders:

Windows: %ProgramFiles%\OSIsoft\Adapters\\Schemas

Linux: /opt/OSIsoft/Adapters//Schemas

Buffering parameters

The following parameters are available for configuring buffering:

1 Buffering to disk - disk is only used if required;

• Data is only written to the disk buffer if queued in the memory buffer for more than 5 seconds.

• The MaxBufferSizeMB is applied per configured endpoint except the health endpoint.

• An adapter creates 20 MB buffer files that are stored in BufferLocation.

• When MaxBufferSizeMB is reached, the oldest buffer file is deleted and a new buffer file is created.

• The health endpoint is fixed at 20 MB. When the health endpoint buffer file becomes full, a new buffer file is created and the previous buffer file is deleted.

  The following rules apply in case of an error when creating a new buffer file:

  • Attempt to delete oldest buffer file and retry.

  • If unable to buffer, errors are logged to indicate data loss.

  • If a buffer file is corrupted, an attempt is made to recover individual records and any failure to recover records is logged.

2 Buffering only to memory:

• The MaxBufferSizeMB is applied per configured endpoint except the health endpoint.

• When MaxBufferSizeMB is reached, the oldest messages in the memory buffer are removed. Depending on the size of a new message, several old messages may be removed.

• The health endpoint is fixed at 20 MB. When the health endpoint buffer file becomes full, the oldest messages in the memory buffer are removed and new messages are added.

Examples

The following examples are buffering configurations made through the curl REST client.

Retrieve the buffering configuration

curl -X GET "http://localhost:5590/api/v1/configuration/system/buffering"

Sample output:

{

"bufferLocation": "C:/ProgramData/aveva/Adapters//Buffers",

"maxBufferSizeMB": 1024,

"enablePersistentBuffering": true

}

200 OK response indicates success.

Update MaxBufferSizeMb parameter

curl -d "{ \"MaxBufferSizeMB\": 100 }" -H "Content-Type: application/json" -X PATCH "http://localhost:5590/api/v1/configuration/system/buffering"

204 No Content response indicates success.

REST URLs