Buffering parameters
- Last UpdatedMar 04, 2025
- 2 minute read
The following parameters are available for configuring buffering:
|
Parameter |
Required |
Type |
Description |
|---|---|---|---|
|
enablePersistentBuffering |
Optional |
boolean |
Enables or disables on-disk buffering Allowed value: true or false Note: If you disable persistent buffering, in-memory buffering is used. On-disk and in-memory buffering are limited by value in the MaxBufferSizeMB property. |
|
maxBufferSizeMB |
Optional |
integer |
Defines the maximum size of the buffer that is persisted on disk 1 or used in memory 2. The unit is specified in MB (1 Megabyte = 1048576 bytes). Consider the capacity and the type of storage medium to determine a suitable value for this parameter. Minimum value: 1 Note: The MaxBufferSizeMB property is applied to each configured endpoint. For example, if you set the MaxBufferSizeMB to 1024 and you configured the adapter to send data to two endpoints (for example, PI Server and CONNECT data services, the total maximum resources used for buffering will be 2048. The Health endpoint is an exception fixed at 20MB. |
|
maxDataBulkTime |
Optional |
string |
Maximum time before batched data in the buffer is sent. The expected format is H:MM:SS Minimum value: 0:00:01 Allowed value: MaxDataBulkTime must have a valid timespan format (for example, hh:mm:ss) and must be between 00:00:01 - 00:10:00. |
|
bufferLocation |
Required |
string |
Defines the location of the buffer files. Absolute paths are required. Consider the access-control list () when you set this parameter. BufferLocation is used to buffer files when EnablePersistentBuffering is true. Allowed value: Valid path to a folder location in the file system Definition: Definition: |
1 Buffering to disk - disk is only used if required;
-
Data is only written to the disk buffer if queued in the memory buffer for more than 5 seconds.
-
The MaxBufferSizeMB is applied per configured endpoint except the Health endpoint.
-
An adapter creates 20MB buffer files that are stored in BufferLocation.
-
When MaxBufferSizeMB is reached, the oldest buffer file is deleted and a new buffer file is created.
-
The Health endpoint is fixed at 20MB. When the Health endpoint buffer file becomes full, a new buffer file is created and the previous buffer file is deleted.
The following rules apply in case of an error when creating a new buffer file:
-
Attempt to delete oldest buffer file and retry.
-
If unable to buffer, errors are logged to indicate data loss.
-
If a buffer file is corrupted, an attempt is made to recover individual records and any failure to recover records is logged.
-
2 Buffering only to memory:
-
The MaxBufferSizeMB is applied per configured endpoint except the Health endpoint.
-
When MaxBufferSizeMB is reached, the oldest messages in the memory buffer are removed. Depending on the size of a new message, several old messages may be removed.
-
The Health endpoint is fixed at 20MB. When the Health endpoint buffer file becomes full, the oldest messages in the memory buffer are removed and new messages are added.