Statistics retention policy
- Last UpdatedMay 21, 2024
- 3 minute read
Statistics retention policy task
The statistics for each view are stored as a separate table in the PIIntegratorStats SQL database. The table name is the view ID.
The statRetentionPolicies task in the maintenanceTasks section of the application configuration file determines when the statistics records are deleted. The following table describes the statRetentionPolicies task parameters.
statRetentionPolicies task parameters
|
Parameter |
Description of parameter |
Default value |
|---|---|---|
|
taskName |
Identifies the name of the statistics retention task. Do not change this value. |
taskName="statRetentionPolicies" |
|
runFrequency |
Frequency with which the statRetentionPolicies task is run. Use PI Time to specify units of time. runFrequency and timeOfDay together determine when the task is run. |
runFrequency="24hours" |
|
timeOfDay |
Time that the statRetentionPolicies task is run. Use PI Time to specify units of time. runFrequency and timeOfDay together determine when the task is run. |
timeOfDay="7:00" |
By default, the statistics records are deleted every 24 hours after startup.
Statistics retention policies
The policies in the statRetentionPolicies section determine which statistics records are deleted. The statistics retention policies assume that views are published at scheduled intervals and the publishing frequency determines which records are deleted. Generally, the more frequently a view is published, more statistics are generated, and you want to delete records more often. This is true for asset views, event views, and scheduled streaming views.
Streaming views that are published in response to changes to key values are not published at regular intervals. Therefore, to effectively manage the amount of statistics data in the database, PI Integrator for Business Analytics makes the assumption that data streaming for these views occurs at the highest possible frequency, that is, at 30-second intervals. Therefore, the policy that includes the publish frequency of 30 seconds is applied to all key-value-triggered streaming views.
Note: The statistics for views that are only published once are kept indefinitely until the view is removed.
The following policies are defined by default:
-
StatsPolicy1 – Defines a policy for views published at a frequency between 1 second and 5 minutes
-
StatsPolicy2 – Defines a policy for views published at a frequency between 5+ minutes and up to 1 hour
-
StatsPolicy3 – Defines a policy for views published at a frequency of 1+ hours
You can edit the default policies or create additional policies and specify whatever level of granularity you require. Check to see that your policies cover all possible time intervals. If a time interval is not included, then the statistics records for views published at the missing interval will be kept indefinitely.
For each policy, you can delete the records for a view based on any combination of the following:
-
The age of the records in the table
-
The total number of records in the table
-
The total size of the table
The following describes the statRetentionPolicies policy parameters and lists the default values for one of the policies, StatsPolicy1.
statRetentionPolicies policy parameters
|
Parameter |
Description of parameter |
Default values for StatsPolicy1 |
|---|---|---|
|
uniquePolicyName |
Name of the policy. Policy names must be unique. If there are multiple policies with the same name, then the configuration file will not run and an error is thrown. |
uniquePolicyName="StatsPolicy1" |
|
filterMinimumRunFrequency |
The lower limit of the frequency of the view publishing. Used with filterMaximumRunFrequency to determine which views this policy applies to. |
filterMinimumRunFrequency="1second" |
|
filterMaximumRunFrequency |
The upper limit of the frequency of the view publishing. Used with filterMinimumRunFrequency to determine which views this policy applies to. |
filterMaximumRunFrequency="5minutes" |
|
maximumTableRecordAge |
The age of the oldest records that are retained. Used with maximumTableRowCount and maximumTableSizeOnDisk to specify the records that are deleted. |
maximumTableRecordAge="24hrs" |
|
maximumTableRowCount |
The maximum number of rows that are retained in the table. If the number of rows exceeds this number, the oldest records are deleted first. Used with maximumTableRecordAge and maximumTableSizeOnDisk to specify the records that are deleted. |
maximumTableRowCount="100000" |
|
maximumTableSizeOnDisk |
The maximum size of the table in megabytes (MB). The oldest records are deleted first. Used with maximumTableRecordAgemaximumTableRecordAge and maximumTableRowCount to specify the records that are deleted. |
maximumTableSizeOnDisk="200" |
The following describes the StatsPolicy1 with its default values. The StatsPolicy1 policy applies to views that are published at a frequency between 1 second and up to 5 minutes. For each view, statistics records up to 24 hours old are kept, the number of rows in the view's table does not exceed 100,000, and the total size of the table does not exceed 200 MB. Records that do not meet all three conditions are deleted. With all retention policies, the oldest records are deleted first.
Note: Every time the scheduled deletion successfully deletes any number of records, the policy will also perform an automatic shrink operation on the database to reduce its size.