Configure an incremental update for a dimension

By default, all records of a dimension are updated at each refresh execution. However, for dimensions with large row counts, you may configure the dimension to update incrementally. Incremental updates are based on the DateTime field of the dimension. The initial population of the dimension only considers records from the data collection start time and then, at each refresh, from records corresponding to the update period. This limits the number of records to be processed at each refresh.

For example, if you configure a dimension with a refresh rate of 15 minutes and an update period of 1 hour, then every 15 minutes, the system will query the past hour of data and update the dimension table accordingly.

Alternatively, when you cannot use an incremental update, you may consider a dimension filter expression to limit the number of records handled at each refresh.

Note:In case you want to leverage Insight Data Source, you must use the [RequestedStartTime] and [RequestedEndTime] parameters in the custom query expressions. If you have not enabled the incremental update feature, then the parameters only return the default value, which is the last round hour.

Note: For the UtilizationEvents custom query expression, the best practice is to use the LastModified parameter in the incremental update feature, instead of StartTime or EndTime parameters. When Incremental update timeframe observes StartTime field, then it considers only events where StartTime matches the incremental update timeframe. Subsequently it excludes all the events that are out of the timeframe, but which have been modified within the same timeframe. Meanwhile, LastModified parameter considers any and all changes with every incremental update. For more information, refer to UtilizationEvents.

Configure an incremental update for a dimension

1. Open the AVEVA BI Gateway Model Builder.

2. Click the Add icon in the Floating Action Buttons menu and then click the Dimension icon. A new dimension is added in the AVEVA BI Gateway model with a default name and the Dimension editor appears in the right pane.

3. Click the Advanced tab.

4. Select the Incremental Update checkbox.

   

5. In the Data collection start list, click and select the data collection start date and time.

   Note: The StartDateTime specified in the custom query expression takes a higher precedence than the datetime value defined in the Data collection start field. For more information about custom query expression, see Use custom queries.

6. In the Time reference field list, click a dimension DateTime reference field for the update period.

   Note: This is the field on which the Data collection start time will be applied. The Time reference field list shows the DateTime fields of the current dimension.

7. In the Incremental Update Period area, do the following:

   1. In the Interval list, click the required update period for the dimension.

   2. In the Unit list, click the required unit of time for the update period. You can provide a minimum update period of one minute and a maximum update period of 366 days. The default update period is one day.

      Ensure that the update period is equal to or more than the refresh rate for the dimension.

8. Click Save.