Virtual tables

Virtual tables enable integration between CONNECT data services and the third-party analytics platform, Databricks.

Virtual tables allow you to:

• Access operations data for analytics initiatives.

• Support the integration of operations data into existing business and financial data infrastructures for data exploration, reporting, and analytics.

• Make data views available on-demand to third-party data and analytics platforms as an external data source.

Your third-party cloud analytics platforms will be able to interact with data from CONNECT data services using the native languages and tools you are familiar with.

For information about data access and permissions in Databricks, read the Databricks documentation.

Current limitations

• Virtual tables partially support community data. The initial creation of a virtual table and subsequent scheduled refreshes of the virtual table will successfully complete. However, any updates made to data in a time range prior to the current refresh interval will not be automatically captured in the virtual table. To capture any changes that have occurred in the past, perform a manual refresh of a set time range of data.

• Virtual tables are derived from data views, which include stream and asset data. Event data is not yet supported in data views or virtual tables.

• CONNECT data services does not have user-facing logs for virtual tables.

• Virtual tables are not backed up. In the case of catastrophic failure, AVEVA will support customers with recreating necessary virtual tables.

  Contact AVEVA customer support for resolution.

• Virtual tables and Databricks currently support timestamps with a maximum precision of microseconds. For more information, see Create a virtual table.

Update processor

In the event that there are stream data changes that affect a virtual table with an interpolated retrieval mode, all intervals for the virtual table between the nearest predecessor and successor events will update at the next refresh. If the nearest predecessor to the data change is more than one month prior, we will will limit the value to one month prior to the data change. For example, if a virtual table has a event schedule of every 2 months, when a data change occurs, you will see updated data for within a one-month window.

If additional previous events need to be updated, you can perform a manual refresh for the time range needed.

Virtual tables workflow

1. If you have none available, create data views.

2. Create a share to connect CONNECT data services to your third-party analytics program.

3. Create a virtual table using your data views and share.

4. Manage permissions for virtual tables if you need to add new roles or adjust existing roles. For more information on roles, see CONNECT data services roles.