Information Standards Data Governance
- Last UpdatedAug 04, 2026
- 11 minute read
This section describes information quality, and maturity, data captures, corresponding identifiers, registers and views.
Data Capture
The Data Capture process is an automation which processes all registers through sub processes of capture, interpret, and consolidation.
Data is consolidated within a specified category such as Tags or Docs (see Fig. 10.1).
Fig. 10.1: Data Capture Process
Corresponding Identifiers
Corresponding identifiers define a system of identifiers, for example, a specific system such as AVEVA P&ID, a project, or some other purpose.
Data is mapped within the EDW using corresponding identifiers defined within the Information Standard.
Note: Corresponding identifiers may be used to map both incoming and outbound data, where outbound may be an external system or a user interface concept such as Engineering Lists.
When corresponding identifiers are used to map incoming data, multiple corresponding identifiers may map to a single target (such as an attribute). When they are used to map outbound data, the first entry will be selected.
Note: The term Nomenclatures has been changed to corresponding identifiers.
Class Mapping
Fig. 10.2 shows the Class Mapping with Corresponding identifiers.
Fig. 10.2: Class Mapping with Corresponding Identifiers
Attribute Mapping
Fig. 10.3 shows the Attribute mapping with corresponding identifiers.
Fig. 10.3: Attribute Mapping with Corresponding Identifiers
Registers
Registers are information interfaces defined within the Information Standard to capture an input of structured data into an Engineering Data Warehouse and describe rules used to determine what the content data is and how it should be mapped.
One data source may have several registers, for example, AVEVA P&ID will define a Main Equipment register, an Instrument registers, an Instrument Loop register, a Line register, a Valve register and so on.
Configuring a Register
Registers for Data Capture are defined under InfoInterface/Data Sources in general ISM extensions, as shown in Fig. 10.4.
Fig. 10.4: Registers for Data Capture
Registers may be nested in sub folders for organisational purposes and to inherit common settings (which may be overridden) but cannot be moved once used as this will define a new register.
Minimum Register Definition
A register must at minimum define the following options:
A category such as Tags, which groups data from different interfaces into a set.
A RegisterType such as Primary which defines how a register should be read.
An IDColumn or IDExpression, which defines how to identify captured records.
A Columns folder with one or more Column entries that are of interest.
Fig. 10.5 shows how register defines the various options.
Fig. 10.5: Defining a Register
Register Columns
Registers must define at least one column, which will be mapped to an actual attribute during interpret phase using Corresponding identifiers mapping, or by finding an attribute with the same ID. If there is no corresponding attribute to map to, the captured data will still be captured, but will not be mapped (so will be ignored unless mapping rules are updated). Fig. 10.6 shows the Register columns.
Fig. 10.6: Register Columns
Register Configuration Options
Options may be inherited from parent folders except where specified otherwise. The following configuration options may be set on a register, as listed in Table 10.1.
Table 10.1: Register Configuration Options
|
Option |
Usage |
|
Capture |
[Default on] Switch to enable/disable capture for an interface. This does not affect any data already captured. on Capture new data for this register on an update off Do not capture new data for this register on an update |
|
Interpret |
[Default on] Switch to enable/disable interpret for an interface. This does not affect any data already interpreted. on Interpret new data for this register on an update off Do not interpret new data for this register on an update |
|
Consolidate |
[Default on] Switch to enable/disable interpret for an interface. This does not affect any data already interpreted on Include interpreted data for this register in the consolidation off Do not include interpreted data for this register in the consolidation |
|
CaptureMethod |
[Default delta] Specifies how data is captured. delta Data is captured as partial updates and will only add or update data. full Data is captured as full updates and any missing data will be marked deleted until subsequently updated. |
|
Category |
[Required] Specifies a category for the register. Records from registers within a single category are combined into a single consolidated record. for example, $[default-register-category]=tags Standard categories are tags|docs|physical|general This is not required if server configuration specifies a default category for all interfaces. |
|
Option |
Usage |
|
FileName |
[Default *.*] Specifies a file name pattern to meet for files to be captured into this register. for example, *tags*.csv |
|
FilePath |
[Default *] Specifies folder path pattern (relative to the root data load folder) to meet for files to be captured into this register. for example, tags\inst |
|
RegisterType |
[Required, cannot be inherited] Specifies the type of register, which determines how data will be captured and interpreted. Primary A primary register captures all fields from source files as attributes and can define record classification and name rules Secondary A secondary register captures all fields from source files as attributes but does not define classification or name rules Attributes An attribute register expects each record in captured data to be one attribute so specifies AttributeNameColumn, AttributeValueColumn and AttributeUnitsColumn so that all attributes for a single ID may be mapped to a single record. Associations |
|
AttributeNameColumn |
[Required for Attribute Registers] Specifies the column used to determine attribute name for each attribute record. input column name |
|
AttributeValueColumn |
[Required for Attribute Registers] Specifies the column used to determine attribute value for each attribute record. input column name |
|
AttributeUnitsColumn |
[Optional for Attribute Registers] Specifies the column used to determine attribute units for each attribute record. Note: If the class library indicates that an attribute is a characteristic and not a property, this will not override that. Instead, if there are units assigned and the attribute is a characteristic, the units will be included in the value when output to for example, EIWM. input column name |
|
IDColumn |
Specifies a column to determine the ID of each record. input column name or expression |
|
Option |
Usage |
|
IDExpression |
Alternative to IDColumn which allows text or an expression to determine the ID of each record. input column name or expression |
|
ClassificationColumn |
[Optional for Primary Registers] Specifies a column in captured data to use in determining the Class of each record. The result of the expression is used to lookup a class in the current class library using lookups in the following order: In order to not use Corresponding identifiers or alias mappings, simply do not assign any. Classification will also be used to determine object context for the main object record ID, which will be looked up from the class in AVEVA:Context (which may be inherited from a parent class). If found, this will indicate a sub context within the asset/project. for example, IPE|Docs. input column name |
|
ClassificationExpression |
[Optional for Primary Registers] As an alternative to ClassificationColumn, an expression may be used. for example, {{ Coalese(@Class1, @Class2) }} |
|
ClassificationDefault |
[Optional for Primary Registers] Specifies a default class to assign to an object record when ClassificationColumn and ClassificationExpression does not assign a class. This default class is checked for validity when publishing the class library (not when processing data). |
|
ClassificationLookups |
[Default id,corresponding identifiers,alias,default for Primary Registers] Mandates specific classification lookup rules and ordering for the interface. for example, corresponding identifiers,default id Find class with matching ID corresponding identifiers Find first class with matching the register’s corresponding identifiers alias Find first class with matching *:alias extension default Find first class with corresponding identifiers matching ClassificationDefault |
|
NameColumn Primary Register only |
[Optional for Primary Registers] Specifies a column to determine a name or description for a record. |
|
NameExpression Primary Register only |
[Optional for Primary Registers] |
|
Option |
Usage |
|
An alternative to NameColumn which allows an expression to determine a name or description for an object record. |
|
|
Columns/Column |
[Required to include at least 1, cannot be inherited] |
|
Columns/Column/ ValueExpression |
[Optional] If set, specifies an expression to use when interpreting captured data. for example, {{ $[attr.value].splitAny(';,') }} Note that there does not need to be a captured column here so a ValueExpression may be used to define an entirely new column. |
|
ContentType |
[Default auto-detect] Best practice is to start with strict modes and only move to relaxed modes as a last resort. auto-detect Use file extension is used to determine content type (the format to read) when capturing data. excel html table Excel files are sometimes HTML files with a .xls extension csv Read as comma separated values in strict RFC 4180 mode tsv|txt|tab Read as tab separated values in strict RFC 4180 mode csv relaxed Read as comma separated values in similar fashion to Excel, acknowledging that this may read some poorly formatted data incorrectly (as Excel would) tsv relaxed Read as tab separated values in similar fashion to Excel, acknowledging that this may read some poorly formatted data incorrectly (as Excel would) eiwm Read EIWM XML (AIM format) buxml Read BUXML (Gatekeeper, Change Manager format) |
|
HeaderRow |
[Default 1 for tabular data sources] Specifies a header row expression where any number less than 1 means no header row. If no header row then the first data row determines dynamic headers F1…FN. To specify a range of rows. Multiple header rows are specified using N:M notation. for example, 6:10 will construct headers from rows 6 through 10 (expected to work for excel sheets with regular merged cells). |
|
FirstDataRow |
[Default 0 for tabular data sources] |
|
Option |
Usage |
|
Specifies the row to start reading data from. If not specified will be the row after the last header row. |
|
|
IncludeSheet |
[Default true for Excel] Specifies expression to limit which sheets are included when reading Excel files. for example, {{ $[sheet.name] like 'line*list' }} |
|
TextDelimiter |
[Default , for CSV or tab for TSV] Specifies the text delimited to use,which may be multiple characters. for example, |!| |
|
TextQuote |
[Default " for text delimited format] Specifies the quoting expression to use to surround text, which may be multiple characters. Set to none to indicate no quoting of values when capturing data. |
|
TextEndOfLine |
[Default CRLF for text delimited format] Specifies the end of line expression to use when capturing data, which may be multiple characters, for example, {{ LF }} |
|
TooFewFields |
[Default Warning for tabular formats] Specifies the action to take when a record has less fields than expected, which is the number of identified headers. ErrorNow Stop immediately and error ErrorLater Process files for this register to the end then error Warning Log a warning, include record, and continue Accept Ignore, include record, and continue |
|
TooManyFields |
[Default Warning for tabular formats] Specifies the action to take when a record has more fields than expected, which is the number of identified headers. ErrorNow Stop immediately and error ErrorLater Process files for this register to the end then error Warning Log a warning, include record, and continue AcceptExpected Ignore excess fields, include record, and continue AcceptExtra Include excess fields using F{N} notation, include record, and continue |
|
Corresponding Identifiers |
[Optional] |
|
Option |
Usage |
|
Specifies the corresponding identifiers this register uses for mapping in data interpretation, typically an identifier for another system. Refer to Mapping Data with Corresponding identifiers. |
|
|
Whitespace |
[Optional] If set, defines a whitespace rule when interpreted data. Whitespace includes spaces, tabs and new lines. By default, incoming attribute data is not modified, however sometimes captured data has strange or excess spacing which is unwanted. trim Whitespace is trimmed from the start and end of attribute values normalise Any consecutive whitespace is converted to a single space and trimmed from the start and end of attribute values. |
|
AssociationDelimiter |
[Optional for Primary, Secondary and Attribute Registers] If specified, indicates a list of single character delimiters used to split reference attribute values on interpretation. Reference attributes are those which are mapped to associations. for example, ,|; A Note: Any register Column may define a ValueExpression with specific split rules for that attribute. |
|
Exclude |
[Optional for Attribute Registers] An expression which indicates source attributes you want to explicitly exclude. This may be used where an attribute register includes attributes you want to ensure are set from a primary register, but since this is an expression may be a more complex rule with conditional logic, and so on. for example, {{ $[attr.name] not in ['status','tag number'] }} |
|
DuplicateRecord |
[Default merge for Primary, Secondary and Attribute Registers] Action to take when capturing data encounters a record with the same ID. Best practice is to leave this defaulted to avoid confusion. first Use the first record last Use the last record reject Reject the duplicate record (clarify meaning) merge Merge the records (last wins) |
|
Option |
Usage |
|
DuplicateAttribute |
[Default replace for Primary, Secondary and Attribute Registers] Action to take if duplicate attributes are found when capturing data. Best practice is to leave this defaulted to avoid confusion. append Append any encountered values, removing duplicates. replace Replace attribute values (last wins). ignore Keep he first value. |
|
Priority |
[Default least-priority] If set to a whole number of 0 or more, specifies a priority for a register when consolidating data. 1 is a higher priority than 2. When consolidating data, if an attribute occurs in 2 or more registers, a higher priority will win over a lower priority, and where there is no difference the record with the latest update timestamp will win. |
Primary and Secondary Registers
Primary and secondary registers will capture all fields from source files (all fields are captured regardless of whether they are marked as interesting register columns).
A primary register is permitted to specify a class and name, noting that if multiple primary registers are defined, consolidation will determine a single winner in each case.
Classification of objects is optional, however, best practice is to ensure a default classification. Where this is not well defined, classes such as XX – Unknown Mechanical, XX – Unknown Instrument, and so on, may be defined which act as filters in which to see poorly defined information. These items may then be examined to determine whether data is incorrect or Information Standard rules require updating to cater for them. Fig.
10.7 shows the Capture Process for Primary and Secondary Registers.
Fig. 10.7: Capture Process for Primary and Secondary Registers
Fig. 10.8 shows the Interpret Process for Primary and Secondary Registers.
Fig. 10.8: Interpret Process for Primary and Secondary Registers
Attribute Registers
Attribute registers are used where incoming data specifies a single object attribute per record in source files, and hence are required to specify Attribute Name, Value and (optionally) Unit columns in addition to an ID.
This type of content is known as a TGA file in Gatekeeper. Fig. 10.9 shows the Capture Process for Attribute Registers.
Fig. 10.9: Capture Process for Attribute Registers
Fig. 10.10 shows the Interpret process for Attribute Registers.
Fig. 10.10: Interpret Process for Attribute Registers
Association Registers
Association registers are used to add and remove associations, though only in the context of the specific register. If an attribute creates an association from another register or another association register creates an association, the association will be included. Fig.
10.11 shows the Capture process for Association Registers.
Fig. 10.11: Capture Process for Association Registers
Fig. 10.12 shows the Interpret process for Association Registers.
Fig. 10.12: Interpret Process for Association Registers
Views
Views are output interfaces and registered under InfoInterface/Views in general extensions.
Configuring Views
Views specify a corresponding identifier, which is used to map attributes into the view, as shown in Fig. 10.13.
Fig. 10.13: Information Interface Views
Fig. 10.14 shows a view for a Pump Register.
Fig. 10.14: A View for a Pump Register
Information Quality and Maturity
The Registers Gateway is used to report the maturity level and completeness of data that is being provided and viewed in AIM. The required attributes are provided within a maturity level following the defined rules in ISM.