Load EIWM

The EIWM setting under the Load menu contains various options to define the settings to store EIWM Output data along with the FileSystem information.

The EIWM setting contains the following options:

• Data Output Type: Normally set to EIWM where the output Engineering data will be written to an EIWM XML file. Selecting None means that no Engineering data is written, for example, when only a CSV report is needed. Configure this in the Load EIWM Configuration file using the dataType element, for example, <dataType value="EIWM" />.

  Note: These values can be a case-insensitive representation of either "EIWM" or "NONE".

• Output Destination: Select whether the output will be written to a File System or an S3 or an AVEVA Cloud Storage folder, or any combination of these. Configure this in the Load EIWM Configuration file using the OutputDestination element, for example, <OutputDestination Destination="FileSystem">.

  Notes:

  • The Destination values can be a case-insensitive representation of either "FileSystem" or "S3", or "Both".

  • File System Details: Under Output Path option, type the output file path or click the Browse for Folder icon to search for the output folder path.

    Note: Output EIWM Registry Location is available in OutputData section in json log.

  • S3: For more information while using S3 as the output destination, see Access an AWS S3 Bucket.

  • AVEVA Cloud Storage: For more information while using ACS as the output destination, see Access AVEVA Cloud Storage.

• Replicate Input Structure: Select this option to generate an output folder structure from the base input path. Any sub-folders below the input locator will be replicated in the output destination, and the files generated from each input file in that structure will be in the corresponding output sub-folders. Configure this using the FolderStructure attribute of the OutputDestination element, for example, <OutputDestination Destination="FileSystem" FolderStructure="keepInputFolderStructureForOutput ">

• Custom Folder Structure: Select this option to enable the File Name and Folder Structure fields. To write all output files to the Output Folder, leave the File Name and Folder Structure fields empty.

  Use the File Name field to create multiple smaller EIWM files.

  Custom File Name Structure

  File Name is configurable and the expected entries in a File Name box can be the same as defined in the Folder Structure. If File Suffix setting is blank (default) then the file's suffix is determined by the project context:

  • A project context of avngate will set the suffix to "_avngate.xml".

  • A project context set to any value other than avngate will set the suffix to "_null.xml".

Scenarios:

• An example configuration that creates an output file for each object in a data source:

  <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
  <S3>
  <Authentication instance="true">
  <CredentialFile path="default" profileName="default" />
  </Authentication>
  <Region />
  <BucketName />
  <OutputDirectory />
  </S3>
  <FileSystem>
  <OutputPath>C:\Test \TestProject\Output</OutputPath>
  </FileSystem>
  <keepInputFolderStructureForOutput />
  <CustomFolderStructureForOutput>
  <FileName value="[eiwm:id]" />
  <FolderStructure value="" />
  </CustomFolderStructureForOutput>
  </OutputDestination>

  In this case, an XML file is created with the eiwm:id prefixed to the file name to generate a unique file name, for example, p-100_avngate.xml, assuming that File Suffix setting was blank and Project Context was set to avngate.

  Note: Performance of the Gateway may be compromised while generating a very large number of files.

• An example configuration that distributes the output across different directories by classification:

  <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
  <S3>
  <Authentication instance="true">
  <CredentialFile path="default" profileName="default" />
  </Authentication>
  <Region />
  <BucketName />
  <OutputDirectory />
  </S3>
  <FileSystem>
  <OutputPath>C:\Users\<user_name>\Desktop\sdfsfgsd\Output</OutputPath>
  </FileSystem>
  <keepInputFolderStructureForOutput />
  <CustomFolderStructureForOutput>
  <FileName value="output" />
  <FolderStructure value="[eiwm:ClassID]" />
  </CustomFolderStructureForOutput>
  </OutputDestination>

  The above results in a sub-directory being created for each classification. Each directory contains a file 'output_avngate.xml', if the Project context is set to avngate. If Project Context is other than avngate, the suffix will be _null.xml, assuming that File Suffix setting was also blank.

• An example configuration that creates a single file per object within the classification sub-directories:

  <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
  <S3>
  <Authentication instance="true">
  <CredentialFile path="default" profileName="default" />
  </Authentication>
  <Region />
  <BucketName />
  <OutputDirectory />
  </S3>
  <FileSystem>
  <OutputPath> C:\Test \TestProject\Output</OutputPath>
  </FileSystem>
  <keepInputFolderStructureForOutput />
  <CustomFolderStructureForOutput>
  <FileName value="[eiwm:id].xml" />
  <FolderStructure value="[eiwm:ClassID]" />
  </CustomFolderStructureForOutput>
  </OutputDestination>

  This setting results in a sub-directory being created for each classification. Each classification directory contains XML files created with the object’s ID matching the classification.

• Produce Revisions in Output File Names:

  If you create an output file per object/document, the Gateway will by default put all revisions of a document into a single file. For example, the following settings produce an output file for all revisions of a document:

  <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
  <S3>
  <Authentication instance="true">
  <CredentialFile path="default" profileName="default" />
  </Authentication>
  <Region />
  <BucketName />
  <OutputDirectory />
  </S3>
  <FileSystem>
  <OutputPath> C:\Test \TestProject\Output</OutputPath>
  </FileSystem>
  <keepInputFolderStructureForOutput />
  <CustomFolderStructureForOutput>
  <FileName value="[eiwm:id] " />
  <FolderStructure value="[eiwm:ClassID]" />
  </CustomFolderStructureForOutput>
  </OutputDestination>

  If you want to have a single output file for each revision, then the configuration section must have the revision added to it:

  <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
  <S3>
  <Authentication instance="true">
  <CredentialFile path="default" profileName="default" />
  </Authentication>
  <Region />
  <BucketName />
  <OutputDirectory />
  </S3>
  <FileSystem>
  <OutputPath> C:\Test \TestProject\Output</OutputPath>
  </FileSystem>
  <keepInputFolderStructureForOutput />
  <CustomFolderStructureForOutput>
  <FileName value="[eiwm:id]_[eiwm:revision] " />
  <FolderStructure value="[eiwm:ClassID]" />
  </CustomFolderStructureForOutput>
  </OutputDestination>

• An example configuration that creates an output folder for the nested context in a data source:

  <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
  <S3>
  <Authentication instance="true">
  <CredentialFile path="default" profileName="default" />
  </Authentication>
  <Region />
  <BucketName />
  <OutputDirectory />
  </S3>
  <FileSystem>
  <OutputPath>C:\Test\TestProject\Output</OutputPath>
  </FileSystem>
  <keepInputFolderStructureForOutput />
  <CustomFolderStructureForOutput>
  <FileName value="[eiwm:id]" />
  <FolderStructure value="[eiwm:context]" />
  </CustomFolderStructureForOutput>
  </OutputDestination>

  In this case, the value for [eiwm:context] is used in the folder structure. If its value is "HigherContext", then a folder C:\Test\TestProject\Output\HigherContext will be created for the output files. If the context is nested, then a folder structure will be created, for example, C:\Test\TestProject\Output\HigherContext\LowerContext, corresponding to the object’s context in the EIWM. For example:

  <Context>

  <ID> HigherContext </ID>

  <Context>

  <ID> LowerContext </ID>

  </Context>

  </Context>

  Custom Folder Structure

  Manage the EIWM files by specifying a customized Folder Structure. Smaller EIWM files are needed to allow more efficient imports or when large EIWM files (approximately > 60 MB) fail to import via the Import Controller.

  The base output path for FileSystem is defined by the Output Path. The base Output Path for S3 is defined by the Output Folder.

Expected Values:

The expected values in a Folder Structure are listed in the following table:

Notes:

• Every "\" in the folder hierarchy is treated as a delimiter and is resolved as a subfolder.

• Configured attributes that resolve to an empty value are skipped during the path generation.

• The values that are not resolved are skipped and a warning is logged.

• Characters, such as ‘/’, ‘:’, ‘*’, ‘?’, ‘"’, ‘<’, ‘>’ and ‘|’, are not allowed and are skipped if found in the resolved attribute values or in the static values. A warning is logged.

• Attributes that need to be resolved must be written within square brackets, for example, [name], [id] or [DateAndTime].

• Create Document Object: A Document Object is the metadata object present in the EIWM that describes the input data source. It holds the information about the data source for extraction like filename, date and time of extraction and so on. Default value of createDocumentObject option is true.

  The configuration entry is as follows.

  <configuration>

  ...

  <createDocumentObject apply="true" />

  ...

  </configuration>

  This option when set to 'true' allows you to include the document object and its related association "is referenced in" with all other objects in the output EIWM file.

  When set to 'false', it directs the EIWM Loader not to include the Document object and its related association "is referenced in" from the Object Model.

  Note: It is possible to modify the same setting from both configuration and the EIWM Loader GUI page.

• Project Context: By default, it is set to Avngate. It controls the EIWM object's output context, also it affects EIWM files suffix when File Suffix is left blank. It can be set to empty resulting in EIWM objects generated with empty context, unless it's defined in base mapping.

• File Suffix: By default, it is set to empty, it controls the generated EIWM files suffixes if not left blank.

  This allows for pre-processing by the AVEVA AIM Import Controller to overwrite Avngate with any Vnet file definition for the project context (refer to the AVEVA AIM User Guide). Characters such as ‘/’, ‘:’, ‘*’, ‘?’, ‘"’, ‘<’, ‘>’ and ‘|’ are not allowed to be used in File Suffix setting.

• Generate Trigger File: After output file generation, to trigger the Import Controller, a trigger.start file is created in the staging area. It passes through three stages: start, process and finish. When the file reaches the finish stage, the output files are imported into AIM.

  Note: Each loader has the option to generate a trigger file in the output folder defined for that loader and must be independently selected to generate it. This means that when a common folder is being used for more than one loader then they need to be set to be consistent with each other, otherwise, ambiguous imports into AIM may result. Note that the trigger file is created after the loader output file, which may occur before the other loader output file is created. In this case a second trigger file would be created, which the Import Controller will then convert to a trigger queue file.

• Create Alias in EIWM Objects: Select this option to create an alias in the EIWM objects from the Gateway. Configure this using the createAlias element, for example, <createAlias apply="false" attribute="" />.

  Note: The apply value must be an xsd schema supported Boolean value. The valid values for xsd:boolean are true, false, 0 and 1. Attribute values that are capitalized (for example, TRUE) or abbreviated (for example, T) are not valid.

  Alias Attribute: If you select Create Alias in EIWM Objects, enter the name of the attribute that is used for the alias. Default value for this option is empty.

  Notes:

  • Alias is created only when the value of the Alias attribute defined in the loader configuration differs from the value of the Object ID in the mapping file.

  • No aliases will be created when the value of the apply attribute is set to false.

  • When the value of an attribute is set to either empty or invalid, the default Alias attribute used is "GlobalID". No aliases will be created if the attribute "GlobalID" is not present.

    Examples:

  • If the source system has an object with attributes "GlobalID" and "AlternateID" with values "3$pEhtFpv31QFndkpGikoC" and "X 101", respectively, the Load configuration file and mapping configuration file are set to the following values:

    Load Configuration file:

    <outputEIWM>
    .........
    <createAlias apply="true" attribute ="AlternateID" />
    ..........
    </outputEIWM>

    Mapping configuration is defined as:

    <Object>
    <ObjectID value="[GlobalID]" />

    <ClassID value="PUMP" />
    </Object>

    This allows the loader to create aliases in the output EIWM objects, where the associated objects have the IDs per alias attribute defined in the configuration, "X 101" in this case.

    The output object should look similar to the following, where the IDs of associated objects are the values of attribute defined in the Alias attribute field in the GUI or the value in the createAlias element in the Load configuration XML.

    <Object>

    <ID>3$pEhtFpv31QFndkpGikoC</ID>
    <Context>
    <ID>Avngate</ID>
    </Context>
    <ClassID>PUMP</ClassID>
    <Association type="is identified by">
    <Object>
    <ID>X 101</ID>
    <Context>
    <ID>Avngate</ID>
    </Context>
    <ClassID>PUMP</ClassID>
    </Object>
    </Association>

    .........

    </Object>

  • If the source system has an object with an attribute "GlobalID" having a value "3$pEhtFpv31QFndkpGikoC" and does not contain the attribute "AlternateID", the Load configuration file and mapping configuration file are set to the following values (when Alias attribute is not present in the source system):

    Load configuration file:

    <outputEIWM>
    .........
    <createAlias apply="true" attribute ="AlternateID" />
    ..........
    </outputEIWM>

    Mapping configuration is defined as:

    <Object>
    <ObjectID value="[GlobalID]_AVA" />

    <ClassID value="PUMP" />
    </Object>

    The output object should look similar to the following, where the default alias attribute "GlobalID" is used to create the alias, as the Alias attribute "AlternateID" is not present in that particular object on the source system.

    <Object>
    <ID>3$pEhtFpv31QFndkpGikoC_AVA</ID>
    <Context>
    <ID>Avngate</ID>
    </Context>
    <ClassID>PUMP</ClassID>
    <Association type="is identified by">
    <Object>
    <ID>3$pEhtFpv31QFndkpGikoC</ID>
    <Context>
    <ID>Avngate</ID>
    </Context>
    <ClassID>PUMP</ClassID>
    </Object>
    </Association>
    ..........
    </Object>

• If the source system has an object with attribute "GlobalID" having a value "3$pEhtFpv31QFndkpGikoC" and the Load configuration file and mapping configuration file are set to the following values:

  Load configuration file:

  <outputEIWM>
  .........
  <createAlias apply="true" attribute ="GlobalID" />
  ..........
  </outputEIWM>

  Mapping configuration is defined as:

  <Object>
  <ObjectID value="[GlobalID]" />

  <ClassID value="PUMP" />
  </Object>

  The output object should look similar to the following, where no alias is created, as both the ObjectID and Alias attribute are mapped to the same attribute "GlobalID".

  <Object>
  <ID>3$pEhtFpv31QFndkpGikoC</ID>
  <Context>
  <ID>Avngate</ID>
  </Context>
  <ClassID>PUMP</ClassID>

  .........

  </Object>

  

• Export Class Definition: Select this option to generate AIM bootstrap files that can be imported into AIM by the Import Controller to generate or update the AIM schema definition for: classes, characteristics, properties, associations and permissible associations. You can configure this by selecting which files to generate, either Class Definition files (aka bootstrap files) or CSV files that contain the same values that might be used in a post process.

  Class Definition files can be used by the bootstrap loader to generate additional class/property/characteristic nodes for each level in the hierarchy, to update and extend the AIM Class Library. Any new Class must be a subclass of one of the existing built-in classes. This is achieved by defining one of the existing class hierarchies to which the parent class of the new class belongs. This node is present in the following models:

  • Cd Class

  • Cd Property Class

  • CdCharacteristicClass

  Structure Example

  

  The full hierarchy of a class, property, or characteristic is defined as a pipe-separated (`|`) string. In the above example this is the ParentClassHierarchy "EQUIPMENT | FUNCTIONAL ARTIFACT | FACILITY". The hierarchy is ordered from root to immediate parent, separated by `|`.

  The ParentClassHierarchy and ParentClassID nodes are required elements that encodes the full parent class chain for a class, property, or characteristic, enabling correct hierarchical relationships in the generated class definition files.

  Warning: Importing bootstrap files into AIM should always be done with caution as an incorrectly configured bootstrap file can corrupt the AIM schema and hence your access to AIM data. You should be thoroughly familiar with bootstrap functionality as described in AIM's documentation before using this Gateway feature. If in any doubt seek guidance and advice from the AVEVA helpdesk.

• CSV Files: When you select the CSV Files button, two configurable options appear:

  • Files Locator: Selects the folder where the output bootstrap CSV files will be generated. By default, this path is set to Configurations folder of the select project.

  • Prefix: Sets the prefix to be used in naming the output CSV files.

    

    When you select Class Definition Files button, one configurable option appears:

  • Configuration File Locator: Selects the location for the Class Definition Configuration xml file. This configuration file contains the details of the names and parameters of the bootstrap files that can be imported into AIM by the ImportController to generate or update the AIM schema definition for: classes, characteristics, properties, associations and permissible associations. The Class Definition Configuration file can be selected by using the browse path button or edited by selecting the edit button. By default, the ClassDefinitionConfiguration.xml file is present in the Configurations folder of the selected project.

    

    The default structure of Class Definition Configuration File is shown as follows:

    

    In the above image, there are five different nodes for: Class, CharacteristicClass, PropertyClass, AssociationClass and PermissibleAssociation. Each node is used to generate the respective output bootstrap files.

    Notes:

    • In each node, the [path] attribute value is mandatory to generate the respective output bootstrap files. It should be kept as empty value, that is, path = “”, if that output bootstrap file generation in not needed. Otherwise, it should point to the already existing bootstrap file for that particular node class, or a new file can be created by giving the full path along with the xml filename.

    • For the Association node, there is only one option to use in the ClassDefinitionConfiguration.xml file, that is, "type" of the association which is the association itself. The same attribute "type" is used to generate the "ID", "SourceRole" and "TargetRole".

    • Update Class Definition Files controls whether new classes identified in the input data are appended to the existing output bootstrap files. If cleared, any existing output bootstrap files of the same name are overwritten.

  • Save Settings: After you have selected the required settings, click Save Settings to save the Engineering project settings.

    The following code shows an example of Load configuration:

    <configuration xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" sourceProductName="AVEVA™ Gateway for 2D Data" componentName="Load" componentVersion="2.10.0.0" >
    <components>
    <component name="LoadEIWM" locator=".\LoadEIWMConfiguration.xml" />
    <component name="LoadSVG" locator=".\LoadSVGConfiguration.xml" />
    <component name="LoadCSV" locator=".\LoadCSVConfiguration.xml" />
    </components>
    </configuration>

The following code shows an example of Load EIWM configuration:

<configuration xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" sourceProductName="AVEVA™ Gateway for 2D Data" componentName="LoadEIWM" componentVersion="2.10.0.0" >
<OutputDestination Destination="S3||FileSystem||S3&FileSystem">
<S3>
<Authentication instance="true||false">
<CredentialFile path="default" profileName="default"/>
</Authentication>
<Region></Region> <!-- Required, defines region of the destination S3 bucket -->
<BucketName></BucketName> <!-- Required, defines the destination S3 bucket -->
<OutputDirectory></OutputDirectory> <!-- Optional, If not defined files will be uploaded to the base of S3 bucket -->
</S3>
<FileSystem>
<OutputPath></OutputPath>
</FileSystem>
</OutputDestination>
<dataType value="EIWM/NONE"/>
<keepInputFolderStructureForOutput apply="true" />
<generateTriggerStart apply="true" />
<addDescriptionToHeader apply="true" />
<projectContext value="Avngate" />
<createAlias apply="true" attribute="GlobalID" />
<file maxObjectCount="0" suffix="_part_" suffixNumberFormat="DDD" />
<annotations level="Basic" />
</configuration>

Note: You need to use tracking of changes provided by EIWM Load on data using annotations feature. For information about annotations, see Appendix C: Tracking Changes in Data.

Output File Format

The output setting allows you to set the format and naming convention of the output XML. This setting can be modified using the below-mentioned attributes in the Loader configuration file:

<file maxObjectCount="0" suffix="_part_"
suffixNumberFormat="DDD" contextSuffix="" />

Modify the output file formatting as follows:

• Context Suffix: Set the EIWM file name's suffix to be appended to the end of the file name. Set this to empty, in which case the EIWM file name suffix will be set to _avngate or _null, depending on the projectContext value.

• Splitting Output File: Split the output across multiple files based on the number of objects identified during conversion as valid EIWM objects. This output setting can be modified using <file maxObjectCount="0" attribute in the configuration file. The <file maxObjectCount= parameter accepts the value as non-negative integers. This parameter defines the maximum number of objects permitted to be serialized in a single file. If the object count exceeds the count defined in the setting, the objects will be serialized across multiple output files such that no file contains more objects than set by this setting. The maxObjectCount value ="0" is the default value and it defines that all the objects should be serialized into a single output file.

• Suffix for multipart output files: Set the suffix to be appended to the multipart output files. Suffix will only be appended to the file as the output file is split based on the exceeding maxObjectCount value or based on the multiple template mapped to the EIWM objects. This setting can be modified using <file ... suffix="_part_" attribute in the loader configuration setting. The output for above-defined setting is as follows:

  • Wall_part_001_null.xml

  • Wall_part_002_null.xml

• Suffix number format: Set the serial number format as required. This setting can be accessed using <file … suffixNumberFormat="DDDD"> attribute in the configuration setting. Using the above settings, the output files should be as follows:

  • Wall_part_0001_null.xml

  • Wall_part_0002_null.xml