Load Settings

The Gateway uses the Load component to load the selected and transformed objects into AIM via EIWM files, including hot-spotting these objects in their XGL/ZGL 3D model representation. Load component also exports the transformed data into various other CSV files.

EIWM

The EIWM setting under the Load menu contains various options to define the settings to store EIWM Output data along with the FileSystem information.

The EIWM setting contains the following options:

• Data Output Type: Normally set to EIWM where the output Engineering data will be written to an EIWM XML file. Selecting None means that no Engineering data is written, for example, when only a CSV report is needed. You can configure this in the Load EIWM Configuration file using the dataType element, for example, <dataType value="EIWM" />.

  Note: These values can be a case-insensitive representation of either "EIWM" or "NONE".

• Output Destination: Select whether the output will be written to a File System or an Amazon Web Services (AWS) S3 Bucket or AVEVA Cloud Storage. You can configure this in the Load EIWM Configuration file using the OutputDestination element, for example, <OutputDestination Destination="FileSystem">.

  Note: The Destination values can be a case-insensitive representation of either "FileSystem" or "S3", or "Both". You can select all these types independently or select one of the S3 and AVEVA Cloud Storage type with FileSystem destination type together.

  • File System: The file system is normally the path that defines the AIM staging area.

    • Output Path: Type the Output file path or click the Browse for Folder tab to enter the output folder path.

  • S3: To access an S3 bucket requires the relevant authentication details like credential file, region information, bucket name and output directory. For more information, see Accessing an AWS S3 Bucket.

    

  • AVEVA Cloud Storage: For more information about accessing AVEVA Cloud Storage, see Accessing AVEVA Cloud Storage.

    

• Add Description to Header: Select this option to add the metadata to a header of EIWM file. You can configure this using the addDescriptionToHeader element, for example, <addDescriptionToHeader apply="false" />.

  Note: The apply value must be an xsd schema supported Boolean value. The valid values for xsd:boolean are true, false, 0 and 1. Attribute values that are capitalized (for example, TRUE) or abbreviated (for example, T) are not valid.

• Replicate Input Structure: Select this option to generate an output folder structure from the base input path. Any sub-folders below the input locator will be replicated in the output destination, and the files generated from the input data in that structure will be in the corresponding output sub-folders. You can configure this using the FolderStructure attribute of the OutputDestination element, for example, <OutputDestination Destination="FileSystem" FolderStructure="keepInputFolderStructureForOutput ">

• Custom Folder Structure: Select this option to enable the File Name and Folder Structure fields. To write all output files to the Output Folder, leave the File Name and Folder Structure fields empty.

  You can use the File Name field to create multiple smaller EIWM files.

  Custom File Name Structure

  File Name is configurable and the expected entries in a File Name field can be the same as defined in the Folder Structure. If the project context is avngate or IPE, then the suffix "_avngate.xml" or "_null.xml" is derived from the default project context value by default. Any other value for the project context results in the EIWM:Context attribute value being used as the suffix to the file name.

• Scenarios:

  • An example configuration that creates an output file for each object in a data source:

    <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
    <S3>
    <Authentication instance="true">
    <CredentialFile path="default" profileName="default" />
    </Authentication>
    <Region />
    <BucketName />
    <OutputDirectory />
    </S3>
    <FileSystem>
    <OutputPath>C:\Test \TestProject\Output</OutputPath>
    </FileSystem>
    <keepInputFolderStructureForOutput />
    <CustomFolderStructureForOutput>
    <FileName value="[eiwm:id]" />
    <FolderStructure value="" />
    </CustomFolderStructureForOutput>
    </OutputDestination>

    In this case, an XML file is created with the eiwm:id prefixed to the file name to generate a unique file name, for example, p-100_avngate.xml.

    Note: Performance of the Gateway may be compromised while generating a very large number of files.

  • An example configuration that distributes the output across different directories by classification:

    <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
    <S3>
    <Authentication instance="true">
    <CredentialFile path="default" profileName="default" />
    </Authentication>
    <Region />
    <BucketName />
    <OutputDirectory />
    </S3>
    <FileSystem>
    <OutputPath>C:\Users\<user_name>\Desktop\sdfsfgsd\Output</OutputPath>
    </FileSystem>
    <keepInputFolderStructureForOutput />
    <CustomFolderStructureForOutput>
    <FileName value="output" />
    <FolderStructure value="[eiwm:ClassID]" />
    </CustomFolderStructureForOutput>
    </OutputDestination>

    The above results in a sub-directory being created for each classification. Each directory contains a file 'output.avngate.xml', if the Project context is set to avngate. If Project context is other than avngate, the suffix will be _null.xml.

  • An example configuration that creates a single file per object within the classification sub-directories:

    <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
    <S3>
    <Authentication instance="true">
    <CredentialFile path="default" profileName="default" />
    </Authentication>
    <Region />
    <BucketName />
    <OutputDirectory />
    </S3>
    <FileSystem>
    <OutputPath> C:\Test \TestProject\Output</OutputPath>
    </FileSystem>
    <keepInputFolderStructureForOutput />
    <CustomFolderStructureForOutput>
    <FileName value="[eiwm:id].xml" />
    <FolderStructure value="[eiwm:ClassID]" />
    </CustomFolderStructureForOutput>
    </OutputDestination>

    The above setting results in a sub-directory being created for each classification. Each classification directory contains XML files created with the object’s ID matching the classification.

  • Produce Revisions in Output File Names:

    If you create an output file per object/document, the Gateway will by default put all revisions of a document into a single file. For example, the following settings produce an output file for all revisions of a document:

    <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
    <S3>
    <Authentication instance="true">
    <CredentialFile path="default" profileName="default" />
    </Authentication>
    <Region />
    <BucketName />
    <OutputDirectory />
    </S3>
    <FileSystem>
    <OutputPath> C:\Test \TestProject\Output</OutputPath>
    </FileSystem>
    <keepInputFolderStructureForOutput />
    <CustomFolderStructureForOutput>
    <FileName value="[eiwm:id] " />
    <FolderStructure value="[eiwm:ClassID]" />
    </CustomFolderStructureForOutput>
    </OutputDestination>

    If you want to have a single output file for each revision, then the configuration section must have the revision added to it:

    <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
    <S3>
    <Authentication instance="true">
    <CredentialFile path="default" profileName="default" />
    </Authentication>
    <Region />
    <BucketName />
    <OutputDirectory />
    </S3>
    <FileSystem>
    <OutputPath> C:\Test \TestProject\Output</OutputPath>
    </FileSystem>
    <keepInputFolderStructureForOutput />
    <CustomFolderStructureForOutput>
    <FileName value="[eiwm:id]_[eiwm:revision] " />
    <FolderStructure value="[eiwm:ClassID]" />
    </CustomFolderStructureForOutput>
    </OutputDestination>

  • An example configuration that creates an output folder for the nested context in a data source:

    <OutputDestination Destination="FileSystem" FolderStructure="CustomFolderStructureForOutput">
    <S3>
    <Authentication instance="true">
    <CredentialFile path="default" profileName="default" />
    </Authentication>
    <Region />
    <BucketName />
    <OutputDirectory />
    </S3>
    <FileSystem>
    <OutputPath>C:\Test\TestProject\Output</OutputPath>
    </FileSystem>
    <keepInputFolderStructureForOutput />
    <CustomFolderStructureForOutput>
    <FileName value="[eiwm:id]" />
    <FolderStructure value="[eiwm:context]" />
    </CustomFolderStructureForOutput>
    </OutputDestination>

    In this case, the value for [eiwm:context] is used in the folder structure. If its value is "HigherContext", then a folder C:\Test\TestProject\Output\HigherContext will be created for the output files. If the context is nested, then a folder structure will be created, for example, C:\Test\TestProject\Output\HigherContext\LowerContext, corresponding to the object’s context in the EIWM. For example:

    <Context>

    <ID> HigherContext </ID>

    <Context>

    <ID> LowerContext </ID>

    </Context>

    </Context>

    Custom Folder Structure

    You can manage the EIWM files by specifying a customized Folder Structure. Smaller EIWM files are needed to allow more efficient imports or when large EIWM files (approximately > 60 MB) fail to import via the AIM Import Controller.

    The base output path for FileSystem is defined by the Output Path. The base Output Path for S3 is defined by the Output Folder.

    Expected Values:

    The expected values in a Folder Structure are listed in the following table:

    Expected Entries in a Folder Structure	Description
    Engineering attributes	Attributes extracted from the data source along with the attributes created during the base mapping can be used as elements in the folder structure, for example, [Tag], [Custom Attribute] and so on. Also reserved attributes, such as [TemplateID], [ObjectID], [ClassID], [ContextID], [ObjectName] and [Revision], can be used. Note: The nested context must be resolved as separate sub folders in the hierarchy.
    EIWM reserved attributes	EIWM reserved attributes can be used as elements in the folder structure: [eiwm:id], [eiwm:classid], [eiwm:name], [eiwm:revision] or [eiwm:context]
    Manifest attributes	For more information about manifest attributes, see Manifest References.
    Reserved placeholders	Reserved placeholders can be used as elements in the folder structure: [Date]: Provides the value of the current date. Default format for date is dd-MM-yyyy. This attribute provides a format string, for example, [Date{d.MMM.yy}]. [Time]: Provides the value of the current time. Default format for date is hh-mm-ss. This attribute provides a format string, for example, [Time{H.mm}]. [DateAndTime]: Provides the value of the current date and time. Default format for date is dd-MM-yyyy hh-mm-ss. This attribute provides a format string, for example, [Date{d.MMM.yy H:mm}]. Note: The formats can be from any of the formats as specified under the MSDN DateTime Formats.
    Static values	The static values are simple text values and are left unchanged and can be defined without any square brackets. Example: <FileName value="output_[ eiwm:id].xml" />. Here "output_" is a static value.

    Notes:

  • Every "\" in the folder hierarchy is treated as a delimiter and is resolved as a subfolder.

  • Configured attributes that resolve to an empty value are skipped during the path generation.

  • The values that are not resolved are skipped and a warning is logged.

  • Characters, such as ‘/’, ‘:’, ‘*’, ‘?’, ‘"’, ‘<’, ‘>’ and ‘|’, are not allowed and are skipped if found in the resolved attribute values or in the static values. A warning is logged.

  • Attributes that need to be resolved must be written within square brackets, for example, [name], [id] or [DateAndTime].

• Create Document Object: A Document Object is the metadata object present in the EIWM that describes the input data source. It holds the information about the data source for extraction like filename, date and time of extraction and so on. Default value of createDocumentObject option is true.

  The configuration entry looks as follow.

  <configuration>

  ...

  <createDocumentObject apply="true" />

  ...

  </configuration>

  This option when set to 'true' allows you to include the document object and its related association "is referenced in" with all other objects in the output EIWM file.

  When set to 'false', it directs the EIWM Loader not to include the Document object and its related association "is referenced in" from the Object Model.

  Note: You can modify the same setting from both configuration and the EIWM Loader GUI page.

• Project Context: By default, it is set to Avngate. If the project context is undefined in the base mapping, the EIWM objects will be written to an <output>_avngate.xml file and each object's context will be set to Avngate. This allows for pre-processing by the AIM Import Controller to overwrite Avngate with a Vnet file definition for the project context (refer to the AIM User Guide).

  If the project context value is left blank or set to any other value, the EIWM objects will be written to an <output>_null.xml file and each object's context will be set to empty if undefined in the base mapping.

• Generate Trigger File: After output file generation, to trigger the Import Controller, a trigger.start file is created in the staging area. It passes through three stages: start, process and finish. When the file reaches the finish stage, the output files are imported into AIM.

  Note: Each loader has the option to generate a trigger file in the output folder defined for that loader and must be independently selected to generate it. This means that when a common folder is being used for more than one loader then they need to be set to be consistent with each other, otherwise ambiguous imports into AIM may result. Note that the trigger file is created after the loader output file, which may occur before the other loader output file is created. In this case a second trigger file would be created, which the Import Controller will then convert to a trigger queue file.

• Create Alias in EIWM Objects: Select this option to create an alias in the EIWM objects from the Gateway. You can configure this using the createAlias element, for example, <createAlias apply="false" attribute="" />.

  Note: The apply value must be an xsd schema supported Boolean value. The valid values for xsd:boolean are true, false, 0 and 1. Attribute values that are capitalized (for example, TRUE) or abbreviated (for example, T) are not valid.

  Alias Attribute: If you select Create Alias in EIWM Objects, you can enter the name of the attribute that is used for the alias. Default value for this option is empty.

  Notes:

  • Alias is created only when the value of the Alias attribute defined in the loader configuration differs from the value of the Object ID in the mapping file.

  • No aliases will be created when the value of the apply attribute is set to false.

  • When the value of an attribute is set to either empty or invalid, the default Alias attribute used is "GlobalID". No aliases will be created if the attribute "GlobalID" is not present.

    Examples:

  • If the source system has an object with attributes "GlobalID" and "AlternateID" with values "3$pEhtFpv31QFndkpGikoC" and "X 101", respectively, the Load configuration file and mapping configuration file are set to the following values:

    Load Configuration file:

    <outputEIWM>
    .........
    <createAlias apply="true" attribute ="AlternateID" />
    ..........
    </outputEIWM>

    Mapping configuration is defined as:

    <Object>
    <ObjectID value="[GlobalID]" />

    <ClassID value="PUMP" />
    </Object>

    This allows the loader to create aliases in the output EIWM objects, where the associated objects have the IDs per alias attribute defined in the configuration, "X 101" in this case.

    The output object should look similar to the following, where the IDs of associated objects are the values of attribute defined in the Alias attribute field in the GUI or the value in the createAlias element in the Load configuration XML.

    <Object>

    <ID>3$pEhtFpv31QFndkpGikoC</ID>
    <Context>
    <ID>Avngate</ID>
    </Context>
    <ClassID>PUMP</ClassID>
    <Association type="is identified by">
    <Object>
    <ID>X 101</ID>
    <Context>
    <ID>Avngate</ID>
    </Context>
    <ClassID>PUMP</ClassID>
    </Object>
    </Association>

    .........

    </Object>

    If the source system has an object with an attribute "GlobalID" having a value "3$pEhtFpv31QFndkpGikoC" and does not contain the attribute "AlternateID", the Load configuration file and mapping configuration file are set to the following values (when Alias attribute is not present in the source system):

    Load configuration file:

    <outputEIWM>
    .........
    <createAlias apply="true" attribute ="AlternateID" />
    ..........
    </outputEIWM>

    Mapping configuration is defined as:

    <Object>
    <ObjectID value="[GlobalID]_AVA" />

    <ClassID value="PUMP" />
    </Object>

    The output object should look similar to the following, where the default alias attribute "GlobalID" is used to create the alias, as the Alias attribute "AlternateID" is not present in that particular object on the source system.

    <Object>
    <ID>3$pEhtFpv31QFndkpGikoC_AVA</ID>
    <Context>
    <ID>Avngate</ID>
    </Context>
    <ClassID>PUMP</ClassID>
    <Association type="is identified by">
    <Object>
    <ID>3$pEhtFpv31QFndkpGikoC</ID>
    <Context>
    <ID>Avngate</ID>
    </Context>
    <ClassID>PUMP</ClassID>
    </Object>
    </Association>
    ..........
    </Object>

  • If the source system has an object with attribute "GlobalID" having a value "3$pEhtFpv31QFndkpGikoC" and the Load configuration file and mapping configuration file are set to the following values:

    Load configuration file:

    <outputEIWM>
    .........
    <createAlias apply="true" attribute ="GlobalID" />
    ..........
    </outputEIWM>

    Mapping configuration is defined as:

    <Object>
    <ObjectID value="[GlobalID]" />

    <ClassID value="PUMP" />
    </Object>

    The output object should look similar to the following, where no alias is created, as both the ObjectID and Alias attribute are mapped to the same attribute "GlobalID".

    <Object>
    <ID>3$pEhtFpv31QFndkpGikoC</ID>
    <Context>
    <ID>Avngate</ID>
    </Context>
    <ClassID>PUMP</ClassID>

    .........

    </Object>

  • Save Settings: After you have selected the required settings, click Save Settings to save the Engineering project settings.

    The following code shows an example of Load configuration:

    <configuration xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" sourceProductName="AVEVA.NET.Gateway.SmartPlant3D" componentName="Load" componentVersion="2.2.0.0" >
    <components>
    <component name="LoadEIWM" locator=".\LoadEIWMConfiguration.xml" />
    <component name="LoadXGL" locator=".\LoadXGLConfiguration.xml" />
    <component name="LoadCSV" locator=".\LoadCSVConfiguration.xml" />
    </components>
    </configuration>

    The following code shows an example of Load EIWM configuration:

    <configuration xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" sourceProductName="AVEVA.NET.Gateway.SmartPlant3D" componentName="LoadEIWM" componentVersion="2.2.0.0" >
    <OutputDestination Destination="S3||FileSystem||S3&FileSystem">
    <S3>
    <Authentication instance="true||false">
    <CredentialFile path="default" profileName="default"/>
    </Authentication>
    <Region></Region> <!-- Required, defines region of the destination S3 bucket -->
    <BucketName></BucketName> <!-- Required, defines the destination S3 bucket -->
    <OutputDirectory></OutputDirectory> <!-- Optional, If not defined files will be uploaded to the base of S3 bucket -->
    </S3>
    <FileSystem>
    <OutputPath></OutputPath>
    </FileSystem>
    </OutputDestination>
    <dataType value="EIWM/NONE"/>
    <keepInputFolderStructureForOutput apply="true" />
    <generateTriggerStart apply="true" />
    <addDescriptionToHeader apply="true" />
    <projectContext value="Avngate" />
    <createAlias apply="true" attribute="GlobalID" />
    <file maxObjectCount="0" suffix="_part_" suffixNumberFormat="DDD" />
    <annotations level="Basic" />
    </configuration>

    Note: You can set tracking of changes provided by EIWM Load on data using annotations feature. For information about annotations, see Appendix C: Tracking Changes in Data.

Output File Format

The output setting allows you to set the format and naming convention of the output XML. This setting can be modified using the below-mentioned attributes in the Loader configuration file: <file maxObjectCount="0" suffix="_part_" suffixNumberFormat="DDD" />

You can modify the output file formatting as follows:

• Splitting Output File: You can split the output across multiple files based on the number of objects identified during conversion as valid EIWM objects. This output setting can be modified using <file maxObjectCount="0" attribute in the configuration file. The <file maxObjectCount= parameter accepts the value as non-negative integers. This parameter defines the maximum number of objects permitted to be serialized in a single file. If the object count exceeds the count defined in the setting, the objects will be serialized across multiple output files such that no file contains more objects than set by this setting. The maxObjectCount value ="0" is the default value and it defines that all the objects should be serialized into a single output file.

• Suffix for multipart output files: You can set the suffix to be appended to the multipart output files. Suffix will only be appended to the file as the output file is split based on the exceeding maxObjectCount value or based on the multiple template mapped to the EIWM objects. This setting can be modified using <file ... suffix="_part_" attribute in the loader configuration setting. The output for above-defined setting is as follows:

  • Wall_part_001_null.xml

  • Wall_part_002_null.xml

• Suffix number format: You can set the serial number format as required. This setting can be accessed using <file … suffixNumberFormat="DDDD"> attribute in the configuration setting. Using the above settings, the output files should be as follows:

  • Wall_part_0001_null.xml

  • Wall_part_0002_null.xml

XGL

Load exports graphical data together with metadata to XGL/ZGL format. You can select the graphical data output type XGL/ZGL format under the Load, XGL settings.

After you select the required settings, click Save Settings to save the Load geometry settings of graphical data.

The XGL setting contains the following options:

• Graphics Output Type: Select the output file type for storing the graphics, such as XGL (XML format)/ZGL (compressed XGL format). You can use None option if no 3D graphical data is needed, for example, if only the EIWM file is needed. Selecting None means that no graphical data is written, for example, if only the engineering data via EIWM XML is to be imported into AIM.

• Output Destination: Select whether the output will be written to a File System or an S3 or AVEVA Cloud Storage. You can select all these types independently or select one of the S3 and AVEVA Cloud Storage type with Filesystem destination type together.

  • File System: The file system is normally the path that defines the AIM staging area.

    • Output Path: Type the Output file path or click Browse for Folder button to enter the output file path.

  • S3: To access an S3 bucket requires the relevant authentication details like credential file, region information, bucket name and output directory. For more information, see Accessing an AWS S3 Bucket.

  • AVEVA Cloud Storage: For more information about accessing AVEVA Cloud Storage, see Accessing AVEVA Cloud Storage.

    

• Background: You can select the background colour of drawings. The background colour of the XGL/ZGL files generated by the project can be selected by any of these methods:

  • Colour: Select the required colour from the Colour list, for example, Black.

  • R, G, B: Enter the RGB values of the colour in the fields provided.

  • Pick Colour: Click Pick Colour and select the required colour from the standard colour selection dialog. A preview of the selected colour is displayed next to the Pick Colour button.

• View Direction: From the View Direction list, select the direction from which the drawings are initially viewed. You can select only one option from the list, for example, View from the Front.

• Save Settings: Click Save Settings to save the settings.

  Note: You can set tracking of changes provided by XGL Load on data using annotations feature. For information about annotations, see Appendix C: Tracking Changes in Data.

CSV

This option enables you to configure the CSV settings. The CSV setting contains the following options:

• Output Destination: Select the output destination type to store the output file type in the Destination folder. The destination type can be Filesystem or S3 or AVEVA Cloud Storage. You can select all these types independently or select one of the S3 and AVEVA Cloud Storage type with Filesystem destination type together.

  • File System: he file system is normally the path that defines the AIM staging area.

    • Output Path: Type the Output file path or click Browse for Folder tab to enter the output file path.

      

  • S3: To access an S3 bucket requires the relevant authentication details like credential file, region information, bucket name and output directory. For more information about S3 Bucket Details, see Accessing an AWS S3 Bucket.

    

  • AVEVA Cloud Storage: For more information about accessing AVEVA Cloud Storage, see Accessing AVEVA Cloud Storage.

    

  • CSV Columns: In the CSV Columns field, add the additional comma separated columns to export the objects' attributes and associations in the CSV export.

  • Generate Headers in CSV: If you select this setting then the names of the attributes or associations are written to the first row of the CSV file. The header level information contains configuration file data and date of creation and so on.

  • Replicate Input Structure: If you select this option, the output file is generated in a duplicate path. This is replicated input folder structure in output folder if you point on input location of input data containing also subfolders with files of such type.

  • Save Settings: After you have selected the required settings, click this to save the CSV settings.