Appendix B: CSV Reporting

The extracted data and the way they are transformed can be inspected at any point during the processing by creating a CSV report from the selected data. The selection of data to be included in CSV reporting is defined by the Transform and Loader configuration extensions. CSV files may include attributes or associations, which are selected in columns keyword in the configuration file. Each transformation configuration extension represents a step in the transformation process, so you should position this CSV reporting configuration at the appropriate transformation step to access the state of the data due to all previous transformation steps. A CSV report of the final transformed data that is converted to EIWM format can also be generated in the loader configuration.

Note: If there are more than one million rows to be written then a warning is logged and the CSV output is segmented into multiple files, with an underscore plus index number added to the extra file names to keep them unique, for example: <filename>_1.csv, <filename>_2.csv, and so on.

CSV report can be configured and generated in the following two scenarios:

• Reporting during transformation configured in Transform configuration file:

  This feature generates the CSV report during transformation and customizes the type of the information written to the CSV file. You can also generate multiple reports by using different suffixes and output paths for CSV files.

Example 1: CSV Reporting section with basic attributes, from Transform configuration file:

<extension name="CSVReporting">

<csvReport

suffix="_AfterTransformation"

addHeaderRow="true"

columns="ClassID, ObjectID, ClassName, Name, #Association:ContainedInStructure"

/>

</extension>

Example 2: CSV Reporting section with tracking attributes, from Transform configuration file:

<extension name="CSVReporting">

<csvReport

suffix="_AfterTransformation"

addHeaderRow="true"

columns="#InternalId#, ClassName">

<column value="#Tracking:

DateTime

ModuleName

EventType

ChangedMember

PrevAttributeName

AttributeName

PrevAttributeValue

AttributeValue

PrevAssociationType

AssociationType

ChangedMember

PrevTargetInternalId

TargetInternalId

#Filter: EventType: AttributeChanged"/>

</csvReport>

</extension>

Note: The optional 'annotations' feature allows you to store additional data about how each object, attribute or association has been changed due to a transformation step. These tracking attributes store data about creation, modification and deletion operations, and can be inspected via CSV reporting.

• Reporting after transformation configured from GUI:

  This CSV report is generated when CSV value is selected in Data Output Type combo list in Load > CSV tab. This report reflects the extracted data state after all transformations. Report file with .csv extension is generated in the location set in Output Path field.

• GUI Settings in the Engineering Tab under Load:

  

Settings:

The following list details about the CSV Report features for the Transform and the Loader configurations:

• columns = "<keywords, names...>": (Mandatory) Sets number of columns, each separated by a comma.

  • #Attribute: <name> - (Optional) Sets the name of the attribute to select the attribute values that will be shown in the CSV column for all extracted data entities. Name of the attribute can also be written without #Attribute. #Attribute is added to maintain consistency with other keywords. Besides standard objects' attributes, you can also collect different data using keywords:

    • #InternalID# - Specifies the Internal Globally Unique Identifier (GUID) of the object.

    • #SourceID# - If available, the source ID of the object specified in the source data, for example, the STEP line number in IFC files.

    • #ObjectType# - Specifies the internal type of the object, for example: EngineeringObject, GraphicalObject2D.

    • #SourceDataName# - Specifies the name of source data like input file.

  • These attributes cannot be changed but may be helpful in tracking the history of the objects or filtering the CSV report by Object Type.

  • #Association: <type> - Checks if specified association type is available in objects. CSV cells indicate AVAILABLE status if the corresponding object contains selected association type.

  • #Associations - Checks if associations are available in objects. All types of associations are present in the extracted data and each CSV column represents one type of association. CSV cells indicate AVAILABLE status if the corresponding object contains association type defined in the CSV column.

  • #Association:<type> and #TargetAttribute:<name> - Used together to first set the name of the association and then the name of the attribute whose value should be shown in the CSV column.

    Note: An association may point to many referenced objects and in this case values of attributes of all these objects are added to the cell in the CSV column, each separated by a comma.

  • #Unit - Used only with one of the three keywords: #Attribute: <name>, #Attributes or #TargetAttribute:<name>. If attribute contains unit, it is added to the value after putting a single space.

  • #Tracking: <tracking attributes names> - This keyword can be used with the list of Tracking attributes' names separated by a blank character (for example, space, tabulation or a new line), where each blank character represents data in a specific column. All recorded changes (Tracking entries) of extracted data are listed in a chronological order in the CSV cell (see Example 2 of CSV Document).

  • #Tracking: <tracking attributes names> #Filter: <tracking attribute name> : <RegEx expression> - The #Filter: keyword includes only those changes in the CSV report where selected tracking attribute name matches the RegEx.

• column = "<keywords, names...>": Separates long and complex descriptions of columns and defines each description in an XML element (see Example 2). The separate "column" element adds more clarity when the column contains a #Filter with a regular expression.

  Note: Columns and column can be used interchangeably or both at the same time.

The following list details about the settings applicable for the CSV Reporting configuration:

• suffix = "<file name suffix>: (Optional) Defines a suffix to add to the CSV file name. It is used only in Transform configuration. It is ignored when outputFilePath is set.

• outputFolder = "<output folder path>": (Optional) Stores the CSV files in the location indicated by this setting. Otherwise, files are passed to the output folder set in Load settings. It is used only in Transform configuration. It is ignored when outputFilePath is set.

• addHeaderRow = "<true/false>"(in Generate Headers in CSV checkbox): (Optional) Sets to "True" to write the names of the attributes or associations to the first row of the CSV file. This setting is enabled by default.

• append = "<true/valse>": (Optional) Adds current CSV data to file set by ouputFilePath. When option addHeaderRow is enabled, then file to append must contain columns with the same names in the header row. If addHeaderRow is disabled, then number of the columns in the file must be the same as in the data to append.

  In append mode, user cannot provide generic types of columns like #Attributes or #Associations because they may deliver different list of attributes and associations between files. In append mode, list of columns for all processed files is constant.

• outputFilePath = “<output file path>”: This attribute must be set when append mode is enabled. It can be provided absolute or path relative to Transform configuration location. When outputFilePath is set then attributes: outputFolder and suffix are ignored. This setting can be also used with disabled append mode.

Tracking attributes names which can be used with keyword #Tracking:

List of the EventType types which can appear in CSV in the EventType column:

• ObjectModified, ObjectAdded and ObjectRemoved

• AssociationAdded, AssociationChanged, AssociationReplaced, AssociationRemoved, AssociationMoved and AssociationsCleared

• AttributeAdded, AttributeChanged, AttributeReplaced, AttributeRemoved, AttributeMoved and AttributesCleared

Example 1 of CSV document viewed in Excel

Example 2 of CSV document with history of changes viewed in Excel

Note: If you want to open CSV reporting in a spreadsheet form, you must extend rows and columns of cells to see complete content.

Object Parameters Available for the CSV Report

The data available to select as columns in the CSV Report are as follows:

• ObjectID

• ClassID

• ClassName

• #Attributes (selects all attributes)

• #Associations (selects all associations)

• #InternalID#

• #SourceID#

• #ObjectType#

• Column name from tabular data source