Connect to an Assai DMS

The Assai sample is a json split in six main sections. Some configuration parts of the sections are already hardcoded in the template so the sample include only some of the configurations, as follows:

• Authentication

• DmsConfiguration

• MetadataAIMContext

• AIMIngestion

• HttpSettings

• PipelineRunnerSettings

Authentication

It is adapted to use Oauth2 with Resources Owner grant type (also known as Password grant type) and Client Credentials grant type. The following values are required:

• AuthorizeApi: Assai URL where the access token is requested.

• BasicLogin

  • username: the Client ID provided by Assai

  • password: the Client Secret provided by Assai

• BodyRequestValues

  • grant_type: the used Oauth2 grant type. (values: password or client_credentials)

  • username: Assai username (only for password grant type)

  • password: Assai password (only for password grant type)

DmsConfiguration

DmsConfiguration is a section where we configure DMS-related settings such as formats, endpoints, query parameters, and so on.

• DMSBaseUrl: represents the base url of the endpoints that are used. It is the common part of all the URLs that are consumed.

• DeltaEndpoint: represents the endpoint that will return an expected list of documents that must be pushed to ingestion. Usually, this endpoint returns a Delta instead of all the documents. Delta represents all the documents that are new or were modified between a certain date and the actual moment. Usually that certain date is transmitted to the endpoint as a query parameter.

• DeltaQueryParameters

  • response_type: the format of the response. Preferred is csv.

  • is_latest: boolean. If true (“Y”) returns the latest versions of the documents and if false (“N”) returns all version.

  • for_polling: boolean. If true (“Y”) activates polling on assai request and if false (“N”) works in the classic way. For the first release for_polling must be N.

  • latest_change_date_subq_after: variable date that always comes from the DB. It will always be set to “%DeltaDate%”

  • project_code: Assai project code to extract the documents from.

• EnableDeleteAudit: Represents a boolean (values: true or false) and provides an ability to process deleted documents.

• DeleteAuditEndpoint: represents the endpoint that will return audit data of deleted documents that should be pushed to the Data Pipeline. The endpoint will become mandatory when “EnableDeleteAudit“ is set to true.

  Notes:

  • Before enabling this setting, ensure you have added the DeleteWhere attribute to the ISM class library register and this change is successfully orchestrated in AIM-A. An example register is as follows:

    <Documents Category="Docs" RegisterType="Tracker" IDColumn="docs_seq_nr"

    Remarks="This tracks the primary key for a document in assai"

    Filepath="ASSAI/Documents" HeaderRow="1" CaptureMethod="Partial"

    DeleteWhere="{{@[Action] starts any ['Del','Rem']}}">

    <Columns>

    <document_nr>

    <capture ValueExpression="{{ coalesce(@document_nr, GUID) }}" />

    </document_nr>

    </Columns>

    </Documents>

  • This functionality does not delete the scraped references as this action is not supported by AIM-A.

  • Historical document deletion cannot be actioned.

• Filtering

  • FilteringItem

    • Identifier: a unique identifier of the filteringItem; an identifier that will be used in the ValidationExpression.

    • Mapping: the actual filtering name.

    • Value: the actual value against which the filtering is done; it can contain a list of values separated by commas.

  • ValidationExpression: a filtering condition based on the identifiers already created and the logical operators in use OR and AND.

    Notes:

  • The supported filters for the Mapping field are: class_code|subclass_code|company_code|document_type|discipline_code|asset_code|status_code|approval_code|publication_file|active_ind.

  • If the filtering section is not present in the onboarding file, then no filtering will be applied and a log stating that the filtering is not enabled is displayed.

  • When filtering is present in the file, then it is applied based on the selected criteria, for example, if mapping is publication_file and value is pdf, then only the files that meet that criteria will be selected.

  • All of the documents that fail to meet the filtering criteria will be excluded from processing. An explicit log will be displayed about the state of the documents filtered out.

  • Filtering is not applied to the dbi_document_revision_digitaltwin (in the Assai API). Filtering is applied after the call to dbi_document_revision_digitaltwin, so is applied to the response from dbi_document_revision_digitaltwin.

  • The Filtering section is not applied to the Delta Query Parameters as filtering is applied after the document revision is retrieved.

    Filtering can be configured by adding the following section within the DmsConfiguration section. The example is shown with sample values which must be replaced with actual values before adding to your configuration.

    "Filtering": {

    "Filteringitems": [

    {

    "identifier": "sampleIdentifier",

    "mapping": "sampleMapping",

    "value": "sampleValue"

    },

    {

    "identifier": "testIdentifier",

    "mapping": "testMapping",

    "value": "testValue"

    }

    ],

    "ValidationExpression": "{sampleIdentifier} or {testIdentifier}"

    }

• DateTimeFormat: is a string representation of the date format supported by DMS. Different DMSs may use different formats so we need to convert our date to the DMS format before starting any request that contains a date. Assai currently use the following format: "yyyy-MMddTHH:mm:ssZ"

  Limitation: The DateTimeFormat for Assai must be "yyyy-MM-ddTHH:mm:ssZ". If this parameter value is not as mentioned, an empty delta is returned, and no document is processed.

• ProjectsEndpoint: endpoint that returns the projects list.

• PackagesEndpoint: endpoint that returns the packages list.

• FileTypeLookup: For Assai, one document may have multiple files. This property defines what kind of files to look for. Each type should be separated by a vertical-bar ('|'). Supported types are:

  • publish_file

  • source_file

    See the following example, which includes the required formatting:

    "FileTypeLookup": "publish_file:pdf|source_file:dwg,dxf"

    The FileTypeLookup value for publish_file can optionally include file extensions, for example:

  • publish_file - all file extensions are considered.

  • publish_file:fileextension1, fileextension2 - multiple file extensions can be specified, separated by comma.

    The FileTypeLookup value for source_file can optionally include file extensions, for example:

  • source_file - all file extensions are considered.

  • source_file:fileextension1, fileextension2 - multiple file extensions can be specified, separated by comma.

• DocumentAPIAddress: the endpoint that return all the document links and if the file ID is appended as a query string, it will download the document link for that specific document.

• LastRefreshDateTimeURL: Is actually the same endpoint as delta but it returns just one document in json. One of the document's metadata is last_refresh_date that represent the last time Assai did a refresh of all the files. We get and store this date in a db in order to use it as a starting date on the next delta call (it will be used as %DeltaDate% parameter).

• ProjectCode: Assai can work with multiple projects but Document Connector can extract documents from just one project. Each tenant must have a single project associated.

• DocumentDownloadEndpoint: This is the endpoint that will be used to download the actual file. The file ID will pe appended to the end of this endpoint in order to generate the final download link.

• DownloadDocumentAsset: Represents a boolean (it can have the value true or false) and marks the ability to download the document links. If the value is true, document links will be extracted and pushed to ingestion. If value is false then no document link will be extracted.

• DownloadProjects: Represents a boolean (it can have the value true or false) and marks the ability to download a list of projects that the user has access to. If the value is true, projects list will be extracted and pushed to ingestion. If value is false then project list will not be extracted.

• DownloadPackages: Represents a boolean (it can have the value true or false) and marks the ability to download a list of packages that the user has access to. If the value is true, packages list will be extracted and pushed to ingestion. If value is false then packages list will not be extracted.

MetadataAIMContext

MetadataAIMContext is a section where we configure what we send to the Ingestion API, such as paths, groups, naming formats, extensions, and so on.

• context: value sent to ingestion and used to group files for clients with multiple tenants.

• ClassLibraryRegisterPath

  • DocumentMetadataRegister: path for the document revisions.

  • ScrapedFileRegister: path for scraped files.

  • DocLinkAIMContext: path for document links.

  • ProjectAIMContext: path for projects list.

  • PackageAIMContext: path for packages list.

• SuffixFileTimeStampFormat: timestamp format that will be used for delta file name.

• FileNamePrefix: file name prefix for regi files, used for identification.

• FileNameSuffixForDelta

  • Suffix: delta filename suffix.

  • Extension: delta extension. If the response_type from DmsConfiguration is set to csv, then the extension must be “csv”. Same for any other response types.

AIMIngestion

AIMIngestion section is about ingestion configuration.

• BaseUrl: represents the base url for ingestion api.

• AssetId: represents the ID of the asset for which the upload is done.

• ProcessingUnit: represents the processing units with the attached allowed extension type.

HttpSettings

HttpSettings section configures timeouts and retries. Most of them are hardcoded in the template, only the DMS timeout must be configured by user.

• DmsTimeout: timeout used for DMS http calls (in minutes - minimum 7, the maximum 30).

PipelineRunnerSettings

PipelineRunnerSettings is a section that will configure the flow of the program. Most of the configurations are already hardcoded in the template.

• DownloadFiles: boolean, switch for toggling on and off the files transfer. If set to FALSE, only delta file and document links will be transferred.