Structured Data Files

Input Directory

Location of the source files to process. FTP servers are not supported.

Note: For the adapter to process data, the adapter service account must have read, write, and delete permissions for the input directory.

Output Directory

Location for the files to be moved to after being processed. FTP servers are not supported.

Note: For the adapter to process data, the adapter service account must have read, write, and delete permissions for the output directory.

Stream Id Prefix

The stream ID prefix applied to all data items collected from the data source.

Note: If you change the StreamIdPrefix of a configured adapter, for example when you delete and add a data source, you need to restart the adapter for the changes to take place. New streams are created on adapter restart and pre-existing streams are no longer updated.

Default Stream Id Pattern

Specifies the default stream ID pattern to use. For example, {FriendlyName}, {ValueField}

Note: The {FriendlyName} parameter can contain any text, the {ValueField} parameter contains JSONPath/XPath symbols, which are replaced according to Text parser.

Friendly Name

The label to use for the data source.

File Name Filter

Pattern used to match files in the InputDirectory for processing.

If no filter is specified, the adapter attempts to process all files in the InputDirectory. Use * as the wildcard character.

Purge Delay

Specifies a time after which processed files are removed from the output directory to save disk space. The value must be a valid time span string (minimum of zero seconds: 0.00:00:00.0000000 or null.

Note: If Purge Delay is set to null, files are never purged from the output directory.

Discovery Directory

Location for the files to be processed during discovery. FTP servers are not supported. For example, C:\\DiscoveryDirectory

Note: For the adapter to process data, the adapter service account must have read, write, and delete permissions for the discovery directory.

Has Header

Select the check box if a header line is present in the file. Only applies to CSV files.

Culture

Locale setting for the input files.

Note: The value associated with the culture parameter are only applied to the values read in from the .csv file and not the headers. For example, a culture of "en-US" parses floats like 1.234, while a culture of "fr-FR" parses floats like 1,234.

Time Zone

Time zone of timestamps in the input files. If specified, the value must be a valid entry from the IANA time zone database.

Default value: Etc/UTC

Example 1: America/Los_Angeles

Example 2: This example represents three dates and a value separated by commas.

Date1,Date2,Date3,Value

03-08-2020T02:59-05:00,03-08-2020T02:59,03-08-2020T02:59Z,12345.6789

Date1 = 03-08-2020T02:59-05:00

this date has a "-05:00" adjustment meaning the date will be UTC-5.

Date2 = 03-08-2020T02:59

this date is displayed with no adjustment and will make use of the default timezone properly.

Date3 = 03-08-2020T02:59Z

this date is displayed with a "Z" so will be UTC.

Value = 12345.6789

Format

Select the file format that the adapter uses to read structured data files from this data source. You can select between Csv, Json, or Xml.

Compression

Input file compression format. Valid selections are None, Zip, Tar, GZip, TarGZip

Encoding

Character encoding used in the input files. Valid selections are UTF8, ASCII, Unicode

Field Separator

Character used to delineate fields in the input files. Only applies to CSV files.

Line Separator

Specifies the character or sequence that marks the end of each line in the source file. Only applies to CSV files.

Select one of the following from the drop-down: CR, LF, CRLF.

These values correspond to the following Structured Data Files valid values:

CR = \r
LF = \n
CRLF = \r\n

Note: Verify the line termination (CR|LF or LF) in the CSV file matches the operating system where the adapter is running and your lineSeparator value is identical to the data source configuration.

If you are using Linux, the parameter should default to \n.

If you are using Windows, the parameter should default to \r\n.