Data collection

When the adapter starts, it begins scanning for files in the input directory that match the configured file name filter. These files are processed in the order of their creation time starting with the oldest. After a file has been processed, it will be moved to the output directory. If the input directory is empty when scanned by the adapter, the adapter waits five seconds before rescanning.

You must assign the following permissions to the adapter service account to successfully process structured data files:

To use discovery, you must also assign the read, write, and delete permissions to the adapter service account on the discovery directory.

If the adapter is not able to open a file from the input directory or move a file to the output directory after processing it, new files will not be processed until the problem is resolved. Do not open a file after it has been placed in the input directory as this could prevent the adapter from opening or moving it.

Supported file types

The adapter supports files that are in CSV, JSON, or XML format. The raw data files can be uncompressed or compressed as a zip, gzip, tar, or tar.gzip files. The files can have UTF-8, ASCII, or Unicode encoding.

Note: Compression is only supported at the individual file level. The adapter does not support compressed archives that contain multiple files.

Supported data types

The following table lists value data types that the adapter supports for data collection and types of streams that will be created.