Configure the Hadoop Distributed File System (HDFS) target
- Last UpdatedMay 21, 2024
- 2 minute read
Complete the procedure below to configure the Hadoop Distributed File System target and verify that the HDFS writer is working.
Create the Hadoop Distributed File System target before you configure it. For more information, see Add a publish target.
-
On the Administration page, click the Targets tab. Then select the target in the Publish Targets list.
Note: If you have already selected your target, skip to the next step.
-
Configure the Hadoop Distributed File System target with the following parameters:
Parameter
Description
HDFS URL
URL to access HDFS through WebHDFS directly, or through the Apache Knox gateway or HttpFS.
Examples:
HDFS – http://host:50070/webhdfs/v1
HttpFS – http://host:14000/webhdfs/v1
Apache Knox – https://host:8443/gateway/default/webhdfs/v1
Verify TLS/SSL Certificate
Select the check box if you are using an TLS/SSL certificate. Clear the check box if you are using a self-signed certificate.
Username
User name used to connect to HDFS. The user must have permissions to read and write to HDFS.
Password
(Optional) Password to authenticate user. It is used when connecting to Apache Knox or when connecting to HttpFS if security is enabled. If a password is provided, then Hadoop HTTP authentication is used. If no password is entered, then PI Integrator for Business Analytics uses Hadoop simple authentication.
Directory
Directory where the data files are created. Click Browse to navigate to the directory where you want the files located.
Append Timestamp
When selected, a time stamp of when the view is published is appended to the file name and a new file is created each time the view is published. If not selected, then the data is appended to the existing file.
Include Header
When selected, column names are added to the beginning of the file.
-
Click Verify HDFS Writer to verify that the HDFS writer can connect to and write to HDFS.
Note: If you are unable to successfully write to the HDFS, refer to Tips for setting up Hadoop Distributed File System (HDFS).
-
Click Save Changes.
-
Give users access to the Hadoop Distributed File System target. For more information, see Grant access to targets.