SAP Data Hub 2.4 Operators Error - Connectivity > Write File (HDFS)

[hwang_in_nam]

Hello

In the modeler, Write File (HDFS) does not work. An error occurs.

Question .
1) How do I check the detailed log?

2) Is there a wrong in setting?

1. Config

- write file config 1

- write file config 2

2. Error message

3. HDFS start

docker run --name hdfs --hostname hdfs -p 22022:22 -p 9000:9000
-p 50010:50010 -p 50020:50020 -p 50070:50070 -p 50075:50075 -p 50090:50090
--add-host=hdfs:192.168.10.221
--add-host=localhost:192.168.10.221
--env ADVERTISED_HOST=192.168.10.221 --env ADVERTISED_PORT=50070
--env ADVERTISED_PORT=9000 -d dockerq/docker-hdfs

docker exec -it hdfs bash

hdfs dfs -mkdir -p /webhdfs/v1

hdfs dfs -chmod -R 777 /webhdfs

best regards.

inam

[michael_eaton3 · Accepted solution]

Hello

Looking at your screenshot, it is trying to write to http://hdfs:50075. The error message is 'no such host'. Is the host reachable (via hostname) from the Data Hub cluster? I see you used an IP address to connect, however WebHDFS write is a 2 step process with a redirect, and the redirect is returning a URL with a hostname rather than the IP address.

Michael

[hwang_in_nam]

Hi .

the problem has been resolved changing the hdfs start option.

docker run --name hdfs --hostname hdfs -p 22022:22 -p 9000:9000

=>

docker run --name hdfs --hostname 192.168.10.221 -p 22022:22 -p 9000:9000

Thanks, Michael