Can anyone please help me here.
On Tue, May 28, 2019 at 2:45 PM Behroz Sikander wrote:
> Hello,
> I have a client application which uses HDFS API to push data. My HDFS is
> setup in HA mode.
> My client application has certain timeout requirements e.g. an upload
> shouldn't take longer than 10 seconds otherwise the operation is considered
> a failure.
>
> Sometimes, I notice the following exceptions in my code
>
> 2019-05-23 14:13:31,356 INFO Thread-11 org.apache.hadoop.hdfs.DFSClient []:
> Exception in createBlockOutputStream
> java.io.IOException: Got error, status message , ack with firstBadLink as
> xx.xx.xx.xx:50010
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:142)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1359)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1262)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:448)2019-05-23
> 14:13:31,357 INFO Thread-11 org.apache.hadoop.hdfs.DFSClient []: Abandoning
> BP-1672040070-127.0.0.1-1527078068582:blk_1073742599_17752019-05-23
> 14:13:31,384 INFO Thread-11 org.apache.hadoop.hdfs.DFSClient []: Excluding
> datanode
> DatanodeInfoWithStorage[xx.xx.xx.xx:50010,DS-4be6740a-bad8-438b-99d6-fbc50d7760dd,DISK]
>
>
> Even though the upload succeeds but takes around *20 seconds*.
>
> I am interested in the configuration property which controls this timeout, so
> I can adjust it
>
> according to my client's requirements. Or is there any configuration which
> says
>
> if the operation is not completed within X seconds, then HDFS API throws an
> exception?
>
> Can it be ipc.client.connect.timeout?
>
>
> Thanks.
>
> Behroz
>
>
