Determine absolute path of a file in hdfs

Hi, I have a situation where I need to determine the absolute path of a file in hadoop as a String. The path should exclude the schema and authority. Eg: /mn/op/data/t1.dat I found the following option in hadoop api: https://hadoop.apache.org/docs/r2.6.2/api/org/apache/hadoop/fs/Path.html#getPath

Socket Timeout Exception while multiple concurrent applications are reading HDFS data through WebHDFS interface

Hi, We are seeing this SocketTImeout exception while a number of concurrent applications (probably, 50 of them) are trying to read HDFS data through WebHDFS interface. Are there any parameters we can tune so it doesn't happen? An exception occurred: java.net.SocketTimeoutException: Read timed ou

Re: Determine absolute path of a file in hdfs

Hello, Yes, path.toUri().getPath() is a reliable way to get the absolute path without scheme or authority as a String from a Path instance. The Hadoop codebase itself uses this same pattern throughout a lot of JUnit tests when we need to run assertions on an absolute path. --Chris Nauroth Fr

Re: Socket Timeout Exception while multiple concurrent applications are reading HDFS data through WebHDFS interface

Hello Krishna, Judging from that stack trace, it looks like you have written a client application that uses Apache HTTP Components to make HTTP calls to WebHDFS. The exception is a client-side timeout. I recommend troubleshooting this from the perspective of that client application. The HDFS

Re: Unsubscribe

On 12/02/2015 03:45 PM, Sath wrote: Namikaze I know its a mailing list. I am not expecting you to do my dirty work. First of all who are you. I didn't address you. Actually, yes, you did address him. By sending an e-mail to this mailing list, you are addressing it to all subscribers to t

How do I run ZKFC on a different node that is not the name node?

I am receiving this error when trying to format zookeeper using the zkfc: Exception in thread "main" org.apache.hadoop.HadoopIllegalArgumentException: Could not get the namenode ID of this node. You may run zkfc on the node other than namenode. at org.apache.hadoop.hdfs.tools.DFSZKFailover

Re: Determine absolute path of a file in hdfs

Thanks Chris for the confirmation. Richards Peter. On Wed, Dec 9, 2015 at 11:59 PM, Chris Nauroth wrote: > Hello, > > Yes, path.toUri().getPath() is a reliable way to get the absolute path > without scheme or authority as a String from a Path instance. The Hadoop > codebase itself uses this sa