Re: Vulnerabilities to UserGroupInformation / credentials in a Spark Cluster

2017-10-31 Thread Ravi Prakash
Hi Blaze! Thanks for the link, although it did not have anything I didn't already know. I'm afraid I don't quite follow what your concern is here. The files are protected using UNIX permissions on the worker nodes. Is that not what you are seeing? Are you using the LinuxContainerExecutor? Are the

Re: Unable to append to a file in HDFS

2017-10-31 Thread Ravi Prakash
HI Tarik! I'm glad you were able to diagnose your issue. Thanks for sharing with the user list. I suspect your writer may have set minimum replication to 3, and since you have only 2 datanodes, the Namenode will not allow you to successfully close the file. You could add another node or reduce

RE: How to add new journal nodes without service downtime?

2017-10-31 Thread Fu, Yong
From Cloudera’s guide, there should have a downtime when moving Jounal Nodes: https://www.cloudera.com/documentation/enterprise/5-7-x/topics/admin_nn_migrate_roles.html#concept_w3h_m2l_2r And a ticket from Community about this problem which is still unresolved:

Duplicate folder creation in hadoop file system configured with blob storage

2017-10-31 Thread Mallieswari Dineshbabu
Hi All, I have a hadoop cluster configured with default file system as wasb(Windows Azure Storage Blob) storage. When i upload a folder from local to HDFS through the hadoop commands it additionally create a file in same name for the folder which is uploaded. Example: *Here 'userdata' is a