RE: Connecting Hadoop HA cluster via java client

Hi. 1. minimal configuration to connect HA namenode is below properties. zookeeper information does not necessary. dfs.nameservices dfs.ha.namenodes.${dfs.nameservices} dfs.namenode.rpc-address.${dfs.nameservices}.nn1 dfs.namenode.rpc-address.${dfs.nameservices}.nn2

Connecting Hadoop HA cluster via java client

Hi, I have two questions pertaining to accessing the hadoop ha cluster from java client. 1. Is it necessary to supply conf.set("dfs.ha.automatic-failover.enabled",true); and conf.set("ha.zookeeper.quorum","zk1.example.com:2181,zk2.example.com:2181,zk3.example.com:2181"); in addition to the

RE: Authentication Failure talking to Ranger KMS

Just for kicks I tried applying the patch in that ticket and it didn't have any effect. It makes sense because my issue is on CREATE, and the ticket only has to do with OPEN. Note that I don't have these issues using WebHDFS, only using httpfs, so it definitely seems like we're on the right

RE: how to add a shareable node label?

Hah, how so? I am confused as I was under impression that I needed sharing but not preemption. Let’s model this out. Assuming I got 4 “normal” machines node1-4, and two special node8 and node9 where JobA can be executed on. And I need two queues, ProdQ and TestQ equally sharing Node1-4, and a

Re: how to add a shareable node label?

Hi Frank Extremely sorry for the delay.. Yes, you are correct. Sharing feature of node label is not needed in your case. Existing node labels and a queue model could solve the problem. Thanks Sunil On Fri, Oct 7, 2016 at 11:59 PM Frank Luo wrote: > That is correct, Sunil.

RE: Authentication Failure talking to Ranger KMS

That seems promising. But shouldn't I be able to work around it by just ensuring that httpfs has all necessary privileges in the KMS service under Ranger? Thanks, Ben From: Wei-Chiu Chuang [weic...@cloudera.com] Sent: Tuesday, October 11, 2016 11:57 AM To:

Re: Authentication Failure talking to Ranger KMS

Somes to me you encountered this bug? HDFS-10481 If you’re using CDH, this is fixed in CDH5.5.5, CDH5.7.2 and CDH5.8.2 Wei-Chiu Chuang A very happy Clouderan > On Oct 11, 2016, at 8:38 AM, Benjamin Ross wrote: > >

Authentication Failure talking to Ranger KMS

All, I'm trying to use httpfs to write to an encryption zone with security off. I can read from an encryption zone, but I can't write to one. Here's the applicable namenode logs. httpfs and root both have all possible privileges in the KMS. What am I missing? 2016-10-07 15:48:16,164 DEBUG