RE: Error in znode creation after adding SASL digest on server and client
Sidebar: -Dcom.sun.management.jmxremote.authenticate=false why did you disable authentication? Martin __ > From: harsha...@gmail.com > Date: Fri, 8 Jul 2016 22:24:25 + > Subject: Re: Error in znode creation after adding SASL digest on server and > client > To: users@kafka.apache.org > > Hi, > So we specifically kept the consumers to world writable in secure > mode. This is to allow zookeeper based consumers to create their own child > nodes under /consumers and they can add their own sasl based acls on top of > it. From the looks of it incase of zookeeper digest based connection it > expects all the nodes to have an ACL on it. This could be an issue with > ZkClient tha we use or we need to navigate this case differently. Can you > file a JIRA for this. > > Thanks, > Harsha > > On Thu, Jul 7, 2016 at 10:48 PM Vipul Sharma> wrote: > > > I am running zookeeper and kafka on local machine. > > This is the user permission on zookeeper > > [zk: localhost:2181(CONNECTED) 0] getAcl / > > 'digest,'broker:TqgUewyrgBbYEWTfsNStYmIfD2Q= > > : cdrwa > > > > I am using the same user in kafka to connect to this local zookeeper > > > > /usr/lib/jvm/java-8-oracle-amd64/bin/java -Xmx200m -Xms200m > > -Djava.security.auth.login.config=/opt/kafka/config/jaas.conf -server > > -Djava.awt.headless=true -XX:PermSize=48m -XX:MaxPermSize=48m -XX:+UseG1GC > > -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 > > -Xloggc:/var/log/kafka/kafka-gc.log -XX:+PrintGCDateStamps > > -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote > > -Dcom.sun.management.jmxremote.authenticate=false > > -Dcom.sun.management.jmxremote.ssl=false > > -Dcom.sun.management.jmxremote.port= > > -Dkafka.logs.dir=/opt/kafka/bin/../logs > > -Dlog4j.configuration=file:/opt/kafka/config/log4j.properties -cp > > :/opt/kafka/bin/../libs/* kafka.Kafka /opt/kafka/config/server.properties > > > > root@default-ubuntu-1404:~# cat /opt/kafka/config/jaas.conf > > Client { > >org.apache.zookeeper.server.auth.DigestLoginModule required > >username=broker > >password=password; > > }; > > > > > > The kafka start fails with these logs > > > > [2016-07-08 05:43:32,326] INFO Client > > > > environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > > (org.apache.zookeeper.ZooKeeper) > > [2016-07-08 05:43:32,327] INFO Client environment:java.io.tmpdir=/tmp > > (org.apache.zookeeper.ZooKeeper) > > [2016-07-08 05:43:32,327] INFO Client environment:java.compiler= > > (org.apache.zookeeper.ZooKeeper) > > [2016-07-08 05:43:32,327] INFO Client environment:os.name=Linux > > (org.apache.zookeeper.ZooKeeper) > > [2016-07-08 05:43:32,328] INFO Client environment:os.arch=amd64 > > (org.apache.zookeeper.ZooKeeper) > > [2016-07-08 05:43:32,328] INFO Client > > environment:os.version=4.2.0-35-generic (org.apache.zookeeper.ZooKeeper) > > [2016-07-08 05:43:32,328] INFO Client environment:user.name=root > > (org.apache.zookeeper.ZooKeeper) > > [2016-07-08 05:43:32,329] INFO Client environment:user.home=/root > > (org.apache.zookeeper.ZooKeeper) > > [2016-07-08 05:43:32,329] INFO Client environment:user.dir=/root > > (org.apache.zookeeper.ZooKeeper) > > [2016-07-08 05:43:32,330] INFO Initiating client connection, > > connectString=default-ubuntu-1404:2181,localhost:2181 sessionTimeout=6000 > > watcher=org.I0Itec.zkclient.ZkClient@bef2d72 > > (org.apache.zookeeper.ZooKeeper) > > [2016-07-08 05:43:32,359] INFO Waiting for keeper state SaslAuthenticated > > (org.I0Itec.zkclient.ZkClient) > > [2016-07-08 05:43:32,362] INFO successfully logged in. > > (org.apache.zookeeper.Login) > > [2016-07-08 05:43:32,363] INFO Client will use DIGEST-MD5 as SASL > > mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient) > > [2016-07-08 05:43:32,507] INFO Opening socket connection to server > > localhost/0:0:0:0:0:0:0:1:2181. Will attempt to SASL-authenticate using > > Login Context section 'Client' (org.apache.zookeeper.ClientCnxn) > > [2016-07-08 05:43:32,519] INFO Socket connection established to > > localhost/0:0:0:0:0:0:0:1:2181, initiating session > > (org.apache.zookeeper.ClientCnxn) > > [2016-07-08 05:43:32,537] INFO Session establishment complete on server > > localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x155c8e99f690005, negotiated > > timeout = 6000 (org.apache.zookeeper.ClientCnxn) > > [2016-07-08 05:43:32,541] INFO zookeeper state changed (SyncConnected) > > (org.I0Itec.zkclient.ZkClient) > > [2016-07-08 05:43:32,564] INFO zookeeper state changed (SaslAuthenticated) > > (org.I0Itec.zkclient.ZkClient) > > [2016-07-08 05:43:32,614] FATAL Fatal error during KafkaServer startup. > > Prepare to shutdown (kafka.server.KafkaServer) > > org.I0Itec.zkclient.exception.ZkException: > > org.apache.zookeeper.KeeperException$NoAuthException:
Re: Error in znode creation after adding SASL digest on server and client
Hi, So we specifically kept the consumers to world writable in secure mode. This is to allow zookeeper based consumers to create their own child nodes under /consumers and they can add their own sasl based acls on top of it. From the looks of it incase of zookeeper digest based connection it expects all the nodes to have an ACL on it. This could be an issue with ZkClient tha we use or we need to navigate this case differently. Can you file a JIRA for this. Thanks, Harsha On Thu, Jul 7, 2016 at 10:48 PM Vipul Sharmawrote: > I am running zookeeper and kafka on local machine. > This is the user permission on zookeeper > [zk: localhost:2181(CONNECTED) 0] getAcl / > 'digest,'broker:TqgUewyrgBbYEWTfsNStYmIfD2Q= > : cdrwa > > I am using the same user in kafka to connect to this local zookeeper > > /usr/lib/jvm/java-8-oracle-amd64/bin/java -Xmx200m -Xms200m > -Djava.security.auth.login.config=/opt/kafka/config/jaas.conf -server > -Djava.awt.headless=true -XX:PermSize=48m -XX:MaxPermSize=48m -XX:+UseG1GC > -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 > -Xloggc:/var/log/kafka/kafka-gc.log -XX:+PrintGCDateStamps > -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote > -Dcom.sun.management.jmxremote.authenticate=false > -Dcom.sun.management.jmxremote.ssl=false > -Dcom.sun.management.jmxremote.port= > -Dkafka.logs.dir=/opt/kafka/bin/../logs > -Dlog4j.configuration=file:/opt/kafka/config/log4j.properties -cp > :/opt/kafka/bin/../libs/* kafka.Kafka /opt/kafka/config/server.properties > > root@default-ubuntu-1404:~# cat /opt/kafka/config/jaas.conf > Client { >org.apache.zookeeper.server.auth.DigestLoginModule required >username=broker >password=password; > }; > > > The kafka start fails with these logs > > [2016-07-08 05:43:32,326] INFO Client > > environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > (org.apache.zookeeper.ZooKeeper) > [2016-07-08 05:43:32,327] INFO Client environment:java.io.tmpdir=/tmp > (org.apache.zookeeper.ZooKeeper) > [2016-07-08 05:43:32,327] INFO Client environment:java.compiler= > (org.apache.zookeeper.ZooKeeper) > [2016-07-08 05:43:32,327] INFO Client environment:os.name=Linux > (org.apache.zookeeper.ZooKeeper) > [2016-07-08 05:43:32,328] INFO Client environment:os.arch=amd64 > (org.apache.zookeeper.ZooKeeper) > [2016-07-08 05:43:32,328] INFO Client > environment:os.version=4.2.0-35-generic (org.apache.zookeeper.ZooKeeper) > [2016-07-08 05:43:32,328] INFO Client environment:user.name=root > (org.apache.zookeeper.ZooKeeper) > [2016-07-08 05:43:32,329] INFO Client environment:user.home=/root > (org.apache.zookeeper.ZooKeeper) > [2016-07-08 05:43:32,329] INFO Client environment:user.dir=/root > (org.apache.zookeeper.ZooKeeper) > [2016-07-08 05:43:32,330] INFO Initiating client connection, > connectString=default-ubuntu-1404:2181,localhost:2181 sessionTimeout=6000 > watcher=org.I0Itec.zkclient.ZkClient@bef2d72 > (org.apache.zookeeper.ZooKeeper) > [2016-07-08 05:43:32,359] INFO Waiting for keeper state SaslAuthenticated > (org.I0Itec.zkclient.ZkClient) > [2016-07-08 05:43:32,362] INFO successfully logged in. > (org.apache.zookeeper.Login) > [2016-07-08 05:43:32,363] INFO Client will use DIGEST-MD5 as SASL > mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient) > [2016-07-08 05:43:32,507] INFO Opening socket connection to server > localhost/0:0:0:0:0:0:0:1:2181. Will attempt to SASL-authenticate using > Login Context section 'Client' (org.apache.zookeeper.ClientCnxn) > [2016-07-08 05:43:32,519] INFO Socket connection established to > localhost/0:0:0:0:0:0:0:1:2181, initiating session > (org.apache.zookeeper.ClientCnxn) > [2016-07-08 05:43:32,537] INFO Session establishment complete on server > localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x155c8e99f690005, negotiated > timeout = 6000 (org.apache.zookeeper.ClientCnxn) > [2016-07-08 05:43:32,541] INFO zookeeper state changed (SyncConnected) > (org.I0Itec.zkclient.ZkClient) > [2016-07-08 05:43:32,564] INFO zookeeper state changed (SaslAuthenticated) > (org.I0Itec.zkclient.ZkClient) > [2016-07-08 05:43:32,614] FATAL Fatal error during KafkaServer startup. > Prepare to shutdown (kafka.server.KafkaServer) > org.I0Itec.zkclient.exception.ZkException: > org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = > NoAuth for /consumers > at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:68) > at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:1000) > at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:527) > at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:293) > at kafka.utils.ZkPath$.createPersistent(ZkUtils.scala:938) > at kafka.utils.ZkUtils.makeSurePersistentPathExists(ZkUtils.scala:340) > at kafka.utils.ZkUtils$$anonfun$setupCommonPaths$1.apply(ZkUtils.scala:175) > at kafka.utils.ZkUtils$$anonfun$setupCommonPaths$1.apply(ZkUtils.scala:174) > at
Re: Error in znode creation after adding SASL digest on server and client
Hi, So we specifically kept the consumers to world writable in secure mode. This is to allow zookeeper based consumers to create their own child nodes under /consumers and they can add their own sasl based acls on top of it. From the looks of it incase of zookeeper digest based connection it expects all the nodes to have an ACL on it. This could be an issue with ZkClient tha we use or we need to navigate this case differently. Can you file a JIRA for this. Thanks, Harsha On Fri, Jul 8, 2016 at 3:24 PM Harsha Chwrote: > Hi, > So we specifically kept the consumers to world writable in secure > mode. This is to allow zookeeper based consumers to create their own child > nodes under /consumers and they can add their own sasl based acls on top of > it. From the looks of it incase of zookeeper digest based connection it > expects all the nodes to have an ACL on it. This could be an issue with > ZkClient tha we use or we need to navigate this case differently. Can you > file a JIRA for this. > > Thanks, > Harsha > > On Thu, Jul 7, 2016 at 10:48 PM Vipul Sharma > wrote: > >> I am running zookeeper and kafka on local machine. >> This is the user permission on zookeeper >> [zk: localhost:2181(CONNECTED) 0] getAcl / >> 'digest,'broker:TqgUewyrgBbYEWTfsNStYmIfD2Q= >> : cdrwa >> >> I am using the same user in kafka to connect to this local zookeeper >> >> /usr/lib/jvm/java-8-oracle-amd64/bin/java -Xmx200m -Xms200m >> -Djava.security.auth.login.config=/opt/kafka/config/jaas.conf -server >> -Djava.awt.headless=true -XX:PermSize=48m -XX:MaxPermSize=48m -XX:+UseG1GC >> -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 >> -Xloggc:/var/log/kafka/kafka-gc.log -XX:+PrintGCDateStamps >> -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote >> -Dcom.sun.management.jmxremote.authenticate=false >> -Dcom.sun.management.jmxremote.ssl=false >> -Dcom.sun.management.jmxremote.port= >> -Dkafka.logs.dir=/opt/kafka/bin/../logs >> -Dlog4j.configuration=file:/opt/kafka/config/log4j.properties -cp >> :/opt/kafka/bin/../libs/* kafka.Kafka /opt/kafka/config/server.properties >> >> root@default-ubuntu-1404:~# cat /opt/kafka/config/jaas.conf >> Client { >>org.apache.zookeeper.server.auth.DigestLoginModule required >>username=broker >>password=password; >> }; >> >> >> The kafka start fails with these logs >> >> [2016-07-08 05:43:32,326] INFO Client >> >> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >> (org.apache.zookeeper.ZooKeeper) >> [2016-07-08 05:43:32,327] INFO Client environment:java.io.tmpdir=/tmp >> (org.apache.zookeeper.ZooKeeper) >> [2016-07-08 05:43:32,327] INFO Client environment:java.compiler= >> (org.apache.zookeeper.ZooKeeper) >> [2016-07-08 05:43:32,327] INFO Client environment:os.name=Linux >> (org.apache.zookeeper.ZooKeeper) >> [2016-07-08 05:43:32,328] INFO Client environment:os.arch=amd64 >> (org.apache.zookeeper.ZooKeeper) >> [2016-07-08 05:43:32,328] INFO Client >> environment:os.version=4.2.0-35-generic (org.apache.zookeeper.ZooKeeper) >> [2016-07-08 05:43:32,328] INFO Client environment:user.name=root >> (org.apache.zookeeper.ZooKeeper) >> [2016-07-08 05:43:32,329] INFO Client environment:user.home=/root >> (org.apache.zookeeper.ZooKeeper) >> [2016-07-08 05:43:32,329] INFO Client environment:user.dir=/root >> (org.apache.zookeeper.ZooKeeper) >> [2016-07-08 05:43:32,330] INFO Initiating client connection, >> connectString=default-ubuntu-1404:2181,localhost:2181 sessionTimeout=6000 >> watcher=org.I0Itec.zkclient.ZkClient@bef2d72 >> (org.apache.zookeeper.ZooKeeper) >> [2016-07-08 05:43:32,359] INFO Waiting for keeper state SaslAuthenticated >> (org.I0Itec.zkclient.ZkClient) >> [2016-07-08 05:43:32,362] INFO successfully logged in. >> (org.apache.zookeeper.Login) >> [2016-07-08 05:43:32,363] INFO Client will use DIGEST-MD5 as SASL >> mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient) >> [2016-07-08 05:43:32,507] INFO Opening socket connection to server >> localhost/0:0:0:0:0:0:0:1:2181. Will attempt to SASL-authenticate using >> Login Context section 'Client' (org.apache.zookeeper.ClientCnxn) >> [2016-07-08 05:43:32,519] INFO Socket connection established to >> localhost/0:0:0:0:0:0:0:1:2181, initiating session >> (org.apache.zookeeper.ClientCnxn) >> [2016-07-08 05:43:32,537] INFO Session establishment complete on server >> localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x155c8e99f690005, negotiated >> timeout = 6000 (org.apache.zookeeper.ClientCnxn) >> [2016-07-08 05:43:32,541] INFO zookeeper state changed (SyncConnected) >> (org.I0Itec.zkclient.ZkClient) >> [2016-07-08 05:43:32,564] INFO zookeeper state changed (SaslAuthenticated) >> (org.I0Itec.zkclient.ZkClient) >> [2016-07-08 05:43:32,614] FATAL Fatal error during KafkaServer startup. >> Prepare to shutdown (kafka.server.KafkaServer) >> org.I0Itec.zkclient.exception.ZkException: >>
Error in znode creation after adding SASL digest on server and client
I am running zookeeper and kafka on local machine. This is the user permission on zookeeper [zk: localhost:2181(CONNECTED) 0] getAcl / 'digest,'broker:TqgUewyrgBbYEWTfsNStYmIfD2Q= : cdrwa I am using the same user in kafka to connect to this local zookeeper /usr/lib/jvm/java-8-oracle-amd64/bin/java -Xmx200m -Xms200m -Djava.security.auth.login.config=/opt/kafka/config/jaas.conf -server -Djava.awt.headless=true -XX:PermSize=48m -XX:MaxPermSize=48m -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -Xloggc:/var/log/kafka/kafka-gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port= -Dkafka.logs.dir=/opt/kafka/bin/../logs -Dlog4j.configuration=file:/opt/kafka/config/log4j.properties -cp :/opt/kafka/bin/../libs/* kafka.Kafka /opt/kafka/config/server.properties root@default-ubuntu-1404:~# cat /opt/kafka/config/jaas.conf Client { org.apache.zookeeper.server.auth.DigestLoginModule required username=broker password=password; }; The kafka start fails with these logs [2016-07-08 05:43:32,326] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper) [2016-07-08 05:43:32,327] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper) [2016-07-08 05:43:32,327] INFO Client environment:java.compiler= (org.apache.zookeeper.ZooKeeper) [2016-07-08 05:43:32,327] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper) [2016-07-08 05:43:32,328] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper) [2016-07-08 05:43:32,328] INFO Client environment:os.version=4.2.0-35-generic (org.apache.zookeeper.ZooKeeper) [2016-07-08 05:43:32,328] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper) [2016-07-08 05:43:32,329] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper) [2016-07-08 05:43:32,329] INFO Client environment:user.dir=/root (org.apache.zookeeper.ZooKeeper) [2016-07-08 05:43:32,330] INFO Initiating client connection, connectString=default-ubuntu-1404:2181,localhost:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@bef2d72 (org.apache.zookeeper.ZooKeeper) [2016-07-08 05:43:32,359] INFO Waiting for keeper state SaslAuthenticated (org.I0Itec.zkclient.ZkClient) [2016-07-08 05:43:32,362] INFO successfully logged in. (org.apache.zookeeper.Login) [2016-07-08 05:43:32,363] INFO Client will use DIGEST-MD5 as SASL mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient) [2016-07-08 05:43:32,507] INFO Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will attempt to SASL-authenticate using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn) [2016-07-08 05:43:32,519] INFO Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session (org.apache.zookeeper.ClientCnxn) [2016-07-08 05:43:32,537] INFO Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x155c8e99f690005, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn) [2016-07-08 05:43:32,541] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient) [2016-07-08 05:43:32,564] INFO zookeeper state changed (SaslAuthenticated) (org.I0Itec.zkclient.ZkClient) [2016-07-08 05:43:32,614] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) org.I0Itec.zkclient.exception.ZkException: org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /consumers at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:68) at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:1000) at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:527) at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:293) at kafka.utils.ZkPath$.createPersistent(ZkUtils.scala:938) at kafka.utils.ZkUtils.makeSurePersistentPathExists(ZkUtils.scala:340) at kafka.utils.ZkUtils$$anonfun$setupCommonPaths$1.apply(ZkUtils.scala:175) at kafka.utils.ZkUtils$$anonfun$setupCommonPaths$1.apply(ZkUtils.scala:174) at scala.collection.immutable.List.foreach(List.scala:381) at kafka.utils.ZkUtils.setupCommonPaths(ZkUtils.scala:174) at kafka.server.KafkaServer.initZk(KafkaServer.scala:298) at kafka.server.KafkaServer.startup(KafkaServer.scala:180) at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37) at kafka.Kafka$.main(Kafka.scala:67) at kafka.Kafka.main(Kafka.scala) Caused by: org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /consumers at org.apache.zookeeper.KeeperException.create(KeeperException.java:113) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:99) at