Not sure how to explain it more clearly. 
You cannot communicate with plain text on a TLS port and vica versa: TLS comm. 
cannot be accepted on plaintext ports. It’s a general thing, not just ZooKeeper.

Andor



> On 2020. Mar 31., at 23:57, karthick rn <karthick.narend...@gmail.com> wrote:
> 
> Hi Andor,
> 
> I've tried the openssl command you shared but unable to get it working, may
> be something to do with converting to PEM format the keys and certs. I'll
> look into this.
> 
>> There’s no point trying non-secure communication on the secure port as
> it’s currently not unified.
> I'm not getting, please can you explain it?
> 
> Thanks,
> Karthick
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Tue, 31 Mar 2020 at 15:50, Patrick Hunt <ph...@apache.org> wrote:
> 
>> We (Karthick can you? :-) ) should add this to the docs. PR would be great!
>> https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute
>> 
>> Thanks,
>> 
>> Patrick
>> 
>> On Tue, Mar 31, 2020 at 7:17 AM Andor Molnar <an...@apache.org> wrote:
>> 
>>> Hi Karthick,
>>> 
>>> The following command works for me on the secure port (1181):
>>> 
>>> (echo "srvr"; sleep 1) | openssl s_client -connect zkhost:1181 -cert
>>> cert.pem -key ./key.pem
>>> 
>>> I had to add sleep, because openssl client closes the connection as soon
>>> as stdin ends.
>>> There’s no point trying non-secure communication on the secure port as
>>> it’s currently not unified.
>>> 
>>> Andor
>>> 
>>> 
>>> 
>>> 
>>> 
>>>> On 2020. Mar 31., at 15:22, karthick rn <karthick.narend...@gmail.com>
>>> wrote:
>>>> 
>>>> Thanks Enrico for sharing the jira. This is great!
>>>> 
>>>> With the below config, I'm now able to run the 4LW commands
>> successfully,
>>>> also the downstream systems that was relying on the 4LW commands
>> started
>>>> displaying the metrics. Thanks for your help.
>>>> 
>>>> #secureClientPort=2281
>>>> 
>>>> clientPort=2281
>>>> 
>>>> client.portUnification=True
>>>> 
>>>> 
>>>> - Karthick
>>>> 
>>>> 
>>>> 
>>>> On Mon, 30 Mar 2020 at 21:59, Enrico Olivelli <eolive...@gmail.com>
>>> wrote:
>>>> 
>>>>> You may be interested in Port unification, contributed by Facebook:
>>>>> 
>>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3388
>>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3371
>>>>> 
>>>>> Enrico
>>>>> 
>>>>> Il giorno lun 30 mar 2020 alle ore 13:33 karthick rn
>>>>> <karthick.narend...@gmail.com> ha scritto:
>>>>>> 
>>>>>> Hi Mate,
>>>>>> 
>>>>>> Thanks for suggesting these options in detail
>>>>>> 
>>>>>> 1) We are already using AdminServer as an alternate to the 4LW,
>>> hopefully
>>>>>> we'll look at modifying the downstream systems to use REST instead of
>>> the
>>>>>> 4LW commands.
>>>>>> 
>>>>>> 2) Added "clientPort=2181" back to the configs and tested "srvr" &
>>> other
>>>>>> whitelisted 4LW commands and they all work now :)
>>>>>> 
>>>>>> 3) When I configure the same port "2281" for both secure and unsecure
>>>>>> communication with "client.portUnification=true", the JVM exits with
>>> Bind
>>>>>> exception stating the "Address already in use" & unable to start ZK.
>>>>>> 
>>>>>> For short term, I think we'd run a mixed-mode communication like you
>>>>>> mentioned in option 2 & whitelist only specific 4LW commands required
>>> and
>>>>>> not all.
>>>>>> 
>>>>>> Appreciate if someone can confirm if the 4LW is expected to work
>>> against
>>>>>> secure client port or not so we can update the doc accordingly.
>> Thanks
>>>>>> again!
>>>>>> 
>>>>>> Regards,
>>>>>> Karthick
>>>>>> 
>>>>>> On Mon, 30 Mar 2020 at 09:30, Szalay-Bekő Máté <
>>>>> szalay.beko.m...@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Hi Karthick,
>>>>>>> 
>>>>>>> I am not sure if "echo srvr | nc localhost 2281" is expected to work
>>>>>>> against the secure client port. I don't think so, but maybe others
>>> know
>>>>>>> better. I think you have the following options:
>>>>>>> 
>>>>>>> 1) use the admin server which is a HTTP interface where the 4LW
>>>>> commands
>>>>>>> are available on a REST protocol (see
>>>>>>> 
>>>>> 
>>> 
>> https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_adminserver
>>>>>>> )
>>>>>>> 
>>>>>>> 2) if AdminServer is not an option for you, then you can configure
>>>>>>> ZooKeeper to use both secure and unsecure ports. And use the
>> unsecure
>>>>> port
>>>>>>> for 4LW commands, while use the secure port for the rest of the
>>>>> traffic.
>>>>>>> E.g.:
>>>>>>> clientPort=2281
>>>>>>> secureClientPort=2282
>>>>>>> 
>>>>>>> 3) you can even configure ZooKeeper to use the same port for both
>> TLS
>>>>> and
>>>>>>> unsecure communication. I haven't used 4LW commands with port
>>>>> unification,
>>>>>>> but I assume it works:
>>>>>>> client.portUnification=true
>>>>>>> 
>>>>>>> I hope some of these options will work for you.
>>>>>>> 
>>>>>>> Kind regards,
>>>>>>> Mate
>>>>>>> 
>>>>>>> On Mon, Mar 30, 2020 at 12:24 AM karthick rn <
>>>>> karthick.narend...@gmail.com
>>>>>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hello,
>>>>>>>> 
>>>>>>>> After configuring TLS, running "echo srvr | nc localhost 2281" or
>> any
>>>>>>> other
>>>>>>>> 4LW doesn’t show any output. The below messages are printed on the
>>>>> ZK log
>>>>>>>> whilst running the ‘srvr’ command. Also tried adding
>>>>>>>> "4lw.commands.whitelist=*" to zoo.cfg but still no difference.
>>>>> However,
>>>>>>>> disabling TLS I'm able to see all 4LW working as expected.
>>>>>>>> 
>>>>>>>> Let me know if this is a known issue when TLS is enabled? I'm using
>>>>> ZK
>>>>>>> v3.6
>>>>>>>> and have seen the same behaviour with v3.5.6 & 3.5.7.
>>>>>>>> 
>>>>>>>> I have shared my Quorum TLS configs at the bottom, in-case if you
>>>>> want to
>>>>>>>> check if I'm missing something. Many thanks
>>>>>>>> 
>>>>>>>> 
>>>>>>>> zookeeper.log:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 2020-03-29 21:09:27,079 [myid:1] - ERROR
>>>>>>>> 
>> [nioEventLoopGroup-4-1:NettyServerCnxnFactory$CertificateVerifier@434
>>> ]
>>>>> -
>>>>>>>> Unsuccessful handshake with session 0x0
>>>>>>>> 
>>>>>>>> 2020-03-29 21:09:27,083 [myid:1] - WARN
>>>>>>>> 
>> [nioEventLoopGroup-4-1:NettyServerCnxnFactory$CnxnChannelHandler@273
>>> ]
>>>>> -
>>>>>>>> Exception caught
>>>>>>>> 
>>>>>>>> io.netty.handler.codec.DecoderException:
>>>>>>>> io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:
>>>>>>>> 737276720a
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
>>>>>>>> 
>>>>>>>>   at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>> 
>>> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>>>>>>>> 
>>>>>>>>   at java.base/java.lang.Thread.run(Thread.java:834)
>>>>>>>> 
>>>>>>>> Caused by: io.netty.handler.ssl.NotSslRecordException: not an
>> SSL/TLS
>>>>>>>> record: 737276720a
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>> 
>>> io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1198)
>>>>>>>> 
>>>>>>>>   at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
>>>>>>>> 
>>>>>>>>   at
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
>>>>>>>> 
>>>>>>>>   ... 17 more
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> conf/zoo.cfg:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> # Server configuration
>>>>>>>> 
>>>>>>>> secureClientPort=2281
>>>>>>>> 
>>>>>>>> 
>> serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> # Quorum configuration
>>>>>>>> 
>>>>>>>> sslQuorum=true
>>>>>>>> 
>>>>>>>> ssl.quorum.keyStore.location=</path/to/keystore.jks>
>>>>>>>> 
>>>>>>>> ssl.quorum.keyStore.password=<password>
>>>>>>>> 
>>>>>>>> ssl.quorum.trustStore.location=</path/to/truststore.jks>
>>>>>>>> 
>>>>>>>> ssl.quorum.trustStore.password=<password>
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> bin/zkEnv.sh
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> SERVER_JVMFLAGS="-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
>>>>>>>> \
>>>>>>>> 
>>>>>>>> -Dzookeeper.ssl.keyStore.location=</path/to/keystore.jks> \
>>>>>>>> 
>>>>>>>> -Dzookeeper.ssl.keyStore.password=<password>\
>>>>>>>> 
>>>>>>>> -Dzookeeper.ssl.trustStore.location=</path/to/truststore.jks> \
>>>>>>>> 
>>>>>>>> -Dzookeeper.ssl.trustStore.password=<password>"
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> CLIENT_JVMFLAGS="-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty
>>>>>>>> \
>>>>>>>> 
>>>>>>>> -Dzookeeper.client.secure=true \
>>>>>>>> 
>>>>>>>> -Dzookeeper.ssl.keyStore.location=</path/to/keystore.jks> \
>>>>>>>> 
>>>>>>>> -Dzookeeper.ssl.keyStore.password=<password>\
>>>>>>>> 
>>>>>>>> -Dzookeeper.ssl.trustStore.location=</path/to/truststore.jks> \
>>>>>>>> 
>>>>>>>> -Dzookeeper.ssl.trustStore.password=<password>"
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> - Karthick
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>>> 
>> 

Reply via email to