Re: Zookeeper 3.5 SSL and Kerberos authentication

2019-12-17 Thread Jörn Franke 
Thanks a lot for the invesrtigation. I did not find again the time to
install it somewhere. I observed the issue with 3.5.5

On Tue, Dec 17, 2019 at 6:58 PM Andor Molnar  wrote:

> "We were using early 3.5.3 or something like that.”
>
> Netty stack had a major refactor in 3.5.5
>
> Andor
>
>
>
> > On 2019. Dec 17., at 16:40, Enrico Olivelli  wrote:
> >
> > Il giorno mar 17 dic 2019 alle ore 16:26 Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com> ha scritto:
> >
> >> I added a comment on Jira. This is something we will also need to fix
> in my
> >> company soon.
> >>
> >> @enrico you wrote:
> >>> in my company we set up some ZK with TLS and SASL, using TLS for
> >> encryption and SASL for auth. We were using early 3.5.3 or something
> like
> >> that.
> >>
> >
> > Unfortunately we do not have that setup anymore, we had to drop it
> because
> > at that time (and still nowadays) from the same JVMs we had also to
> connect
> > to an HBase cluster with ZK 3.4
> > that does not support TLS.
> >
> > Currently we are using only SASL and not TLS
> > Sorry
> >
> > Enrico
> >
> >
> >>
> >> According to this, the scenario should work. Maybe we just misconfigured
> >> something, or this was something got broken in a later version? Can you
> >> share the config you use? Maybe you are setting
> `zookeeper.ssl.clientAuth`
> >> and `zookeeper.ssl.quorum.clientAuth` to `none` or `want` ?
> >>
> >> Regards,
> >> Mate
> >>
> >> On Tue, Dec 17, 2019 at 10:48 AM Andor Molnar  wrote:
> >>
> >>> Hi Jorn,
> >>>
> >>> Sorry for coming back late to this. I’ve just validated the scenario on
> >> my
> >>> test cluster. Looks like the issue is valid: Kerberos auth and SSL are
> >>> mutually exclusive currently. When Kerberos is set up and trying to
> >> connect
> >>> to secure port I got an infinite loop on client side:
> >>>
> >>> 2019-12-17 01:43:30,984 [myid:barbaresco-1.vpc.cloudera.com:2182] -
> WARN
> >>> [Thread-39:Login$1@197] - TGT renewal thread has been interrupted and
> >>> will exit.
> >>> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] -
> INFO
> >>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182):Login@302] -
> Client
> >>> successfully logged in.
> >>> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] -
> INFO
> >>> [Thread-40:Login$1@135] - TGT refresh thread started.
> >>> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] -
> INFO
> >>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
> ):SecurityUtils$1@124
> >> ]
> >>> - Client will use GSSAPI as SASL mechanism.
> >>> 2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] -
> INFO
> >>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
> >>> ):ClientCnxn$SendThread@1112] - Opening socket connection to server
> >>> barbaresco-1.vpc.cloudera.com/10.65.25.98:2182. Will attempt to
> >>> SASL-authenticate using Login Context section 'Client'
> >>> 2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] -
> INFO
> >>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
> >>> ):ClientCnxn$SendThread@959] - Socket connection established,
> initiating
> >>> session, client: /10.65.25.98:45362, server:
> >>> barbaresco-1.vpc.cloudera.com/10.65.25.98:2182
> >>> 2019-12-17
> >>> 
> >>> 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> >>> [Thread-40:Login@320] - TGT valid starting at:Tue Dec 17
> >> 01:43:30
> >>> PST 2019
> >>> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] -
> INFO
> >>> [Thread-40:Login@321] - TGT expires:  Thu Jan 16
> >> 01:43:30
> >>> PST 2020
> >>> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] -
> INFO
> >>> [Thread-40:Login$1@193] - TGT refresh sleeping until: Fri Jan 10
> >> 20:23:33
> >>> PST 2020
> >>> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] -
> INFO
> >>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
> >>> ):ClientCnxn$SendThread@1240] - Unable to read additional data from
> >>> server sessionid 0x0, likely server has closed socket, closing socket
> >>> connection and attempting reconnect
> >>>
> >>> And the following error on server side:
> >>>
> >>> 2019-12-17 01:43:33,002 INFO
> >>> org.apache.zookeeper.server.NettyServerCnxnFactory: SSL handler added
> for
> >>> channel: [id: 0xcf37c14b, L:/10.65.25.98:2182 - R:/10.65.25.98:45380]
> >>> 2019-12-17 01:43:33,003 ERROR
> >>> org.apache.zookeeper.server.NettyServerCnxnFactory: Unsuccessful
> >> handshake
> >>> with session 0x0
> >>> 2019-12-17 01:43:33,003 WARN
> >>> org.apache.zookeeper.server.NettyServerCnxnFactory: Exception caught
> >>> io.netty.handler.codec.DecoderException:
> >>> io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:
> >>>
> >>
> 002d7530001000
> >>>at
> >>>
> >>
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(Byte

Re: Zookeeper 3.5 SSL and Kerberos authentication

2019-12-17 Thread Andor Molnar 
"We were using early 3.5.3 or something like that.”

Netty stack had a major refactor in 3.5.5

Andor



> On 2019. Dec 17., at 16:40, Enrico Olivelli  wrote:
> 
> Il giorno mar 17 dic 2019 alle ore 16:26 Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com> ha scritto:
> 
>> I added a comment on Jira. This is something we will also need to fix in my
>> company soon.
>> 
>> @enrico you wrote:
>>> in my company we set up some ZK with TLS and SASL, using TLS for
>> encryption and SASL for auth. We were using early 3.5.3 or something like
>> that.
>> 
> 
> Unfortunately we do not have that setup anymore, we had to drop it because
> at that time (and still nowadays) from the same JVMs we had also to connect
> to an HBase cluster with ZK 3.4
> that does not support TLS.
> 
> Currently we are using only SASL and not TLS
> Sorry
> 
> Enrico
> 
> 
>> 
>> According to this, the scenario should work. Maybe we just misconfigured
>> something, or this was something got broken in a later version? Can you
>> share the config you use? Maybe you are setting `zookeeper.ssl.clientAuth`
>> and `zookeeper.ssl.quorum.clientAuth` to `none` or `want` ?
>> 
>> Regards,
>> Mate
>> 
>> On Tue, Dec 17, 2019 at 10:48 AM Andor Molnar  wrote:
>> 
>>> Hi Jorn,
>>> 
>>> Sorry for coming back late to this. I’ve just validated the scenario on
>> my
>>> test cluster. Looks like the issue is valid: Kerberos auth and SSL are
>>> mutually exclusive currently. When Kerberos is set up and trying to
>> connect
>>> to secure port I got an infinite loop on client side:
>>> 
>>> 2019-12-17 01:43:30,984 [myid:barbaresco-1.vpc.cloudera.com:2182] - WARN
>>> [Thread-39:Login$1@197] - TGT renewal thread has been interrupted and
>>> will exit.
>>> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
>>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182):Login@302] - Client
>>> successfully logged in.
>>> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
>>> [Thread-40:Login$1@135] - TGT refresh thread started.
>>> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
>>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182):SecurityUtils$1@124
>> ]
>>> - Client will use GSSAPI as SASL mechanism.
>>> 2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
>>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
>>> ):ClientCnxn$SendThread@1112] - Opening socket connection to server
>>> barbaresco-1.vpc.cloudera.com/10.65.25.98:2182. Will attempt to
>>> SASL-authenticate using Login Context section 'Client'
>>> 2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
>>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
>>> ):ClientCnxn$SendThread@959] - Socket connection established, initiating
>>> session, client: /10.65.25.98:45362, server:
>>> barbaresco-1.vpc.cloudera.com/10.65.25.98:2182
>>> 2019-12-17
>>> 
>>> 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
>>> [Thread-40:Login@320] - TGT valid starting at:Tue Dec 17
>> 01:43:30
>>> PST 2019
>>> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
>>> [Thread-40:Login@321] - TGT expires:  Thu Jan 16
>> 01:43:30
>>> PST 2020
>>> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
>>> [Thread-40:Login$1@193] - TGT refresh sleeping until: Fri Jan 10
>> 20:23:33
>>> PST 2020
>>> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
>>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
>>> ):ClientCnxn$SendThread@1240] - Unable to read additional data from
>>> server sessionid 0x0, likely server has closed socket, closing socket
>>> connection and attempting reconnect
>>> 
>>> And the following error on server side:
>>> 
>>> 2019-12-17 01:43:33,002 INFO
>>> org.apache.zookeeper.server.NettyServerCnxnFactory: SSL handler added for
>>> channel: [id: 0xcf37c14b, L:/10.65.25.98:2182 - R:/10.65.25.98:45380]
>>> 2019-12-17 01:43:33,003 ERROR
>>> org.apache.zookeeper.server.NettyServerCnxnFactory: Unsuccessful
>> handshake
>>> with session 0x0
>>> 2019-12-17 01:43:33,003 WARN
>>> org.apache.zookeeper.server.NettyServerCnxnFactory: Exception caught
>>> io.netty.handler.codec.DecoderException:
>>> io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:
>>> 
>> 002d7530001000
>>>at
>>> 
>> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:475)
>>>at
>>> 
>> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
>>>at
>>> 
>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
>>>at
>>> 
>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
>>>at
>>> 
>> io.netty.c

Re: Zookeeper 3.5 SSL and Kerberos authentication

2019-12-17 Thread Enrico Olivelli 
Il giorno mar 17 dic 2019 alle ore 16:26 Szalay-Bekő Máté <
szalay.beko.m...@gmail.com> ha scritto:

> I added a comment on Jira. This is something we will also need to fix in my
> company soon.
>
> @enrico you wrote:
> > in my company we set up some ZK with TLS and SASL, using TLS for
> encryption and SASL for auth. We were using early 3.5.3 or something like
> that.
>

Unfortunately we do not have that setup anymore, we had to drop it because
at that time (and still nowadays) from the same JVMs we had also to connect
to an HBase cluster with ZK 3.4
that does not support TLS.

Currently we are using only SASL and not TLS
Sorry

Enrico


>
> According to this, the scenario should work. Maybe we just misconfigured
> something, or this was something got broken in a later version? Can you
> share the config you use? Maybe you are setting `zookeeper.ssl.clientAuth`
> and `zookeeper.ssl.quorum.clientAuth` to `none` or `want` ?
>
> Regards,
> Mate
>
> On Tue, Dec 17, 2019 at 10:48 AM Andor Molnar  wrote:
>
> > Hi Jorn,
> >
> > Sorry for coming back late to this. I’ve just validated the scenario on
> my
> > test cluster. Looks like the issue is valid: Kerberos auth and SSL are
> > mutually exclusive currently. When Kerberos is set up and trying to
> connect
> > to secure port I got an infinite loop on client side:
> >
> > 2019-12-17 01:43:30,984 [myid:barbaresco-1.vpc.cloudera.com:2182] - WARN
> > [Thread-39:Login$1@197] - TGT renewal thread has been interrupted and
> > will exit.
> > 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> > [main-SendThread(barbaresco-1.vpc.cloudera.com:2182):Login@302] - Client
> > successfully logged in.
> > 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> > [Thread-40:Login$1@135] - TGT refresh thread started.
> > 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> > [main-SendThread(barbaresco-1.vpc.cloudera.com:2182):SecurityUtils$1@124
> ]
> > - Client will use GSSAPI as SASL mechanism.
> > 2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> > [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
> > ):ClientCnxn$SendThread@1112] - Opening socket connection to server
> > barbaresco-1.vpc.cloudera.com/10.65.25.98:2182. Will attempt to
> > SASL-authenticate using Login Context section 'Client'
> > 2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> > [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
> > ):ClientCnxn$SendThread@959] - Socket connection established, initiating
> > session, client: /10.65.25.98:45362, server:
> > barbaresco-1.vpc.cloudera.com/10.65.25.98:2182
> > 2019-12-17
> > 
> > 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> > [Thread-40:Login@320] - TGT valid starting at:Tue Dec 17
> 01:43:30
> > PST 2019
> > 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> > [Thread-40:Login@321] - TGT expires:  Thu Jan 16
> 01:43:30
> > PST 2020
> > 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> > [Thread-40:Login$1@193] - TGT refresh sleeping until: Fri Jan 10
> 20:23:33
> > PST 2020
> > 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> > [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
> > ):ClientCnxn$SendThread@1240] - Unable to read additional data from
> > server sessionid 0x0, likely server has closed socket, closing socket
> > connection and attempting reconnect
> >
> > And the following error on server side:
> >
> > 2019-12-17 01:43:33,002 INFO
> > org.apache.zookeeper.server.NettyServerCnxnFactory: SSL handler added for
> > channel: [id: 0xcf37c14b, L:/10.65.25.98:2182 - R:/10.65.25.98:45380]
> > 2019-12-17 01:43:33,003 ERROR
> > org.apache.zookeeper.server.NettyServerCnxnFactory: Unsuccessful
> handshake
> > with session 0x0
> > 2019-12-17 01:43:33,003 WARN
> > org.apache.zookeeper.server.NettyServerCnxnFactory: Exception caught
> > io.netty.handler.codec.DecoderException:
> > io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:
> >
> 002d7530001000
> > at
> >
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:475)
> > at
> >
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
> > at
> >
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> > at
> >
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> > at
> >
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> > at
> >
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> > at
> >
> i

Re: Zookeeper 3.5 SSL and Kerberos authentication

2019-12-17 Thread Szalay-Bekő Máté 
I added a comment on Jira. This is something we will also need to fix in my
company soon.

@enrico you wrote:
> in my company we set up some ZK with TLS and SASL, using TLS for
encryption and SASL for auth. We were using early 3.5.3 or something like
that.

According to this, the scenario should work. Maybe we just misconfigured
something, or this was something got broken in a later version? Can you
share the config you use? Maybe you are setting `zookeeper.ssl.clientAuth`
and `zookeeper.ssl.quorum.clientAuth` to `none` or `want` ?

Regards,
Mate

On Tue, Dec 17, 2019 at 10:48 AM Andor Molnar  wrote:

> Hi Jorn,
>
> Sorry for coming back late to this. I’ve just validated the scenario on my
> test cluster. Looks like the issue is valid: Kerberos auth and SSL are
> mutually exclusive currently. When Kerberos is set up and trying to connect
> to secure port I got an infinite loop on client side:
>
> 2019-12-17 01:43:30,984 [myid:barbaresco-1.vpc.cloudera.com:2182] - WARN
> [Thread-39:Login$1@197] - TGT renewal thread has been interrupted and
> will exit.
> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182):Login@302] - Client
> successfully logged in.
> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> [Thread-40:Login$1@135] - TGT refresh thread started.
> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182):SecurityUtils$1@124]
> - Client will use GSSAPI as SASL mechanism.
> 2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
> ):ClientCnxn$SendThread@1112] - Opening socket connection to server
> barbaresco-1.vpc.cloudera.com/10.65.25.98:2182. Will attempt to
> SASL-authenticate using Login Context section 'Client'
> 2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
> ):ClientCnxn$SendThread@959] - Socket connection established, initiating
> session, client: /10.65.25.98:45362, server:
> barbaresco-1.vpc.cloudera.com/10.65.25.98:2182
> 2019-12-17
> 
> 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> [Thread-40:Login@320] - TGT valid starting at:Tue Dec 17 01:43:30
> PST 2019
> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> [Thread-40:Login@321] - TGT expires:  Thu Jan 16 01:43:30
> PST 2020
> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> [Thread-40:Login$1@193] - TGT refresh sleeping until: Fri Jan 10 20:23:33
> PST 2020
> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO
> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182
> ):ClientCnxn$SendThread@1240] - Unable to read additional data from
> server sessionid 0x0, likely server has closed socket, closing socket
> connection and attempting reconnect
>
> And the following error on server side:
>
> 2019-12-17 01:43:33,002 INFO
> org.apache.zookeeper.server.NettyServerCnxnFactory: SSL handler added for
> channel: [id: 0xcf37c14b, L:/10.65.25.98:2182 - R:/10.65.25.98:45380]
> 2019-12-17 01:43:33,003 ERROR
> org.apache.zookeeper.server.NettyServerCnxnFactory: Unsuccessful handshake
> with session 0x0
> 2019-12-17 01:43:33,003 WARN
> org.apache.zookeeper.server.NettyServerCnxnFactory: Exception caught
> io.netty.handler.codec.DecoderException:
> io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:
> 002d7530001000
> at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:475)
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:483)
> at
> io.netty.channe

Re: Zookeeper 3.5 SSL and Kerberos authentication

2019-12-17 Thread Andor Molnar 
Hi Jorn,

Sorry for coming back late to this. I’ve just validated the scenario on my test 
cluster. Looks like the issue is valid: Kerberos auth and SSL are mutually 
exclusive currently. When Kerberos is set up and trying to connect to secure 
port I got an infinite loop on client side:

2019-12-17 01:43:30,984 [myid:barbaresco-1.vpc.cloudera.com:2182] - WARN  
[Thread-39:Login$1@197] - TGT renewal thread has been interrupted and will exit.
2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO  
[main-SendThread(barbaresco-1.vpc.cloudera.com:2182):Login@302] - Client 
successfully logged in.
2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO  
[Thread-40:Login$1@135] - TGT refresh thread started.
2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO  
[main-SendThread(barbaresco-1.vpc.cloudera.com:2182):SecurityUtils$1@124] - 
Client will use GSSAPI as SASL mechanism.
2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO  
[main-SendThread(barbaresco-1.vpc.cloudera.com:2182):ClientCnxn$SendThread@1112]
 - Opening socket connection to server 
barbaresco-1.vpc.cloudera.com/10.65.25.98:2182. Will attempt to 
SASL-authenticate using Login Context section 'Client'
2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO  
[main-SendThread(barbaresco-1.vpc.cloudera.com:2182):ClientCnxn$SendThread@959] 
- Socket connection established, initiating session, client: 
/10.65.25.98:45362, server: barbaresco-1.vpc.cloudera.com/10.65.25.98:2182
2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO  
[Thread-40:Login@320] - TGT valid starting at:Tue Dec 17 01:43:30 PST 
2019
2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO  
[Thread-40:Login@321] - TGT expires:  Thu Jan 16 01:43:30 PST 
2020
2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO  
[Thread-40:Login$1@193] - TGT refresh sleeping until: Fri Jan 10 20:23:33 PST 
2020
2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO  
[main-SendThread(barbaresco-1.vpc.cloudera.com:2182):ClientCnxn$SendThread@1240]
 - Unable to read additional data from server sessionid 0x0, likely server has 
closed socket, closing socket connection and attempting reconnect

And the following error on server side:

2019-12-17 01:43:33,002 INFO 
org.apache.zookeeper.server.NettyServerCnxnFactory: SSL handler added for 
channel: [id: 0xcf37c14b, L:/10.65.25.98:2182 - R:/10.65.25.98:45380]
2019-12-17 01:43:33,003 ERROR 
org.apache.zookeeper.server.NettyServerCnxnFactory: Unsuccessful handshake with 
session 0x0
2019-12-17 01:43:33,003 WARN 
org.apache.zookeeper.server.NettyServerCnxnFactory: Exception caught
io.netty.handler.codec.DecoderException: 
io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 
002d7530001000
at 
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:475)
at 
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
at 
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
at 
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
at 
io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
at 
io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:483)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:383)
at 
io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
at 
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at 
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)

I will update the Jira too.

Andor





> On 2019. Nov 8., at 20:31, Jörn Franke  wrote:
> 
> Thanks. Can you please share the configuration file?
> 
> I tried with 3.5.5 - without SSL Kerberos works, but once I configured client 
> ssl it said authentication fail (I have to check if I can dig up the log 
> files) and as far as I remember this was related to x509 authentication. The 
> cert

Re: Zookeeper 3.5 SSL and Kerberos authentication

2019-11-08 Thread Jörn Franke 
Thanks. Can you please share the configuration file?

I tried with 3.5.5 - without SSL Kerberos works, but once I configured client 
ssl it said authentication fail (I have to check if I can dig up the log files) 
and as far as I remember this was related to x509 authentication. The 
certificate and truststore themselves are fine (I think I needed to convert the 
truststore to jks).
Sorry it was some time ago, I should have separated the log files. 
For me it did not matter that the ports are separated, but it worked on the 
non-ssl port fine.

> Am 06.11.2019 um 23:08 schrieb Enrico Olivelli :
> 
> Jorn,
> IIRC in my company we set up some ZK with TLS and SASL, using TLS for
> encryption and SASL for auth.
> We were using early 3.5.3 or something like that.
> 
> Do you have a specific error?
> 
> I can also add that in 3.6.0 we will have port-unification, this way you
> can configure only one client port and accept plain text and TLS connection
> from clients (this helps the ttransition to TLS)
> 
> Enrico
> 
> Il mer 6 nov 2019, 22:28 Jörn Franke  ha scritto:
> 
>> Dear all,
>> 
>> it seems that ZooKeeper 3.5 with SSL enabled does not support Kerberos
>> authentication, but only X509 authentication. Kerberos is used in many
>> Enterprise environments and is supported by Apache Solr. Is this a bug? Or
>> am I missing something?
>> 
>> 
>> I created a Jira for this:
>> https://issues.apache.org/jira/browse/ZOOKEEPER-3482
>> 
>> 
>> thank you.
>> 
>> best regards
>>

Re: Zookeeper 3.5 SSL and Kerberos authentication

2019-11-08 Thread Andor Molnar 
Hi Jörn!

Thanks for reporting the problem, I answered the Jira.
I'll set up a small test environment soon, but we need more specifics
on how you set up your environment and what is the failure exactly.

Regards,
Andor



-Original Message-
From: Jörn Franke 
Reply-To: user@zookeeper.apache.org
To: user@zookeeper.apache.org
Subject: Zookeeper 3.5 SSL and Kerberos authentication
Date: Wed, 6 Nov 2019 22:28:32 +0100

Dear all,

it seems that ZooKeeper 3.5 with SSL enabled does not support Kerberos
authentication, but only X509 authentication. Kerberos is used in many
Enterprise environments and is supported by Apache Solr. Is this a bug?
Or
am I missing something?


I created a Jira for this:
https://issues.apache.org/jira/browse/ZOOKEEPER-3482


thank you.

best regards

Re: Zookeeper 3.5 SSL and Kerberos authentication

2019-11-06 Thread Enrico Olivelli 
Jorn,
IIRC in my company we set up some ZK with TLS and SASL, using TLS for
encryption and SASL for auth.
We were using early 3.5.3 or something like that.

Do you have a specific error?

I can also add that in 3.6.0 we will have port-unification, this way you
can configure only one client port and accept plain text and TLS connection
from clients (this helps the ttransition to TLS)

Enrico

Il mer 6 nov 2019, 22:28 Jörn Franke  ha scritto:

> Dear all,
>
> it seems that ZooKeeper 3.5 with SSL enabled does not support Kerberos
> authentication, but only X509 authentication. Kerberos is used in many
> Enterprise environments and is supported by Apache Solr. Is this a bug? Or
> am I missing something?
>
>
> I created a Jira for this:
> https://issues.apache.org/jira/browse/ZOOKEEPER-3482
>
>
> thank you.
>
> best regards
>