Re: Lack of support for TLS-only ZK cluster

2024-02-29 Thread Abhilash Kishore 
Hi,

Gentle reminder to kindly review the PR - Support TLS-only ZK server
.

Regards,
Abhilash Kishore



On Thu, 22 Feb 2024 at 13:28, Andor Molnar  wrote:

> Yes, I'll take a look also soon.
>
> Andor
>
>
>
> On Wed, 2024-02-21 at 13:03 -0800, Abhilash Kishore wrote:
> > Thanks for the review Enrico!
> >
> > Do we need another +1 or are we ready to merge?
> >
> > Regards,
> > Abhilash Kishore
> >
> >
> > On Tue, 20 Feb 2024 at 13:59, Enrico Olivelli 
> > wrote:
> > > Il Mar 20 Feb 2024, 22:43 Abhilash Kishore 
> > > ha
> > > scritto:
> > >
> > > > Yes, that's the PR <
> > > https://github.com/apache/zookeeper/pull/2117>;.
> > > >
> > > > It's ready now. Do you mind taking a look?
> > > >
> > >
> > >
> > > Reviewed.
> > > Thanks
> > >
> > > Enrico
> > >
> > >
> > > > Regards,
> > > > Abhilash Kishore
> > > >
> > > >
> > > > On Tue, 13 Feb 2024 at 02:28, Andor Molnar 
> > > wrote:
> > > >
> > > > > Hi Abhilash,
> > > > >
> > > > > Is this the patch that you're working on?
> > > > >
> > > > > https://github.com/apache/zookeeper/pull/2117
> > > > >
> > > > > I see it's still draft, are u going to finish it soon?
> > > > >
> > > > > Andor
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Mon, 2024-01-08 at 18:46 -0800, Abhilash Kishore wrote:
> > > > > > Thanks Andor, that makes sense. I agree with you, this is a
> > > simpler
> > > > > > and
> > > > > > cleaner solution.
> > > > > >
> > > > > > I'll work on the changes and will try to keep it backwards
> > > > > > compatible.
> > > > > >
> > > > > > Regards,
> > > > > > Abhilash Kishore
> > > > > >
> > > > > >
> > > > > > On Fri, 5 Jan 2024 at 09:00, Andor Molnar 
> > > wrote:
> > > > > >
> > > > > > > Hi Abhilash,
> > > > > > >
> > > > > > > Thanks for looking into this issue.
> > > > > > >
> > > > > > > I wouldn't complicate things by trying to get reconfig
> > > parameters
> > > > > > > aligned and mixed with clientPort/secureClientPort. Since
> > > the
> > > > > > > documentation says these options are already deprecated I
> > > suggest
> > > > > > > to
> > > > > > > upgrade Reconfig config line to support secure client port
> > > as well.
> > > > > > >
> > > > > > > So, the following reconfig line:
> > > > > > >
> > > > > > > "server.1=abhilash-
> > > ubuntu:3183:4183:participant;0.0.0.0:2181"
> > > > > > >
> > > > > > > will become:
> > > > > > >
> > > > > > > "server.1=abhilash-
> > > > > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > > > > 82".
> > > > > > >
> > > > > > > The 3 scenarios will become:
> > > > > > >
> > > > > > > 1. Non-TLS only:
> > > > > > >
> > > > > > > "server.1=abhilash-
> > > ubuntu:3183:4183:participant;0.0.0.0:2181;"
> > > > > > >
> > > > > > > 2. TLS-only:
> > > > > > >
> > > > > > > "server.1=abhilash-
> > > ubuntu:3183:4183:participant;;0.0.0.0:2182".
> > > > > > >
> > > > > > > 3. TLS/non-TLS mixed:
> > > > > > >
> > > > > > > "server.1=abhilash-
> > > > > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > > > > 82".
> > > > > > >
> > > > > > > In addition to that I would force the user to use either
> > > the
> > > > > > > deprecated
> > > > > > > settings (clientPort/secureClientPort) OR reconfig lines,
> > > but not
> > > > > > > both.
> > > > > > > Throw an exception and halt the server if both options are
> > > > > > > specified at
> > > > > > > the same time.
> > > > > > >
> > > > > > > Thoughts?
> > > > > > >
> > > > > > > Regards,
> > > > > > > Andor
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> > > > > > > > Many organizations, large and small, have strict security
> > > and
> > > > > > > > compliance
> > > > > > > > requirements to only accept encrypted/TLS connections and
> > > not
> > > > > > > > plain
> > > > > > > > text
> > > > > > > > connections.
> > > > > > > >
> > > > > > > > I'd like to discuss an issue which is preventing us from
> > > starting
> > > > > > > > our
> > > > > > > > ZK
> > > > > > > > clusters in TLS only mode (for client traffic).
> > > > > > > >
> > > > > > > > As per dynamic reconfig doc
> > > > > > > > <
> > > https://zookeeper.apache.org/doc/current/zookeeperReconfig.html>
> > > > > > > > ;;,
> > > > > > > >
> > > > > > > > > Starting with 3.5.0 the *clientPort* and
> > > *clientPortAddress*
> > > > > > > > > configuration
> > > > > > > > > parameters should no longer be used. Instead, this
> > > information
> > > > > > > > > is
> > > > > > > > > now part
> > > > > > > > > of the server keyword specification, which becomes as
> > > follows:
> > > > > > > > > server. =
> > > > > > > > > ::[:role];[ > > > > > > > > port
> > > > > > > > > address>:]
> > > > > > > >
> > > > > > > > Let's say the dynamic config entry of a server is
> > > > > > > > "server.1=abhilash-
> > > ubuntu:3183:4183:participant;0.0.0.0:2181".
> > > > > > > > The
> > > > > > > > server
> > > > > > > > starts up with a (plaintext) clientPort listener on 2181.
> > >

Re: Lack of support for TLS-only ZK cluster

2024-02-22 Thread Andor Molnar 
Yes, I'll take a look also soon.

Andor



On Wed, 2024-02-21 at 13:03 -0800, Abhilash Kishore wrote:
> Thanks for the review Enrico! 
> 
> Do we need another +1 or are we ready to merge? 
> 
> Regards,
> Abhilash Kishore
> 
> 
> On Tue, 20 Feb 2024 at 13:59, Enrico Olivelli 
> wrote:
> > Il Mar 20 Feb 2024, 22:43 Abhilash Kishore 
> > ha
> > scritto:
> > 
> > > Yes, that's the PR <
> > https://github.com/apache/zookeeper/pull/2117>;.
> > >
> > > It's ready now. Do you mind taking a look?
> > >
> > 
> > 
> > Reviewed.
> > Thanks
> > 
> > Enrico
> > 
> > 
> > > Regards,
> > > Abhilash Kishore
> > >
> > >
> > > On Tue, 13 Feb 2024 at 02:28, Andor Molnar 
> > wrote:
> > >
> > > > Hi Abhilash,
> > > >
> > > > Is this the patch that you're working on?
> > > >
> > > > https://github.com/apache/zookeeper/pull/2117
> > > >
> > > > I see it's still draft, are u going to finish it soon?
> > > >
> > > > Andor
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, 2024-01-08 at 18:46 -0800, Abhilash Kishore wrote:
> > > > > Thanks Andor, that makes sense. I agree with you, this is a
> > simpler
> > > > > and
> > > > > cleaner solution.
> > > > >
> > > > > I'll work on the changes and will try to keep it backwards
> > > > > compatible.
> > > > >
> > > > > Regards,
> > > > > Abhilash Kishore
> > > > >
> > > > >
> > > > > On Fri, 5 Jan 2024 at 09:00, Andor Molnar 
> > wrote:
> > > > >
> > > > > > Hi Abhilash,
> > > > > >
> > > > > > Thanks for looking into this issue.
> > > > > >
> > > > > > I wouldn't complicate things by trying to get reconfig
> > parameters
> > > > > > aligned and mixed with clientPort/secureClientPort. Since
> > the
> > > > > > documentation says these options are already deprecated I
> > suggest
> > > > > > to
> > > > > > upgrade Reconfig config line to support secure client port
> > as well.
> > > > > >
> > > > > > So, the following reconfig line:
> > > > > >
> > > > > > "server.1=abhilash-
> > ubuntu:3183:4183:participant;0.0.0.0:2181"
> > > > > >
> > > > > > will become:
> > > > > >
> > > > > > "server.1=abhilash-
> > > > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > > > 82".
> > > > > >
> > > > > > The 3 scenarios will become:
> > > > > >
> > > > > > 1. Non-TLS only:
> > > > > >
> > > > > > "server.1=abhilash-
> > ubuntu:3183:4183:participant;0.0.0.0:2181;"
> > > > > >
> > > > > > 2. TLS-only:
> > > > > >
> > > > > > "server.1=abhilash-
> > ubuntu:3183:4183:participant;;0.0.0.0:2182".
> > > > > >
> > > > > > 3. TLS/non-TLS mixed:
> > > > > >
> > > > > > "server.1=abhilash-
> > > > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > > > 82".
> > > > > >
> > > > > > In addition to that I would force the user to use either
> > the
> > > > > > deprecated
> > > > > > settings (clientPort/secureClientPort) OR reconfig lines,
> > but not
> > > > > > both.
> > > > > > Throw an exception and halt the server if both options are
> > > > > > specified at
> > > > > > the same time.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > Regards,
> > > > > > Andor
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> > > > > > > Many organizations, large and small, have strict security
> > and
> > > > > > > compliance
> > > > > > > requirements to only accept encrypted/TLS connections and
> > not
> > > > > > > plain
> > > > > > > text
> > > > > > > connections.
> > > > > > >
> > > > > > > I'd like to discuss an issue which is preventing us from
> > starting
> > > > > > > our
> > > > > > > ZK
> > > > > > > clusters in TLS only mode (for client traffic).
> > > > > > >
> > > > > > > As per dynamic reconfig doc
> > > > > > > <
> > https://zookeeper.apache.org/doc/current/zookeeperReconfig.html>
> > > > > > > ;;,
> > > > > > >
> > > > > > > > Starting with 3.5.0 the *clientPort* and
> > *clientPortAddress*
> > > > > > > > configuration
> > > > > > > > parameters should no longer be used. Instead, this
> > information
> > > > > > > > is
> > > > > > > > now part
> > > > > > > > of the server keyword specification, which becomes as
> > follows:
> > > > > > > > server. =
> > > > > > > > ::[:role];[ > > > > > > > port
> > > > > > > > address>:]
> > > > > > >
> > > > > > > Let's say the dynamic config entry of a server is
> > > > > > > "server.1=abhilash-
> > ubuntu:3183:4183:participant;0.0.0.0:2181".
> > > > > > > The
> > > > > > > server
> > > > > > > starts up with a (plaintext) clientPort listener on 2181.
> > > > > > >
> > > > > > > Now, if we want to make this server TLS-only, what
> > options do we
> > > > > > > have? We
> > > > > > > want to stop accepting plaintext traffic on 2181 and make
> > the
> > > > > > > same
> > > > > > > port
> > > > > > > accept TLS connections only (make clientPort as
> > > > > > > secureClientPort).
> > > > > > >
> > > > > > > If we add "secureClientPort=2181" in zoo.cfg, then ZK
> > server
> > > > > > > first
> > > > > > > starts a
> > > > > > > plaintext listener on 2181 because of ";0.0.0.0:2181"

Re: Lack of support for TLS-only ZK cluster

2024-02-21 Thread Abhilash Kishore 
Thanks for the review Enrico!

Do we need another +1 or are we ready to merge?

Regards,
Abhilash Kishore


On Tue, 20 Feb 2024 at 13:59, Enrico Olivelli  wrote:

> Il Mar 20 Feb 2024, 22:43 Abhilash Kishore  ha
> scritto:
>
> > Yes, that's the PR .
> >
> > It's ready now. Do you mind taking a look?
> >
>
>
> Reviewed.
> Thanks
>
> Enrico
>
>
> > Regards,
> > Abhilash Kishore
> >
> >
> > On Tue, 13 Feb 2024 at 02:28, Andor Molnar  wrote:
> >
> > > Hi Abhilash,
> > >
> > > Is this the patch that you're working on?
> > >
> > > https://github.com/apache/zookeeper/pull/2117
> > >
> > > I see it's still draft, are u going to finish it soon?
> > >
> > > Andor
> > >
> > >
> > >
> > >
> > > On Mon, 2024-01-08 at 18:46 -0800, Abhilash Kishore wrote:
> > > > Thanks Andor, that makes sense. I agree with you, this is a simpler
> > > > and
> > > > cleaner solution.
> > > >
> > > > I'll work on the changes and will try to keep it backwards
> > > > compatible.
> > > >
> > > > Regards,
> > > > Abhilash Kishore
> > > >
> > > >
> > > > On Fri, 5 Jan 2024 at 09:00, Andor Molnar  wrote:
> > > >
> > > > > Hi Abhilash,
> > > > >
> > > > > Thanks for looking into this issue.
> > > > >
> > > > > I wouldn't complicate things by trying to get reconfig parameters
> > > > > aligned and mixed with clientPort/secureClientPort. Since the
> > > > > documentation says these options are already deprecated I suggest
> > > > > to
> > > > > upgrade Reconfig config line to support secure client port as well.
> > > > >
> > > > > So, the following reconfig line:
> > > > >
> > > > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181"
> > > > >
> > > > > will become:
> > > > >
> > > > > "server.1=abhilash-
> > > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > > 82".
> > > > >
> > > > > The 3 scenarios will become:
> > > > >
> > > > > 1. Non-TLS only:
> > > > >
> > > > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;"
> > > > >
> > > > > 2. TLS-only:
> > > > >
> > > > > "server.1=abhilash-ubuntu:3183:4183:participant;;0.0.0.0:2182".
> > > > >
> > > > > 3. TLS/non-TLS mixed:
> > > > >
> > > > > "server.1=abhilash-
> > > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > > 82".
> > > > >
> > > > > In addition to that I would force the user to use either the
> > > > > deprecated
> > > > > settings (clientPort/secureClientPort) OR reconfig lines, but not
> > > > > both.
> > > > > Throw an exception and halt the server if both options are
> > > > > specified at
> > > > > the same time.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > Regards,
> > > > > Andor
> > > > >
> > > > >
> > > > >
> > > > > On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> > > > > > Many organizations, large and small, have strict security and
> > > > > > compliance
> > > > > > requirements to only accept encrypted/TLS connections and not
> > > > > > plain
> > > > > > text
> > > > > > connections.
> > > > > >
> > > > > > I'd like to discuss an issue which is preventing us from starting
> > > > > > our
> > > > > > ZK
> > > > > > clusters in TLS only mode (for client traffic).
> > > > > >
> > > > > > As per dynamic reconfig doc
> > > > > >  >
> > > > > > ;;,
> > > > > >
> > > > > > > Starting with 3.5.0 the *clientPort* and *clientPortAddress*
> > > > > > > configuration
> > > > > > > parameters should no longer be used. Instead, this information
> > > > > > > is
> > > > > > > now part
> > > > > > > of the server keyword specification, which becomes as follows:
> > > > > > > server. =
> > > > > > > ::[:role];[ > > > > > > port
> > > > > > > address>:]
> > > > > >
> > > > > > Let's say the dynamic config entry of a server is
> > > > > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181".
> > > > > > The
> > > > > > server
> > > > > > starts up with a (plaintext) clientPort listener on 2181.
> > > > > >
> > > > > > Now, if we want to make this server TLS-only, what options do we
> > > > > > have? We
> > > > > > want to stop accepting plaintext traffic on 2181 and make the
> > > > > > same
> > > > > > port
> > > > > > accept TLS connections only (make clientPort as
> > > > > > secureClientPort).
> > > > > >
> > > > > > If we add "secureClientPort=2181" in zoo.cfg, then ZK server
> > > > > > first
> > > > > > starts a
> > > > > > plaintext listener on 2181 because of ";0.0.0.0:2181" in
> > > > > > "server.1"
> > > > > > dynamic
> > > > > > config entry and then attempts to start a TLS client listener on
> > > > > > the
> > > > > > same
> > > > > > port (2181) and fails. The reason for this behavior is already
> > > > > > described in
> > > > > > ZOOKEEPER-4276 <
> > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4276'
> > > > > > > (highly
> > > > > > recommended pre-read).
> > > > > >
> > > > > > It is not possible to just remove the "" part from
> > > > > > the
> > > > > > "server.1"

Re: Lack of support for TLS-only ZK cluster

2024-02-20 Thread Enrico Olivelli 
Il Mar 20 Feb 2024, 22:43 Abhilash Kishore  ha
scritto:

> Yes, that's the PR .
>
> It's ready now. Do you mind taking a look?
>


Reviewed.
Thanks

Enrico


> Regards,
> Abhilash Kishore
>
>
> On Tue, 13 Feb 2024 at 02:28, Andor Molnar  wrote:
>
> > Hi Abhilash,
> >
> > Is this the patch that you're working on?
> >
> > https://github.com/apache/zookeeper/pull/2117
> >
> > I see it's still draft, are u going to finish it soon?
> >
> > Andor
> >
> >
> >
> >
> > On Mon, 2024-01-08 at 18:46 -0800, Abhilash Kishore wrote:
> > > Thanks Andor, that makes sense. I agree with you, this is a simpler
> > > and
> > > cleaner solution.
> > >
> > > I'll work on the changes and will try to keep it backwards
> > > compatible.
> > >
> > > Regards,
> > > Abhilash Kishore
> > >
> > >
> > > On Fri, 5 Jan 2024 at 09:00, Andor Molnar  wrote:
> > >
> > > > Hi Abhilash,
> > > >
> > > > Thanks for looking into this issue.
> > > >
> > > > I wouldn't complicate things by trying to get reconfig parameters
> > > > aligned and mixed with clientPort/secureClientPort. Since the
> > > > documentation says these options are already deprecated I suggest
> > > > to
> > > > upgrade Reconfig config line to support secure client port as well.
> > > >
> > > > So, the following reconfig line:
> > > >
> > > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181"
> > > >
> > > > will become:
> > > >
> > > > "server.1=abhilash-
> > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > 82".
> > > >
> > > > The 3 scenarios will become:
> > > >
> > > > 1. Non-TLS only:
> > > >
> > > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;"
> > > >
> > > > 2. TLS-only:
> > > >
> > > > "server.1=abhilash-ubuntu:3183:4183:participant;;0.0.0.0:2182".
> > > >
> > > > 3. TLS/non-TLS mixed:
> > > >
> > > > "server.1=abhilash-
> > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > 82".
> > > >
> > > > In addition to that I would force the user to use either the
> > > > deprecated
> > > > settings (clientPort/secureClientPort) OR reconfig lines, but not
> > > > both.
> > > > Throw an exception and halt the server if both options are
> > > > specified at
> > > > the same time.
> > > >
> > > > Thoughts?
> > > >
> > > > Regards,
> > > > Andor
> > > >
> > > >
> > > >
> > > > On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> > > > > Many organizations, large and small, have strict security and
> > > > > compliance
> > > > > requirements to only accept encrypted/TLS connections and not
> > > > > plain
> > > > > text
> > > > > connections.
> > > > >
> > > > > I'd like to discuss an issue which is preventing us from starting
> > > > > our
> > > > > ZK
> > > > > clusters in TLS only mode (for client traffic).
> > > > >
> > > > > As per dynamic reconfig doc
> > > > > 
> > > > > ;;,
> > > > >
> > > > > > Starting with 3.5.0 the *clientPort* and *clientPortAddress*
> > > > > > configuration
> > > > > > parameters should no longer be used. Instead, this information
> > > > > > is
> > > > > > now part
> > > > > > of the server keyword specification, which becomes as follows:
> > > > > > server. =
> > > > > > ::[:role];[ > > > > > port
> > > > > > address>:]
> > > > >
> > > > > Let's say the dynamic config entry of a server is
> > > > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181".
> > > > > The
> > > > > server
> > > > > starts up with a (plaintext) clientPort listener on 2181.
> > > > >
> > > > > Now, if we want to make this server TLS-only, what options do we
> > > > > have? We
> > > > > want to stop accepting plaintext traffic on 2181 and make the
> > > > > same
> > > > > port
> > > > > accept TLS connections only (make clientPort as
> > > > > secureClientPort).
> > > > >
> > > > > If we add "secureClientPort=2181" in zoo.cfg, then ZK server
> > > > > first
> > > > > starts a
> > > > > plaintext listener on 2181 because of ";0.0.0.0:2181" in
> > > > > "server.1"
> > > > > dynamic
> > > > > config entry and then attempts to start a TLS client listener on
> > > > > the
> > > > > same
> > > > > port (2181) and fails. The reason for this behavior is already
> > > > > described in
> > > > > ZOOKEEPER-4276 <
> > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4276'
> > > > > > (highly
> > > > > recommended pre-read).
> > > > >
> > > > > It is not possible to just remove the "" part from
> > > > > the
> > > > > "server.1" entry as well (I believe it is mandatory from v3.5). I
> > > > > tried:
> > > > >
> > > > > [zk: localhost:2181(CONNECTED) 4] reconfig -remove 1
> > > > > [zk: localhost:2181(CONNECTED) 5] reconfig -add
> > > > > server.1=abhilash-ubuntu:3183:4183:participant
> > > > > Arguments are not valid :
> > > > >
> > > > >
> > > > > The reconfig command does not allow us to add a server entry
> > > > > without
> > > > > ";[ > > > > port address>:]".
> > > > >
> > > > > How do we

Re: Lack of support for TLS-only ZK cluster

2024-02-20 Thread Abhilash Kishore 
Yes, that's the PR .

It's ready now. Do you mind taking a look?

Regards,
Abhilash Kishore


On Tue, 13 Feb 2024 at 02:28, Andor Molnar  wrote:

> Hi Abhilash,
>
> Is this the patch that you're working on?
>
> https://github.com/apache/zookeeper/pull/2117
>
> I see it's still draft, are u going to finish it soon?
>
> Andor
>
>
>
>
> On Mon, 2024-01-08 at 18:46 -0800, Abhilash Kishore wrote:
> > Thanks Andor, that makes sense. I agree with you, this is a simpler
> > and
> > cleaner solution.
> >
> > I'll work on the changes and will try to keep it backwards
> > compatible.
> >
> > Regards,
> > Abhilash Kishore
> >
> >
> > On Fri, 5 Jan 2024 at 09:00, Andor Molnar  wrote:
> >
> > > Hi Abhilash,
> > >
> > > Thanks for looking into this issue.
> > >
> > > I wouldn't complicate things by trying to get reconfig parameters
> > > aligned and mixed with clientPort/secureClientPort. Since the
> > > documentation says these options are already deprecated I suggest
> > > to
> > > upgrade Reconfig config line to support secure client port as well.
> > >
> > > So, the following reconfig line:
> > >
> > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181"
> > >
> > > will become:
> > >
> > > "server.1=abhilash-
> > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > 82".
> > >
> > > The 3 scenarios will become:
> > >
> > > 1. Non-TLS only:
> > >
> > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;"
> > >
> > > 2. TLS-only:
> > >
> > > "server.1=abhilash-ubuntu:3183:4183:participant;;0.0.0.0:2182".
> > >
> > > 3. TLS/non-TLS mixed:
> > >
> > > "server.1=abhilash-
> > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > 82".
> > >
> > > In addition to that I would force the user to use either the
> > > deprecated
> > > settings (clientPort/secureClientPort) OR reconfig lines, but not
> > > both.
> > > Throw an exception and halt the server if both options are
> > > specified at
> > > the same time.
> > >
> > > Thoughts?
> > >
> > > Regards,
> > > Andor
> > >
> > >
> > >
> > > On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> > > > Many organizations, large and small, have strict security and
> > > > compliance
> > > > requirements to only accept encrypted/TLS connections and not
> > > > plain
> > > > text
> > > > connections.
> > > >
> > > > I'd like to discuss an issue which is preventing us from starting
> > > > our
> > > > ZK
> > > > clusters in TLS only mode (for client traffic).
> > > >
> > > > As per dynamic reconfig doc
> > > > 
> > > > ;;,
> > > >
> > > > > Starting with 3.5.0 the *clientPort* and *clientPortAddress*
> > > > > configuration
> > > > > parameters should no longer be used. Instead, this information
> > > > > is
> > > > > now part
> > > > > of the server keyword specification, which becomes as follows:
> > > > > server. =
> > > > > ::[:role];[ > > > > port
> > > > > address>:]
> > > >
> > > > Let's say the dynamic config entry of a server is
> > > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181".
> > > > The
> > > > server
> > > > starts up with a (plaintext) clientPort listener on 2181.
> > > >
> > > > Now, if we want to make this server TLS-only, what options do we
> > > > have? We
> > > > want to stop accepting plaintext traffic on 2181 and make the
> > > > same
> > > > port
> > > > accept TLS connections only (make clientPort as
> > > > secureClientPort).
> > > >
> > > > If we add "secureClientPort=2181" in zoo.cfg, then ZK server
> > > > first
> > > > starts a
> > > > plaintext listener on 2181 because of ";0.0.0.0:2181" in
> > > > "server.1"
> > > > dynamic
> > > > config entry and then attempts to start a TLS client listener on
> > > > the
> > > > same
> > > > port (2181) and fails. The reason for this behavior is already
> > > > described in
> > > > ZOOKEEPER-4276 <
> > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4276'
> > > > > (highly
> > > > recommended pre-read).
> > > >
> > > > It is not possible to just remove the "" part from
> > > > the
> > > > "server.1" entry as well (I believe it is mandatory from v3.5). I
> > > > tried:
> > > >
> > > > [zk: localhost:2181(CONNECTED) 4] reconfig -remove 1
> > > > [zk: localhost:2181(CONNECTED) 5] reconfig -add
> > > > server.1=abhilash-ubuntu:3183:4183:participant
> > > > Arguments are not valid :
> > > >
> > > >
> > > > The reconfig command does not allow us to add a server entry
> > > > without
> > > > ";[ > > > port address>:]".
> > > >
> > > > How do we support a "TLS-only" cluster in this case?
> > > >
> > > > My recommendation:
> > > >
> > > >1. If both clientPort and secureClientPort are not set in
> > > > zoo.cfg,
> > > > then
> > > >use the client port address from dynamic config.
> > > >2. If only clientPort is set in zoo.cfg, then it has to match
> > > > the
> > > > port
> > > >in dynamic config and ZK starts a plaintext listener on

Re: Lack of support for TLS-only ZK cluster

2024-02-13 Thread Andor Molnar 
Hi Abhilash,

Is this the patch that you're working on?

https://github.com/apache/zookeeper/pull/2117

I see it's still draft, are u going to finish it soon?

Andor




On Mon, 2024-01-08 at 18:46 -0800, Abhilash Kishore wrote:
> Thanks Andor, that makes sense. I agree with you, this is a simpler
> and
> cleaner solution.
> 
> I'll work on the changes and will try to keep it backwards
> compatible.
> 
> Regards,
> Abhilash Kishore
> 
> 
> On Fri, 5 Jan 2024 at 09:00, Andor Molnar  wrote:
> 
> > Hi Abhilash,
> > 
> > Thanks for looking into this issue.
> > 
> > I wouldn't complicate things by trying to get reconfig parameters
> > aligned and mixed with clientPort/secureClientPort. Since the
> > documentation says these options are already deprecated I suggest
> > to
> > upgrade Reconfig config line to support secure client port as well.
> > 
> > So, the following reconfig line:
> > 
> > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181"
> > 
> > will become:
> > 
> > "server.1=abhilash-
> > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > 82".
> > 
> > The 3 scenarios will become:
> > 
> > 1. Non-TLS only:
> > 
> > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;"
> > 
> > 2. TLS-only:
> > 
> > "server.1=abhilash-ubuntu:3183:4183:participant;;0.0.0.0:2182".
> > 
> > 3. TLS/non-TLS mixed:
> > 
> > "server.1=abhilash-
> > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > 82".
> > 
> > In addition to that I would force the user to use either the
> > deprecated
> > settings (clientPort/secureClientPort) OR reconfig lines, but not
> > both.
> > Throw an exception and halt the server if both options are
> > specified at
> > the same time.
> > 
> > Thoughts?
> > 
> > Regards,
> > Andor
> > 
> > 
> > 
> > On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> > > Many organizations, large and small, have strict security and
> > > compliance
> > > requirements to only accept encrypted/TLS connections and not
> > > plain
> > > text
> > > connections.
> > > 
> > > I'd like to discuss an issue which is preventing us from starting
> > > our
> > > ZK
> > > clusters in TLS only mode (for client traffic).
> > > 
> > > As per dynamic reconfig doc
> > > 
> > > ;;,
> > > 
> > > > Starting with 3.5.0 the *clientPort* and *clientPortAddress*
> > > > configuration
> > > > parameters should no longer be used. Instead, this information
> > > > is
> > > > now part
> > > > of the server keyword specification, which becomes as follows:
> > > > server. =
> > > > ::[:role];[ > > > port
> > > > address>:]
> > > 
> > > Let's say the dynamic config entry of a server is
> > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181".
> > > The
> > > server
> > > starts up with a (plaintext) clientPort listener on 2181.
> > > 
> > > Now, if we want to make this server TLS-only, what options do we
> > > have? We
> > > want to stop accepting plaintext traffic on 2181 and make the
> > > same
> > > port
> > > accept TLS connections only (make clientPort as
> > > secureClientPort).
> > > 
> > > If we add "secureClientPort=2181" in zoo.cfg, then ZK server
> > > first
> > > starts a
> > > plaintext listener on 2181 because of ";0.0.0.0:2181" in
> > > "server.1"
> > > dynamic
> > > config entry and then attempts to start a TLS client listener on
> > > the
> > > same
> > > port (2181) and fails. The reason for this behavior is already
> > > described in
> > > ZOOKEEPER-4276 <
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-4276'
> > > > (highly
> > > recommended pre-read).
> > > 
> > > It is not possible to just remove the "" part from
> > > the
> > > "server.1" entry as well (I believe it is mandatory from v3.5). I
> > > tried:
> > > 
> > > [zk: localhost:2181(CONNECTED) 4] reconfig -remove 1
> > > [zk: localhost:2181(CONNECTED) 5] reconfig -add
> > > server.1=abhilash-ubuntu:3183:4183:participant
> > > Arguments are not valid :
> > > 
> > > 
> > > The reconfig command does not allow us to add a server entry
> > > without
> > > ";[ > > port address>:]".
> > > 
> > > How do we support a "TLS-only" cluster in this case?
> > > 
> > > My recommendation:
> > > 
> > >1. If both clientPort and secureClientPort are not set in
> > > zoo.cfg,
> > > then
> > >use the client port address from dynamic config.
> > >2. If only clientPort is set in zoo.cfg, then it has to match
> > > the
> > > port
> > >in dynamic config and ZK starts a plaintext listener on this
> > > port.
> > >3. If only secureClientPort is set in zoo.cfg, then it has to
> > > match the
> > >port in dynamic config and ZK starts a TLS listener on this
> > > port.
> > >4. If both clientPort and secureClientPort are set in zoo.cfg,
> > > then the
> > >client port in zoo.cfg should match the port in dynamic
> > > config. ZK
> > > starts a
> > >plaintext listener on clientPort and TLS listener on
> > > secureClientPort (dual
> > >mode).
> >

Re: Lack of support for TLS-only ZK cluster

2024-01-08 Thread Abhilash Kishore 
Thanks Andor, that makes sense. I agree with you, this is a simpler and
cleaner solution.

I'll work on the changes and will try to keep it backwards compatible.

Regards,
Abhilash Kishore


On Fri, 5 Jan 2024 at 09:00, Andor Molnar  wrote:

> Hi Abhilash,
>
> Thanks for looking into this issue.
>
> I wouldn't complicate things by trying to get reconfig parameters
> aligned and mixed with clientPort/secureClientPort. Since the
> documentation says these options are already deprecated I suggest to
> upgrade Reconfig config line to support secure client port as well.
>
> So, the following reconfig line:
>
> "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181"
>
> will become:
>
> "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> 82".
>
> The 3 scenarios will become:
>
> 1. Non-TLS only:
>
> "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;"
>
> 2. TLS-only:
>
> "server.1=abhilash-ubuntu:3183:4183:participant;;0.0.0.0:2182".
>
> 3. TLS/non-TLS mixed:
>
> "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> 82".
>
> In addition to that I would force the user to use either the deprecated
> settings (clientPort/secureClientPort) OR reconfig lines, but not both.
> Throw an exception and halt the server if both options are specified at
> the same time.
>
> Thoughts?
>
> Regards,
> Andor
>
>
>
> On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> > Many organizations, large and small, have strict security and
> > compliance
> > requirements to only accept encrypted/TLS connections and not plain
> > text
> > connections.
> >
> > I'd like to discuss an issue which is preventing us from starting our
> > ZK
> > clusters in TLS only mode (for client traffic).
> >
> > As per dynamic reconfig doc
> > ;,
> >
> > > Starting with 3.5.0 the *clientPort* and *clientPortAddress*
> > > configuration
> > > parameters should no longer be used. Instead, this information is
> > > now part
> > > of the server keyword specification, which becomes as follows:
> > > server. = ::[:role];[ > > port
> > > address>:]
> >
> >
> > Let's say the dynamic config entry of a server is
> > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181". The
> > server
> > starts up with a (plaintext) clientPort listener on 2181.
> >
> > Now, if we want to make this server TLS-only, what options do we
> > have? We
> > want to stop accepting plaintext traffic on 2181 and make the same
> > port
> > accept TLS connections only (make clientPort as secureClientPort).
> >
> > If we add "secureClientPort=2181" in zoo.cfg, then ZK server first
> > starts a
> > plaintext listener on 2181 because of ";0.0.0.0:2181" in "server.1"
> > dynamic
> > config entry and then attempts to start a TLS client listener on the
> > same
> > port (2181) and fails. The reason for this behavior is already
> > described in
> > ZOOKEEPER-4276  > > (highly
> > recommended pre-read).
> >
> > It is not possible to just remove the "" part from the
> > "server.1" entry as well (I believe it is mandatory from v3.5). I
> > tried:
> >
> > [zk: localhost:2181(CONNECTED) 4] reconfig -remove 1
> > [zk: localhost:2181(CONNECTED) 5] reconfig -add
> > server.1=abhilash-ubuntu:3183:4183:participant
> > Arguments are not valid :
> >
> >
> > The reconfig command does not allow us to add a server entry without
> > ";[ > port address>:]".
> >
> > How do we support a "TLS-only" cluster in this case?
> >
> > My recommendation:
> >
> >1. If both clientPort and secureClientPort are not set in zoo.cfg,
> > then
> >use the client port address from dynamic config.
> >2. If only clientPort is set in zoo.cfg, then it has to match the
> > port
> >in dynamic config and ZK starts a plaintext listener on this port.
> >3. If only secureClientPort is set in zoo.cfg, then it has to
> > match the
> >port in dynamic config and ZK starts a TLS listener on this port.
> >4. If both clientPort and secureClientPort are set in zoo.cfg,
> > then the
> >client port in zoo.cfg should match the port in dynamic config. ZK
> > starts a
> >plaintext listener on clientPort and TLS listener on
> > secureClientPort (dual
> >mode).
> >
> >
> > This would reintroduce the requirement to set "clientPort" in zoo.cfg
> > if
> > someone wants to start the cluster in dual mode.
> >
> > For example,
> >
> > secureClientPort=2182
> > server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181
> >
> > will no longer be a valid config because of rule 3 above.
> >
> > It has to be:
> >
> > clientPort=2181
> > secureClientPort=2182
> > server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181
> >
> >
> > I can create a PR to make the above changes, but first I'd like to
> > know
> > your thoughts on this and discuss further on whether there's a better
> > way
> > to handle this.
> >
> > Regards,
> > Abhilash

Re: Lack of support for TLS-only ZK cluster

2024-01-05 Thread Andor Molnar 
Hi Abhilash,

Thanks for looking into this issue.

I wouldn't complicate things by trying to get reconfig parameters
aligned and mixed with clientPort/secureClientPort. Since the
documentation says these options are already deprecated I suggest to
upgrade Reconfig config line to support secure client port as well.

So, the following reconfig line:

"server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181"

will become:

"server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
82".

The 3 scenarios will become:

1. Non-TLS only:

"server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;"

2. TLS-only:

"server.1=abhilash-ubuntu:3183:4183:participant;;0.0.0.0:2182".

3. TLS/non-TLS mixed:

"server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
82".

In addition to that I would force the user to use either the deprecated
settings (clientPort/secureClientPort) OR reconfig lines, but not both.
Throw an exception and halt the server if both options are specified at
the same time.

Thoughts?

Regards,
Andor



On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> Many organizations, large and small, have strict security and
> compliance
> requirements to only accept encrypted/TLS connections and not plain
> text
> connections.
> 
> I'd like to discuss an issue which is preventing us from starting our
> ZK
> clusters in TLS only mode (for client traffic).
> 
> As per dynamic reconfig doc
> ;,
> 
> > Starting with 3.5.0 the *clientPort* and *clientPortAddress*
> > configuration
> > parameters should no longer be used. Instead, this information is
> > now part
> > of the server keyword specification, which becomes as follows:
> > server. = ::[:role];[ > port
> > address>:]
> 
> 
> Let's say the dynamic config entry of a server is
> "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181". The
> server
> starts up with a (plaintext) clientPort listener on 2181.
> 
> Now, if we want to make this server TLS-only, what options do we
> have? We
> want to stop accepting plaintext traffic on 2181 and make the same
> port
> accept TLS connections only (make clientPort as secureClientPort).
> 
> If we add "secureClientPort=2181" in zoo.cfg, then ZK server first
> starts a
> plaintext listener on 2181 because of ";0.0.0.0:2181" in "server.1"
> dynamic
> config entry and then attempts to start a TLS client listener on the
> same
> port (2181) and fails. The reason for this behavior is already
> described in
> ZOOKEEPER-4276  > (highly
> recommended pre-read).
> 
> It is not possible to just remove the "" part from the
> "server.1" entry as well (I believe it is mandatory from v3.5). I
> tried:
> 
> [zk: localhost:2181(CONNECTED) 4] reconfig -remove 1
> [zk: localhost:2181(CONNECTED) 5] reconfig -add
> server.1=abhilash-ubuntu:3183:4183:participant
> Arguments are not valid :
> 
> 
> The reconfig command does not allow us to add a server entry without
> ";[ port address>:]".
> 
> How do we support a "TLS-only" cluster in this case?
> 
> My recommendation:
> 
>1. If both clientPort and secureClientPort are not set in zoo.cfg,
> then
>use the client port address from dynamic config.
>2. If only clientPort is set in zoo.cfg, then it has to match the
> port
>in dynamic config and ZK starts a plaintext listener on this port.
>3. If only secureClientPort is set in zoo.cfg, then it has to
> match the
>port in dynamic config and ZK starts a TLS listener on this port.
>4. If both clientPort and secureClientPort are set in zoo.cfg,
> then the
>client port in zoo.cfg should match the port in dynamic config. ZK
> starts a
>plaintext listener on clientPort and TLS listener on
> secureClientPort (dual
>mode).
> 
> 
> This would reintroduce the requirement to set "clientPort" in zoo.cfg
> if
> someone wants to start the cluster in dual mode.
> 
> For example,
> 
> secureClientPort=2182
> server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181
> 
> will no longer be a valid config because of rule 3 above.
> 
> It has to be:
> 
> clientPort=2181
> secureClientPort=2182
> server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181
> 
> 
> I can create a PR to make the above changes, but first I'd like to
> know
> your thoughts on this and discuss further on whether there's a better
> way
> to handle this.
> 
> Regards,
> Abhilash Kishore

Lack of support for TLS-only ZK cluster

2024-01-02 Thread Abhilash Kishore 
Many organizations, large and small, have strict security and compliance
requirements to only accept encrypted/TLS connections and not plain text
connections.

I'd like to discuss an issue which is preventing us from starting our ZK
clusters in TLS only mode (for client traffic).

As per dynamic reconfig doc
,

> Starting with 3.5.0 the *clientPort* and *clientPortAddress* configuration
> parameters should no longer be used. Instead, this information is now part
> of the server keyword specification, which becomes as follows:
> server. = ::[:role];[ address>:]



Let's say the dynamic config entry of a server is
"server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181". The server
starts up with a (plaintext) clientPort listener on 2181.

Now, if we want to make this server TLS-only, what options do we have? We
want to stop accepting plaintext traffic on 2181 and make the same port
accept TLS connections only (make clientPort as secureClientPort).

If we add "secureClientPort=2181" in zoo.cfg, then ZK server first starts a
plaintext listener on 2181 because of ";0.0.0.0:2181" in "server.1" dynamic
config entry and then attempts to start a TLS client listener on the same
port (2181) and fails. The reason for this behavior is already described in
ZOOKEEPER-4276  (highly
recommended pre-read).

It is not possible to just remove the "" part from the
"server.1" entry as well (I believe it is mandatory from v3.5). I tried:

[zk: localhost:2181(CONNECTED) 4] reconfig -remove 1
[zk: localhost:2181(CONNECTED) 5] reconfig -add
server.1=abhilash-ubuntu:3183:4183:participant
Arguments are not valid :


The reconfig command does not allow us to add a server entry without ";[:]".

How do we support a "TLS-only" cluster in this case?

My recommendation:

   1. If both clientPort and secureClientPort are not set in zoo.cfg, then
   use the client port address from dynamic config.
   2. If only clientPort is set in zoo.cfg, then it has to match the port
   in dynamic config and ZK starts a plaintext listener on this port.
   3. If only secureClientPort is set in zoo.cfg, then it has to match the
   port in dynamic config and ZK starts a TLS listener on this port.
   4. If both clientPort and secureClientPort are set in zoo.cfg, then the
   client port in zoo.cfg should match the port in dynamic config. ZK starts a
   plaintext listener on clientPort and TLS listener on secureClientPort (dual
   mode).


This would reintroduce the requirement to set "clientPort" in zoo.cfg if
someone wants to start the cluster in dual mode.

For example,

secureClientPort=2182
server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181

will no longer be a valid config because of rule 3 above.

It has to be:

clientPort=2181
secureClientPort=2182
server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181


I can create a PR to make the above changes, but first I'd like to know
your thoughts on this and discuss further on whether there's a better way
to handle this.

Regards,
Abhilash Kishore