RE: [EXTERNAL] Re: Zookeeper on Kubernetes and Presistent Volumes
Thanks you Steph, thank you Enrico. I'll share your feedback with our architects Regards, Rémi -Message d'origine- De : Steph van Schalkwyk Envoyé : Monday, September 14, 2020 18:53 À : user@zookeeper.apache.org Cc : Stephane Galles Objet : [EXTERNAL] Re: Zookeeper on Kubernetes and Presistent Volumes Remi We are using ZK in k8s with SOLR and Fusion. We are using PVs. I cannot see how one could not use PVs. Nonetheless, we are using 5 zk instead of the usual 3 we use outside of k8s, and we are working on an elaborate restore schema in case the quorum is broken. Hope this helps. Steph *steph van schalkwyk+1.314.452.2896 (Tel/SMS)* On Mon, Sep 14, 2020 at 10:21 AM Enrico Olivelli wrote: > Remi, > sorry for the late reply. > > Your cluster would be able to work if and only if at least X/2 + 1 > servers are up and running and properly connected to the other peers. > any server that is restarted will rejoin from scratch. > It is very dangerous ! you won't be able to recover and probably it > will be very hard to understand what happened (you are going to lose > all of your tx > logs) > > I don't have experience with Kubernetes and non persistent volumes but > as far as I know it is not supposed to work > > Enrico > > > Il giorno ven 11 set 2020 alle ore 14:21 Remi Serrano > ha scritto: > > > Hello mailing list, > > > > We are assessing running Zookeeper in Kubernetes. There are a bunch > > of examples around and they all use Kubernetes Persistent Volumes. > > For some underlying technical reasons, we would like to avoid the > > use of Kubernetes Persistent Volumes. > > What is the risk to setup a ZK cluster on Kubernetes without > > persistent volume ? > > Sub-question, what happen if a ZK node without persistent get killed > > and rebooted by Kubernetes (ie: without any data) ? > > > > Thanks for your help > > > > Rémi > > > > >
Zookeeper on Kubernetes and Presistent Volumes
Hello mailing list, We are assessing running Zookeeper in Kubernetes. There are a bunch of examples around and they all use Kubernetes Persistent Volumes. For some underlying technical reasons, we would like to avoid the use of Kubernetes Persistent Volumes. What is the risk to setup a ZK cluster on Kubernetes without persistent volume ? Sub-question, what happen if a ZK node without persistent get killed and rebooted by Kubernetes (ie: without any data) ? Thanks for your help Rémi
RE: Client-Server authentication with DIGEST-MD5
Perfect. Thanks Enrico. It is the 'setAcl / ' that I was missing. Rémi -Message d'origine- De : Enrico Olivelli [mailto:eolive...@gmail.com] Envoyé : Wednesday, April 11, 2018 11:12 À : UserZooKeeper <user@zookeeper.apache.org> Objet : Re: Client-Server authentication with DIGEST-MD5 2018-04-11 11:08 GMT+02:00 Remi Serrano <rserr...@pros.com>: > Thank you very much Enrico, > > So let's move at ACL level. If I create a new node as : > > Create /mynode content sasl:myuser:mydigest:crdwa > > Indeed only the authenticated myuser is able to READ /mynode... BUT > any other non authenticated user can DELETE the node. How can I prevent this ? > I Could not find explicit solution in the doc. > I am not sure but I think that in order to prevent deletion you have to set ACLs on the parent, in this case '/', and I don't know if is is possible. If a node has children it cannot be deleted, so maybe the solution for you is to create a special "root" node, like /myapp and set ACLs on it and on every children. This is actually what I am doing. Hope that helps Enrico > > Regards, > > Rémi > > -Message d'origine- > De : Enrico Olivelli [mailto:eolive...@gmail.com] Envoyé : Tuesday, > April 10, 2018 15:51 À : UserZooKeeper <user@zookeeper.apache.org> > Objet : Re: Client-Server authentication with DIGEST-MD5 > > 2018-04-10 15:22 GMT+02:00 Remi Serrano <rserr...@pros.com>: > > > Hello > > > > I'm trying to secure my ZK cluster. To do so I'm trying to leverage > > both > : > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi > > ki > > .apache.org%2Fconfluence%2Fdisplay%2FZOOKEEPER%2F=02%7C01%7Crse > > rr > > ano%40pros.com%7Cb7666ab58a2b4380d6a108d59eea2387%7C094cfb7ad1314637 > > 90 > > 47e339e7d04359%7C0%7C0%7C636589650815046832=kKnxsghiwmRKgCdwTZ > > XV > > 88thlMICx%2BF8Ha38ESUW9Zc%3D=0 > > Server-Server+mutual+authentication > > and > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi > > ki > > .apache.org%2Fconfluence%2Fdisplay%2FZOOKEEPER%2F=02%7C01%7Crse > > rr > > ano%40pros.com%7Cb7666ab58a2b4380d6a108d59eea2387%7C094cfb7ad1314637 > > 90 > > 47e339e7d04359%7C0%7C0%7C636589650815046832=kKnxsghiwmRKgCdwTZ > > XV > > 88thlMICx%2BF8Ha38ESUW9Zc%3D=0 > > Client-Server+mutual+authentication > > > > The Server to Server works fine. However, the Client to Server seems > > to be useless as here is the behavior I get : > > > > * Client using a declared user on the server + good password CAN > > connect > > * Client using a declared user on the server + bad password CANNOT > > connect > > * Client using a non declared user on the Server CANNOT connect > > so far so good... but : > > > > * Client using NO user at all CAN connect !!! > > > > > This is expected. Client auth is mostly used together with ACLs, > otherwise AFAIK is pretty useless in ZK. > > Please not that MD5 is not "secure" at all, and consider using > SASL/Kerberos for a production environment. > > Cheers > Enrico > > > > > > Any hint ? > > > > >
RE: stable 3.5 release
Thanks Andor ! -Message d'origine- De : Andor Molnar [mailto:an...@cloudera.com] Envoyé : Wednesday, April 11, 2018 11:17 À : user@zookeeper.apache.org Objet : Re: stable 3.5 release The optimistic answer is couple of months I would say. But given that how reluctant the community is these days, I'm afraid of being too optimistic. Anyway I think this is very important and valuable for the users, therefore I'll make every effort to get it out as soon as possible. Will see how it goes. Regards, Andor On Wed, Apr 11, 2018 at 9:44 AM, Remi Serrano <rserr...@pros.com> wrote: > Thank you very much Andor. > Just to have an estimate, are we talking couple of weeks ? couple of > months ? or probably more than 3 months ? > > Thanks > > Rémi > > -Message d'origine- > De : Andor Molnar [mailto:an...@cloudera.com] Envoyé : Tuesday, April > 10, 2018 17:35 À : user@zookeeper.apache.org Objet : Re: stable 3.5 > release > > Hi Remi / Ansel, > > We have a quite impressive list of blockers to get out of the way: > > https://na01.safelinks.protection.outlook.com/?url= > https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FZOOKEEPER-1549%3Ffil > ter% 3D12343244=02%7C01%7Crserrano%40pros.com% > 7C40524549c0af49856e4308d59ef8a24a%7C094cfb7ad13146379047e339e7d0 > 4359%7C0%7C0%7C636589713067158062=EPudToyCNDaDB4Avg7SC9LFMfgbbhN > W5q4kMmFbK%2F0Q%3D=0 > > Additionally we have to get the following PRs merged: > > https://na01.safelinks.protection.outlook.com/?url= > https%3A%2F%2Fgithub.com%2Fapache%2Fzookeeper%2Fpull% > 2F377=02%7C01%7Crserrano%40pros.com%7C40524549c0af49856e4308d59ef > 8 a24a%7C094cfb7ad13146379047e339e7d04359%7C0%7C0% > 7C636589713067158062=aZjNc5uks6yWMPPksZzBOjX7yt9b3V > bMB4e06rpKvPg%3D=0 > https://na01.safelinks.protection.outlook.com/?url= > https%3A%2F%2Fgithub.com%2Fapache%2Fzookeeper%2Fpull% > 2F184=02%7C01%7Crserrano%40pros.com%7C40524549c0af49856e4308d59ef > 8 a24a%7C094cfb7ad13146379047e339e7d04359%7C0%7C0% > 7C636589713067158062=Z2qGgfBye9Z4%2FqL9CVN3n% > 2F2iQc6IU4MSAyxnjGQKDhs%3D=0 > > Personally, I'm working on getting it released as soon as possible, > but presumably this won't happen tomorrow. > > Regards, > Andor > > > > > On Tue, Apr 10, 2018 at 10:40 AM, Ansel Zandegran < > ansel.zandeg...@infor.com > > wrote: > > > Yes please, If anyone knows. > > > > // Ansel > > > > > On 10 Apr 2018, at 10:04, Remi Serrano <rserr...@pros.com> wrote: > > > > > > Hello, > > > > > > Does anyone know when will be released the stable 3.5 version ? > > > > > > Rémi > > > > > > > >
RE: Client-Server authentication with DIGEST-MD5
Thank you very much Enrico, So let's move at ACL level. If I create a new node as : Create /mynode content sasl:myuser:mydigest:crdwa Indeed only the authenticated myuser is able to READ /mynode... BUT any other non authenticated user can DELETE the node. How can I prevent this ? I Could not find explicit solution in the doc. Regards, Rémi -Message d'origine- De : Enrico Olivelli [mailto:eolive...@gmail.com] Envoyé : Tuesday, April 10, 2018 15:51 À : UserZooKeeper <user@zookeeper.apache.org> Objet : Re: Client-Server authentication with DIGEST-MD5 2018-04-10 15:22 GMT+02:00 Remi Serrano <rserr...@pros.com>: > Hello > > I'm trying to secure my ZK cluster. To do so I'm trying to leverage both : > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki > .apache.org%2Fconfluence%2Fdisplay%2FZOOKEEPER%2F=02%7C01%7Crserr > ano%40pros.com%7Cb7666ab58a2b4380d6a108d59eea2387%7C094cfb7ad131463790 > 47e339e7d04359%7C0%7C0%7C636589650815046832=kKnxsghiwmRKgCdwTZXV > 88thlMICx%2BF8Ha38ESUW9Zc%3D=0 > Server-Server+mutual+authentication > and > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki > .apache.org%2Fconfluence%2Fdisplay%2FZOOKEEPER%2F=02%7C01%7Crserr > ano%40pros.com%7Cb7666ab58a2b4380d6a108d59eea2387%7C094cfb7ad131463790 > 47e339e7d04359%7C0%7C0%7C636589650815046832=kKnxsghiwmRKgCdwTZXV > 88thlMICx%2BF8Ha38ESUW9Zc%3D=0 > Client-Server+mutual+authentication > > The Server to Server works fine. However, the Client to Server seems > to be useless as here is the behavior I get : > > * Client using a declared user on the server + good password CAN > connect > * Client using a declared user on the server + bad password CANNOT > connect > * Client using a non declared user on the Server CANNOT connect > so far so good... but : > > * Client using NO user at all CAN connect !!! > This is expected. Client auth is mostly used together with ACLs, otherwise AFAIK is pretty useless in ZK. Please not that MD5 is not "secure" at all, and consider using SASL/Kerberos for a production environment. Cheers Enrico > > Any hint ? > >
RE: stable 3.5 release
Thank you very much Andor. Just to have an estimate, are we talking couple of weeks ? couple of months ? or probably more than 3 months ? Thanks Rémi -Message d'origine- De : Andor Molnar [mailto:an...@cloudera.com] Envoyé : Tuesday, April 10, 2018 17:35 À : user@zookeeper.apache.org Objet : Re: stable 3.5 release Hi Remi / Ansel, We have a quite impressive list of blockers to get out of the way: https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FZOOKEEPER-1549%3Ffilter%3D12343244=02%7C01%7Crserrano%40pros.com%7C40524549c0af49856e4308d59ef8a24a%7C094cfb7ad13146379047e339e7d04359%7C0%7C0%7C636589713067158062=EPudToyCNDaDB4Avg7SC9LFMfgbbhNW5q4kMmFbK%2F0Q%3D=0 Additionally we have to get the following PRs merged: https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fzookeeper%2Fpull%2F377=02%7C01%7Crserrano%40pros.com%7C40524549c0af49856e4308d59ef8a24a%7C094cfb7ad13146379047e339e7d04359%7C0%7C0%7C636589713067158062=aZjNc5uks6yWMPPksZzBOjX7yt9b3VbMB4e06rpKvPg%3D=0 https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fzookeeper%2Fpull%2F184=02%7C01%7Crserrano%40pros.com%7C40524549c0af49856e4308d59ef8a24a%7C094cfb7ad13146379047e339e7d04359%7C0%7C0%7C636589713067158062=Z2qGgfBye9Z4%2FqL9CVN3n%2F2iQc6IU4MSAyxnjGQKDhs%3D=0 Personally, I'm working on getting it released as soon as possible, but presumably this won't happen tomorrow. Regards, Andor On Tue, Apr 10, 2018 at 10:40 AM, Ansel Zandegran <ansel.zandeg...@infor.com > wrote: > Yes please, If anyone knows. > > // Ansel > > > On 10 Apr 2018, at 10:04, Remi Serrano <rserr...@pros.com> wrote: > > > > Hello, > > > > Does anyone know when will be released the stable 3.5 version ? > > > > Rémi > > > >
Client-Server authentication with DIGEST-MD5
Hello I'm trying to secure my ZK cluster. To do so I'm trying to leverage both : https://cwiki.apache.org/confluence/display/ZOOKEEPER/Server-Server+mutual+authentication and https://cwiki.apache.org/confluence/display/ZOOKEEPER/Client-Server+mutual+authentication The Server to Server works fine. However, the Client to Server seems to be useless as here is the behavior I get : * Client using a declared user on the server + good password CAN connect * Client using a declared user on the server + bad password CANNOT connect * Client using a non declared user on the Server CANNOT connect so far so good... but : * Client using NO user at all CAN connect !!! Any hint ?
stable 3.5 release
Hello, Does anyone know when will be released the stable 3.5 version ? Rémi
