Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
https://issues.apache.org/jira/browse/ZOOKEEPER-3511 Andor -Original Message-From: Patrick Hunt Reply- To: user@zookeeper.apache.orgTo: UserZooKeeper < user@zookeeper.apache.org>Subject: Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0Date: Mon, 5 Aug 2019 09:04:44 -0700 It sounds to me like a regression. We always had the properties format for4lw, this (membership:) breaks that. I'd recommend fixing it in the next3.5/3.6. ie. output the membership on a single line "membership: \n".Should be a pretty simple change - anyone interested in taking it on? Also agree that folks should move off 4lw to the new (better) options, espas we plan to deprecate 4lw at some point. Patrick On Sun, Aug 4, 2019 at 12:15 PM Enrico Olivelli wrote: > Il sab 3 ago 2019, 21:41 Shawn Heisey ha > scritto: > > On 8/2/2019 10:33 AM, Patrick Hunt wrote: > > > Right, it prints the membership of the quorum, see (for majority > > > case > > which > > > is > > > typical):org.apache.zookeeper.server.quorum.flexible.QuorumMaj#to > > > String > https://github.com/apache/zookeeper/blob/faa7cec71fddfb959a7d67923acffdb67d93c953/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/flexible/QuorumMaj.java#L112 > > For our purposes (the Solr project) the output of the "conf" 4lw > > commandis inconsistent, changing when there is a multi-server > > ensemble. All ofthe lines except the "membership: " one use an > > equals sign as aseparator. Our parsing code fails on that line > > because there is noequals sign. > > Whether or not the ZK project should consider this a bug is the > > questionthat I am asking. > > While getting to the bottom of that question, another one > > arises: Whoare the intended audiences of the "conf" 4lw > > output? If one of thoseaudiences is ZK itself, then the output of > > the command probably willwork perfectly for that audience, as ZK > > uses Java's "properties" API toread its config file, which means > > that both = and : will work asseparators. > > The current output also works great for a human audience. Humans > > arequite flexible. > > The difficulty is machine-based parsers like the one in Solr, which > > isvery simple and just splits lines on an equal sign. How > > muchconsistency can an audience like this expect? I would > > personally saythat the way "membership: " is output is a bug. That > > line probablyshould be entirely removed, or the colon could be > > replaced with an equalsign. I think that the line only makes sense > > for a human audience, andthat audience probably doesn't really need > > it. > > An alternate path: One statement in the documentation would remove > > alldifficulty, without any code changes in ZK: > > "The output from the conf 4lw command should be parsed by the > > JavaProperties API for best results." > > I think the best option is to switch to the Admin, HTTP + json based, > as itis possible to integrate better with other automatic tools.We > are working on docs for 3.6 (expecially the http admin server).We > also added many new 'commands' to the admin API, which is supposed to > bethe future for the mid/long term > Enrico > > > > If that statement is added, then Solr just needs to utilize > > theProperties API, which is very easy to do, and all is well again. > > So... I'm thinking we should open an issue in Jira, and then leave > > it upto the ZK committers whether it's better to change the output > > or adjustthe documentation. I can supply a patch either way. What > > does thecommunity think? > > Thanks,Shawn
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
It sounds to me like a regression. We always had the properties format for 4lw, this (membership:) breaks that. I'd recommend fixing it in the next 3.5/3.6. ie. output the membership on a single line "membership: \n". Should be a pretty simple change - anyone interested in taking it on? Also agree that folks should move off 4lw to the new (better) options, esp as we plan to deprecate 4lw at some point. Patrick On Sun, Aug 4, 2019 at 12:15 PM Enrico Olivelli wrote: > Il sab 3 ago 2019, 21:41 Shawn Heisey ha scritto: > > > On 8/2/2019 10:33 AM, Patrick Hunt wrote: > > > Right, it prints the membership of the quorum, see (for majority case > > which > > > is typical): > > > org.apache.zookeeper.server.quorum.flexible.QuorumMaj#toString > > > > > > https://github.com/apache/zookeeper/blob/faa7cec71fddfb959a7d67923acffdb67d93c953/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/flexible/QuorumMaj.java#L112 > > > > For our purposes (the Solr project) the output of the "conf" 4lw command > > is inconsistent, changing when there is a multi-server ensemble. All of > > the lines except the "membership: " one use an equals sign as a > > separator. Our parsing code fails on that line because there is no > > equals sign. > > > > Whether or not the ZK project should consider this a bug is the question > > that I am asking. > > > > While getting to the bottom of that question, another one arises: Who > > are the intended audiences of the "conf" 4lw output? If one of those > > audiences is ZK itself, then the output of the command probably will > > work perfectly for that audience, as ZK uses Java's "properties" API to > > read its config file, which means that both = and : will work as > > separators. > > > > The current output also works great for a human audience. Humans are > > quite flexible. > > > > The difficulty is machine-based parsers like the one in Solr, which is > > very simple and just splits lines on an equal sign. How much > > consistency can an audience like this expect? I would personally say > > that the way "membership: " is output is a bug. That line probably > > should be entirely removed, or the colon could be replaced with an equal > > sign. I think that the line only makes sense for a human audience, and > > that audience probably doesn't really need it. > > > > An alternate path: One statement in the documentation would remove all > > difficulty, without any code changes in ZK: > > > > "The output from the conf 4lw command should be parsed by the Java > > Properties API for best results." > > > > I think the best option is to switch to the Admin, HTTP + json based, as it > is possible to integrate better with other automatic tools. > We are working on docs for 3.6 (expecially the http admin server). > We also added many new 'commands' to the admin API, which is supposed to be > the future for the mid/long term > > Enrico > > > > > If that statement is added, then Solr just needs to utilize the > > Properties API, which is very easy to do, and all is well again. > > > > So... I'm thinking we should open an issue in Jira, and then leave it up > > to the ZK committers whether it's better to change the output or adjust > > the documentation. I can supply a patch either way. What does the > > community think? > > > > Thanks, > > Shawn > > >
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
Il lun 5 ago 2019, 00:57 Jan Høydahl ha scritto: > Will admin server be folded in and exposed on same port as main client > port in the future? If not, clients will need to have one config for zkHost > plus one more for zkAdminServer. Personally I hope we won't do this. I hope we continue investing in the client endpoint performances and mixing it with an HTTP server will complicate things. That said, it is possible in theory to merge them I asked in another thread of admin server port number will have a > better/more unique default than 8080 in the future, such as 2188 or > whatever? > +1 I don't know how much this can impact downstream bundles. I am not an user of admin server yet, I will switch as soon as 3.6 is out. Enrico > Jan Høydahl > > > 4. aug. 2019 kl. 21:15 skrev Enrico Olivelli : > > > > Il sab 3 ago 2019, 21:41 Shawn Heisey ha scritto: > > > >>> On 8/2/2019 10:33 AM, Patrick Hunt wrote: > >>> Right, it prints the membership of the quorum, see (for majority case > >> which > >>> is typical): > >>> org.apache.zookeeper.server.quorum.flexible.QuorumMaj#toString > >>> > >> > https://github.com/apache/zookeeper/blob/faa7cec71fddfb959a7d67923acffdb67d93c953/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/flexible/QuorumMaj.java#L112 > >> > >> For our purposes (the Solr project) the output of the "conf" 4lw command > >> is inconsistent, changing when there is a multi-server ensemble. All of > >> the lines except the "membership: " one use an equals sign as a > >> separator. Our parsing code fails on that line because there is no > >> equals sign. > >> > >> Whether or not the ZK project should consider this a bug is the question > >> that I am asking. > >> > >> While getting to the bottom of that question, another one arises: Who > >> are the intended audiences of the "conf" 4lw output? If one of those > >> audiences is ZK itself, then the output of the command probably will > >> work perfectly for that audience, as ZK uses Java's "properties" API to > >> read its config file, which means that both = and : will work as > >> separators. > >> > >> The current output also works great for a human audience. Humans are > >> quite flexible. > >> > >> The difficulty is machine-based parsers like the one in Solr, which is > >> very simple and just splits lines on an equal sign. How much > >> consistency can an audience like this expect? I would personally say > >> that the way "membership: " is output is a bug. That line probably > >> should be entirely removed, or the colon could be replaced with an equal > >> sign. I think that the line only makes sense for a human audience, and > >> that audience probably doesn't really need it. > >> > >> An alternate path: One statement in the documentation would remove all > >> difficulty, without any code changes in ZK: > >> > >> "The output from the conf 4lw command should be parsed by the Java > >> Properties API for best results." > >> > > > > I think the best option is to switch to the Admin, HTTP + json based, as > it > > is possible to integrate better with other automatic tools. > > We are working on docs for 3.6 (expecially the http admin server). > > We also added many new 'commands' to the admin API, which is supposed to > be > > the future for the mid/long term > > > > Enrico > > > > > > > >> If that statement is added, then Solr just needs to utilize the > >> Properties API, which is very easy to do, and all is well again. > >> > >> So... I'm thinking we should open an issue in Jira, and then leave it up > >> to the ZK committers whether it's better to change the output or adjust > >> the documentation. I can supply a patch either way. What does the > >> community think? > >> > >> Thanks, > >> Shawn > >> >
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
Will admin server be folded in and exposed on same port as main client port in the future? If not, clients will need to have one config for zkHost plus one more for zkAdminServer. I asked in another thread of admin server port number will have a better/more unique default than 8080 in the future, such as 2188 or whatever? Jan Høydahl > 4. aug. 2019 kl. 21:15 skrev Enrico Olivelli : > > Il sab 3 ago 2019, 21:41 Shawn Heisey ha scritto: > >>> On 8/2/2019 10:33 AM, Patrick Hunt wrote: >>> Right, it prints the membership of the quorum, see (for majority case >> which >>> is typical): >>> org.apache.zookeeper.server.quorum.flexible.QuorumMaj#toString >>> >> https://github.com/apache/zookeeper/blob/faa7cec71fddfb959a7d67923acffdb67d93c953/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/flexible/QuorumMaj.java#L112 >> >> For our purposes (the Solr project) the output of the "conf" 4lw command >> is inconsistent, changing when there is a multi-server ensemble. All of >> the lines except the "membership: " one use an equals sign as a >> separator. Our parsing code fails on that line because there is no >> equals sign. >> >> Whether or not the ZK project should consider this a bug is the question >> that I am asking. >> >> While getting to the bottom of that question, another one arises: Who >> are the intended audiences of the "conf" 4lw output? If one of those >> audiences is ZK itself, then the output of the command probably will >> work perfectly for that audience, as ZK uses Java's "properties" API to >> read its config file, which means that both = and : will work as >> separators. >> >> The current output also works great for a human audience. Humans are >> quite flexible. >> >> The difficulty is machine-based parsers like the one in Solr, which is >> very simple and just splits lines on an equal sign. How much >> consistency can an audience like this expect? I would personally say >> that the way "membership: " is output is a bug. That line probably >> should be entirely removed, or the colon could be replaced with an equal >> sign. I think that the line only makes sense for a human audience, and >> that audience probably doesn't really need it. >> >> An alternate path: One statement in the documentation would remove all >> difficulty, without any code changes in ZK: >> >> "The output from the conf 4lw command should be parsed by the Java >> Properties API for best results." >> > > I think the best option is to switch to the Admin, HTTP + json based, as it > is possible to integrate better with other automatic tools. > We are working on docs for 3.6 (expecially the http admin server). > We also added many new 'commands' to the admin API, which is supposed to be > the future for the mid/long term > > Enrico > > > >> If that statement is added, then Solr just needs to utilize the >> Properties API, which is very easy to do, and all is well again. >> >> So... I'm thinking we should open an issue in Jira, and then leave it up >> to the ZK committers whether it's better to change the output or adjust >> the documentation. I can supply a patch either way. What does the >> community think? >> >> Thanks, >> Shawn >>
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
Il sab 3 ago 2019, 21:41 Shawn Heisey ha scritto: > On 8/2/2019 10:33 AM, Patrick Hunt wrote: > > Right, it prints the membership of the quorum, see (for majority case > which > > is typical): > > org.apache.zookeeper.server.quorum.flexible.QuorumMaj#toString > > > https://github.com/apache/zookeeper/blob/faa7cec71fddfb959a7d67923acffdb67d93c953/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/flexible/QuorumMaj.java#L112 > > For our purposes (the Solr project) the output of the "conf" 4lw command > is inconsistent, changing when there is a multi-server ensemble. All of > the lines except the "membership: " one use an equals sign as a > separator. Our parsing code fails on that line because there is no > equals sign. > > Whether or not the ZK project should consider this a bug is the question > that I am asking. > > While getting to the bottom of that question, another one arises: Who > are the intended audiences of the "conf" 4lw output? If one of those > audiences is ZK itself, then the output of the command probably will > work perfectly for that audience, as ZK uses Java's "properties" API to > read its config file, which means that both = and : will work as > separators. > > The current output also works great for a human audience. Humans are > quite flexible. > > The difficulty is machine-based parsers like the one in Solr, which is > very simple and just splits lines on an equal sign. How much > consistency can an audience like this expect? I would personally say > that the way "membership: " is output is a bug. That line probably > should be entirely removed, or the colon could be replaced with an equal > sign. I think that the line only makes sense for a human audience, and > that audience probably doesn't really need it. > > An alternate path: One statement in the documentation would remove all > difficulty, without any code changes in ZK: > > "The output from the conf 4lw command should be parsed by the Java > Properties API for best results." > I think the best option is to switch to the Admin, HTTP + json based, as it is possible to integrate better with other automatic tools. We are working on docs for 3.6 (expecially the http admin server). We also added many new 'commands' to the admin API, which is supposed to be the future for the mid/long term Enrico > If that statement is added, then Solr just needs to utilize the > Properties API, which is very easy to do, and all is well again. > > So... I'm thinking we should open an issue in Jira, and then leave it up > to the ZK committers whether it's better to change the output or adjust > the documentation. I can supply a patch either way. What does the > community think? > > Thanks, > Shawn >
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
On 8/2/2019 10:33 AM, Patrick Hunt wrote: Right, it prints the membership of the quorum, see (for majority case which is typical): org.apache.zookeeper.server.quorum.flexible.QuorumMaj#toString https://github.com/apache/zookeeper/blob/faa7cec71fddfb959a7d67923acffdb67d93c953/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/flexible/QuorumMaj.java#L112 For our purposes (the Solr project) the output of the "conf" 4lw command is inconsistent, changing when there is a multi-server ensemble. All of the lines except the "membership: " one use an equals sign as a separator. Our parsing code fails on that line because there is no equals sign. Whether or not the ZK project should consider this a bug is the question that I am asking. While getting to the bottom of that question, another one arises: Who are the intended audiences of the "conf" 4lw output? If one of those audiences is ZK itself, then the output of the command probably will work perfectly for that audience, as ZK uses Java's "properties" API to read its config file, which means that both = and : will work as separators. The current output also works great for a human audience. Humans are quite flexible. The difficulty is machine-based parsers like the one in Solr, which is very simple and just splits lines on an equal sign. How much consistency can an audience like this expect? I would personally say that the way "membership: " is output is a bug. That line probably should be entirely removed, or the colon could be replaced with an equal sign. I think that the line only makes sense for a human audience, and that audience probably doesn't really need it. An alternate path: One statement in the documentation would remove all difficulty, without any code changes in ZK: "The output from the conf 4lw command should be parsed by the Java Properties API for best results." If that statement is added, then Solr just needs to utilize the Properties API, which is very easy to do, and all is well again. So... I'm thinking we should open an issue in Jira, and then leave it up to the ZK committers whether it's better to change the output or adjust the documentation. I can supply a patch either way. What does the community think? Thanks, Shawn
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
On Fri, Aug 2, 2019 at 7:12 AM Jan Høydahl wrote: > So it turned out that Solr failed parsing the 'conf' response from > Zookeeper since there was one line that did not conform to the key=value > syntax: > > ... > quorumPort=2888 > peerType=0 > membership: > server.1=zoo1:2888:3888:participant;0.0.0.0:2181 > server.2=zoo2:2888:3888:participant;0.0.0.0:2181 > server.3=zoo3:2888:3888:participant;0.0.0.0:2181 > ... > > The "membership: " line was the offender here. Can anyone shed some light > on the purpose of that line? Is it a header for what comes in the lines > below or is it a key with wrong delimiter (: instead of =)? and empty value? > > Right, it prints the membership of the quorum, see (for majority case which is typical): org.apache.zookeeper.server.quorum.flexible.QuorumMaj#toString https://github.com/apache/zookeeper/blob/faa7cec71fddfb959a7d67923acffdb67d93c953/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/flexible/QuorumMaj.java#L112 Seems like the header, it came in from a change by @Alexander Shraer who may have more insight. Patrick > We'll fix the parsing in Solr to disregard this line for now. > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > > 31. jul. 2019 kl. 08:34 skrev Zheng Lin Edwin Yeo >: > > > > Sure. I have another thread on this in the Solr mailing list. > > > > Regards, > > Edwin > > > > On Wed, 31 Jul 2019 at 14:04, Jörn Franke wrote: > > > >> Could be a Solr issue. Check the Solr mailing list - they can help you > >> there better on Solr issues. > >> > >>> Am 31.07.2019 um 04:27 schrieb Zheng Lin Edwin Yeo < > edwinye...@gmail.com > >>> : > >>> > >>> Thanks everyone for your reply. > >>> > >>> I have tried to put the following in zoo.cfg under ZooKeeper: > >>> 4lw.commands.whitelist=mntr,conf,ruok > >>> > >>> But it is still showing this error. > >>> *"Errors: - membership: Check 4lq.commands.whitelist setting in > zookeeper > >>> configuration file."* > >>> > >>> As I am using SolrCloud, the collection config can still be loaded to > >>> ZooKeeper as per normal. But if I tried to create a collection, I will > >> get > >>> the following error: > >>> > >>> { > >>> "responseHeader":{ > >>> "status":400, > >>> "QTime":686}, > >>> "failure":{ > >>> "192.168.1.2:8983 > >> _solr":"org.apache.solr.client.solrj.SolrServerException:IOException > >>> occurred when talking to server at: http://192.168.1.2:8983/solr";, > >>> "192.168.1.2:8984 > >> _solr":"org.apache.solr.client.solrj.SolrServerException:IOException > >>> occurred when talking to server at: http://192.168.1.2:8984/solr"}, > >>> "Operation create caused > >>> > >> > exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: > >>> Underlying core creation failed while creating collection: > collection1", > >>> "exception":{f > >>> "msg":"Underlying core creation failed while creating collection: > >>> collection1", > >>> "rspCode":400}, > >>> "error":{ > >>> "metadata":[ > >>> "error-class","org.apache.solr.common.SolrException", > >>> "root-error-class","org.apache.solr.common.SolrException"], > >>> "msg":"Underlying core creation failed while creating collection: > >>> collection1", > >>> "code":400}} > >>> > >>> Is there anything which I may have missed out? > >>> > >>> Regards, > >>> Edwin > >>> > On Wed, 31 Jul 2019 at 00:12, Shawn Heisey > wrote: > > > On 7/29/2019 11:45 PM, Enrico Olivelli wrote: > > Due to potential security risks since ZK 3.5 you have to explicitly > > whitelist some commands. > > The 3.5.5 documentation says that "*" can be used to whitelist all > commands. > > But what you just said seems to contradict that. If your statement is > more accurate, then the documentation should be updated to list the > commands that are NOT enabled when using a wildcard. > > There is a SOLR issue to upgrade the client in Solr to 3.5.5: > > https://issues.apache.org/jira/browse/SOLR-8346 > > A comment was made on this issue saying that the following config is > needed when the server is running 3.5.x: > > 4lw.commands.whitelist=mntr,conf,ruok > > Thanks, > Shawn > > >> > >
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
So it turned out that Solr failed parsing the 'conf' response from Zookeeper since there was one line that did not conform to the key=value syntax: ... quorumPort=2888 peerType=0 membership: server.1=zoo1:2888:3888:participant;0.0.0.0:2181 server.2=zoo2:2888:3888:participant;0.0.0.0:2181 server.3=zoo3:2888:3888:participant;0.0.0.0:2181 ... The "membership: " line was the offender here. Can anyone shed some light on the purpose of that line? Is it a header for what comes in the lines below or is it a key with wrong delimiter (: instead of =)? and empty value? We'll fix the parsing in Solr to disregard this line for now. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 31. jul. 2019 kl. 08:34 skrev Zheng Lin Edwin Yeo : > > Sure. I have another thread on this in the Solr mailing list. > > Regards, > Edwin > > On Wed, 31 Jul 2019 at 14:04, Jörn Franke wrote: > >> Could be a Solr issue. Check the Solr mailing list - they can help you >> there better on Solr issues. >> >>> Am 31.07.2019 um 04:27 schrieb Zheng Lin Edwin Yeo >> : >>> >>> Thanks everyone for your reply. >>> >>> I have tried to put the following in zoo.cfg under ZooKeeper: >>> 4lw.commands.whitelist=mntr,conf,ruok >>> >>> But it is still showing this error. >>> *"Errors: - membership: Check 4lq.commands.whitelist setting in zookeeper >>> configuration file."* >>> >>> As I am using SolrCloud, the collection config can still be loaded to >>> ZooKeeper as per normal. But if I tried to create a collection, I will >> get >>> the following error: >>> >>> { >>> "responseHeader":{ >>> "status":400, >>> "QTime":686}, >>> "failure":{ >>> "192.168.1.2:8983 >> _solr":"org.apache.solr.client.solrj.SolrServerException:IOException >>> occurred when talking to server at: http://192.168.1.2:8983/solr";, >>> "192.168.1.2:8984 >> _solr":"org.apache.solr.client.solrj.SolrServerException:IOException >>> occurred when talking to server at: http://192.168.1.2:8984/solr"}, >>> "Operation create caused >>> >> exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: >>> Underlying core creation failed while creating collection: collection1", >>> "exception":{f >>> "msg":"Underlying core creation failed while creating collection: >>> collection1", >>> "rspCode":400}, >>> "error":{ >>> "metadata":[ >>> "error-class","org.apache.solr.common.SolrException", >>> "root-error-class","org.apache.solr.common.SolrException"], >>> "msg":"Underlying core creation failed while creating collection: >>> collection1", >>> "code":400}} >>> >>> Is there anything which I may have missed out? >>> >>> Regards, >>> Edwin >>> On Wed, 31 Jul 2019 at 00:12, Shawn Heisey wrote: > On 7/29/2019 11:45 PM, Enrico Olivelli wrote: > Due to potential security risks since ZK 3.5 you have to explicitly > whitelist some commands. The 3.5.5 documentation says that "*" can be used to whitelist all commands. But what you just said seems to contradict that. If your statement is more accurate, then the documentation should be updated to list the commands that are NOT enabled when using a wildcard. There is a SOLR issue to upgrade the client in Solr to 3.5.5: https://issues.apache.org/jira/browse/SOLR-8346 A comment was made on this issue saying that the following config is needed when the server is running 3.5.x: 4lw.commands.whitelist=mntr,conf,ruok Thanks, Shawn >>
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
Sure. I have another thread on this in the Solr mailing list. Regards, Edwin On Wed, 31 Jul 2019 at 14:04, Jörn Franke wrote: > Could be a Solr issue. Check the Solr mailing list - they can help you > there better on Solr issues. > > > Am 31.07.2019 um 04:27 schrieb Zheng Lin Edwin Yeo >: > > > > Thanks everyone for your reply. > > > > I have tried to put the following in zoo.cfg under ZooKeeper: > > 4lw.commands.whitelist=mntr,conf,ruok > > > > But it is still showing this error. > > *"Errors: - membership: Check 4lq.commands.whitelist setting in zookeeper > > configuration file."* > > > > As I am using SolrCloud, the collection config can still be loaded to > > ZooKeeper as per normal. But if I tried to create a collection, I will > get > > the following error: > > > > { > > "responseHeader":{ > >"status":400, > >"QTime":686}, > > "failure":{ > >"192.168.1.2:8983 > _solr":"org.apache.solr.client.solrj.SolrServerException:IOException > > occurred when talking to server at: http://192.168.1.2:8983/solr";, > >"192.168.1.2:8984 > _solr":"org.apache.solr.client.solrj.SolrServerException:IOException > > occurred when talking to server at: http://192.168.1.2:8984/solr"}, > > "Operation create caused > > > exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: > > Underlying core creation failed while creating collection: collection1", > > "exception":{f > >"msg":"Underlying core creation failed while creating collection: > > collection1", > >"rspCode":400}, > > "error":{ > >"metadata":[ > > "error-class","org.apache.solr.common.SolrException", > > "root-error-class","org.apache.solr.common.SolrException"], > >"msg":"Underlying core creation failed while creating collection: > > collection1", > >"code":400}} > > > > Is there anything which I may have missed out? > > > > Regards, > > Edwin > > > >> On Wed, 31 Jul 2019 at 00:12, Shawn Heisey wrote: > >> > >>> On 7/29/2019 11:45 PM, Enrico Olivelli wrote: > >>> Due to potential security risks since ZK 3.5 you have to explicitly > >>> whitelist some commands. > >> > >> The 3.5.5 documentation says that "*" can be used to whitelist all > >> commands. > >> > >> But what you just said seems to contradict that. If your statement is > >> more accurate, then the documentation should be updated to list the > >> commands that are NOT enabled when using a wildcard. > >> > >> There is a SOLR issue to upgrade the client in Solr to 3.5.5: > >> > >> https://issues.apache.org/jira/browse/SOLR-8346 > >> > >> A comment was made on this issue saying that the following config is > >> needed when the server is running 3.5.x: > >> > >> 4lw.commands.whitelist=mntr,conf,ruok > >> > >> Thanks, > >> Shawn > >> >
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
Could be a Solr issue. Check the Solr mailing list - they can help you there better on Solr issues. > Am 31.07.2019 um 04:27 schrieb Zheng Lin Edwin Yeo : > > Thanks everyone for your reply. > > I have tried to put the following in zoo.cfg under ZooKeeper: > 4lw.commands.whitelist=mntr,conf,ruok > > But it is still showing this error. > *"Errors: - membership: Check 4lq.commands.whitelist setting in zookeeper > configuration file."* > > As I am using SolrCloud, the collection config can still be loaded to > ZooKeeper as per normal. But if I tried to create a collection, I will get > the following error: > > { > "responseHeader":{ >"status":400, >"QTime":686}, > "failure":{ > > "192.168.1.2:8983_solr":"org.apache.solr.client.solrj.SolrServerException:IOException > occurred when talking to server at: http://192.168.1.2:8983/solr";, > > "192.168.1.2:8984_solr":"org.apache.solr.client.solrj.SolrServerException:IOException > occurred when talking to server at: http://192.168.1.2:8984/solr"}, > "Operation create caused > exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: > Underlying core creation failed while creating collection: collection1", > "exception":{f >"msg":"Underlying core creation failed while creating collection: > collection1", >"rspCode":400}, > "error":{ >"metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","org.apache.solr.common.SolrException"], >"msg":"Underlying core creation failed while creating collection: > collection1", >"code":400}} > > Is there anything which I may have missed out? > > Regards, > Edwin > >> On Wed, 31 Jul 2019 at 00:12, Shawn Heisey wrote: >> >>> On 7/29/2019 11:45 PM, Enrico Olivelli wrote: >>> Due to potential security risks since ZK 3.5 you have to explicitly >>> whitelist some commands. >> >> The 3.5.5 documentation says that "*" can be used to whitelist all >> commands. >> >> But what you just said seems to contradict that. If your statement is >> more accurate, then the documentation should be updated to list the >> commands that are NOT enabled when using a wildcard. >> >> There is a SOLR issue to upgrade the client in Solr to 3.5.5: >> >> https://issues.apache.org/jira/browse/SOLR-8346 >> >> A comment was made on this issue saying that the following config is >> needed when the server is running 3.5.x: >> >> 4lw.commands.whitelist=mntr,conf,ruok >> >> Thanks, >> Shawn >>
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
Thanks everyone for your reply. I have tried to put the following in zoo.cfg under ZooKeeper: 4lw.commands.whitelist=mntr,conf,ruok But it is still showing this error. *"Errors: - membership: Check 4lq.commands.whitelist setting in zookeeper configuration file."* As I am using SolrCloud, the collection config can still be loaded to ZooKeeper as per normal. But if I tried to create a collection, I will get the following error: { "responseHeader":{ "status":400, "QTime":686}, "failure":{ "192.168.1.2:8983_solr":"org.apache.solr.client.solrj.SolrServerException:IOException occurred when talking to server at: http://192.168.1.2:8983/solr";, "192.168.1.2:8984_solr":"org.apache.solr.client.solrj.SolrServerException:IOException occurred when talking to server at: http://192.168.1.2:8984/solr"}, "Operation create caused exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Underlying core creation failed while creating collection: collection1", "exception":{f "msg":"Underlying core creation failed while creating collection: collection1", "rspCode":400}, "error":{ "metadata":[ "error-class","org.apache.solr.common.SolrException", "root-error-class","org.apache.solr.common.SolrException"], "msg":"Underlying core creation failed while creating collection: collection1", "code":400}} Is there anything which I may have missed out? Regards, Edwin On Wed, 31 Jul 2019 at 00:12, Shawn Heisey wrote: > On 7/29/2019 11:45 PM, Enrico Olivelli wrote: > > Due to potential security risks since ZK 3.5 you have to explicitly > > whitelist some commands. > > The 3.5.5 documentation says that "*" can be used to whitelist all > commands. > > But what you just said seems to contradict that. If your statement is > more accurate, then the documentation should be updated to list the > commands that are NOT enabled when using a wildcard. > > There is a SOLR issue to upgrade the client in Solr to 3.5.5: > > https://issues.apache.org/jira/browse/SOLR-8346 > > A comment was made on this issue saying that the following config is > needed when the server is running 3.5.x: > > 4lw.commands.whitelist=mntr,conf,ruok > > Thanks, > Shawn >
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
On 7/29/2019 11:45 PM, Enrico Olivelli wrote: Due to potential security risks since ZK 3.5 you have to explicitly whitelist some commands. The 3.5.5 documentation says that "*" can be used to whitelist all commands. But what you just said seems to contradict that. If your statement is more accurate, then the documentation should be updated to list the commands that are NOT enabled when using a wildcard. There is a SOLR issue to upgrade the client in Solr to 3.5.5: https://issues.apache.org/jira/browse/SOLR-8346 A comment was made on this issue saying that the following config is needed when the server is running 3.5.x: 4lw.commands.whitelist=mntr,conf,ruok Thanks, Shawn
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
Due to potential security risks since ZK 3.5 you have to explicitly whitelist some commands. I think you should contact SolrCloud I hope that helps Enrico Il mar 30 lug 2019, 04:08 Zheng Lin Edwin Yeo ha scritto: > Hi, > > I am using the new Solr 8.2.0 with SolrCloud and external ZooKeeper 3.5.5. > > However, after adding in the line under zoo.cfg > *4lw.commands.whitelist=** > > I get the error under Cloud -> ZK Status in Solr > *"Errors: - membership: Check 4lq.commands.whitelist setting in zookeeper > configuration file."* > > I have noticed that the issue is cause by adding the "conf" in the > whitelist. But if I do not add the "conf" to the whitelist, I will get the > following error: > *"Errors: - conf is not executed because it is not in the whitelist. Check > 4lw.commands.whitelist setting in zookeeper configuration file."* > > What could be the issue that cause this error, and how can we resolve it? > > Thank you. > > Regards, > Edwin >
Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0
Maybe the Solr mailing list should address this. > Am 30.07.2019 um 04:08 schrieb Zheng Lin Edwin Yeo : > > Hi, > > I am using the new Solr 8.2.0 with SolrCloud and external ZooKeeper 3.5.5. > > However, after adding in the line under zoo.cfg > *4lw.commands.whitelist=** > > I get the error under Cloud -> ZK Status in Solr > *"Errors: - membership: Check 4lq.commands.whitelist setting in zookeeper > configuration file."* > > I have noticed that the issue is cause by adding the "conf" in the > whitelist. But if I do not add the "conf" to the whitelist, I will get the > following error: > *"Errors: - conf is not executed because it is not in the whitelist. Check > 4lw.commands.whitelist setting in zookeeper configuration file."* > > What could be the issue that cause this error, and how can we resolve it? > > Thank you. > > Regards, > Edwin