Re: What OpenJDK Version Can I Use?

2023-09-22 Thread Shawn Heisey
On 9/21/23 12:51, DeLella, David wrote: To all, I am contemplating a Java upgrade on some local instances of Zookeeper. I cannot find any documentation online about the highest compatible version of Java I can use. The version numbers of both my Solr and Zookeeper instance are fixed and

Re: Is the zookeeper client 3.5x compatible with 3.4 server

2023-06-03 Thread Shawn Heisey
On 6/2/23 00:00, Xie wrote: No infomation found in the release note. But we do find some use who complained the backward compatibility to 3.4 server. So, do we have any tests about this? And which functions are broken when use 3.5+ client against the 3.4 server? Yes. There is compatibility

Re: Support for Java 17

2023-04-18 Thread Shawn Heisey
On 4/17/2023 7:21 AM, Wilson Leao wrote: Are there any plans to support Java 17+ ? I can't speak directly for the ZK project... but I am running a development version of Solr, in cloud mode, with the embedded ZK server, on OpenJDK 17. So it is running both the ZK server and the ZK client.

Re: Backup and restore Solr 8.11.2 collections and configsets in Zookeeper version: 3.7.0

2022-09-16 Thread Shawn Heisey
On 9/16/22 09:37, Szalay-Bekő Máté wrote: But actually much better would be to do the backup and restore on Solr level. Solr doesn't currently have this capability.  We do have functionality that can download index configs from ZK to the filesystem, but not all the cluster contents in ZK.

Re: Apache ZooKeeper Consistency with Majority Failure

2022-07-28 Thread Shawn Heisey
On 7/28/22 13:33, Shawn Heisey wrote: Node 1 is most likely informed that its database is now out of date (or it decides that for itself) so it syncs the whole DB from the current leader, which will not know about the znode created in step B. Not in any way a ZK expert.  But that seems like

Re: Apache ZooKeeper Consistency with Majority Failure

2022-07-28 Thread Shawn Heisey
On 7/28/22 08:35, José Armando García Sancio wrote: B) I started the majority of the nodes (1, 2). The ensemble was established and I was able to create a znode using the CLI. C) I shutdown all of the nodes (1, 2 since I never started node 3). To simulate a disk failure I deleted the content of

Re: What does Apache ZooKeeper do?

2022-05-18 Thread Shawn Heisey
On 5/18/2022 8:07 AM, Turritopsis Dohrnii Teo En Ming wrote: I notice my company/organization is using Apache ZooKeeper. What does it do? Did you visit the website?  It's got a pretty good summary.  And following that, there is link to the wiki, which has even more detail.

Re: TLS quorum host name verification issue with docker-compose

2022-04-06 Thread Shawn Heisey
On 2022-04-06 14:54, René Buffat wrote: javax.net.ssl.SSLPeerUnverifiedException: Certificate for doesn't match common name of the certificate subject: zookeeper2 java.security.cert.CertificateException: Failed to verify both host address and host name Generally speaking, I have never

Re: 回复: why zookeeper server count is odd !!!

2021-09-01 Thread Shawn Heisey
On 9/1/2021 6:32 PM, 一直以来 wrote: second email i want know : if i have 7 zookeeper server, if internet error, <123>server in one small master-replica, <4567> server in one small master-replica too, so i want know : <123>server can have new leader?? and <4567>server can have new leader?? I

Re: why zookeeper server count is odd !!!

2021-09-01 Thread Shawn Heisey
On 8/31/2021 7:59 PM, 一直以来 wrote: can has a url address ?? or at zookeeper.apache.org site doc ?? An odd number of servers is recommended because you don't gain anything from having one more (making it an even number).

Re: Zookeeper 3.4.5 with Solr 8.8.0

2021-03-01 Thread Shawn Heisey
On 3/1/2021 6:51 AM, Subhajit Das wrote: I noticed, that Solr 8.8.0 uses Zookeeper 3.6.2 client, while Solr 6.3.0 uses Zookeeper 3.4.6 client. Is this a client bug or mismatch issue? If so, how to fix this? The ZK project guarantees that each minor version (X.Y.Z, where Y is the same) will

Re: zookeeper / solr cloud problems

2019-12-13 Thread Shawn Heisey
On 12/13/2019 11:01 AM, Kojo wrote: We had already changed SO configuration before the last crash, so I think that the problem is not there. ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size

Re: zookeeper / solr cloud problems

2019-12-13 Thread Shawn Heisey
On 12/13/2019 9:47 AM, Kojo wrote: My setup is Solr Cloud (two shards) and Zookeeper (one instance) in the same box. I am having some problems (OutOfMemory) on Solr. This is the solr oom log: java.lang.OutOfMemoryError: unable to create new native thread Solr tried to start a new thread.

Re: One node crashing in 3.4.11 triggered a full ensemble restart

2019-10-03 Thread Shawn Heisey
On 10/3/2019 2:45 AM, Norbert Kalmar wrote: As for running a mixed version of 3.5 and 3.4 quorum - I'm afraid it will not work. From 3.5 we have a check on PROTOCOL_VERSION. 3.4 did not have this protocol version, so when the nodes try to communicate it will throw an exception. Plus, it is not a

Re: Zookeeper client with single address pointing to multiple servers

2019-09-27 Thread Shawn Heisey
On 9/27/2019 9:24 AM, Benjamin Reed wrote: are you making the assumption that you have a single machine that will always be up? that is not a common assumption these days, which is why solr might be resistant to such a change. you can have a single DNS name resolve to multiple IP addresses and

Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0

2019-08-03 Thread Shawn Heisey
On 8/2/2019 10:33 AM, Patrick Hunt wrote: Right, it prints the membership of the quorum, see (for majority case which is typical): org.apache.zookeeper.server.quorum.flexible.QuorumMaj#toString

Can SSL capability be satisfied by a smaller dependency than netty-all?

2019-07-30 Thread Shawn Heisey
We neglected to notice that netty is a required dependency for ZK SSL when we upgraded to ZK 3.5.5 in Solr. We have an issue to track this: https://issues.apache.org/jira/browse/SOLR-13665 I was noticing that the netty-all jar included in ZK is nearly 4MB ... and we will have to include it

Re: Issues with using ZooKeeper 3.5.5 together with Solr 8.2.0

2019-07-30 Thread Shawn Heisey
On 7/29/2019 11:45 PM, Enrico Olivelli wrote: Due to potential security risks since ZK 3.5 you have to explicitly whitelist some commands. The 3.5.5 documentation says that "*" can be used to whitelist all commands. But what you just said seems to contradict that. If your statement is more

Re: Does mentioning port number with FQDN resolve multiple connections

2018-10-09 Thread Shawn Heisey
On 10/9/2018 12:54 PM, Karthik K G wrote: We have a scenario where we give the Zookeeper FQDN to our Solr Application. When we use this we are seeing that zookeeper is accepting connections from all Solr nodes every minute. I'm more familiar with the Solr side than the ZK side. Here's a ZKHOST

Re: ZooKeeper monuturize clients

2018-10-08 Thread Shawn Heisey
On 10/8/2018 9:53 AM, Celso Diogo da Silva Batista (Academia) wrote: I have a zookeeper group with 3 nodes, with two clients there connected, kafka and flink. I wonder if there is any way to monuturize which clients are connected in real time to the zookeeper and know their status. I'm

Re: Does Apache ZooKeeper really need a JDK? Is JRE sufficient?

2018-09-15 Thread Shawn Heisey
On 9/15/2018 10:15 AM, M.P. Ardhanareeswaran wrote: Does Apache ZooKeeper need a JDK? Is a JRE sufficient? For running a binary release, JRE is sufficient. If you want to compile it from source, you'll need the JDK. Thanks, Shawn

Re: can not know the process name from zk log

2018-09-12 Thread Shawn Heisey
On 9/12/2018 2:33 AM, wangyongqiang0...@163.com wrote: from zk log, i can get the ip and port, i think if zk can print the process info with the ip and port , will help us in some cases What precisely are you after?  A java program can typically report what PID its process has, but I don't

Re: Port 3888 closed on Leader

2018-08-23 Thread Shawn Heisey
On 8/15/2018 7:46 AM, harish lohar wrote: In a deployment of 3 Node Zk Cluster we have seen that sometime port 3888 is absent after the cluster is formed , this causes Follower node to not able to connect to leader if they restart. Don't leader itself should come out of clustering if this

Re: ZooKeeper in different datacenters

2018-08-22 Thread Shawn Heisey
On 8/22/2018 11:10 AM, ilango dhandapani wrote: So, if I have 3 zk servers on 1st DC, 2 solr servers on 1st DC and 2 solr servers on 2nd DC, this will work right ? Other than network/latency between DC1 and DC2 for solr replication. No.  This is what I was trying to tell you. With half your

Re: ZooKeeper in different datacenters

2018-08-22 Thread Shawn Heisey
On 8/22/2018 10:02 AM, ilango dhandapani wrote: 1. To have disaster recovery, planning to have 2 solr servers on 1st DC and other 2 solr servers on 2nd DC. Seems there should not be any issue here. Each shard will have 1st node in 1st DC and 2nd node in 2nd DC. For Solr nodes in a SolrCloud

Re: Configuring SolrCloud with Redundancy on Two Physical Frames

2018-05-02 Thread Shawn Heisey
On 5/2/2018 11:44 AM, Adam Blank wrote: > Shawn - Sorry if I mixed up terminology, but by standalone I meant having a > single Zk instead of a Zk ensemble. So I would reconfigure the remaining > Zk and Solr node to only use that single Zk. To your point about the > config data being stored in

Re: Configuring SolrCloud with Redundancy on Two Physical Frames

2018-05-02 Thread Shawn Heisey
On 5/2/2018 7:07 AM, Adam Blank wrote: > Thank you everyone for the useful information. Would it be easy to > reconfigure an existing clustered deployment to a standalone deployment? > This is what I'm thinking: > > I have two physical servers. I would have one Zk installed on server 1 and > two

Re: Need help installing Zookeeper service in Ubuntu 16.04

2018-04-10 Thread Shawn Heisey
On 4/10/2018 7:43 PM, Gregorius Soedharmo wrote: Thank you for your help, but unfortunately, it all sounds gibberish to me. As stated in the stack exchange question, I'm a complete Linux newbie that couldn't even properly install a piece of software in Ubuntu. I did include all of my efforts so

Re: Need help installing Zookeeper service in Ubuntu 16.04

2018-04-10 Thread Shawn Heisey
On 4/10/2018 10:39 AM, Gregorius Soedharmo wrote: > I'm having problem installing Zookeeper as a service in Ubuntu, can you > help by answering either of these stack exchange question? > >

Re: Is the current max packet length available via the API?

2018-04-07 Thread Shawn Heisey
On 4/6/2018 6:46 AM, Martin Gainty wrote: ZOOMAIN="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=$JMXPORT -Dcom.sun.management.jmxremote.authenticate=$JMXAUTH -Dcom.sun.management.jmxremote.ssl=$JMXSSL -Dzookeeper.jmx.log4j.disable=$JMXLOG4J

Re: Is the current max packet length available via the API?

2018-04-05 Thread Shawn Heisey
On 4/5/2018 3:44 AM, Andor Molnar wrote: You can get the current jute.maxbuffer setting from a running ZooKeeper instance by querying ZooKeeperServerBean via JMX. I'm not sure how I would do that in a client program.  It might be trivial, but it's not something I've ever done. Currently

Is the current max packet length available via the API?

2018-04-04 Thread Shawn Heisey
Is it possible to get the current max packet length from the API? (version 3.4.x) If not, I'm guessing that I need to look for the jute.maxbuffer system property and fallback to ZkClientConfig.CLIENT_MAX_PACKET_LENGTH_DEFAULT if it's not defined. What I'm trying to do is log a useful error

Re: Upgrade required Java version to 1.8 on 3.5+

2018-03-08 Thread Shawn Heisey
On 3/7/2018 1:21 PM, Jeff Widman wrote: > +1 from me to using Java 8 or even going all the way to 9 for the 3.5 > release branch. I don't think it would be a good idea to require Java 9 at this time.  It's probably already an uphill battle for sysadmins to get approval to jump ONE major version. 

Re: Upgrade required Java version to 1.8 on 3.5+

2018-03-07 Thread Shawn Heisey
On 3/7/2018 4:04 AM, Andor Molnar wrote: I've quickly checked some of the major components that are heavy Zk clients: Hadoop/HDFS = 1.8 required HBase = 1.8 required Kafka = 1.7 required (has some 1.8 and 1.9 bindings) Hive = 1.8 required Curator = 1.7 required (has 1.8-only async module to

Re: "ant eclipse" in source code fails, easy fix

2018-03-04 Thread Shawn Heisey
On 3/3/2018 10:10 PM, Edward Ribeiro wrote: I recommend you open a JIRA issue at https://issues.apache.org/jira/projects/ZOOKEEPER and then open a PR to https://www.github.com/apache/zookeeper , please. This is clearly a bug and the fix is trivial so you can bypass any dev mailing list

"ant eclipse" in source code fails, easy fix

2018-03-03 Thread Shawn Heisey
I know I really should be putting this on the dev list.  The reason I'm not doing so is because I'm already subscribed to far too many mailing lists.  I don't expect to be making a ton of contributions to ZK, so I don't want to join another mailing list for one little discussion.  If at some

Re: Ensemble fails when one node looses connectivity

2018-03-02 Thread Shawn Heisey
On 3/2/2018 6:54 AM, Jim Keeney wrote: Thanks for jumping in on the ZK side as well. I will take a hard look at my config files but I checked and I do not have any one file over 1MB. The combined files (10 indexes) is 2.2MB. I am using micros for the nodes which are very limited in memory.

Re: Ensemble fails when one node looses connectivity

2018-03-01 Thread Shawn Heisey
On 3/1/2018 7:59 PM, Jim Keeney wrote: Read about the maxbuffer and am pretty sure that this might explain the behavior we are seeing since it occurs when there has been a significant reboot of all the servers. We have over 2 mb of config files for all of our indexes and if all the Solr nodes

Re: Zookeeper session expiration

2017-12-04 Thread Shawn Heisey
On 12/4/2017 12:51 PM, Anthony Shaya wrote: Thanks Shawn, should I message the developer mailing list for a more definitive answer? The ZK dev list is for discussion around the development of ZK itself, NOT for development of software that uses ZK. For the latter kind of development, you

Re: Zookeeper session expiration

2017-12-04 Thread Shawn Heisey
On 12/4/2017 8:22 AM, Anthony Shaya wrote: My question is related to how session expiration works, I noticed on many of the client machines the times across these machines were all off (by anywhere from 1 minute to 20 minutes - which was resolved after discovery - haven't verified this

Re: ...likely client has closed socket...

2017-07-14 Thread Shawn Heisey
On 7/14/2017 5:14 AM, mosto...@gmail.com wrote: > Using zookeeper 3.5.3-beta we are getting a few log lines like: > >[2017-07-14 13:02:30,588] WARN Unable to read additional data from >client sessionid 0xc00bee319ed0004, likely client has closed socket >

Re: How to add nodes to a Zookeeper 3.5.3-beta ensemble with reconfigEnabled=false

2017-06-23 Thread Shawn Heisey
On 6/22/2017 11:39 PM, Alexander Shraer wrote: > The described behavior is the intended one - in 3.5 configuration is > part of the synced state and is updated when the server syncs with the > leader. The only rolling upgrade I tested was to upgrade the software > version of the servers - this

Re: Yet another "two datacenter" discussion

2017-05-26 Thread Shawn Heisey
On 5/26/2017 9:48 AM, Jordan Zimmerman wrote: > In ZK 3.4.x if you have configuration differences amongst your instances you > are susceptible to a split brain. See this email thread, "Rolling Config > Change Considered Harmful": > >

Yet another "two datacenter" discussion

2017-05-26 Thread Shawn Heisey
I feel fairly certain that this thread willbe an annoyance. I don't know enough about zookeeper to answer the questions that are being asked, so I apologize about needing to relay questions about ZK fault tolerance in two datacenters. It seems that everyone wants to avoid the expense of a

Re: odd issue after enabling the firewall

2017-05-10 Thread Shawn Heisey
On 5/10/2017 11:40 AM, msouthwick wrote: > I have 2 zookeepers, 2 shards and 2 replica shards in my setup. Followup, noticed this after I had sent the previous reply: A ZK ensemble of two servers is LESS fault tolerant than a single server. If *either* server were to go down, you would lose

Re: odd issue after enabling the firewall

2017-05-10 Thread Shawn Heisey
On 5/10/2017 11:40 AM, msouthwick wrote: > I have 2 zookeepers, 2 shards and 2 replica shards in my setup. Everything > was working just fine until I enabled the firewall. I started by allowing > ports: 1099, 2181, 2888, 3888, 8983. Now I get the following in the > zookeeper log. > > 2017-05-10

Re: Memory requirement

2017-04-27 Thread Shawn Heisey
On 4/26/2017 12:25 PM, Daniel Chan wrote: > We have a Zookeeper (3.4.6) data store with > zk_approximate_data_size 1.88G > zk_znode_count4.43 millions > > 99% of the znodes has dataLen around 600 bytes. > > The Zookeeper instance is configured with "-Xms4G -Xmx4G" but it failed on >

Re: Query: 3.5.x version as alpha

2017-02-22 Thread Shawn Heisey
On 2/20/2017 2:19 AM, Deepti Sharma S wrote: > Can anyone confirm, is there any specific reason in the naming > convention in this release and to make it as "alpha". As per my > understanding Alpha means its beta release and not recommended to use > in production, however as per below mail it

Re: Zookeeper Ensemble Automation

2017-01-05 Thread Shawn Heisey
On 1/5/2017 11:19 AM, Washko, Daniel wrote: > Thanks for the reply Shawn. I would like to clarify something though. > Right now, the Dynamic Reconfiguration of Zookeeper works for > Zookeeper – that is adding/removing nodes automatically without having > to reconfigure each zookeeper node

Re: Zookeeper Ensemble Automation

2017-01-05 Thread Shawn Heisey
On 1/5/2017 10:28 AM, Washko, Daniel wrote: > Good day, I am soliciting advice for how to automate setting up and > maintaining a Zookeeper ensemble. In our environment we try to > automate everything. We are currently operating out of AWS using > Scalr. Our goal for Zookeeper would be to automate

Re: Is SSL supported in 3.4.9?

2016-12-07 Thread Shawn Heisey
On 12/7/2016 7:44 AM, Dan Langille wrote: > I'm getting mixed messages from the documentation, and I'm unable to get > Zookeeper > to talk on a secureClientPort. > > Is SSL supported in Zookeeper 3.4.9? > > At https://zookeeper.apache.org/doc/r3.4.9/zookeeperAdmin.html >

Re: Split a large ZooKeeper cluster into multiple separate clusters

2016-09-07 Thread Shawn Heisey
On 9/7/2016 5:15 PM, Eric Young wrote: > Also, adding/removing ZooKeeper nodes can be problematic to manage > over a large cluster (partly because 3.4.6 doesn't support live config > changes b3.5.0+ does). When an actual stable 3.5 release comes out, Solr can begin the process of upgrading its

Re: Split a large ZooKeeper cluster into multiple separate clusters

2016-09-07 Thread Shawn Heisey
On 9/7/2016 4:04 PM, Shawn Heisey wrote: > You *might* be able to just use the DELETE action on the Collections API > to delete collections instead of manually editing clusterstate, but I'm > not 100% positive about that. On second thought, DON'T TRY THIS. I wouldn't want to take t

Re: Split a large ZooKeeper cluster into multiple separate clusters

2016-09-07 Thread Shawn Heisey
On 9/7/2016 3:19 PM, Eric Young wrote: > I have a very large ZooKeeper cluster which manages config and replication > for multiple SolrCloud clusters. I want to split the monolithic ZooKeeper > cluster into smaller, more manageable clusters in a live migration (i.e. > minimal or no downtime).

Re: zookeeper deployment strategy for multi data centers

2016-06-03 Thread Shawn Heisey
On 6/3/2016 1:44 PM, Nomar Morado wrote: > Is there any settings to override the quorum rule? Would you know the > rationale behind it? Ideally, you will want to operate the application > even if at least one data center is up. I do not know if the quorum rule can be overridden, or whether your

Re: zookeeper deployment strategy for multi data centers

2016-06-03 Thread Shawn Heisey
On 6/2/2016 4:06 PM, J316 Services wrote: > We have two data centers and got two servers at each. At an event of a > data center failure, with the quorum majority rule - the other > surviving data center seems to be no use at all and we'll be out of luck. You are correct -- the scenario you've

Re: Zookeeper with SSL release date

2016-04-01 Thread Shawn Heisey
On 4/1/2016 10:18 AM, Alexander Shraer wrote: > Because using reconfig without ACLs any client can remove the servers (or > replace them with a different set of servers > or change their configuration parameters) and break the system. This is a potential worry even without reconfig -- a malicious

Re: Is it a accepted practice to share Zookeeper ensemble among Kafka, Storm and Solr

2016-03-30 Thread Shawn Heisey
On 3/30/2016 12:08 AM, Flavio Junqueira wrote: > Sharing is definitely ok, and I'd say that is common practice, but it really > depends on the size of the cluster you're talking about and your workload. > Storm can be quite demanding on ZK, and Kafka typically isn't except during > fail-over. I

Re: from zookeper embedded to standalone

2016-03-15 Thread Shawn Heisey
On 3/15/2016 10:13 AM, Flavio Junqueira wrote: > Have you asked on the Solr list? They are probably better equipped to answer > your question. > > On our end, if you are switching to a new set of servers, it is unclear how > you're going to safely migrate your data from the old ensemble to the

Re: Multi DC ( DC-1 and DC-2) zookeeper setup

2016-03-08 Thread Shawn Heisey
On 3/8/2016 3:40 PM, s influxdb wrote: > How does the client failover to the DC2 if DC1 is down ? Does the services > registered on DC1 for example with ephemeral nodes have to re-register with > DC2 ? Even though Flavio and Camille have both said this, I'm not sure whether the posters on this

Re: Java version and zookeper

2016-01-28 Thread Shawn Heisey
On 1/28/2016 8:54 AM, Muresanu A.V. (Andrei Valentin) wrote: what is the "supported" oracle jdk version that is supported by zookeeper 3.4.6 ? In the Administrator's guide: http://zookeeper.apache.org/doc/r3.4.6/zookeeperAdmin.html This text is included, under "Required Software":

3.4.6 download hard to find

2015-12-11 Thread Shawn Heisey
Someone on the #solr IRC channel asked about where they can get ZK version 3.4.6, since that's what's included in the most recent version of Solr (5.4.0 is a day or two from release). When I first looked, and couldn't even find it on the "archives" link that's present on each mirror, I initially

Prevent a znode from exceeding jute.maxbuffer

2015-10-01 Thread Shawn Heisey
I was going to open an issue in Jira for this, but I figured I should discuss it here before I do that, to make sure that's a reasonable course of action. I was thinking about a problem that we encounter with SolrCloud, where our overseer queue (stored in zookeeper) will greatly exceed the

Re: Prevent a znode from exceeding jute.maxbuffer

2015-10-01 Thread Shawn Heisey
On 10/1/2015 6:35 PM, Edward Ribeiro wrote: > I agree with you, and I think > https://issues.apache.org/jira/browse/ZOOKEEPER-2260 comes close to the > second approach you suggested. wdyt? Interesting! That could be helpful. I think it would require changes to the user application code, to

Set source address in zookeeper client?

2015-09-08 Thread Shawn Heisey
A user wrote to the solr-user mailing list asking how they could set the source address for zookeeper connections from their multi-homed SolrCloud install. Solr doesn't have any way to configure this, but I was wondering whether there's a system property honored by zookeeper that would bind the