ZooKeeper_branch33_solaris - Build # 695 - Still Failing
See https://builds.apache.org/job/ZooKeeper_branch33_solaris/695/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 105430 lines...] [junit] 2013-11-02 07:09:03,781 - INFO [main:ZooKeeperServer@154] - Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 6 datadir /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper_branch33_solaris/trunk/build/test/tmp/test4784044279050288340.junit.dir/version-2 snapdir /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper_branch33_solaris/trunk/build/test/tmp/test4784044279050288340.junit.dir/version-2 [junit] 2013-11-02 07:09:03,782 - INFO [main:NIOServerCnxn$Factory@143] - binding to port 0.0.0.0/0.0.0.0:11221 [junit] 2013-11-02 07:09:03,784 - INFO [main:FileSnap@82] - Reading snapshot /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper_branch33_solaris/trunk/build/test/tmp/test4784044279050288340.junit.dir/version-2/snapshot.0 [junit] 2013-11-02 07:09:03,787 - INFO [main:FileTxnSnapLog@256] - Snapshotting: b [junit] 2013-11-02 07:09:03,789 - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] 2013-11-02 07:09:03,790 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:43150 [junit] 2013-11-02 07:09:03,791 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@1237] - Processing stat command from /127.0.0.1:43150 [junit] 2013-11-02 07:09:03,792 - INFO [Thread-4:NIOServerCnxn$StatCommand@1153] - Stat command output [junit] 2013-11-02 07:09:03,793 - INFO [Thread-4:NIOServerCnxn@1435] - Closed socket connection for client /127.0.0.1:43150 (no session established for client) [junit] ensureOnly:[InMemoryDataTree, StandaloneServer_port] [junit] expect:InMemoryDataTree [junit] found:InMemoryDataTree org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree [junit] expect:StandaloneServer_port [junit] found:StandaloneServer_port org.apache.ZooKeeperService:name0=StandaloneServer_port-1 [junit] 2013-11-02 07:09:03,795 - INFO [main:ClientBase@408] - STOPPING server [junit] 2013-11-02 07:09:03,797 - INFO [ProcessThread:-1:PrepRequestProcessor@128] - PrepRequestProcessor exited loop! [junit] 2013-11-02 07:09:03,797 - INFO [SyncThread:0:SyncRequestProcessor@151] - SyncRequestProcessor exited! [junit] 2013-11-02 07:09:03,798 - INFO [main:FinalRequestProcessor@370] - shutdown of request processor complete [junit] 2013-11-02 07:09:03,799 - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] ensureOnly:[] [junit] 2013-11-02 07:09:03,801 - INFO [main:ClientBase@401] - STARTING server [junit] 2013-11-02 07:09:03,802 - INFO [main:ZooKeeperServer@154] - Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 6 datadir /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper_branch33_solaris/trunk/build/test/tmp/test4784044279050288340.junit.dir/version-2 snapdir /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper_branch33_solaris/trunk/build/test/tmp/test4784044279050288340.junit.dir/version-2 [junit] 2013-11-02 07:09:03,803 - INFO [main:NIOServerCnxn$Factory@143] - binding to port 0.0.0.0/0.0.0.0:11221 [junit] 2013-11-02 07:09:03,804 - INFO [main:FileSnap@82] - Reading snapshot /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper_branch33_solaris/trunk/build/test/tmp/test4784044279050288340.junit.dir/version-2/snapshot.b [junit] 2013-11-02 07:09:03,807 - INFO [main:FileTxnSnapLog@256] - Snapshotting: b [junit] 2013-11-02 07:09:03,808 - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] 2013-11-02 07:09:03,809 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:43152 [junit] 2013-11-02 07:09:03,810 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@1237] - Processing stat command from /127.0.0.1:43152 [junit] 2013-11-02 07:09:03,811 - INFO [Thread-5:NIOServerCnxn$StatCommand@1153] - Stat command output [junit] 2013-11-02 07:09:03,811 - INFO [Thread-5:NIOServerCnxn@1435] - Closed socket connection for client /127.0.0.1:43152 (no session established for client) [junit] ensureOnly:[InMemoryDataTree, StandaloneServer_port] [junit] expect:InMemoryDataTree [junit] found:InMemoryDataTree org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree [junit] expect:StandaloneServer_port [junit] found:StandaloneServer_port org.apache.ZooKeeperService:name0=StandaloneServer_port-1 [junit] 2013-11-02 07:09:03,814 - INFO [main:Clie
[jira] [Commented] (ZOOKEEPER-1807) Observers spam each other creating connections to the election addr
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811902#comment-13811902
]
Alexander Shraer commented on ZOOKEEPER-1807:
-
Thanks Raul. This seems like a bit of an overkill - you're eliminating observer
to observer responses. Would be better to understand what causes it to spin and
to send notifications in normal rate like for participants.
> Observers spam each other creating connections to the election addr
> ---
>
> Key: ZOOKEEPER-1807
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1807
> Project: ZooKeeper
> Issue Type: Bug
>Reporter: Raul Gutierrez Segales
>Assignee: Raul Gutierrez Segales
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-1807.patch
>
>
> Hey [~shralex],
> I noticed today that my Observers are spamming each other trying to open
> connections to the election port. I've got tons of these:
> {noformat}
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 9
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 10
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 6
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 12
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 14
> {noformat}
> and so and so on ad nauseam.
> Now, looking around I found this inside FastLeaderElection.java from when you
> committed ZOOKEEPER-107:
> {noformat}
> private void sendNotifications() {
> -for (QuorumServer server : self.getVotingView().values()) {
> -long sid = server.id;
> -
> +for (long sid : self.getAllKnownServerIds()) {
> +QuorumVerifier qv = self.getQuorumVerifier();
> {noformat}
> Is that really desired? I suspect that is what's causing Observers to try to
> connect to each other (as opposed as just connecting to participants). I'll
> give it a try now and let you know. (Also, we use observer ids that are > 0,
> and I saw some parts of the code that might not deal with that assumption -
> so it could be that too..).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
ZooKeeper_branch34 - Build # 782 - Failure
See https://builds.apache.org/job/ZooKeeper_branch34/782/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 156536 lines...] [junit] 2013-11-02 08:53:46,565 [myid:] - INFO [main:JMXEnv@133] - ensureOnly:[InMemoryDataTree, StandaloneServer_port] [junit] 2013-11-02 08:53:46,567 [myid:] - INFO [main:JMXEnv@105] - expect:InMemoryDataTree [junit] 2013-11-02 08:53:46,567 [myid:] - INFO [main:JMXEnv@108] - found:InMemoryDataTree org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree [junit] 2013-11-02 08:53:46,567 [myid:] - INFO [main:JMXEnv@105] - expect:StandaloneServer_port [junit] 2013-11-02 08:53:46,567 [myid:] - INFO [main:JMXEnv@108] - found:StandaloneServer_port org.apache.ZooKeeperService:name0=StandaloneServer_port-1 [junit] 2013-11-02 08:53:46,568 [myid:] - INFO [main:ClientBase@421] - STOPPING server [junit] 2013-11-02 08:53:46,568 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method [junit] 2013-11-02 08:53:46,568 [myid:] - INFO [main:ZooKeeperServer@441] - shutting down [junit] 2013-11-02 08:53:46,568 [myid:] - INFO [main:SessionTrackerImpl@225] - Shutting down [junit] 2013-11-02 08:53:46,568 [myid:] - INFO [main:PrepRequestProcessor@761] - Shutting down [junit] 2013-11-02 08:53:46,569 [myid:] - INFO [main:SyncRequestProcessor@209] - Shutting down [junit] 2013-11-02 08:53:46,569 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop! [junit] 2013-11-02 08:53:46,569 [myid:] - INFO [SyncThread:0:SyncRequestProcessor@187] - SyncRequestProcessor exited! [junit] 2013-11-02 08:53:46,569 [myid:] - INFO [main:FinalRequestProcessor@415] - shutdown of request processor complete [junit] 2013-11-02 08:53:46,570 [myid:] - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] 2013-11-02 08:53:46,570 [myid:] - INFO [main:JMXEnv@133] - ensureOnly:[] [junit] 2013-11-02 08:53:46,571 [myid:] - INFO [main:ClientBase@414] - STARTING server [junit] 2013-11-02 08:53:46,572 [myid:] - INFO [main:ZooKeeperServer@162] - Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 6 datadir /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34/branch-3.4/build/test/tmp/test1861913488959022062.junit.dir/version-2 snapdir /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34/branch-3.4/build/test/tmp/test1861913488959022062.junit.dir/version-2 [junit] 2013-11-02 08:53:46,572 [myid:] - INFO [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:11221 [junit] 2013-11-02 08:53:46,576 [myid:] - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] 2013-11-02 08:53:46,577 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:39010 [junit] 2013-11-02 08:53:46,577 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@817] - Processing stat command from /127.0.0.1:39010 [junit] 2013-11-02 08:53:46,577 [myid:] - INFO [Thread-5:NIOServerCnxn$StatCommand@653] - Stat command output [junit] 2013-11-02 08:53:46,578 [myid:] - INFO [Thread-5:NIOServerCnxn@997] - Closed socket connection for client /127.0.0.1:39010 (no session established for client) [junit] 2013-11-02 08:53:46,578 [myid:] - INFO [main:JMXEnv@133] - ensureOnly:[InMemoryDataTree, StandaloneServer_port] [junit] 2013-11-02 08:53:46,579 [myid:] - INFO [main:JMXEnv@105] - expect:InMemoryDataTree [junit] 2013-11-02 08:53:46,579 [myid:] - INFO [main:JMXEnv@108] - found:InMemoryDataTree org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree [junit] 2013-11-02 08:53:46,579 [myid:] - INFO [main:JMXEnv@105] - expect:StandaloneServer_port [junit] 2013-11-02 08:53:46,580 [myid:] - INFO [main:JMXEnv@108] - found:StandaloneServer_port org.apache.ZooKeeperService:name0=StandaloneServer_port-1 [junit] 2013-11-02 08:53:46,580 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@57] - FINISHED TEST METHOD testQuota [junit] 2013-11-02 08:53:46,580 [myid:] - INFO [main:ClientBase@451] - tearDown starting [junit] 2013-11-02 08:53:46,655 [myid:] - INFO [main:ZooKeeper@684] - Session: 0x1421804a2f4 closed [junit] 2013-11-02 08:53:46,655 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@509] - EventThread shut down [junit] 2013-11-02 08:53:46,655 [myid:] - INFO [main:ClientBase@421] - STOPPING server [junit] 2013-11-02 08:53:46,655 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method [junit] 2013-11-02 08:53:46,655 [myi
ZooKeeper_branch33 - Build # 1123 - Failure
See https://builds.apache.org/job/ZooKeeper_branch33/1123/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 103360 lines...] [junit] 2013-11-02 09:52:46,156 - INFO [main:ZooKeeperServer@154] - Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 6 datadir /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch33/branch-3.3/build/test/tmp/test8499908235553169282.junit.dir/version-2 snapdir /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch33/branch-3.3/build/test/tmp/test8499908235553169282.junit.dir/version-2 [junit] 2013-11-02 09:52:46,157 - INFO [main:NIOServerCnxn$Factory@143] - binding to port 0.0.0.0/0.0.0.0:11221 [junit] 2013-11-02 09:52:46,159 - INFO [main:FileSnap@82] - Reading snapshot /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch33/branch-3.3/build/test/tmp/test8499908235553169282.junit.dir/version-2/snapshot.0 [junit] 2013-11-02 09:52:46,162 - INFO [main:FileTxnSnapLog@256] - Snapshotting: b [junit] 2013-11-02 09:52:46,164 - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] 2013-11-02 09:52:46,165 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:36066 [junit] 2013-11-02 09:52:46,165 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@1237] - Processing stat command from /127.0.0.1:36066 [junit] 2013-11-02 09:52:46,166 - INFO [Thread-4:NIOServerCnxn$StatCommand@1153] - Stat command output [junit] 2013-11-02 09:52:46,166 - INFO [Thread-4:NIOServerCnxn@1435] - Closed socket connection for client /127.0.0.1:36066 (no session established for client) [junit] ensureOnly:[InMemoryDataTree, StandaloneServer_port] [junit] expect:InMemoryDataTree [junit] found:InMemoryDataTree org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree [junit] expect:StandaloneServer_port [junit] found:StandaloneServer_port org.apache.ZooKeeperService:name0=StandaloneServer_port-1 [junit] 2013-11-02 09:52:46,168 - INFO [main:ClientBase@408] - STOPPING server [junit] 2013-11-02 09:52:46,168 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn$Factory@278] - NIOServerCnxn factory exited run method [junit] 2013-11-02 09:52:46,169 - INFO [SyncThread:0:SyncRequestProcessor@151] - SyncRequestProcessor exited! [junit] 2013-11-02 09:52:46,169 - INFO [main:FinalRequestProcessor@370] - shutdown of request processor complete [junit] 2013-11-02 09:52:46,169 - INFO [ProcessThread:-1:PrepRequestProcessor@128] - PrepRequestProcessor exited loop! [junit] 2013-11-02 09:52:46,170 - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] ensureOnly:[] [junit] 2013-11-02 09:52:46,171 - INFO [main:ClientBase@401] - STARTING server [junit] 2013-11-02 09:52:46,172 - INFO [main:ZooKeeperServer@154] - Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 6 datadir /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch33/branch-3.3/build/test/tmp/test8499908235553169282.junit.dir/version-2 snapdir /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch33/branch-3.3/build/test/tmp/test8499908235553169282.junit.dir/version-2 [junit] 2013-11-02 09:52:46,172 - INFO [main:NIOServerCnxn$Factory@143] - binding to port 0.0.0.0/0.0.0.0:11221 [junit] 2013-11-02 09:52:46,173 - INFO [main:FileSnap@82] - Reading snapshot /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch33/branch-3.3/build/test/tmp/test8499908235553169282.junit.dir/version-2/snapshot.b [junit] 2013-11-02 09:52:46,176 - INFO [main:FileTxnSnapLog@256] - Snapshotting: b [junit] 2013-11-02 09:52:46,178 - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] 2013-11-02 09:52:46,178 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:36068 [junit] 2013-11-02 09:52:46,179 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@1237] - Processing stat command from /127.0.0.1:36068 [junit] 2013-11-02 09:52:46,179 - INFO [Thread-5:NIOServerCnxn$StatCommand@1153] - Stat command output [junit] 2013-11-02 09:52:46,179 - INFO [Thread-5:NIOServerCnxn@1435] - Closed socket connection for client /127.0.0.1:36068 (no session established for client) [junit] ensureOnly:[InMemoryDataTree, StandaloneServer_port] [junit] expect:InMemoryDataTree [junit] found:InMemoryDataTree org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree [junit] expect:StandaloneServer_port [junit] found:StandaloneServer_port org.apache.ZooKeeperService:name0=StandaloneServer_port-1 [junit] 2013-11-02 09:52:46,181 - INFO [main:ClientBase@431] - te
ZooKeeper-3.4-WinVS2008_java - Build # 341 - Still Failing
See https://builds.apache.org/job/ZooKeeper-3.4-WinVS2008_java/341/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 206108 lines...] [junit] 2013-11-02 10:25:20,210 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@224] - NIOServerCnxn factory exited run method [junit] 2013-11-02 10:25:20,125 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@509] - EventThread shut down [junit] 2013-11-02 10:25:20,210 [myid:] - INFO [main:ZooKeeperServer@441] - shutting down [junit] 2013-11-02 10:25:20,211 [myid:] - INFO [main:SessionTrackerImpl@225] - Shutting down [junit] 2013-11-02 10:25:20,211 [myid:] - INFO [main:PrepRequestProcessor@761] - Shutting down [junit] 2013-11-02 10:25:20,310 [myid:] - INFO [main:SyncRequestProcessor@209] - Shutting down [junit] 2013-11-02 10:25:20,310 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop! [junit] 2013-11-02 10:25:20,310 [myid:] - INFO [SyncThread:0:SyncRequestProcessor@187] - SyncRequestProcessor exited! [junit] 2013-11-02 10:25:20,310 [myid:] - INFO [main:FinalRequestProcessor@415] - shutdown of request processor complete [junit] 2013-11-02 10:25:20,312 [myid:] - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] 2013-11-02 10:25:21,000 [myid:] - INFO [SessionTracker:SessionTrackerImpl@162] - SessionTrackerImpl exited loop! [junit] 2013-11-02 10:25:21,001 [myid:] - INFO [SessionTracker:SessionTrackerImpl@162] - SessionTrackerImpl exited loop! [junit] 2013-11-02 10:25:21,309 [myid:] - INFO [main:JMXEnv@133] - ensureOnly:[] [junit] 2013-11-02 10:25:21,313 [myid:] - INFO [main:ZKTestCase$1@65] - FAILED testQuota [junit] junit.framework.AssertionFailedError: expected:<0> but was:<1> [junit] at junit.framework.Assert.fail(Assert.java:47) [junit] at junit.framework.Assert.failNotEquals(Assert.java:283) [junit] at junit.framework.Assert.assertEquals(Assert.java:64) [junit] at junit.framework.Assert.assertEquals(Assert.java:195) [junit] at junit.framework.Assert.assertEquals(Assert.java:201) [junit] at org.apache.zookeeper.test.JMXEnv.ensureOnly(JMXEnv.java:138) [junit] at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:417) [junit] at org.apache.zookeeper.test.ZooKeeperQuotaTest.testQuota(ZooKeeperQuotaTest.java:80) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) [junit] at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) [junit] at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) [junit] at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) [junit] at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52) [junit] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) [junit] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) [junit] at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) [junit] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) [junit] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) [junit] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) [junit] at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) [junit] at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) [junit] at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) [junit] at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) [junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:236) [junit] at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) [junit] 2013-11-02 10:25:21,317 [myid:] - INFO [main:ZKTestCase$1@55] - FINISHED testQuota [junit] Tests run:
ZooKeeper-trunk-solaris - Build # 719 - Still Failing
See https://builds.apache.org/job/ZooKeeper-trunk-solaris/719/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 222336 lines...] [junit] 2013-11-02 11:21:44,327 [myid:] - INFO [NIOServerCxnFactory.SelectorThread-1:NIOServerCnxnFactory$SelectorThread@420] - selector thread exitted run method [junit] 2013-11-02 11:21:44,328 [myid:] - INFO [main:ZooKeeperServer@428] - shutting down [junit] 2013-11-02 11:21:44,328 [myid:] - INFO [main:SessionTrackerImpl@183] - Shutting down [junit] 2013-11-02 11:21:44,329 [myid:] - INFO [main:PrepRequestProcessor@972] - Shutting down [junit] 2013-11-02 11:21:44,329 [myid:] - INFO [main:SyncRequestProcessor@190] - Shutting down [junit] 2013-11-02 11:21:44,329 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@156] - PrepRequestProcessor exited loop! [junit] 2013-11-02 11:21:44,329 [myid:] - INFO [SyncThread:0:SyncRequestProcessor@168] - SyncRequestProcessor exited! [junit] 2013-11-02 11:21:44,329 [myid:] - INFO [main:FinalRequestProcessor@442] - shutdown of request processor complete [junit] 2013-11-02 11:21:44,330 [myid:] - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] 2013-11-02 11:21:44,330 [myid:] - INFO [main:JMXEnv@133] - ensureOnly:[] [junit] 2013-11-02 11:21:44,331 [myid:] - INFO [main:ClientBase@414] - STARTING server [junit] 2013-11-02 11:21:44,332 [myid:] - INFO [main:ZooKeeperServer@149] - Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 6 datadir /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8989929064579792712.junit.dir/version-2 snapdir /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8989929064579792712.junit.dir/version-2 [junit] 2013-11-02 11:21:44,332 [myid:] - INFO [main:NIOServerCnxnFactory@670] - Configuring NIO connection handler with 10s sessionless connection timeout, 2 selector thread(s), 16 worker threads, and 64 kB direct buffers. [junit] 2013-11-02 11:21:44,333 [myid:] - INFO [main:NIOServerCnxnFactory@683] - binding to port 0.0.0.0/0.0.0.0:11221 [junit] 2013-11-02 11:21:44,333 [myid:] - INFO [main:FileSnap@83] - Reading snapshot /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8989929064579792712.junit.dir/version-2/snapshot.b [junit] 2013-11-02 11:21:44,336 [myid:] - INFO [main:FileTxnSnapLog@297] - Snapshotting: 0xb to /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8989929064579792712.junit.dir/version-2/snapshot.b [junit] 2013-11-02 11:21:44,337 [myid:] - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] 2013-11-02 11:21:44,338 [myid:] - INFO [NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory$AcceptThread@296] - Accepted socket connection from /127.0.0.1:41944 [junit] 2013-11-02 11:21:44,339 [myid:] - INFO [NIOWorkerThread-1:NIOServerCnxn@828] - Processing stat command from /127.0.0.1:41944 [junit] 2013-11-02 11:21:44,339 [myid:] - INFO [NIOWorkerThread-1:NIOServerCnxn$StatCommand@677] - Stat command output [junit] 2013-11-02 11:21:44,339 [myid:] - INFO [NIOWorkerThread-1:NIOServerCnxn@999] - Closed socket connection for client /127.0.0.1:41944 (no session established for client) [junit] 2013-11-02 11:21:44,339 [myid:] - INFO [main:JMXEnv@133] - ensureOnly:[InMemoryDataTree, StandaloneServer_port] [junit] 2013-11-02 11:21:44,341 [myid:] - INFO [main:JMXEnv@105] - expect:InMemoryDataTree [junit] 2013-11-02 11:21:44,341 [myid:] - INFO [main:JMXEnv@108] - found:InMemoryDataTree org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree [junit] 2013-11-02 11:21:44,341 [myid:] - INFO [main:JMXEnv@105] - expect:StandaloneServer_port [junit] 2013-11-02 11:21:44,341 [myid:] - INFO [main:JMXEnv@108] - found:StandaloneServer_port org.apache.ZooKeeperService:name0=StandaloneServer_port-1 [junit] 2013-11-02 11:21:44,342 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@57] - FINISHED TEST METHOD testQuota [junit] 2013-11-02 11:21:44,342 [myid:] - INFO [main:ClientBase@451] - tearDown starting [junit] 2013-11-02 11:21:44,432 [myid:] - INFO [main:ZooKeeper@777] - Session: 0x142188c11d7 closed [junit] 2013-11-02 11:21:44,433 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down [junit] 2013-11-02 11:21:44,433 [myid:] - INFO [main:ClientBase@421] - STOPPING server [junit] 2013-11-02 11:21:44,433 [myid:] - INFO [NIOServerCxnFactory.SelectorThread-0:NIOServerCnxnFactory$SelectorThread@420]
[jira] [Commented] (ZOOKEEPER-1807) Observers spam each other creating connections to the election addr
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811993#comment-13811993
]
Raul Gutierrez Segales commented on ZOOKEEPER-1807:
---
Well, if we really need observer to observer responses, for reconfig purposes I
presume, then should we be sending them to observers not in LOOKING state? See
the conditions that apply when responding to participants in the lines below my
patch.
But even still with that being correct it might be too much overhead for large
Observers deployments. Should this be optional?
> Observers spam each other creating connections to the election addr
> ---
>
> Key: ZOOKEEPER-1807
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1807
> Project: ZooKeeper
> Issue Type: Bug
>Reporter: Raul Gutierrez Segales
>Assignee: Raul Gutierrez Segales
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-1807.patch
>
>
> Hey [~shralex],
> I noticed today that my Observers are spamming each other trying to open
> connections to the election port. I've got tons of these:
> {noformat}
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 9
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 10
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 6
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 12
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 14
> {noformat}
> and so and so on ad nauseam.
> Now, looking around I found this inside FastLeaderElection.java from when you
> committed ZOOKEEPER-107:
> {noformat}
> private void sendNotifications() {
> -for (QuorumServer server : self.getVotingView().values()) {
> -long sid = server.id;
> -
> +for (long sid : self.getAllKnownServerIds()) {
> +QuorumVerifier qv = self.getQuorumVerifier();
> {noformat}
> Is that really desired? I suspect that is what's causing Observers to try to
> connect to each other (as opposed as just connecting to participants). I'll
> give it a try now and let you know. (Also, we use observer ids that are > 0,
> and I saw some parts of the code that might not deal with that assumption -
> so it could be that too..).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (ZOOKEEPER-1430) add maven deploy support to the build
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13812029#comment-13812029
]
Patrick Hunt commented on ZOOKEEPER-1430:
-
[~gkesavan] I applied the patch to 3.4 branch (minor conflict, see attachment)
but it fails when running mvn-deploy as follows: (I did not try running with
trunk)
{noformat}
[artifact:mvn] [INFO]
[artifact:mvn] [ERROR] BUILD FAILURE
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] Invalid task '': you must specify a valid lifecycle
phase, or a goal in the format plugin:goal or
pluginGroupId:pluginArtifactId:pluginVersion:goal
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] For more information, run Maven with the -e switch
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] Total time: 50 seconds
[artifact:mvn] [INFO] Finished at: Sat Nov 02 08:30:38 PDT 2013
[artifact:mvn] [INFO] Final Memory: 32M/351M
[artifact:mvn] [INFO]
[artifact:mvn] Java Result: 1
[artifact:mvn] [INFO] Scanning for projects...
[artifact:mvn] [INFO]
[artifact:mvn] [ERROR] BUILD FAILURE
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] Invalid task '': you must specify a valid lifecycle
phase, or a goal in the format plugin:goal or
pluginGroupId:pluginArtifactId:pluginVersion:goal
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] For more information, run Maven with the -e switch
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] Total time: < 1 second
[artifact:mvn] [INFO] Finished at: Sat Nov 02 08:30:39 PDT 2013
[artifact:mvn] [INFO] Final Memory: 32M/352M
[artifact:mvn] [INFO]
[artifact:mvn] Java Result: 1
[artifact:mvn] [INFO] Scanning for projects...
[artifact:mvn] [INFO]
[artifact:mvn] [ERROR] BUILD FAILURE
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] Invalid task '': you must specify a valid lifecycle
phase, or a goal in the format plugin:goal or
pluginGroupId:pluginArtifactId:pluginVersion:goal
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] For more information, run Maven with the -e switch
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] Total time: < 1 second
[artifact:mvn] [INFO] Finished at: Sat Nov 02 08:30:40 PDT 2013
[artifact:mvn] [INFO] Final Memory: 33M/352M
[artifact:mvn] [INFO]
[artifact:mvn] Java Result: 1
[artifact:mvn] [INFO] Scanning for projects...
[artifact:mvn] [INFO]
[artifact:mvn] [ERROR] BUILD FAILURE
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] Invalid task '': you must specify a valid lifecycle
phase, or a goal in the format plugin:goal or
pluginGroupId:pluginArtifactId:pluginVersion:goal
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] For more information, run Maven with the -e switch
[artifact:mvn] [INFO]
[artifact:mvn] [INFO] Total time: < 1 second
[artifact:mvn] [INFO] Finished at: Sat Nov 02 08:30:41 PDT 2013
[artifact:mvn] [INFO] Final Memory: 33M/342M
[artifact:mvn] [INFO]
[artifact:mvn] Java Result: 1
{noformat}
> add maven deploy support to the build
> -
>
> Key: ZOOKEEPER-1430
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1430
> Project: ZooKeeper
> Issue Type: Task
> Components: build
>Affects Versions: 3.4.4, 3.5.0
>Reporter: Patrick Hunt
>Assignee: Giridharan Kesavan
>Priority: Blocker
> Fix For: 3.4.6, 3.5.0
>
> Attachments: ZOOKEEPER-1430-3.4.patch, ZOOKEEPER-1430-V1.PATCH,
> ZOOKEEPER-1430-V2.PATCH, ZOOKEEPER-1430-V3.patch, ZOOKEEPER-1430.patch,
>
[jira] [Commented] (ZOOKEEPER-1430) add maven deploy support to the build
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13812030#comment-13812030 ] Patrick Hunt commented on ZOOKEEPER-1430: - also note that our "version" field in build.xml is not setup properly. it's currently "3.4.5" or "3.5.0" for branch34/trunk respectively. IMO We should update the version number in the build.xml to "3.4.6-SNAPSHOT" and "3.5.0-SNAPSHOT" respectively as part of this patch. During the release process we need to be sure to use "3.4.6"/"3.5.0" when building the release, then update branch34/trunk to "3.4.7-SNAPSHOT"/"3.6.0-SNAPSHOT" thereafter. > add maven deploy support to the build > - > > Key: ZOOKEEPER-1430 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1430 > Project: ZooKeeper > Issue Type: Task > Components: build >Affects Versions: 3.4.4, 3.5.0 >Reporter: Patrick Hunt >Assignee: Giridharan Kesavan >Priority: Blocker > Fix For: 3.4.6, 3.5.0 > > Attachments: ZOOKEEPER-1430-3.4.patch, ZOOKEEPER-1430-V1.PATCH, > ZOOKEEPER-1430-V2.PATCH, ZOOKEEPER-1430-V3.patch, > ZOOKEEPER-1430-br34-V3.patch, ZOOKEEPER-1430.patch, ZOOKEEPER-1430.patch > > > Infra is phasing out the current mechanism we use to deploy maven artifacts. > We need to move to repository.apache.org (nexus). > In particular we need to ensure the following artifacts get published: > * zookeeper-3.x.y.jar > * zookeeper-3.x.y-sources.jar > * zookeeper-3.x.y-tests.jar > * zookeeper-3.x.y-javadoc.jar > In 3.4.4/3.4.5 we missed the tests jar which caused headaches for downstream > users, such as Hadoop. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (ZOOKEEPER-1430) add maven deploy support to the build
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Hunt updated ZOOKEEPER-1430: Attachment: ZOOKEEPER-1430-br34-V3.patch > add maven deploy support to the build > - > > Key: ZOOKEEPER-1430 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1430 > Project: ZooKeeper > Issue Type: Task > Components: build >Affects Versions: 3.4.4, 3.5.0 >Reporter: Patrick Hunt >Assignee: Giridharan Kesavan >Priority: Blocker > Fix For: 3.4.6, 3.5.0 > > Attachments: ZOOKEEPER-1430-3.4.patch, ZOOKEEPER-1430-V1.PATCH, > ZOOKEEPER-1430-V2.PATCH, ZOOKEEPER-1430-V3.patch, > ZOOKEEPER-1430-br34-V3.patch, ZOOKEEPER-1430.patch, ZOOKEEPER-1430.patch > > > Infra is phasing out the current mechanism we use to deploy maven artifacts. > We need to move to repository.apache.org (nexus). > In particular we need to ensure the following artifacts get published: > * zookeeper-3.x.y.jar > * zookeeper-3.x.y-sources.jar > * zookeeper-3.x.y-tests.jar > * zookeeper-3.x.y-javadoc.jar > In 3.4.4/3.4.5 we missed the tests jar which caused headaches for downstream > users, such as Hadoop. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (ZOOKEEPER-1807) Observers spam each other creating connections to the election addr
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13812073#comment-13812073
]
Alexander Shraer commented on ZOOKEEPER-1807:
-
what if we remove the "if(!self.getVotingView().containsKey(response.sid)){"
and always run the else code ?
> Observers spam each other creating connections to the election addr
> ---
>
> Key: ZOOKEEPER-1807
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1807
> Project: ZooKeeper
> Issue Type: Bug
>Reporter: Raul Gutierrez Segales
>Assignee: Raul Gutierrez Segales
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-1807.patch
>
>
> Hey [~shralex],
> I noticed today that my Observers are spamming each other trying to open
> connections to the election port. I've got tons of these:
> {noformat}
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 9
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 10
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 6
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 12
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 14
> {noformat}
> and so and so on ad nauseam.
> Now, looking around I found this inside FastLeaderElection.java from when you
> committed ZOOKEEPER-107:
> {noformat}
> private void sendNotifications() {
> -for (QuorumServer server : self.getVotingView().values()) {
> -long sid = server.id;
> -
> +for (long sid : self.getAllKnownServerIds()) {
> +QuorumVerifier qv = self.getQuorumVerifier();
> {noformat}
> Is that really desired? I suspect that is what's causing Observers to try to
> connect to each other (as opposed as just connecting to participants). I'll
> give it a try now and let you know. (Also, we use observer ids that are > 0,
> and I saw some parts of the code that might not deal with that assumption -
> so it could be that too..).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (ZOOKEEPER-1807) Observers spam each other creating connections to the election addr
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13812074#comment-13812074
]
Alexander Shraer commented on ZOOKEEPER-1807:
-
regarding overhead - if I understand the "else" code correctly, it will only
send a message if one of them is LOOKING, so I'm not sure that the overhead is
excessive.
> Observers spam each other creating connections to the election addr
> ---
>
> Key: ZOOKEEPER-1807
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1807
> Project: ZooKeeper
> Issue Type: Bug
>Reporter: Raul Gutierrez Segales
>Assignee: Raul Gutierrez Segales
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-1807.patch
>
>
> Hey [~shralex],
> I noticed today that my Observers are spamming each other trying to open
> connections to the election port. I've got tons of these:
> {noformat}
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 9
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 10
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 6
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 12
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 14
> {noformat}
> and so and so on ad nauseam.
> Now, looking around I found this inside FastLeaderElection.java from when you
> committed ZOOKEEPER-107:
> {noformat}
> private void sendNotifications() {
> -for (QuorumServer server : self.getVotingView().values()) {
> -long sid = server.id;
> -
> +for (long sid : self.getAllKnownServerIds()) {
> +QuorumVerifier qv = self.getQuorumVerifier();
> {noformat}
> Is that really desired? I suspect that is what's causing Observers to try to
> connect to each other (as opposed as just connecting to participants). I'll
> give it a try now and let you know. (Also, we use observer ids that are > 0,
> and I saw some parts of the code that might not deal with that assumption -
> so it could be that too..).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (ZOOKEEPER-1790) Deal with special ObserverId in QuorumCnxManager.receiveConnection
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13812078#comment-13812078 ] Alexander Shraer commented on ZOOKEEPER-1790: - The problem described in this JIRA turned out not be a problem as I mention in the first comment. [~thawan], I'm actually not sure if -1 would work or not - there aren't any tests for this in 3.4 or 3.5. I can't think of anything specific that changed that would prevent it from working, but I may be wrong. As you pointed out you wouldn't be able to dynamically change the parameters of such observers. > Deal with special ObserverId in QuorumCnxManager.receiveConnection > -- > > Key: ZOOKEEPER-1790 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1790 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.4.6, 3.5.0 >Reporter: Alexander Shraer >Assignee: Alexander Shraer > Fix For: 3.4.6, 3.5.0 > > > QuorumCnxManager.receiveConnection assumes that a negative sid means that > this is a 3.5.0 server, which has a different communication protocol. This > doesn't account for the fact that ObserverId = -1 is a special id that may be > used by observers and is also negative. > This requires a fix to trunk and a separate fix to 3.4 branch, where this > function is different (see ZOOKEEPER-1633) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (ZOOKEEPER-1807) Observers spam each other creating connections to the election addr
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13812100#comment-13812100
]
Raul Gutierrez Segales commented on ZOOKEEPER-1807:
---
Yeah - I think you are right. In this ZOOKEEPER-107 world in which observers
can be promoted, etc the initial if() doesn't make sense anymore. I'll submit a
new patch so we can think about it a bit more.
With regards of the overhead and making all of this optional, well if you have
> 100 observers restarted at once you'll have a large of notifications traffic.
But I guess within the limits of tolerable.
> Observers spam each other creating connections to the election addr
> ---
>
> Key: ZOOKEEPER-1807
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1807
> Project: ZooKeeper
> Issue Type: Bug
>Reporter: Raul Gutierrez Segales
>Assignee: Raul Gutierrez Segales
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-1807.patch
>
>
> Hey [~shralex],
> I noticed today that my Observers are spamming each other trying to open
> connections to the election port. I've got tons of these:
> {noformat}
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 9
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 10
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 6
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 12
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 14
> {noformat}
> and so and so on ad nauseam.
> Now, looking around I found this inside FastLeaderElection.java from when you
> committed ZOOKEEPER-107:
> {noformat}
> private void sendNotifications() {
> -for (QuorumServer server : self.getVotingView().values()) {
> -long sid = server.id;
> -
> +for (long sid : self.getAllKnownServerIds()) {
> +QuorumVerifier qv = self.getQuorumVerifier();
> {noformat}
> Is that really desired? I suspect that is what's causing Observers to try to
> connect to each other (as opposed as just connecting to participants). I'll
> give it a try now and let you know. (Also, we use observer ids that are > 0,
> and I saw some parts of the code that might not deal with that assumption -
> so it could be that too..).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (ZOOKEEPER-1807) Observers spam each other creating connections to the election addr
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raul Gutierrez Segales updated ZOOKEEPER-1807:
--
Attachment: (was: ZOOKEEPER-1807.patch)
> Observers spam each other creating connections to the election addr
> ---
>
> Key: ZOOKEEPER-1807
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1807
> Project: ZooKeeper
> Issue Type: Bug
>Reporter: Raul Gutierrez Segales
>Assignee: Raul Gutierrez Segales
> Fix For: 3.5.0
>
>
> Hey [~shralex],
> I noticed today that my Observers are spamming each other trying to open
> connections to the election port. I've got tons of these:
> {noformat}
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 9
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 10
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 6
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 12
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 14
> {noformat}
> and so and so on ad nauseam.
> Now, looking around I found this inside FastLeaderElection.java from when you
> committed ZOOKEEPER-107:
> {noformat}
> private void sendNotifications() {
> -for (QuorumServer server : self.getVotingView().values()) {
> -long sid = server.id;
> -
> +for (long sid : self.getAllKnownServerIds()) {
> +QuorumVerifier qv = self.getQuorumVerifier();
> {noformat}
> Is that really desired? I suspect that is what's causing Observers to try to
> connect to each other (as opposed as just connecting to participants). I'll
> give it a try now and let you know. (Also, we use observer ids that are > 0,
> and I saw some parts of the code that might not deal with that assumption -
> so it could be that too..).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (ZOOKEEPER-1807) Observers spam each other creating connections to the election addr
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raul Gutierrez Segales updated ZOOKEEPER-1807:
--
Attachment: ZOOKEEPER-1807.patch
As discussed with [~shralex], we now need to apply the same response logic for
voting and non-voting members.
[~fpj], [~thawan] - thoughts?
> Observers spam each other creating connections to the election addr
> ---
>
> Key: ZOOKEEPER-1807
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1807
> Project: ZooKeeper
> Issue Type: Bug
>Reporter: Raul Gutierrez Segales
>Assignee: Raul Gutierrez Segales
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-1807.patch
>
>
> Hey [~shralex],
> I noticed today that my Observers are spamming each other trying to open
> connections to the election port. I've got tons of these:
> {noformat}
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 9
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 10
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 6
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 12
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 14
> {noformat}
> and so and so on ad nauseam.
> Now, looking around I found this inside FastLeaderElection.java from when you
> committed ZOOKEEPER-107:
> {noformat}
> private void sendNotifications() {
> -for (QuorumServer server : self.getVotingView().values()) {
> -long sid = server.id;
> -
> +for (long sid : self.getAllKnownServerIds()) {
> +QuorumVerifier qv = self.getQuorumVerifier();
> {noformat}
> Is that really desired? I suspect that is what's causing Observers to try to
> connect to each other (as opposed as just connecting to participants). I'll
> give it a try now and let you know. (Also, we use observer ids that are > 0,
> and I saw some parts of the code that might not deal with that assumption -
> so it could be that too..).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
Failed: ZOOKEEPER-9478 PreCommit Build #1734
Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-9478 Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1734/ ### ## LAST 60 LINES OF THE CONSOLE ### Started by remote host 127.0.0.1 Building remotely on hadoop9 in workspace /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build Reverting /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build/trunk to depth infinity with ignoreExternals: false Updating http://svn.apache.org/repos/asf/zookeeper/trunk at revision '2013-11-02T18:51:12.814 +' At revision 1538251 no change for http://svn.apache.org/repos/asf/zookeeper/trunk since the previous build No emails were triggered. [PreCommit-ZOOKEEPER-Build] $ /bin/bash /tmp/hudson6297660280696756512.sh /home/jenkins/tools/java/latest/bin/java Buildfile: build.xml check-for-findbugs: findbugs.check: forrest.check: hudson-test-patch: [exec] [exec] [exec] == [exec] == [exec] Testing patch for ZOOKEEPER-9478. [exec] == [exec] == [exec] [exec] [exec] At revision 1538252. [exec] ZOOKEEPER-9478 is not "Patch Available". Exiting. [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] BUILD SUCCESSFUL Total time: 2 seconds Archiving artifacts ERROR: No artifacts found that match the file pattern "trunk/build/test/findbugs/newPatchFindbugsWarnings.html,trunk/patchprocess/*.txt,trunk/patchprocess/*Warnings.xml,trunk/build/test/test-cppunit/*.txt,trunk/build/tmp/zk.log". Configuration error? ERROR: ?trunk/build/test/findbugs/newPatchFindbugsWarnings.html? doesn?t match anything: ?trunk? exists but not ?trunk/build/test/findbugs/newPatchFindbugsWarnings.html? Build step 'Archive the artifacts' changed build result to FAILURE Recording test results Description set: ZOOKEEPER-9478 Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
[jira] [Commented] (ZOOKEEPER-1807) Observers spam each other creating connections to the election addr
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13812123#comment-13812123
]
Hadoop QA commented on ZOOKEEPER-1807:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12611781/ZOOKEEPER-1807.patch
against trunk revision 1535491.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1735//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1735//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1735//console
This message is automatically generated.
> Observers spam each other creating connections to the election addr
> ---
>
> Key: ZOOKEEPER-1807
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1807
> Project: ZooKeeper
> Issue Type: Bug
>Reporter: Raul Gutierrez Segales
>Assignee: Raul Gutierrez Segales
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-1807.patch
>
>
> Hey [~shralex],
> I noticed today that my Observers are spamming each other trying to open
> connections to the election port. I've got tons of these:
> {noformat}
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 9
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 10
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 6
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 12
> 2013-11-01 22:19:45,819 - DEBUG [WorkerSender[myid=13]] - There is a
> connection already for server 14
> {noformat}
> and so and so on ad nauseam.
> Now, looking around I found this inside FastLeaderElection.java from when you
> committed ZOOKEEPER-107:
> {noformat}
> private void sendNotifications() {
> -for (QuorumServer server : self.getVotingView().values()) {
> -long sid = server.id;
> -
> +for (long sid : self.getAllKnownServerIds()) {
> +QuorumVerifier qv = self.getQuorumVerifier();
> {noformat}
> Is that really desired? I suspect that is what's causing Observers to try to
> connect to each other (as opposed as just connecting to participants). I'll
> give it a try now and let you know. (Also, we use observer ids that are > 0,
> and I saw some parts of the code that might not deal with that assumption -
> so it could be that too..).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
Failed: ZOOKEEPER-1807 PreCommit Build #1735
Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-1807 Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1735/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 235264 lines...] [exec] [exec] -1 overall. Here are the results of testing the latest attachment [exec] http://issues.apache.org/jira/secure/attachment/12611781/ZOOKEEPER-1807.patch [exec] against trunk revision 1535491. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] -1 core tests. The patch failed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1735//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1735//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1735//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] d00201ee5dc84bd2891c716c3e2b923e5186c4c0 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] BUILD FAILED /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build/trunk/build.xml:1623: exec returned: 2 Total time: 28 minutes 56 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results Description set: ZOOKEEPER-1807 Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## 1 tests failed. REGRESSION: org.apache.zookeeper.server.ZxidRolloverTest.testMultipleRollover Error Message: KeeperErrorCode = Session expired for /foofoofoo-connected Stack Trace: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /foofoofoo-connected at org.apache.zookeeper.KeeperException.create(KeeperException.java:131) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1270) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1298) at org.apache.zookeeper.server.ZxidRolloverTest.checkClientConnected(ZxidRolloverTest.java:103) at org.apache.zookeeper.server.ZxidRolloverTest.checkClientsConnected(ZxidRolloverTest.java:88) at org.apache.zookeeper.server.ZxidRolloverTest.startAll(ZxidRolloverTest.java:152) at org.apache.zookeeper.server.ZxidRolloverTest.testMultipleRollover(ZxidRolloverTest.java:425)
