ZooKeeper-trunk-solaris - Build # 660 - Still Failing
See https://builds.apache.org/job/ZooKeeper-trunk-solaris/660/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 197967 lines...] [junit] 2013-09-04 09:07:38,202 [myid:] - INFO [main:PrepRequestProcessor@929] - Shutting down [junit] 2013-09-04 09:07:38,203 [myid:] - INFO [main:SyncRequestProcessor@175] - Shutting down [junit] 2013-09-04 09:07:38,203 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@156] - PrepRequestProcessor exited loop! [junit] 2013-09-04 09:07:38,203 [myid:] - INFO [SyncThread:0:SyncRequestProcessor@155] - SyncRequestProcessor exited! [junit] 2013-09-04 09:07:38,203 [myid:] - INFO [main:FinalRequestProcessor@427] - shutdown of request processor complete [junit] 2013-09-04 09:07:38,204 [myid:] - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] 2013-09-04 09:07:38,204 [myid:] - INFO [main:JMXEnv@133] - ensureOnly:[] [junit] 2013-09-04 09:07:38,205 [myid:] - INFO [main:ClientBase@414] - STARTING server [junit] 2013-09-04 09:07:38,206 [myid:] - INFO [main:ZooKeeperServer@149] - Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 6 datadir /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8183074854923355150.junit.dir/version-2 snapdir /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8183074854923355150.junit.dir/version-2 [junit] 2013-09-04 09:07:38,206 [myid:] - INFO [main:NIOServerCnxnFactory@670] - Configuring NIO connection handler with 10s sessionless connection timeout, 2 selector thread(s), 16 worker threads, and 64 kB direct buffers. [junit] 2013-09-04 09:07:38,207 [myid:] - INFO [main:NIOServerCnxnFactory@683] - binding to port 0.0.0.0/0.0.0.0:11221 [junit] 2013-09-04 09:07:38,208 [myid:] - INFO [main:FileSnap@83] - Reading snapshot /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8183074854923355150.junit.dir/version-2/snapshot.b [junit] 2013-09-04 09:07:38,211 [myid:] - INFO [main:FileTxnSnapLog@297] - Snapshotting: 0xb to /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8183074854923355150.junit.dir/version-2/snapshot.b [junit] 2013-09-04 09:07:38,213 [myid:] - INFO [main:FourLetterWordMain@43] - connecting to 127.0.0.1 11221 [junit] 2013-09-04 09:07:38,213 [myid:] - INFO [NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory$AcceptThread@296] - Accepted socket connection from /127.0.0.1:64360 [junit] 2013-09-04 09:07:38,214 [myid:] - INFO [NIOWorkerThread-1:NIOServerCnxn@829] - Processing stat command from /127.0.0.1:64360 [junit] 2013-09-04 09:07:38,215 [myid:] - INFO [NIOWorkerThread-1:NIOServerCnxn$StatCommand@678] - Stat command output [junit] 2013-09-04 09:07:38,215 [myid:] - INFO [NIOWorkerThread-1:NIOServerCnxn@1000] - Closed socket connection for client /127.0.0.1:64360 (no session established for client) [junit] 2013-09-04 09:07:38,215 [myid:] - INFO [main:JMXEnv@133] - ensureOnly:[InMemoryDataTree, StandaloneServer_port] [junit] 2013-09-04 09:07:38,217 [myid:] - INFO [main:JMXEnv@105] - expect:InMemoryDataTree [junit] 2013-09-04 09:07:38,217 [myid:] - INFO [main:JMXEnv@108] - found:InMemoryDataTree org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree [junit] 2013-09-04 09:07:38,217 [myid:] - INFO [main:JMXEnv@105] - expect:StandaloneServer_port [junit] 2013-09-04 09:07:38,217 [myid:] - INFO [main:JMXEnv@108] - found:StandaloneServer_port org.apache.ZooKeeperService:name0=StandaloneServer_port-1 [junit] 2013-09-04 09:07:38,217 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@57] - FINISHED TEST METHOD testQuota [junit] 2013-09-04 09:07:38,218 [myid:] - INFO [main:ClientBase@451] - tearDown starting [junit] 2013-09-04 09:07:38,286 [myid:] - INFO [main:ZooKeeper@777] - Session: 0x140e83a1119 closed [junit] 2013-09-04 09:07:38,286 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down [junit] 2013-09-04 09:07:38,287 [myid:] - INFO [main:ClientBase@421] - STOPPING server [junit] 2013-09-04 09:07:38,288 [myid:] - INFO [NIOServerCxnFactory.SelectorThread-1:NIOServerCnxnFactory$SelectorThread@420] - selector thread exitted run method [junit] 2013-09-04 09:07:38,288 [myid:] - INFO [NIOServerCxnFactory.SelectorThread-0:NIOServerCnxnFactory$SelectorThread@420] - selector thread exitted run method [junit] 2013-09-04 09:07:38,288 [myid:] - INFO [NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory$AcceptThread@219] - acce
[jira] [Commented] (ZOOKEEPER-1462) Read-only server does not initialize database properly
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757614#comment-13757614 ] Germán Blanco commented on ZOOKEEPER-1462: -- After looking a bit more, it seems that the Observer doesn't write the last transactions in the transaction log and that is the reason for not being up-to-date when it restarts. If the Observer is started using the transaction log of a Follower, then it works. Anybody can tell why is it that the Observer doesn't write those transactions? > Read-only server does not initialize database properly > -- > > Key: ZOOKEEPER-1462 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1462 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.4.3 >Reporter: Thawan Kooburat >Assignee: Thawan Kooburat >Priority: Critical > Fix For: 3.4.6 > > Attachments: ZOOKEEPER-1462.patch > > > Brief Description: > When a participant or observer get partitioned and restart as Read-only > server. ZkDb doesn't get reinitialized. This causes the RO server to drop any > incoming request with zxid > 0 > Error message: > Refusing session request for client /xx.xx.xx.xx:39875 > as it has seen zxid 0x2e00405fd9 our last zxid is 0x0 client must try another > server > Steps to reproduce: > Start an RO-enabled observer connecting to an ensemble. Kill the ensemble and > wait until the observer restart in RO mode. Zxid of this observer should be 0. > Description: > Before a server transition into LOOKING state, its database get closed as > part of shutdown sequence. The database of leader, follower and observer get > initialized as a side effect of participating in leader election protocol. > (eg. observer will call registerWithLeader() and call getLastLoggedZxid() > which initialize the db if not already). > However, RO server does not participate in this protocol so its DB doesn't > get initialized properly > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1670) zookeeper should set a default value for SERVER_JVMFLAGS and CLIENT_JVMFLAGS so that memory usage is controlled
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757660#comment-13757660
]
Hudson commented on ZOOKEEPER-1670:
---
SUCCESS: Integrated in ZooKeeper-trunk #2044 (See
[https://builds.apache.org/job/ZooKeeper-trunk/2044/])
ZOOKEEPER-1670. zookeeper should set a default value for SERVER_JVMFLAGS and
CLIENT_JVMFLAGS so that memory usage is controlled (Arpit Gupta via fpj) (fpj:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519655)
* /zookeeper/trunk/CHANGES.txt
* /zookeeper/trunk/bin/zkEnv.sh
> zookeeper should set a default value for SERVER_JVMFLAGS and CLIENT_JVMFLAGS
> so that memory usage is controlled
> ---
>
> Key: ZOOKEEPER-1670
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1670
> Project: ZooKeeper
> Issue Type: Bug
>Affects Versions: 3.4.5
>Reporter: Arpit Gupta
>Assignee: Flavio Junqueira
> Fix For: 3.5.0, 3.4.6
>
> Attachments: ZOOKEEPER-1670.patch, ZOOKEEPER-1670.patch,
> ZOOKEEPER-1670.patch, ZOOKEEPER-1670.patch, ZOOKEEPER-1670.patch
>
>
> We noticed this with jdk 1.6 where if no heap size is set the process takes
> up to 1/4 of mem available on the machine.
> More info
> http://stackoverflow.com/questions/3428251/is-there-a-default-xmx-setting-for-java-1-5
> You can run the following command to see what are the defaults for your
> machine
> {code}
> java -XX:+PrintFlagsFinal -version 2>&1 | grep -i -E
> 'heapsize|permsize|version'
> {code}
> And we noticed on two different class of machines that this was 1/4th of
> total memory on the machine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1462) Read-only server does not initialize database properly
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757697#comment-13757697 ] Germán Blanco commented on ZOOKEEPER-1462: -- It seems to me that the Observer is not sending anything through SyncRequestProcessor ... is that so? It seems that if I add "syncProcessor.processRequest(request);" as the first line of ObserverZooKeeperServer.commitRequest, the problem is solved. I will start preparing a patch with a test case, but given the amount of guessing that I am doing for this I would really appreciate some feedback :-) > Read-only server does not initialize database properly > -- > > Key: ZOOKEEPER-1462 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1462 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.4.3 >Reporter: Thawan Kooburat >Assignee: Thawan Kooburat >Priority: Critical > Fix For: 3.4.6 > > Attachments: ZOOKEEPER-1462.patch > > > Brief Description: > When a participant or observer get partitioned and restart as Read-only > server. ZkDb doesn't get reinitialized. This causes the RO server to drop any > incoming request with zxid > 0 > Error message: > Refusing session request for client /xx.xx.xx.xx:39875 > as it has seen zxid 0x2e00405fd9 our last zxid is 0x0 client must try another > server > Steps to reproduce: > Start an RO-enabled observer connecting to an ensemble. Kill the ensemble and > wait until the observer restart in RO mode. Zxid of this observer should be 0. > Description: > Before a server transition into LOOKING state, its database get closed as > part of shutdown sequence. The database of leader, follower and observer get > initialized as a side effect of participating in leader election protocol. > (eg. observer will call registerWithLeader() and call getLastLoggedZxid() > which initialize the db if not already). > However, RO server does not participate in this protocol so its DB doesn't > get initialized properly > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1462) Read-only server does not initialize database properly
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757911#comment-13757911 ] Flavio Junqueira commented on ZOOKEEPER-1462: - bq. It seems to me that the Observer is not sending anything through SyncRequestProcessor ... is that so? This is related to ZOOKEEPER-1552. > Read-only server does not initialize database properly > -- > > Key: ZOOKEEPER-1462 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1462 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.4.3 >Reporter: Thawan Kooburat >Assignee: Thawan Kooburat >Priority: Critical > Fix For: 3.4.6 > > Attachments: ZOOKEEPER-1462.patch > > > Brief Description: > When a participant or observer get partitioned and restart as Read-only > server. ZkDb doesn't get reinitialized. This causes the RO server to drop any > incoming request with zxid > 0 > Error message: > Refusing session request for client /xx.xx.xx.xx:39875 > as it has seen zxid 0x2e00405fd9 our last zxid is 0x0 client must try another > server > Steps to reproduce: > Start an RO-enabled observer connecting to an ensemble. Kill the ensemble and > wait until the observer restart in RO mode. Zxid of this observer should be 0. > Description: > Before a server transition into LOOKING state, its database get closed as > part of shutdown sequence. The database of leader, follower and observer get > initialized as a side effect of participating in leader election protocol. > (eg. observer will call registerWithLeader() and call getLastLoggedZxid() > which initialize the db if not already). > However, RO server does not participate in this protocol so its DB doesn't > get initialized properly > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-679) Bookie should exit with non-zero if NIOServer crashes with Error
[ https://issues.apache.org/jira/browse/BOOKKEEPER-679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Kelly updated BOOKKEEPER-679: -- Attachment: 0001-BOOKKEEPER-679-Bookie-should-exit-with-non-zero-if-N.patch > Bookie should exit with non-zero if NIOServer crashes with Error > > > Key: BOOKKEEPER-679 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-679 > Project: Bookkeeper > Issue Type: Bug >Reporter: Ivan Kelly >Assignee: Ivan Kelly >Priority: Critical > Fix For: 4.2.2 > > Attachments: > 0001-BOOKKEEPER-679-Bookie-should-exit-with-non-zero-if-N.patch > > > Currently, if NIOServerFactory throws something like OutOfMemoryError, bookie > process will exit with 0. This is bad, as monitoring tools will take it to > mean a normal shutdown and not restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
