ZooKeeper-trunk-solaris - Build # 528 - Still Failing
See https://builds.apache.org/job/ZooKeeper-trunk-solaris/528/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 14411 lines...] [junit] 2013-04-17 10:32:01,662 [myid:] - INFO [CommitProcWorkThread-1:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178ceee80003 type:getData cxid:0xa zxid:0xfffe txntype:unknown reqpath:/session13e178ceee80003-3 [junit] 2013-04-17 10:32:01,782 [myid:] - INFO [CommitProcWorkThread-5:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178d028c type:getData cxid:0xc zxid:0xfffe txntype:unknown reqpath:/session13e178d028c-4 [junit] 2013-04-17 10:32:01,792 [myid:] - INFO [CommitProcWorkThread-1:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178ceee80002 type:create cxid:0x9 zxid:0x13 txntype:1 reqpath:n/a [junit] 2013-04-17 10:32:01,882 [myid:] - INFO [CommitProcessor:1:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178cd5b20007 type:getData cxid:0x3 zxid:0xfffe txntype:unknown reqpath:/session13e178cd5b20007-1 [junit] 2013-04-17 10:32:01,922 [myid:] - INFO [CommitProcWorkThread-1:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178ceee80004 type:getData cxid:0x7 zxid:0xfffe txntype:unknown reqpath:/session13e178ceee80004-2 [junit] 2013-04-17 10:32:01,942 [myid:] - INFO [CommitProcessor:1:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178cd5b20009 type:create cxid:0xb zxid:0x22 txntype:1 reqpath:n/a [junit] 2013-04-17 10:32:02,072 [myid:] - INFO [CommitProcWorkThread-7:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178d028c0002 type:create cxid:0xb zxid:0xa txntype:1 reqpath:n/a [junit] 2013-04-17 10:32:02,082 [myid:] - INFO [CommitProcWorkThread-1:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178ceee80005 type:getData cxid:0x6 zxid:0xfffe txntype:unknown reqpath:/session13e178ceee80005-1 [junit] 2013-04-17 10:32:02,102 [myid:] - INFO [CommitProcWorkThread-7:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178d028c0002 type:getData cxid:0xc zxid:0xfffe txntype:unknown reqpath:/session13e178d028c0002-3 [junit] 2013-04-17 10:32:02,162 [myid:] - INFO [CommitProcessor:1:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178cd5b20002 type:getData cxid:0x2 zxid:0xfffe txntype:unknown reqpath:/session13e178cd5b20002-1 [junit] 2013-04-17 10:32:02,252 [myid:] - INFO [CommitProcWorkThread-1:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178ceee80002 type:getData cxid:0xa zxid:0xfffe txntype:unknown reqpath:/session13e178ceee80002-5 [junit] 2013-04-17 10:32:02,272 [myid:] - INFO [CommitProcessor:1:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178cd5b20006 type:create cxid:0x6 zxid:0x23 txntype:1 reqpath:n/a [junit] 2013-04-17 10:32:02,312 [myid:] - INFO [CommitProcWorkThread-1:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178ceee80004 type:getData cxid:0x8 zxid:0xfffe txntype:unknown reqpath:/session13e178ceee80004-2 [junit] 2013-04-17 10:32:02,332 [myid:] - INFO [main:CommitProcessorTest@103] - tearDown starting [junit] 2013-04-17 10:32:02,332 [myid:] - INFO [main:ZooKeeperServer@398] - shutting down [junit] 2013-04-17 10:32:02,332 [myid:] - INFO [main:SessionTrackerImpl@180] - Shutting down [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 18.136 sec [junit] 2013-04-17 10:32:02,372 [myid:] - INFO [CommitProcWorkThread-5:CommitProcessorTest$ValidateProcessor@384] - Got request sessionid:0x13e178d028c type:create cxid:0xd zxid:0xb txntype:1 reqpath:n/a [junit] Running org.apache.zookeeper.server.quorum.LearnerTest [junit] 2013-04-17 10:32:03,693 [myid:] - INFO [main:ZKTestCase$1@51] - STARTING syncTest [junit] 2013-04-17 10:32:03,699 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@50] - RUNNING TEST METHOD syncTest [junit] 2013-04-17 10:32:03,904 [myid:] - INFO [main:Learner@92] - TCP NoDelay set to: true [junit] 2013-04-17 10:32:03,970 [myid:] - INFO [main:Environment@99] - Server environment:zookeeper.version=3.5.0-1468824, built on 04/17/2013 10:27 GMT [junit] 2013-04-17 10:32:03,970 [myid:] - INFO [main:Environment@99] - Server environment:host.name=hudson-solaris [junit] 2013-04-17 10:32:03,970 [myid:] - INFO [main:Environment@99] - Server environment:java.version=1.6.0_26 [junit] 2013-04-17 10:32:03,970 [myid:] - INFO [main:Environment@99] - Server environment:java.vendor=Sun Microsystems Inc. [junit] 2013-04-17 10:32:03,970
Zookeeper wire protocol details
Hello What is the best place to get details about the protocol zookeeper uses for client communication ? Thanks, -- Radu Stoenescu
Re: Zookeeper wire protocol details
On Wed, Apr 17, 2013 at 10:36 PM, Radu Stoenescu radu.s...@gmail.comwrote: communicatio check out these links http://zookeeper.apache.org/doc/r3.1.2/zookeeperInternals.html http://zookeeper.apache.org/doc/trunk/releasenotes.html http://zookeeper.apache.org/doc/trunk/releasenotes.html and last but not the least Zookeeper source... :) *Thanks Regards* ∞ Shashwat Shriparv
Re: Zookeeper wire protocol details
for the packet layout see the jute file at the root of the source. On Wed, Apr 17, 2013 at 10:49 AM, shashwat shriparv dwivedishash...@gmail.com wrote: On Wed, Apr 17, 2013 at 10:36 PM, Radu Stoenescu radu.s...@gmail.com wrote: communicatio check out these links http://zookeeper.apache.org/doc/r3.1.2/zookeeperInternals.html http://zookeeper.apache.org/doc/trunk/releasenotes.html http://zookeeper.apache.org/doc/trunk/releasenotes.html and last but not the least Zookeeper source... :) *Thanks Regards* ∞ Shashwat Shriparv
Re: Lua Binding For Apache Zookeeper.
This is very nice haiping fu, thanks! https://twitter.com/phunt/status/32458784961408 I'd suggest that you reference from our client binding page: https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZKClientBindings Of late we've been pointing to projects such as yours, rather than adding to contrib, it's just not manageable in the large. Regards, Patrick On Sat, Apr 13, 2013 at 11:58 PM, haiping fu haipi...@gmail.com wrote: Hi guys, I have make a Lua binding for apache zookeeper these days(I call it zklua), it seems everything works fine to me and I have finished most of the APIs binding(except for two zookeeper_interest() and zookeeper_process(), which are not that useful in lua in my opinion), the zklua consists of synchronous and asynchronous APIs as well as some useful auxiliary function. you may take a look at zklua in my github repos: https://github.com/forhappy/zklua. I am wondering what I can do to make zklua to be a contrib of zookeeper like zkpython or zkperl. If anyone knows please helps me(any helpful links would be appreciated ;-)), I will try best to improve zklua (coding style, more tests, docs, examples) and I'm also very pleased to contribute my code to zookeeper, thanks in advance. -- 责任与感恩 祝好 中国科学院计算技术研究所 中关村科学院南路6号 北京市海淀区 邮编100190
ZooKeeper-trunk-WinVS2008 - Build # 802 - Still Failing
See https://builds.apache.org/job/ZooKeeper-trunk-WinVS2008/802/ ### ## LAST 60 LINES OF THE CONSOLE ### Started by timer Started by timer Started by upstream project ZooKeeper-trunk build number 1897 originally caused by: Started by timer Started by timer Started by upstream project ZooKeeper-trunk build number 1898 originally caused by: Started by timer Started by timer Started by upstream project ZooKeeper-trunk build number 1899 originally caused by: Started by timer Started by timer Started by upstream project ZooKeeper-trunk build number 1900 originally caused by: Started by timer Started by timer Started by upstream project ZooKeeper-trunk build number 1901 originally caused by: Started by timer Started by timer Building remotely on windows1 in workspace f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008 FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.call(Request.java:174) at hudson.remoting.Channel.call(Channel.java:672) at hudson.FilePath.act(FilePath.java:854) at hudson.FilePath.act(FilePath.java:838) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:743) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:685) at hudson.model.AbstractProject.checkout(AbstractProject.java:1331) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:682) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:587) at hudson.model.Run.execute(Run.java:1568) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:236) Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:299) at hudson.remoting.Channel.terminate(Channel.java:732) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2577) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1315) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369) at hudson.remoting.Command.readFrom(Command.java:92) at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48) ### ## FAILED TESTS (if any) ## No tests ran.
[jira] [Commented] (ZOOKEEPER-1683) ZooKeeper client NPE when updating server list on disconnected client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634365#comment-13634365 ] Shevek commented on ZOOKEEPER-1683: --- I think it looks OK. We put the code into production on several clusters on Monday afternoon, and we're waiting on results. It's a bit early to promise anything from tests. I'm still somewhat unhappy with the client code overall, as I don't think the contracts of many of the states are clear enough to definitively claim correctness. ZooKeeper client NPE when updating server list on disconnected client - Key: ZOOKEEPER-1683 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1683 Project: ZooKeeper Issue Type: Bug Components: java client Affects Versions: 3.5.0 Reporter: Shevek Assignee: Alexander Shraer Fix For: 3.5.0 Attachments: ZOOKEEPER-1683.patch, ZOOKEEPER-1683-ver1.patch, ZOOKEEPER-1683-ver2.patch, ZOOKEEPER-1683-ver2.patch 2013-04-04 22:16:15,872 ERROR [pool-4-thread-1] com.netflix.curator.ConnectionState.getZooKeeper (ConnectionState.java:84) - Background exception caught java.lang.NullPointerException at org.apache.zookeeper.client.StaticHostProvider.updateServerList(StaticHostProvider.java:161) ~[zookeeper-3.5.0.jar:3.5.0--1] at org.apache.zookeeper.ZooKeeper.updateServerList(ZooKeeper.java:183) ~[zookeeper-3.5.0.jar:3.5.0--1] at com.netflix.curator.HandleHolder$1$1.setConnectionString(HandleHolder.java:121) ~[curator-client-1.3.5-SNAPSHOT.jar:?] The duff code is this: ClientCnxnSocket clientCnxnSocket = cnxn.sendThread.getClientCnxnSocket(); InetSocketAddress currentHost = (InetSocketAddress) clientCnxnSocket.getRemoteSocketAddress(); boolean reconfigMode = hostProvider.updateServerList(serverAddresses, currentHost); Now, currentHost might be null, if we're not yet connected. But StaticHostProvider.updateServerList indirects on it unconditionally. This would be caught by findbugs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1683) ZooKeeper client NPE when updating server list on disconnected client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634385#comment-13634385 ] Alexander Shraer commented on ZOOKEEPER-1683: - very glad to hear you're running client-side reconfiguration in production!! ZooKeeper client NPE when updating server list on disconnected client - Key: ZOOKEEPER-1683 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1683 Project: ZooKeeper Issue Type: Bug Components: java client Affects Versions: 3.5.0 Reporter: Shevek Assignee: Alexander Shraer Fix For: 3.5.0 Attachments: ZOOKEEPER-1683.patch, ZOOKEEPER-1683-ver1.patch, ZOOKEEPER-1683-ver2.patch, ZOOKEEPER-1683-ver2.patch 2013-04-04 22:16:15,872 ERROR [pool-4-thread-1] com.netflix.curator.ConnectionState.getZooKeeper (ConnectionState.java:84) - Background exception caught java.lang.NullPointerException at org.apache.zookeeper.client.StaticHostProvider.updateServerList(StaticHostProvider.java:161) ~[zookeeper-3.5.0.jar:3.5.0--1] at org.apache.zookeeper.ZooKeeper.updateServerList(ZooKeeper.java:183) ~[zookeeper-3.5.0.jar:3.5.0--1] at com.netflix.curator.HandleHolder$1$1.setConnectionString(HandleHolder.java:121) ~[curator-client-1.3.5-SNAPSHOT.jar:?] The duff code is this: ClientCnxnSocket clientCnxnSocket = cnxn.sendThread.getClientCnxnSocket(); InetSocketAddress currentHost = (InetSocketAddress) clientCnxnSocket.getRemoteSocketAddress(); boolean reconfigMode = hostProvider.updateServerList(serverAddresses, currentHost); Now, currentHost might be null, if we're not yet connected. But StaticHostProvider.updateServerList indirects on it unconditionally. This would be caught by findbugs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-572) Make the journal a write ahead log
[
https://issues.apache.org/jira/browse/BOOKKEEPER-572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634019#comment-13634019
]
Ivan Kelly commented on BOOKKEEPER-572:
---
{quote}
the problem is you only get the exception after added the entry to journal. so
the entry lives in journal. when bookie started, replaying this journal tried
to add the entry to ledger storage, but it would failed. so the bookie could
not start.
{quote}
And this can be handled at that point. We should separate what we consider
writing errors (errors with the write medium) and data errors (errors with the
entry). Data errors can be logged as a warn. Whats more, we can simply
prevalidate the entries. The contents of an entry should really be opaque to
the ledger storage, it's an unfortunate historical fact that they are not. The
ledger storage tries to read the ledger id the entry id and stores based on
this. We can prevalidate to ensure that everything we pass to ledger storage
has at least 16 bytes of length.
{quote}the COW is just for ledger index. it works for both skiplists and
interleaved.{quote}
This is true. It would mean either having a separate set of pages for read and
write, or that we have a lookaside cache for index entries. Have you done any
implementation on this? I think it would be a good thing to have whether this
goes in or not. If so, have you measured how much memory it uses?
Make the journal a write ahead log
--
Key: BOOKKEEPER-572
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-572
Project: Bookkeeper
Issue Type: Bug
Reporter: Ivan Kelly
Assignee: Ivan Kelly
Fix For: 4.3.0
Attachments:
0001-BOOKKEEPER-572-Write-to-the-journal-before-writing-t.patch,
0001-BOOKKEEPER-572-Write-to-the-journal-before-writing-t.patch,
0001-BOOKKEEPER-572-Write-to-the-journal-before-writing-t.patch,
0001-BOOKKEEPER-572-Write-to-the-journal-before-writing-t.patch,
0003-BOOKKEEPER-572-Write-to-the-journal-before-writing-t.patch,
0003-BOOKKEEPER-572-Write-to-the-journal-before-writing-t.patch,
BookieServer-2013-02-22.snapshot
A bookie adds to the LedgerStorage before writing to the journal. This is the
fundamental problem behind BOOKKEEPER-447 and blocks a nice solution to
BOOKKEEPER-530. By writing to the memory state before the journal, we exposed
ourselves to bugs if the bookie crashed before we wrote to the journal. The
entry may exist in index, but not in the entrylog, a situation which cannot
be distinguished from an I/O error. The comments on BOOKKEEPER-447 goes into
more details.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-564) Better checkpoint mechanism
[
https://issues.apache.org/jira/browse/BOOKKEEPER-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634026#comment-13634026
]
Ivan Kelly commented on BOOKKEEPER-564:
---
{quote}
How it worked in previous flow on testing LedgerStorage in isolation? I don't
quite understand about this part.
{quote}
We don't. We always construct a full bookie, because it's impossible to test
the ledger storage without the bookie.
{code}
grep -r new InterleavedLedgerStorage bookkeeper-server/src/test/java
{code}
Another place I came across this was with the bkvhbase benchmark. I had to
implement my own SyncThread.
{quote}
in previous flow, sync thread just checkpoint blindly. in current flow, ledger
storage would tell the syncthread (the checkpointer) the point which is a
better point to do checkpoint. the only thing that ledger storage gave out is
the point to guide checkpoint. but if you mean the guide is coupling, I have
nothing more to say.{quote}
It's a command rather than a guide. And how the ledger storage behaves is
dependent on the sync thread. This is coupling.
Better checkpoint mechanism
---
Key: BOOKKEEPER-564
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-564
Project: Bookkeeper
Issue Type: Improvement
Components: bookkeeper-server
Reporter: Sijie Guo
Assignee: Sijie Guo
Fix For: 4.3.0
Attachments: 0001-BOOKKEEPER-564-Better-checkpoint-mechanism.patch,
0001-BOOKKEEPER-564-Better-checkpoint-mechanism.patch,
0002-BOOKKEEPER-564-Better-checkpoint-mechanism.patch, BOOKKEEPER-564.patch,
BOOKKEEPER-564.patch
Currently, SyncThread made a checkpoint too frequently, which affects
performance. data is writing to entry logger file might be blocked by syncing
same entry logger file, which affect bookie to achieve higher throughput. We
could schedule checkpoint only when rotating an entry log file. so new
incoming entries would be written to newer entry log file and old entry log
file could be synced.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
