[jira] Created: (ZOOKEEPER-753) log4j dependency in the pom needs to have exclusion lists
log4j dependency in the pom needs to have exclusion lists - Key: ZOOKEEPER-753 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-753 Project: Zookeeper Issue Type: Bug Reporter: Kay Kay Fix For: 3.3.1 http://repo2.maven.org/maven2/org/apache/hadoop/zookeeper/3.3.0/zookeeper-3.3.0.pom The pom contains log4j dependency as itself. dependency groupIdlog4j/groupId artifactIdlog4j/artifactId version1.2.15/version scopecompile/scope /dependency This is broken without an exclusion list, since the pending dependencies of javax.mail. etc. are not necessary for the most part. Please fix this along with 3.3.1 and republish new dependencies , since at its current state , it is usable by some projects (to host in central , say). Correct dependency for log4j: dependency groupIdlog4j/groupId artifactIdlog4j/artifactId version1.2.15/version scopecompile/scope exclusions exclusion groupIdjavax.mail/groupId artifactIdmail/artifactId /exclusion exclusion groupIdjavax.jms/groupId artifactIdjms/artifactId /exclusion exclusion groupIdcom.sun.jdmk/groupId artifactIdjmxtools/artifactId /exclusion exclusion groupIdcom.sun.jmx/groupId artifactIdjmxri/artifactId /exclusion /exclusions /dependency -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-753) log4j dependency in the pom needs to have exclusion lists
[ https://issues.apache.org/jira/browse/ZOOKEEPER-753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Kay updated ZOOKEEPER-753: -- Fix Version/s: 3.4.0 (was: 3.3.1) Reassigning versions to 3.4.0 , since a workaround is available, which is basically to paste another dependency of the log4j exclusion in the pom along with zookeeper. Not pleasant, but not urgent for the next minor release. It seems an ivy-mvn conversion is happening in zk. Need to work on the same to provide a patch to incorporate this later. log4j dependency in the pom needs to have exclusion lists - Key: ZOOKEEPER-753 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-753 Project: Zookeeper Issue Type: Bug Reporter: Kay Kay Fix For: 3.4.0 http://repo2.maven.org/maven2/org/apache/hadoop/zookeeper/3.3.0/zookeeper-3.3.0.pom The pom contains log4j dependency as itself. dependency groupIdlog4j/groupId artifactIdlog4j/artifactId version1.2.15/version scopecompile/scope /dependency This is broken without an exclusion list, since the pending dependencies of javax.mail. etc. are not necessary for the most part. Please fix this along with 3.3.1 and republish new dependencies , since at its current state , it is usable by some projects (to host in central , say). Correct dependency for log4j: dependency groupIdlog4j/groupId artifactIdlog4j/artifactId version1.2.15/version scopecompile/scope exclusions exclusion groupIdjavax.mail/groupId artifactIdmail/artifactId /exclusion exclusion groupIdjavax.jms/groupId artifactIdjms/artifactId /exclusion exclusion groupIdcom.sun.jdmk/groupId artifactIdjmxtools/artifactId /exclusion exclusion groupIdcom.sun.jmx/groupId artifactIdjmxri/artifactId /exclusion /exclusions /dependency -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-464) Need procedure to garbage collect ledgers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12860970#action_12860970 ] Flavio Paiva Junqueira commented on ZOOKEEPER-464: -- Hi Erwin, Sorry about the delay to review it. I think this patch is pretty much ready to get in. I have just a couple of tiny requests: # I noticed that the two commands you modified in the documentation (bookkeeperStated.xml and bookkeeperConfig.xml) are going over the paper margin for the generated pdfs. Could you please break up the lines? # In LedgerDeleteTest, you're using // for multi-line comments. I believe we have been using /**/ instead. This is just to keep it consistent with comments in other parts of the code. Otherwise, it looks good. If you fix these two things, then I think we will be able to get it in shortly after. Need procedure to garbage collect ledgers - Key: ZOOKEEPER-464 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-464 Project: Zookeeper Issue Type: New Feature Components: contrib-bookkeeper Reporter: Flavio Paiva Junqueira Assignee: Erwin Tam Fix For: 3.4.0 Attachments: zookeeper-464-log.txt, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch An application using BookKeeper is likely to use a large number of ledgers over time. Such an application might not need all ledgers created over time and might want to delete some of these ledgers to free up some space on bookies. The idea of this jira is to implement a procedure that enables an application to garbage-collect unwanted ledgers. To garbage-collect a ledger, we need to delete the ledger metadata on ZooKeeper, and delete the ledger data on corresponding bookies. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-678) Browser application to view and edit the contents of a zookeeper instance
[ https://issues.apache.org/jira/browse/ZOOKEEPER-678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12860997#action_12860997 ] Thomas Koch commented on ZOOKEEPER-678: --- Could you please be so kind to specify the exact URL, where the icons are from? It's best to include the link also in the NOTICE.txt file. I need this information for the Debian package of Zookeeper. Thanks! Browser application to view and edit the contents of a zookeeper instance - Key: ZOOKEEPER-678 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-678 Project: Zookeeper Issue Type: New Feature Affects Versions: 3.3.0 Reporter: Colin Goodheart-Smithe Assignee: Colin Goodheart-Smithe Fix For: 3.3.0 Attachments: zooInspector.sh, ZooInspector.zip, ZooInspector.zip, ZooInspector.zip An application which shows a tree view of the nodes currently in a zookeeper instance and allow the user to view and update the contents of the nodes as well as allowing users to add and remove nodes from the tree, similar in use to the Luke application in the Lucene project. I have a list of other features that I want to add to this application but I wanted to gauge the response before I implemented them all. I have found this useful when debugging my application and thought that it may be useful to others. I was going to submit this as a patch file but I have used some icon files and one library which isn't available in the maven/ivy repositories and these don't seem to work when creating a patch file using subversion. Because of this I have attached a zip containing this application to this issue. If there is a better way to submit this please let me know. The zip contains two directories, the src directory contains the source as it would be added to the contrib folder and the build folder contains a build version of the with a runnable jar. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-737) some 4 letter words may fail with netcat (nc)
[
https://issues.apache.org/jira/browse/ZOOKEEPER-737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Hunt updated ZOOKEEPER-737:
---
Status: Open (was: Patch Available)
Mahadev, this looks great, a few issues I noticed:
If the pwriter flush/close calls fail for any reason we will not close the
socket. In cleanupWriterSocket I think you should have a try/finally around the
pwriter conditional block. This will ensure that no matter what happens we will
close the socket.
What happens if cancel key fails? Is this really a debug level message? What
would happen if cancel fails but later the socket is explicitly closed (after
command returns)?
+if (k != null) {
+try {
+k.cancel();
+} catch(Exception e) {
+LOG.debug(Error cancelling command selection key , e);
+}
Are these errors or info messages? Seems like both are LOG.error? (ie we'd
really want to know if this happened)
LOG.info(Error sending data synchronously , ie);
LOG.info(Error closing a command socket , e);
some 4 letter words may fail with netcat (nc)
-
Key: ZOOKEEPER-737
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-737
Project: Zookeeper
Issue Type: Bug
Components: server
Affects Versions: 3.3.0
Reporter: Patrick Hunt
Assignee: Mahadev konar
Priority: Blocker
Fix For: 3.3.1, 3.4.0
Attachments: ZOOKEEPER-737.patch, ZOOKEEPER-737.patch,
ZOOKEEPER-737.patch, ZOOKEEPER-737.patch
nc closes the write channel as soon as it's sent it's information, for
example echo stat|nc localhost 2181
in general this is fine, however the server code will close the socket as
soon as it receives notice that nc has
closed it's write channel. if not all the 4 letter word result has been
written back to the client yet, this will cause
some or all of the result to be lost - ie the client will not see the full
result. this was introduced in 3.3.0 as part
of a change to reduce blocking of the selector by long running 4letter words.
here's an example of the logs from the server during this
echo -n stat | nc localhost 2181
2010-04-09 21:55:36,124 - INFO
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioservercnxn$fact...@251] -
Accepted socket connection from /127.0.0.1:42179
2010-04-09 21:55:36,124 - INFO
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@968] - Processing
stat command from /127.0.0.1:42179
2010-04-09 21:55:36,125 - WARN
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@606] -
EndOfStreamException: Unable to read additional data from client sessionid
0x0, likely client has closed socket
2010-04-09 21:55:36,125 - INFO
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@1286] - Closed
socket connection for client /127.0.0.1:42179 (no session established for
client)
[ph...@gsbl90850 zookeeper-3.3.0]$ 2010-04-09 21:55:36,126 - ERROR
[Thread-15:nioserverc...@422] - Unexpected Exception:
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59)
at
org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:395)
at
org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:907)
at
org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945)
at java.io.BufferedWriter.flush(BufferedWriter.java:236)
at java.io.PrintWriter.flush(PrintWriter.java:276)
at
org.apache.zookeeper.server.NIOServerCnxn$2.run(NIOServerCnxn.java:1089)
2010-04-09 21:55:36,126 - ERROR [Thread-15:nioservercnxn$factor...@82] -
Thread Thread[Thread-15,5,main] died
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:64)
at
org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.wakeup(NIOServerCnxn.java:927)
at
org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:909)
at
org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945)
at java.io.BufferedWriter.flush(BufferedWriter.java:236)
at java.io.PrintWriter.flush(PrintWriter.java:276)
at
org.apache.zookeeper.server.NIOServerCnxn$2.run(NIOServerCnxn.java:1089)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-738) zookeeper.jute.h fails to compile with -pedantic
[ https://issues.apache.org/jira/browse/ZOOKEEPER-738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Hunt updated ZOOKEEPER-738: --- Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed +1, looks good, thanks Jozef! zookeeper.jute.h fails to compile with -pedantic - Key: ZOOKEEPER-738 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-738 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.3.0 Reporter: Patrick Hunt Assignee: Jozef Hatala Fix For: 3.3.1, 3.4.0 Attachments: semi.patch /home/y/include/zookeeper/zookeeper.jute.h:96: error: extra semicolon /home/y/include/zookeeper/zookeeper.jute.h:158: error: extra semicolon /home/y/include/zookeeper/zookeeper.jute.h:288: error: extra semicolon the code generator needs to be updated to not output a naked semi -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-734) QuorumPeerTestBase.java and ZooKeeperServerMainTest.java do not handle windows path correctly
[
https://issues.apache.org/jira/browse/ZOOKEEPER-734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Hunt updated ZOOKEEPER-734:
---
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
+1 thanks Vishal!
QuorumPeerTestBase.java and ZooKeeperServerMainTest.java do not handle
windows path correctly
-
Key: ZOOKEEPER-734
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-734
Project: Zookeeper
Issue Type: Bug
Components: tests
Affects Versions: 3.3.0
Environment: Windows 32-bit
Reporter: Vishal K
Assignee: Vishal K
Fix For: 3.3.1, 3.4.0
Attachments: ZOOKEEPER-734.patch
While runniing ant test-core-java QuorumPeerTestBase.java and
ZooKeeperServerMainTest.java fail. The problem seems to be in
ZookeeperserverMainTest.java:MainThread():66 and in
QuorumPeerBaseTest.java:MainThread:76.
FileWriter.write() writes windows path to the conf file. Java does not like
windows path. Therefore, the test complains that it cannot find myid and
fails.
Solution - convert windows path to UNIX path. This worked for me on windows.
Diffs are attached below. Solution not tested on Linux since for some reason
build is failing (due to problems not related to this change).
vmc-floorb-dhcp116-114:/opt/zksrc/zookeeper-3.3.0/src/java/test/org/apache/zookeeper/server
# svn diff
Index: ZooKeeperServerMainTest.java
===
--- ZooKeeperServerMainTest.java (revision 931240)
+++ ZooKeeperServerMainTest.java (working copy)
@@ -61,7 +61,8 @@
if (!dataDir.mkdir()) {
throw new IOException(unable to mkdir + dataDir);
}
-fwriter.write(dataDir= + dataDir.toString() + \n);
+String data = dataDir.toString().replace('\\', '/');
+fwriter.write(dataDir= + data + \n);
fwriter.write(clientPort= + clientPort + \n);
fwriter.flush();
Index: quorum/QuorumPeerTestBase.java
===
--- quorum/QuorumPeerTestBase.java(revision 931240)
+++ quorum/QuorumPeerTestBase.java(working copy)
@@ -73,7 +73,8 @@
if (!dataDir.mkdir()) {
throw new IOException(Unable to mkdir + dataDir);
}
-fwriter.write(dataDir= + dataDir.toString() + \n);
+String data = dataDir.toString().replace('\\', '/');
+fwriter.write(dataDir= + data + \n);
fwriter.write(clientPort= + clientPort + \n);
fwriter.write(quorumCfgSection + \n);
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-464) Need procedure to garbage collect ledgers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erwin Tam updated ZOOKEEPER-464: Status: Open (was: Patch Available) Need procedure to garbage collect ledgers - Key: ZOOKEEPER-464 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-464 Project: Zookeeper Issue Type: New Feature Components: contrib-bookkeeper Reporter: Flavio Paiva Junqueira Assignee: Erwin Tam Fix For: 3.4.0 Attachments: zookeeper-464-log.txt, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch An application using BookKeeper is likely to use a large number of ledgers over time. Such an application might not need all ledgers created over time and might want to delete some of these ledgers to free up some space on bookies. The idea of this jira is to implement a procedure that enables an application to garbage-collect unwanted ledgers. To garbage-collect a ledger, we need to delete the ledger metadata on ZooKeeper, and delete the ledger data on corresponding bookies. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-464) Need procedure to garbage collect ledgers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erwin Tam updated ZOOKEEPER-464: Attachment: ZOOKEEPER-464.patch Updated patch with formatting changes requested. Need procedure to garbage collect ledgers - Key: ZOOKEEPER-464 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-464 Project: Zookeeper Issue Type: New Feature Components: contrib-bookkeeper Reporter: Flavio Paiva Junqueira Assignee: Erwin Tam Fix For: 3.4.0 Attachments: zookeeper-464-log.txt, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch An application using BookKeeper is likely to use a large number of ledgers over time. Such an application might not need all ledgers created over time and might want to delete some of these ledgers to free up some space on bookies. The idea of this jira is to implement a procedure that enables an application to garbage-collect unwanted ledgers. To garbage-collect a ledger, we need to delete the ledger metadata on ZooKeeper, and delete the ledger data on corresponding bookies. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-464) Need procedure to garbage collect ledgers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erwin Tam updated ZOOKEEPER-464: Status: Patch Available (was: Open) Uploaded a new patch with the formatting changes. Note that the generated pdf's previously already had the issue with a line going beyond the paper margin. I modified this but let me know if it isn't done correctly. Need procedure to garbage collect ledgers - Key: ZOOKEEPER-464 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-464 Project: Zookeeper Issue Type: New Feature Components: contrib-bookkeeper Reporter: Flavio Paiva Junqueira Assignee: Erwin Tam Fix For: 3.4.0 Attachments: zookeeper-464-log.txt, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch An application using BookKeeper is likely to use a large number of ledgers over time. Such an application might not need all ledgers created over time and might want to delete some of these ledgers to free up some space on bookies. The idea of this jira is to implement a procedure that enables an application to garbage-collect unwanted ledgers. To garbage-collect a ledger, we need to delete the ledger metadata on ZooKeeper, and delete the ledger data on corresponding bookies. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-737) some 4 letter words may fail with netcat (nc)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated ZOOKEEPER-737: Attachment: ZOOKEEPER-737.patch addressed pat's comments on the logging and try and finally block. If cancel key fails, the socket will be closed later to make sure there is no resource leak. some 4 letter words may fail with netcat (nc) - Key: ZOOKEEPER-737 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-737 Project: Zookeeper Issue Type: Bug Components: server Affects Versions: 3.3.0 Reporter: Patrick Hunt Assignee: Mahadev konar Priority: Blocker Fix For: 3.3.1, 3.4.0 Attachments: ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch nc closes the write channel as soon as it's sent it's information, for example echo stat|nc localhost 2181 in general this is fine, however the server code will close the socket as soon as it receives notice that nc has closed it's write channel. if not all the 4 letter word result has been written back to the client yet, this will cause some or all of the result to be lost - ie the client will not see the full result. this was introduced in 3.3.0 as part of a change to reduce blocking of the selector by long running 4letter words. here's an example of the logs from the server during this echo -n stat | nc localhost 2181 2010-04-09 21:55:36,124 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioservercnxn$fact...@251] - Accepted socket connection from /127.0.0.1:42179 2010-04-09 21:55:36,124 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@968] - Processing stat command from /127.0.0.1:42179 2010-04-09 21:55:36,125 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@606] - EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket 2010-04-09 21:55:36,125 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@1286] - Closed socket connection for client /127.0.0.1:42179 (no session established for client) [ph...@gsbl90850 zookeeper-3.3.0]$ 2010-04-09 21:55:36,126 - ERROR [Thread-15:nioserverc...@422] - Unexpected Exception: java.nio.channels.CancelledKeyException at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55) at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59) at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:395) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:907) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945) at java.io.BufferedWriter.flush(BufferedWriter.java:236) at java.io.PrintWriter.flush(PrintWriter.java:276) at org.apache.zookeeper.server.NIOServerCnxn$2.run(NIOServerCnxn.java:1089) 2010-04-09 21:55:36,126 - ERROR [Thread-15:nioservercnxn$factor...@82] - Thread Thread[Thread-15,5,main] died java.nio.channels.CancelledKeyException at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55) at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:64) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.wakeup(NIOServerCnxn.java:927) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:909) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945) at java.io.BufferedWriter.flush(BufferedWriter.java:236) at java.io.PrintWriter.flush(PrintWriter.java:276) at org.apache.zookeeper.server.NIOServerCnxn$2.run(NIOServerCnxn.java:1089) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-737) some 4 letter words may fail with netcat (nc)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated ZOOKEEPER-737: Status: Patch Available (was: Open) some 4 letter words may fail with netcat (nc) - Key: ZOOKEEPER-737 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-737 Project: Zookeeper Issue Type: Bug Components: server Affects Versions: 3.3.0 Reporter: Patrick Hunt Assignee: Mahadev konar Priority: Blocker Fix For: 3.3.1, 3.4.0 Attachments: ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch nc closes the write channel as soon as it's sent it's information, for example echo stat|nc localhost 2181 in general this is fine, however the server code will close the socket as soon as it receives notice that nc has closed it's write channel. if not all the 4 letter word result has been written back to the client yet, this will cause some or all of the result to be lost - ie the client will not see the full result. this was introduced in 3.3.0 as part of a change to reduce blocking of the selector by long running 4letter words. here's an example of the logs from the server during this echo -n stat | nc localhost 2181 2010-04-09 21:55:36,124 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioservercnxn$fact...@251] - Accepted socket connection from /127.0.0.1:42179 2010-04-09 21:55:36,124 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@968] - Processing stat command from /127.0.0.1:42179 2010-04-09 21:55:36,125 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@606] - EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket 2010-04-09 21:55:36,125 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@1286] - Closed socket connection for client /127.0.0.1:42179 (no session established for client) [ph...@gsbl90850 zookeeper-3.3.0]$ 2010-04-09 21:55:36,126 - ERROR [Thread-15:nioserverc...@422] - Unexpected Exception: java.nio.channels.CancelledKeyException at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55) at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59) at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:395) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:907) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945) at java.io.BufferedWriter.flush(BufferedWriter.java:236) at java.io.PrintWriter.flush(PrintWriter.java:276) at org.apache.zookeeper.server.NIOServerCnxn$2.run(NIOServerCnxn.java:1089) 2010-04-09 21:55:36,126 - ERROR [Thread-15:nioservercnxn$factor...@82] - Thread Thread[Thread-15,5,main] died java.nio.channels.CancelledKeyException at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55) at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:64) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.wakeup(NIOServerCnxn.java:927) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:909) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945) at java.io.BufferedWriter.flush(BufferedWriter.java:236) at java.io.PrintWriter.flush(PrintWriter.java:276) at org.apache.zookeeper.server.NIOServerCnxn$2.run(NIOServerCnxn.java:1089) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-464) Need procedure to garbage collect ledgers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12861135#action_12861135 ] Hadoop QA commented on ZOOKEEPER-464: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12442901/ZOOKEEPER-464.patch against trunk revision 938212. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/74/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/74/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/74/console This message is automatically generated. Need procedure to garbage collect ledgers - Key: ZOOKEEPER-464 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-464 Project: Zookeeper Issue Type: New Feature Components: contrib-bookkeeper Reporter: Flavio Paiva Junqueira Assignee: Erwin Tam Fix For: 3.4.0 Attachments: zookeeper-464-log.txt, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch, ZOOKEEPER-464.patch An application using BookKeeper is likely to use a large number of ledgers over time. Such an application might not need all ledgers created over time and might want to delete some of these ledgers to free up some space on bookies. The idea of this jira is to implement a procedure that enables an application to garbage-collect unwanted ledgers. To garbage-collect a ledger, we need to delete the ledger metadata on ZooKeeper, and delete the ledger data on corresponding bookies. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-737) some 4 letter words may fail with netcat (nc)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12861138#action_12861138 ] Hadoop QA commented on ZOOKEEPER-737: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12442903/ZOOKEEPER-737.patch against trunk revision 938212. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/148/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/148/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/148/console This message is automatically generated. some 4 letter words may fail with netcat (nc) - Key: ZOOKEEPER-737 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-737 Project: Zookeeper Issue Type: Bug Components: server Affects Versions: 3.3.0 Reporter: Patrick Hunt Assignee: Mahadev konar Priority: Blocker Fix For: 3.3.1, 3.4.0 Attachments: ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch nc closes the write channel as soon as it's sent it's information, for example echo stat|nc localhost 2181 in general this is fine, however the server code will close the socket as soon as it receives notice that nc has closed it's write channel. if not all the 4 letter word result has been written back to the client yet, this will cause some or all of the result to be lost - ie the client will not see the full result. this was introduced in 3.3.0 as part of a change to reduce blocking of the selector by long running 4letter words. here's an example of the logs from the server during this echo -n stat | nc localhost 2181 2010-04-09 21:55:36,124 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioservercnxn$fact...@251] - Accepted socket connection from /127.0.0.1:42179 2010-04-09 21:55:36,124 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@968] - Processing stat command from /127.0.0.1:42179 2010-04-09 21:55:36,125 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@606] - EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket 2010-04-09 21:55:36,125 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@1286] - Closed socket connection for client /127.0.0.1:42179 (no session established for client) [ph...@gsbl90850 zookeeper-3.3.0]$ 2010-04-09 21:55:36,126 - ERROR [Thread-15:nioserverc...@422] - Unexpected Exception: java.nio.channels.CancelledKeyException at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55) at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59) at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:395) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:907) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945) at java.io.BufferedWriter.flush(BufferedWriter.java:236) at java.io.PrintWriter.flush(PrintWriter.java:276) at org.apache.zookeeper.server.NIOServerCnxn$2.run(NIOServerCnxn.java:1089) 2010-04-09 21:55:36,126 - ERROR [Thread-15:nioservercnxn$factor...@82] - Thread Thread[Thread-15,5,main] died java.nio.channels.CancelledKeyException at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55) at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:64) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.wakeup(NIOServerCnxn.java:927) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:909) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945) at java.io.BufferedWriter.flush(BufferedWriter.java:236) at java.io.PrintWriter.flush(PrintWriter.java:276) at
[jira] Updated: (ZOOKEEPER-737) some 4 letter words may fail with netcat (nc)
[
https://issues.apache.org/jira/browse/ZOOKEEPER-737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Hunt updated ZOOKEEPER-737:
---
Status: Open (was: Patch Available)
Seems like this could infinite loop in some bad cases:
sendbuffersync
while(bb.remaining() != 0) {
if (sock != null) {
sock.write(bb);
}
}
say the client requests, then never reads the response (and the response is
very large, so it fills up the tcp buffers and such).
some 4 letter words may fail with netcat (nc)
-
Key: ZOOKEEPER-737
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-737
Project: Zookeeper
Issue Type: Bug
Components: server
Affects Versions: 3.3.0
Reporter: Patrick Hunt
Assignee: Mahadev konar
Priority: Blocker
Fix For: 3.3.1, 3.4.0
Attachments: ZOOKEEPER-737.patch, ZOOKEEPER-737.patch,
ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch
nc closes the write channel as soon as it's sent it's information, for
example echo stat|nc localhost 2181
in general this is fine, however the server code will close the socket as
soon as it receives notice that nc has
closed it's write channel. if not all the 4 letter word result has been
written back to the client yet, this will cause
some or all of the result to be lost - ie the client will not see the full
result. this was introduced in 3.3.0 as part
of a change to reduce blocking of the selector by long running 4letter words.
here's an example of the logs from the server during this
echo -n stat | nc localhost 2181
2010-04-09 21:55:36,124 - INFO
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioservercnxn$fact...@251] -
Accepted socket connection from /127.0.0.1:42179
2010-04-09 21:55:36,124 - INFO
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@968] - Processing
stat command from /127.0.0.1:42179
2010-04-09 21:55:36,125 - WARN
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@606] -
EndOfStreamException: Unable to read additional data from client sessionid
0x0, likely client has closed socket
2010-04-09 21:55:36,125 - INFO
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@1286] - Closed
socket connection for client /127.0.0.1:42179 (no session established for
client)
[ph...@gsbl90850 zookeeper-3.3.0]$ 2010-04-09 21:55:36,126 - ERROR
[Thread-15:nioserverc...@422] - Unexpected Exception:
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59)
at
org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:395)
at
org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:907)
at
org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945)
at java.io.BufferedWriter.flush(BufferedWriter.java:236)
at java.io.PrintWriter.flush(PrintWriter.java:276)
at
org.apache.zookeeper.server.NIOServerCnxn$2.run(NIOServerCnxn.java:1089)
2010-04-09 21:55:36,126 - ERROR [Thread-15:nioservercnxn$factor...@82] -
Thread Thread[Thread-15,5,main] died
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:64)
at
org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.wakeup(NIOServerCnxn.java:927)
at
org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:909)
at
org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945)
at java.io.BufferedWriter.flush(BufferedWriter.java:236)
at java.io.PrintWriter.flush(PrintWriter.java:276)
at
org.apache.zookeeper.server.NIOServerCnxn$2.run(NIOServerCnxn.java:1089)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-737) some 4 letter words may fail with netcat (nc)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12861156#action_12861156 ] Mahadev konar commented on ZOOKEEPER-737: - this issue could be possible only if the client's tcp buffer is filled up and the write buffer is also filled up. The defaults for WAN specifications are: ||tcp read/write sysctl config||min||default||max|| |net.ipv4.tcp_rmem|4096|87380|174760| |net.ipv4.tcp_wmem|4096|16384|131072| In a non WAN tcp connection the read buffer should be atleast 87KB and write buffer should be 16KB, which gives us a total of 100KB, which should be good enough (except for huge ephemeral dumps on a large tree). Even in that case according to my calculation we should be able to support 50, 000 ephemeral nodes, in which case the client will not be reading the data but the server is still writitng it out! Also, much more importantly all commands will be called via nc/telnet which should not be malformed (not reading off the socket). some 4 letter words may fail with netcat (nc) - Key: ZOOKEEPER-737 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-737 Project: Zookeeper Issue Type: Bug Components: server Affects Versions: 3.3.0 Reporter: Patrick Hunt Assignee: Mahadev konar Priority: Blocker Fix For: 3.3.1, 3.4.0 Attachments: ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch, ZOOKEEPER-737.patch nc closes the write channel as soon as it's sent it's information, for example echo stat|nc localhost 2181 in general this is fine, however the server code will close the socket as soon as it receives notice that nc has closed it's write channel. if not all the 4 letter word result has been written back to the client yet, this will cause some or all of the result to be lost - ie the client will not see the full result. this was introduced in 3.3.0 as part of a change to reduce blocking of the selector by long running 4letter words. here's an example of the logs from the server during this echo -n stat | nc localhost 2181 2010-04-09 21:55:36,124 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioservercnxn$fact...@251] - Accepted socket connection from /127.0.0.1:42179 2010-04-09 21:55:36,124 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@968] - Processing stat command from /127.0.0.1:42179 2010-04-09 21:55:36,125 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@606] - EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket 2010-04-09 21:55:36,125 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:nioserverc...@1286] - Closed socket connection for client /127.0.0.1:42179 (no session established for client) [ph...@gsbl90850 zookeeper-3.3.0]$ 2010-04-09 21:55:36,126 - ERROR [Thread-15:nioserverc...@422] - Unexpected Exception: java.nio.channels.CancelledKeyException at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55) at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59) at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:395) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:907) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945) at java.io.BufferedWriter.flush(BufferedWriter.java:236) at java.io.PrintWriter.flush(PrintWriter.java:276) at org.apache.zookeeper.server.NIOServerCnxn$2.run(NIOServerCnxn.java:1089) 2010-04-09 21:55:36,126 - ERROR [Thread-15:nioservercnxn$factor...@82] - Thread Thread[Thread-15,5,main] died java.nio.channels.CancelledKeyException at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55) at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:64) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.wakeup(NIOServerCnxn.java:927) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(NIOServerCnxn.java:909) at org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.flush(NIOServerCnxn.java:945) at java.io.BufferedWriter.flush(BufferedWriter.java:236) at java.io.PrintWriter.flush(PrintWriter.java:276) at org.apache.zookeeper.server.NIOServerCnxn$2.run(NIOServerCnxn.java:1089) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
ZooKeeper gets three Google Summer of Code students
Hi - Just wanted to announce to the community that we are lucky to have three talented students working on Google's Summer of Code projects directly related to ZooKeeper. Andrei Savu will be working with Patrick Hunt on a Web-based Administrative Interface, extending and improving Patrick's Django-based front end. Abmar Barros will be working with Flavio Junqueira on improving ZooKeeper's failure detector module - making the code cleaner and easier to try out new implementations, as well as implementing a few failure detection algorithms himself! Finally, Sergey Doroshenko will be working with me on a Read-Only Mode for ZooKeeper, which will help bolster ZK's availability in certain circumstances when a network partition is detected, as well as potentially optimising the read-path. (The full list of 450 GSoC students is here: http://socghop.appspot.com/gsoc/program/list_projects/google/gsoc2010) Congratulations to all three - we look forward to seeing what you produce over the summer. Thanks to everyone who applied, suggested projects and offered to mentor students; this program will have a big effect on ZooKeeper's visibility and community, as well as hopefully producing some great code! cheers, Henry -- Henry Robinson Software Engineer Cloudera 415-994-6679
