Re: Program runs fine in NetBeans fails the shutdown properly running via Maven
Right again. I changed the package name to "jaredwinick" and the "sealing violation" issue was resolved. But the JIRA ticket comments were correct about the code's being brittle. It has a String literal that refers to "org.apache.accumulo.core.zookeeper.ZooSession"which does not exist in Accumulo v1.6.1. On Sun, Nov 30, 2014 at 12:05 AM, Josh Elser wrote: > I believe that means you re-used the package org.apache.accumulo.core.util > which was already defined in accumulo-core.jar. > > Don't do that. > > > David Medinets wrote: >> >> Thanks Josh. That explanation makes sense. I added the referenced >> ClientThreads.shutdownNow() method to my code and ran into the >> following: >> >> java.lang.SecurityException: sealing violation: package >> org.apache.accumulo.core.util is sealed >> >> >> I can live with this issue until v2.0 since it happens when the JVM is >> exiting. And I'm just writing demonstration code for no particular >> reason. At the least, we'll have a small project to test the fix >> against. >> >> >> On Sat, Nov 29, 2014 at 10:35 PM, Josh Elser wrote: >>> >>> From http://mojo.codehaus.org/exec-maven-plugin/usage.html, >>> >>> "By default daemon threads are joined and interrupted once all known non >>> daemon threads have quitted" >>> >>> So, your main thread finished, but before the exec plugin stopped the >>> application, it tried to interrupt the other non-daemon threads that were >>> still running (which should be a sign for them quit on their own). >>> >>> However, it's a known issue[1] that we have ZooKeeper and Thrift >>> resources >>> that are not cleaned up (a problem which is slated to be fixed by >>> Christopher's 2.0 API refactoring). >>> >>> Hope that makes sense. >>> >>> [1] https://issues.apache.org/jira/browse/ACCUMULO-2113 >>> >>> >>> David Medinets wrote: This might be a stupid David issue but it's at least marginally related to Accumulo so I'm hoping someone will recognize the problem and help. I am seeing this error when my program is run via maven: java.lang.InterruptedException: sleep interrupted The program connects to a MiniAccumuloCluster and creates a table if it does not exist. The program works fine inside Netbeans. In both NetBeans and Maven, the program connects to the MiniAccumuloCluster just fine. The shutdown is the problem. I have a github project which should demonstrate the issue. # First clone the project git clone https://github.com/medined/jodoc.git cd jodoc # Then build the java image cd java_zulu7 ./build_image.sh # Then compile and create a jar file mvn package # Then build the jodoc image ./build_image.sh # And run the jodoc image. This image runs a MiniAccumuloCluster # using port 2 for Zookeeper. ./run_image.sh # Finally run the client program via Maven mvn exec:java -Dexec.mainClass="com.codebits.jodoc.WriteAndReadDriver" -DZOOKEEPER_PORT=2 Here is the full error stack: java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.accumulo.core.client.impl.ThriftTransportPool$Closer.closeConnections(ThriftTransportPool.java:137) at org.apache.accumulo.core.client.impl.ThriftTransportPool$Closer.run(ThriftTransportPool.java:147) at java.lang.Thread.run(Thread.java:745) 2014-11-29 21:05:18,011 ERROR [com.codebits.jodoc.WriteAndReadDriver.main()-EventThread] zookeeper.ClientCnxn (ClientCnxn.java:run(513)) - Event thread exiting due to interruption java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) 2014-11-29 21:05:18,012 INFO [com.codebits.jodoc.WriteAndReadDriver.main()-EventThread] zookeeper.ClientCnxn (ClientCnxn.java:run(516)) - EventThread shut down 2014-11-29 21:05:28,010 WARN [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:run(1185)) - Session 0x149fe6a2c5a0007 for server localhost/127.0.0.1:2, unexpected error, closing socket connection and attempting reconnect java.nio.channels.ClosedByInterruptException at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:477) at org.apache.zookee
Re: Program runs fine in NetBeans fails the shutdown properly running via Maven
I experimented a bit more. In the waitForZooKeeperClientThreads method, I added a "thread.interrupt" call inside the "while (thread.isAlive())" loop. It seems like the thread is interrupted after a few seconds, but then is reconnected. Darn that resiliency! I've watched it reconnect four times before I killed the process. Can I reset the retry number somehow? Am I too far down the rabbit hole? java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1045) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1101) 2014-11-30 09:29:32,596 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:startConnect(1058)) - Opening socket connection to server localhost/127.0.0.1:2 2014-11-30 09:29:32,597 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(947)) - Socket connection established to localhost/127.0.0.1:2, initiating session 2014-11-30 09:29:32,599 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:readConnectResult(736)) - Session establishment complete on server localhost/127.0.0.1:2, sessionid = 0x1 On Sun, Nov 30, 2014 at 9:10 AM, David Medinets wrote: > Right again. I changed the package name to "jaredwinick" and the > "sealing violation" issue was resolved. But the JIRA ticket comments > were correct about the code's being brittle. It has a String literal > that refers to "org.apache.accumulo.core.zookeeper.ZooSession"which > does not exist in Accumulo v1.6.1. > > On Sun, Nov 30, 2014 at 12:05 AM, Josh Elser wrote: >> I believe that means you re-used the package org.apache.accumulo.core.util >> which was already defined in accumulo-core.jar. >> >> Don't do that. >> >> >> David Medinets wrote: >>> >>> Thanks Josh. That explanation makes sense. I added the referenced >>> ClientThreads.shutdownNow() method to my code and ran into the >>> following: >>> >>> java.lang.SecurityException: sealing violation: package >>> org.apache.accumulo.core.util is sealed >>> >>> >>> I can live with this issue until v2.0 since it happens when the JVM is >>> exiting. And I'm just writing demonstration code for no particular >>> reason. At the least, we'll have a small project to test the fix >>> against. >>> >>> >>> On Sat, Nov 29, 2014 at 10:35 PM, Josh Elser wrote: From http://mojo.codehaus.org/exec-maven-plugin/usage.html, "By default daemon threads are joined and interrupted once all known non daemon threads have quitted" So, your main thread finished, but before the exec plugin stopped the application, it tried to interrupt the other non-daemon threads that were still running (which should be a sign for them quit on their own). However, it's a known issue[1] that we have ZooKeeper and Thrift resources that are not cleaned up (a problem which is slated to be fixed by Christopher's 2.0 API refactoring). Hope that makes sense. [1] https://issues.apache.org/jira/browse/ACCUMULO-2113 David Medinets wrote: > > This might be a stupid David issue but it's at least marginally > related to Accumulo so I'm hoping someone will recognize the problem > and help. > > I am seeing this error when my program is run via maven: > > java.lang.InterruptedException: sleep interrupted > > The program connects to a MiniAccumuloCluster and creates a table if > it does not exist. The program works fine inside Netbeans. > > In both NetBeans and Maven, the program connects to the > MiniAccumuloCluster just fine. The shutdown is the problem. > > I have a github project which should demonstrate the issue. > > # First clone the project > git clone https://github.com/medined/jodoc.git > cd jodoc > > # Then build the java image > cd java_zulu7 > ./build_image.sh > > # Then compile and create a jar file > mvn package > > # Then build the jodoc image > ./build_image.sh > > # And run the jodoc image. This image runs a MiniAccumuloCluster > # using port 2 for Zookeeper. > ./run_image.sh > > # Finally run the client program via Maven > mvn exec:java -Dexec.mainClass="com.codebits.jodoc.WriteAndReadDriver" > -DZOOKEEPER_PORT=2 > > Here is the full error stack: > > java.lang.InterruptedException: sleep interrupted > at java.lang.Thread.sleep(Native Method) > at > > org.apache.accumulo.core.client.impl.ThriftTransportPool$Closer.closeConnections(ThriftTransportPool.java:137) > at > > org.apache.accumulo.core.client.impl.ThriftTransportPool$Closer.run(ThriftTransportPool.java:147) > at java.lang.Thre
Can MiniAccumuloCluster reuse directory?
When starting MiniAccumuloCluster, can I point to a directory used by a previous MiniAccumuloCluster? If not, would it be infeasible to do so?
Re: Program runs fine in NetBeans fails the shutdown properly running via Maven
David Medinets wrote: I experimented a bit more. In the waitForZooKeeperClientThreads method, I added a "thread.interrupt" call inside the "while (thread.isAlive())" loop. It seems like the thread is interrupted after a few seconds, but then is reconnected. Darn that resiliency! I've watched it reconnect four times before I killed the process. Can I reset the retry number somehow? Am I too far down the rabbit hole? You shouldn't have to interrupt the ZK threads (I'm not sure what it's doing underneath the hood when you do so). As the comment in that method says, ZooKeeper.close() is non-blocking, so that loop will finish gracefully (given ZooKeeper.close() is performing how we expect it to). java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1045) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1101) 2014-11-30 09:29:32,596 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:startConnect(1058)) - Opening socket connection to server localhost/127.0.0.1:2 2014-11-30 09:29:32,597 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(947)) - Socket connection established to localhost/127.0.0.1:2, initiating session 2014-11-30 09:29:32,599 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:readConnectResult(736)) - Session establishment complete on server localhost/127.0.0.1:2, sessionid = 0x1 On Sun, Nov 30, 2014 at 9:10 AM, David Medinets wrote: Right again. I changed the package name to "jaredwinick" and the "sealing violation" issue was resolved. But the JIRA ticket comments were correct about the code's being brittle. It has a String literal that refers to "org.apache.accumulo.core.zookeeper.ZooSession"which does not exist in Accumulo v1.6.1. I'm sure Jared is pleased with your alternative package name. Did you copy/paste the code Jared had on his Github rather than using the variant we have in the codebase (org.apache.accumulo.core.util.CleanUp)?
Re: Can MiniAccumuloCluster reuse directory?
Not sure if it already can. I believe it would be trivial to do so. David Medinets wrote: When starting MiniAccumuloCluster, can I point to a directory used by a previous MiniAccumuloCluster? If not, would it be infeasible to do so?
Re: Program runs fine in NetBeans fails the shutdown properly running via Maven
Of course I did. Using CleanUp.shutdownNow(); works perfectly from inside the "mvn exec:java" command. Excellent. On Sun, Nov 30, 2014 at 6:54 PM, Josh Elser wrote: > David Medinets wrote: >> >> I experimented a bit more. In the waitForZooKeeperClientThreads >> method, I added a "thread.interrupt" call inside the "while >> (thread.isAlive())" loop. It seems like the thread is interrupted >> after a few seconds, but then is reconnected. Darn that resiliency! >> I've watched it reconnect four times before I killed the process. Can >> I reset the retry number somehow? Am I too far down the rabbit hole? > > > You shouldn't have to interrupt the ZK threads (I'm not sure what it's doing > underneath the hood when you do so). As the comment in that method says, > ZooKeeper.close() is non-blocking, so that loop will finish gracefully > (given ZooKeeper.close() is performing how we expect it to). > > >> java.lang.InterruptedException: sleep interrupted >> at java.lang.Thread.sleep(Native Method) >> at >> org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1045) >> at >> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1101) >> 2014-11-30 09:29:32,596 INFO >> [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] >> zookeeper.ClientCnxn (ClientCnxn.java:startConnect(1058)) - Opening >> socket connection to server localhost/127.0.0.1:2 >> 2014-11-30 09:29:32,597 INFO >> [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] >> zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(947)) - Socket >> connection established to localhost/127.0.0.1:2, initiating >> session >> 2014-11-30 09:29:32,599 INFO >> [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] >> zookeeper.ClientCnxn (ClientCnxn.java:readConnectResult(736)) - >> Session establishment complete on server localhost/127.0.0.1:2, >> sessionid = 0x1 >> >> >> On Sun, Nov 30, 2014 at 9:10 AM, David Medinets >> wrote: >>> >>> Right again. I changed the package name to "jaredwinick" and the >>> "sealing violation" issue was resolved. But the JIRA ticket comments >>> were correct about the code's being brittle. It has a String literal >>> that refers to "org.apache.accumulo.core.zookeeper.ZooSession"which >>> does not exist in Accumulo v1.6.1. > > > I'm sure Jared is pleased with your alternative package name. > > Did you copy/paste the code Jared had on his Github rather than using the > variant we have in the codebase (org.apache.accumulo.core.util.CleanUp)?
Re: Program runs fine in NetBeans fails the shutdown properly running via Maven
Great. Glad it's working as expected. David Medinets wrote: Of course I did. Using CleanUp.shutdownNow(); works perfectly from inside the "mvn exec:java" command. Excellent. On Sun, Nov 30, 2014 at 6:54 PM, Josh Elser wrote: David Medinets wrote: I experimented a bit more. In the waitForZooKeeperClientThreads method, I added a "thread.interrupt" call inside the "while (thread.isAlive())" loop. It seems like the thread is interrupted after a few seconds, but then is reconnected. Darn that resiliency! I've watched it reconnect four times before I killed the process. Can I reset the retry number somehow? Am I too far down the rabbit hole? You shouldn't have to interrupt the ZK threads (I'm not sure what it's doing underneath the hood when you do so). As the comment in that method says, ZooKeeper.close() is non-blocking, so that loop will finish gracefully (given ZooKeeper.close() is performing how we expect it to). java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1045) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1101) 2014-11-30 09:29:32,596 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:startConnect(1058)) - Opening socket connection to server localhost/127.0.0.1:2 2014-11-30 09:29:32,597 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(947)) - Socket connection established to localhost/127.0.0.1:2, initiating session 2014-11-30 09:29:32,599 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:readConnectResult(736)) - Session establishment complete on server localhost/127.0.0.1:2, sessionid = 0x1 On Sun, Nov 30, 2014 at 9:10 AM, David Medinets wrote: Right again. I changed the package name to "jaredwinick" and the "sealing violation" issue was resolved. But the JIRA ticket comments were correct about the code's being brittle. It has a String literal that refers to "org.apache.accumulo.core.zookeeper.ZooSession"which does not exist in Accumulo v1.6.1. I'm sure Jared is pleased with your alternative package name. Did you copy/paste the code Jared had on his Github rather than using the variant we have in the codebase (org.apache.accumulo.core.util.CleanUp)?
Re: Can MiniAccumuloCluster reuse directory?
I had a ticket for that awhile back and I don't believe it was ever completed. By default, it wants to dump out new config files for everything- have it reusing a config file would mean not re-initializing each time and reusing the same instance id + rfiles. ACCUMULO-1378 was the it and it looks like it's still open. https://issues.apache.org/jira/browse/ACCUMULO-1378?filter=-1 On Sun, Nov 30, 2014 at 7:09 PM, Josh Elser wrote: > Not sure if it already can. I believe it would be trivial to do so. > > > David Medinets wrote: > >> When starting MiniAccumuloCluster, can I point to a directory used by >> a previous MiniAccumuloCluster? >> >> If not, would it be infeasible to do so? >> >
Re: Program runs fine in NetBeans fails the shutdown properly running via Maven
Please only use the "jaredwinick" package name for code that works perfectly. Thanks. ;-) On Sun, Nov 30, 2014 at 5:34 PM, Josh Elser wrote: > Great. Glad it's working as expected. > > > David Medinets wrote: > >> Of course I did. Using CleanUp.shutdownNow(); works perfectly from >> inside the "mvn exec:java" command. Excellent. >> >> On Sun, Nov 30, 2014 at 6:54 PM, Josh Elser wrote: >> >>> David Medinets wrote: >>> I experimented a bit more. In the waitForZooKeeperClientThreads method, I added a "thread.interrupt" call inside the "while (thread.isAlive())" loop. It seems like the thread is interrupted after a few seconds, but then is reconnected. Darn that resiliency! I've watched it reconnect four times before I killed the process. Can I reset the retry number somehow? Am I too far down the rabbit hole? >>> >>> You shouldn't have to interrupt the ZK threads (I'm not sure what it's >>> doing >>> underneath the hood when you do so). As the comment in that method says, >>> ZooKeeper.close() is non-blocking, so that loop will finish gracefully >>> (given ZooKeeper.close() is performing how we expect it to). >>> >>> >>> java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread. startConnect(ClientCnxn.java:1045) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1101) 2014-11-30 09:29:32,596 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:startConnect(1058)) - Opening socket connection to server localhost/127.0.0.1:2 2014-11-30 09:29:32,597 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(947)) - Socket connection established to localhost/127.0.0.1:2, initiating session 2014-11-30 09:29:32,599 INFO [com.codebits.jodoc.WriteAndReadDriver.main(localhost:2)] zookeeper.ClientCnxn (ClientCnxn.java:readConnectResult(736)) - Session establishment complete on server localhost/127.0.0.1:2, sessionid = 0x1 On Sun, Nov 30, 2014 at 9:10 AM, David Medinets wrote: > Right again. I changed the package name to "jaredwinick" and the > "sealing violation" issue was resolved. But the JIRA ticket comments > were correct about the code's being brittle. It has a String literal > that refers to "org.apache.accumulo.core.zookeeper.ZooSession"which > does not exist in Accumulo v1.6.1. > >>> I'm sure Jared is pleased with your alternative package name. >>> >>> Did you copy/paste the code Jared had on his Github rather than using the >>> variant we have in the codebase (org.apache.accumulo.core.util.CleanUp)? >>> >>
Re: Can MiniAccumuloCluster reuse directory?
I added a note to that ticket about the jodoc Docker image I created. It starts a MAC instance in less than 1.4 seconds. I can see use for the MAC beyond testing now. It can be easily used in education settings and for small development projects. On Sun, Nov 30, 2014 at 8:01 PM, Corey Nolet wrote: > I had a ticket for that awhile back and I don't believe it was ever > completed. By default, it wants to dump out new config files for everything- > have it reusing a config file would mean not re-initializing each time and > reusing the same instance id + rfiles. > ACCUMULO-1378 was the it and it looks like it's still open. > > https://issues.apache.org/jira/browse/ACCUMULO-1378?filter=-1 > > On Sun, Nov 30, 2014 at 7:09 PM, Josh Elser wrote: >> >> Not sure if it already can. I believe it would be trivial to do so. >> >> >> David Medinets wrote: >>> >>> When starting MiniAccumuloCluster, can I point to a directory used by >>> a previous MiniAccumuloCluster? >>> >>> If not, would it be infeasible to do so? > >
Change the Log Level in MiniAccumuloCluster be changed?
I'd like to move the log level from INFO to WARN to reduce the chattiness. Has anyone tried to connect a Monitor process to the MAC?
Re: Change the Log Level in MiniAccumuloCluster be changed?
There's at least one IT that starts a Monitor process. You should just be able to MiniAccumuloClusterImpl.exec(Monitor.class) David Medinets wrote: I'd like to move the log level from INFO to WARN to reduce the chattiness. Has anyone tried to connect a Monitor process to the MAC?
