Hi, Thanks a lot. Now I understand better how it works. The loop was here to make this error to happen quicker.
Julien Ruchaud Le Fri, 9 Oct 2009 14:16:07 -0400, Jean-Daniel Cryans <[email protected]> a écrit : > Julien, > > Closing the HTable itself doesn't close the connections, it just > flushes anything that may still be in the write buffer (isn't your > case). To close all connections, call > HConnectionManager.deleteAllConnections(true). > > A client connection to Zookeeper is made by the Master, the > RegionServer, the HBase clients and in version 0.20.0 by the > RegionHistorian. Some of those even carry more than 1 connection so 5 > connections is too low. > > WRT the "Exception 2" thing, I tried and got it some times. This is an > expected behavior, at least it's by design. If you delete and put too > fast, you may be able to have both operations at the same timestamp in > which case the delete wins... so your row appears to be deleted. If > you need a higher level of precision, you have to provide your own > timestamps or put a sleep of some milliseconds after the deletes. > > Also, in your code you delete 10 times the same row. Maybe you already > know that, but deleting the row key deletes the whole row with all the > data you inserted so you only need to delete once. > > J-D > > On Fri, Oct 9, 2009 at 1:06 PM, Julien Ruchaud > <[email protected]> wrote: > > I am continue test, I have set > > base.zookeeper.property.maxClientCnxns = 5 and close htable each > > get, put and delete, I have got the same error. is it normal ? > > > > Julien Ruchaud > > > > Le Fri, 9 Oct 2009 17:48:37 +0200, > > Julien Ruchaud <[email protected]> a écrit : > > > >> J-D, > >> > >> Yes, it is the only test. I am downloading here > >> http://mir2.ovh.net/ftp.apache.org/dist/hadoop/hbase/ the 0.20.0 > >> release. > >> > >> Julien > >> > >> Le Fri, 9 Oct 2009 11:47:45 -0400, > >> Jean-Daniel Cryans <[email protected]> a écrit : > >> > >> > Julien, > >> > > >> > Is it the only test you are running? If so, you shouldn't have to > >> > close after each operation but only at the end of the test. Are > >> > you sure this is the official 0.20.0 release? > >> > > >> > J-D > >> > > >> > On Fri, Oct 9, 2009 at 11:35 AM, Julien Ruchaud > >> > <[email protected]> wrote: > >> > > Hi, > >> > > > >> > > If hbase.zookeeper.property.maxClientCnxns increase to 300, the > >> > > test pass. So I test the second solution to close connection > >> > > each put, get and delete, the test pass too. How do you know > >> > > when to close connection ? each get, delete or put ? is it > >> > > better to instanciate a new HTable each time ? > >> > > > >> > > Thanks > >> > > > >> > > Julien Ruchaud > >> > > > >> > > Le Fri, 9 Oct 2009 11:07:55 -0400, > >> > > Jean-Daniel Cryans <[email protected]> a écrit : > >> > > > >> > >> Julien, > >> > >> > >> > >> This is a Zookeeper error that can be related to the number of > >> > >> connections a ZK server can handle. To verify, set > >> > >> hbase.zookeeper.property.maxClientCnxns to something a lot > >> > >> bigger than the default 30 and restart HBase. > >> > >> > >> > >> This could happen because you run a lot of unit tests > >> > >> sequentially without closing the connections. > >> > >> > >> > >> J-D > >> > >> > >> > >> On Fri, Oct 9, 2009 at 10:53 AM, Julien Ruchaud > >> > >> <[email protected]> wrote: > >> > >> > Hi, > >> > >> > > >> > >> > I am a newbie on hbase. Currently I am testing hbase 0.20.0, > >> > >> > but I don't understand why my test don't run correctly > >> > >> > sometimes : > >> > >> > > >> > >> > public void testLoop() throws Exception { > >> > >> > HTable hTable = new HTable(new HBaseConfiguration(), > >> > >> > Bytes.toBytes("Table")); byte[] rowId = > >> > >> > Bytes.toBytes("Test"); > >> > >> > > >> > >> > for (int j = 0; j < 10; j++) { > >> > >> > Put put = new Put(rowId); > >> > >> > put.add(Bytes.toBytes("data"), > >> > >> > Bytes.toBytes("1:" + j), Bytes.toBytes(j)); hTable.put(put); > >> > >> > } > >> > >> > > >> > >> > for (int j = 0; j < 10; j++) { > >> > >> > Get get = new Get(rowId); > >> > >> > Result row = hTable.get(get); > >> > >> > if(row.getValue(Bytes.toBytes("data"), > >> > >> > Bytes.toBytes("1:" + j)) == null) { throw new > >> > >> > Exception("Error 1"); } > >> > >> > } > >> > >> > > >> > >> > for (int j = 0; j < 10; j++) { > >> > >> > Delete delete = new Delete(rowId); > >> > >> > hTable.delete(delete); > >> > >> > } > >> > >> > > >> > >> > for (int j = 0; j < 10; j++) { > >> > >> > Put put = new Put(rowId); > >> > >> > put.add(Bytes.toBytes("data"), > >> > >> > Bytes.toBytes("2:" + j), Bytes.toBytes(j)); hTable.put(put); > >> > >> > } > >> > >> > > >> > >> > for (int j = 0; j < 10; j++) { > >> > >> > Get get = new Get(rowId); > >> > >> > Result row = hTable.get(get); > >> > >> > if(row.getValue(Bytes.toBytes("data"), > >> > >> > Bytes.toBytes("2:" + j)) == null) { throw new > >> > >> > Exception("Error 2"); } > >> > >> > } > >> > >> > } > >> > >> > > >> > >> > In fact I have got exception "Error 2" with a exception in > >> > >> > hbase log : > >> > >> > > >> > >> > 2009-10-09 16:39:34,829 WARN > >> > >> > org.apache.zookeeper.server.NIOServerCnxn: Exception causing > >> > >> > close of session 0x124393fc5390046 due to > >> > >> > java.io.IOException: Read error 2009-10-09 16:39:34,829 > >> > >> > DEBUG org.apache.zookeeper.server.NIOServerCnxn: > >> > >> > IOException stack trace java.io.IOException: Read error at > >> > >> > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:366) > >> > >> > at > >> > >> > org.apache.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:220) > >> > >> > 2009-10-09 16:39:34,830 INFO > >> > >> > org.apache.zookeeper.server.NIOServerCnxn: closing > >> > >> > session:0x124393fc5390046 NIOServerCnxn: > >> > >> > java.nio.channels.SocketChannel[connected > >> > >> > local=/127.0.0.1:2181 remote=/127.0.0.1:48870] 2009-10-09 > >> > >> > 16:39:34,830 DEBUG > >> > >> > org.apache.zookeeper.server.NIOServerCnxn: ignoring > >> > >> > exception during input shutdown java.net.SocketException: > >> > >> > Transport endpoint is not connected at > >> > >> > sun.nio.ch.SocketChannelImpl.shutdown(Native Method) at > >> > >> > sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640) > >> > >> > at > >> > >> > sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360) > >> > >> > at > >> > >> > org.apache.zookeeper.server.NIOServerCnxn.close(NIOServerCnxn.java:841) > >> > >> > at > >> > >> > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:492) > >> > >> > at > >> > >> > org.apache.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:220) > >> > >> > > >> > >> > Thanks in advance for your input > >> > >> > > >> > >> > Julien Ruchaud > >> > >> > > >> > > > >
