On 20/08/11 20:50, Ioan Eugen Stan wrote:
Hello,
Things are looking better and better. I have refactored a lot of code, mainly:
- reduced number of columns to 3 in MESSAGES table. I have plans to
get them down to 2, but it's ok for now.
- created Convertor classes for Flags and Properties, as Eric once
suggested. this also lead to the following benefit:
- HBaseUtils is much more clean. Another sweep is around the corner.
As it stands now, the test results are:
Tests run: 291, Failures: 0, Errors: 80, Skipped: 0
Well done :)
I get a lot of java.nio.BufferOverflowException and I got some other
stuff to fix in the flags area, but I think things are going well. One
thing that kept me was the fact that I was returning mssage UID's in
the wrong order. The server was expecting them in ascending order and
because I store them using Long.MAX_VALUE - uid i was getting them in
descending order. Now it's fixed. I'm using Collections.reverse() to
switch them.
Why about storing them directly in correct order?
The test runs for about 50 min. I don't know why it takes so much but
I think it has to do something with the large amount of debugging
output that's coming out of HBase.
Cluster start/stop and table enable on each test takes time.
You can mitigate it via a junit suite that start hbase cluster once
(don't forget to fix the maven-surefire-plugin in pom.xml to only
include **/*SuiteTest.java.
You will have to delete/create the table before each test (or delete
everything). See also open thread [2] on hbase mailing list to further
optimize the process.
[1]
http://svn.apache.org/repos/asf/james/server/trunk/hbase/src/test/java/org/apache/james/JamesServerHBaseSuiteTest.java
[2] http://markmail.org/message/o77ldtza4ojbyoxv
Stay tuned for more information and check out he latest version in the repo:
http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/
and:
http://code.google.com/a/apache-extras.org/p/hbase-mailbox-integration-test/
p.s. a quick code review will be appreciated.
I went through the last version, but don't have much time today...
Quick feedback:
- deleteAllMemberships and deleteAllMailboxes: you delete/recreate the
tables instead of removing the content. why not. you could centralize
the table creation code somewhere in a lazy singleton.
- HBaseUtils: I would separate the constants definitions in to a new
interface and leave the methods in the class. Also, you can use inner
interface to make it still more readable.
- the mm file should not be under src/main/resources, but under
src/site/resource for example.
Thanks,
--
Eric
http://about.echarles.net
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
