Hi there, 2011/7/31 Ioan Eugen Stan <[email protected]>: > Hello everybody, > > I'm back from vacation and ready to get some work done. >
Hope you enjoyed your trip .. > >> Hi Ioan, >> >> First feedback's: >> >> - Build and test run fine. >> >> - Test take time, this is due to minicluster start/stopn, but I see luster >> is already launched with a @BeforeClass. maybe some tests could be grouped >> in a TestSuite, the TestSuite being responsible to launch the minicluster. >> this would need some > > I will try to group them. I didn't do it because it works and it > wasn't a priority. Like you said not a big prio atm as far as I can tell.. > >> - HBaseSubscription: don't throw NullPointerException, but >> IllegalArgumentException >> >> - HBaseSubscription: There's a TODO move all HBase related operations into >> it's own class. Do you mean you will move SUBSCRIPTION_CF, MARKER and toPut >> to another class? Indeed, now, you create the Put in HBaseSubscription and >> Delete in HBaseSubscriptionMapper. >> >> - Everywhere: when you open a HTable, you need to close() it in a finally >> block. >> >> - HBaseMailboxManager: I would get rid of useStreaming. >> >> - HBaseMailbox: UUID already ensure you some kind of unique id. This is a >> classical way to create a unique id in nosql. >> >> - I was confused with HBaseUtils.mailboxRowKey(HBaseMailbox). You are using >> it to transform the UUID and create the needed Put, Delete..., and when you >> recreate a HBaseMailbox, you use UUIDFromRowKey. I suppose you gain place >> using the Most and Least Significant Bits. Is this the goal ? If yes, you >> can document it in javadoc and even have a special class UUIDRowKeyConverter >> to make this conversion instead of having it in HBaseUtils. I would even >> split the public final constants in a separate interfaces and have the util >> methods in their own classes. > > I agree that this will be more readable. I kept them like this because > they are small and kind of related. Another reason is that the code is > possible to evolve and things will get changed (even the UUID rowkey, > although unlikely). Once I finish with MessageMapper I will take care > of these as well. It should not be long. +1 > >> - You don't use the XML configuration previously parsed with jaxme? How are >> you going to inject the HBaseMailboxSessionMapperFactory with its >> Configuration conf. This is classically done in James via Spring. You can >> try with mailbox-spring to be sure we've got a way to do it. >> >> When you will get futher in MessageManager, you will have to pay attention >> to spring wiring and mailbox integration tests, so still much work (the >> suggested 'pencils down' date being 16/8). >> >> Good work! > > > Thanks, also for the input, Eric. I will take care of everything. I > hope we can have a meeting to sort of the details. > I also found some other things to improve (use HTablePool instead of HTable). Makes sense, as without pooling you will hit some bottleneck soon. > > The main problem that I am facing is how to stream messages to HBase > and back, taking into account big sized messages. I am thinking to > split the message in many parts and ship those parts individually in > each put, as we discussed. When retrieving the message I will use > batch operations > (http://ofps.oreilly.com/titles/9781449396107/clientapisbasics.html) > to get just a few columns at a time, so we don't run into memory > issues. > I had exactly the same problem when writing the proof of concept for the cassandra mailbox (not opensource (yet?)). I splitted the message in 1mb chunks and build an InputStream/Outputstream around the fetch operations. Maybe you get the idea from this: https://github.com/rantav/hector/tree/master/core/src/main/java/me/prettyprint/cassandra/io > I hope things will be more clear once I understand better how messages > are stored and retrieved. > Feel free to contact me/us if you have more questions.. > > Bye, > > -- > Ioan Eugen Stan > http://ieugen.blogspot.com/ Bye, Norman --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
