Re: Cell Encoders and usage of Cell

2013-04-23 Thread ramkrishna vasudevan
Until then, you are left w/ some sort of hack on KV. Yes. If all was Cell on the read/write path, I'd imagine it'd be easy enough getting your tags in the mix. Am checking on this also. May take some time till i finalise on the changes. Will discuss with Andy offline and get back on some

Re: Cell Encoders and usage of Cell

2013-04-22 Thread Stack
On Thu, Apr 18, 2013 at 8:19 PM, ramkrishna vasudevan ramkrishna.s.vasude...@gmail.com wrote: My questions were mainly because, if i have the current code and i would want to introduce tags in it, where would i do it? Good question. If the read/write path currently does not support

Re: Cell Encoders and usage of Cell

2013-04-22 Thread Andrew Purtell
On HBASE-7544 I added encryption to HFile starting with the encoder and decoder contexts and working outward, so I can attest it is possible to add features that wrap *around* KV serialization without changing the KV serialization itself. On trunk the block encoder stuff has been at least

Re: Cell Encoders and usage of Cell

2013-04-21 Thread Matt Corgan
fyi Ram - i started adding the Cell interface to the read path of the delta encoders in HBASE-7323 https://issues.apache.org/jira/browse/HBASE-7323. It's one possible place to start working on it. On Thu, Apr 18, 2013 at 8:19 PM, ramkrishna vasudevan ramkrishna.s.vasude...@gmail.com wrote:

Re: Cell Encoders and usage of Cell

2013-04-21 Thread Nick Dimiduk
A related question. Can you clarify the distinction between a Cell and a KeyValue as pertains to the timestamp? That is, which of these two concepts carries the timestamp as a component of its coordinates? Does a Cell contain multiple KeyValue versions or does a KeyValue contain multiple Cell

Re: Cell Encoders and usage of Cell

2013-04-21 Thread ramkrishna vasudevan
Just adding to what Matt said, Cell and KeyValue are the same. Just that in cell you have individual byte arrays carrying the Row, family, qualifier, Type and timestamp. So it is basically saving us from the internal size it occupies. Also it helps us to use the same interface between the RPC

Re: Cell Encoders and usage of Cell

2013-04-18 Thread ramkrishna vasudevan
Ping? Could someone clarify on this pls? Regards Ram On Wed, Apr 17, 2013 at 10:46 PM, ramkrishna vasudevan ramkrishna.s.vasude...@gmail.com wrote: Hi With the introduction of the new Cell Interface we are providing a way where both the RPC usage of cell and the usage of Cell in HFile are

Re: Cell Encoders and usage of Cell

2013-04-18 Thread Stack
On Wed, Apr 17, 2013 at 10:16 AM, ramkrishna vasudevan ramkrishna.s.vasude...@gmail.com wrote: Hi With the introduction of the new Cell Interface we are providing a way where both the RPC usage of cell and the usage of Cell in HFile are unified.(abstracted) The current block encoder which

Re: Cell Encoders and usage of Cell

2013-04-18 Thread ramkrishna vasudevan
Thanks for your reply Stack. I think so. hfile APIs are about KVs. Should be about Cell I'd think. Yes. This is what i too think. If you need the above, you are no doing Cell right I'd argue. The very idea of Cell is a disconnect between how it is stored and Cell use. Yes Stack. I

Cell Encoders and usage of Cell

2013-04-17 Thread ramkrishna vasudevan
Hi With the introduction of the new Cell Interface we are providing a way where both the RPC usage of cell and the usage of Cell in HFile are unified.(abstracted) The current block encoder which encodes the kvs into hfile blocks will be enhanced may be BlockEncode2 which will deal with Cell