> Instead of a custom cell, could you use a regular cell with a custom > operation attribute (see OperationWithAttributes). Pardon me, I didn't get what you said.
On 2017-09-30 04:31, Andrew Purtell <apurt...@apache.org> wrote: > Instead of a custom cell, could you use a regular cell with a custom > operation attribute (see OperationWithAttributes). > > On Fri, Sep 29, 2017 at 1:28 PM, Chia-Ping Tsai <chia7...@apache.org> wrote: > > > The custom cell help us to save memory consumption. We don't have own > > serialization/deserialization mechanism, hence to transform data from > > client to server needs many conversion phase (user data -> Put/Cell -> pb > > object). The cost of conversion is large in transferring bulk data. In > > fact, we also have custom mutation to manage the memory usage of inner cell > > collection. > > > > On 2017-09-30 02:43, Andrew Purtell <apurt...@apache.org> wrote: > > > What are the use cases for a custom cell? It seems a dangerously low > > level > > > thing to attempt and perhaps we should unwind support for it. But perhaps > > > there is a compelling justification. > > > > > > > > > On Thu, Sep 28, 2017 at 10:20 PM, Chia-Ping Tsai <chia7...@apache.org> > > > wrote: > > > > > > > Thanks for all comment. > > > > > > > > The problem i want to resolve is the valid code should be exposed as > > > > IA.Public. Otherwise, end user have to access the IA.Private class to > > build > > > > the custom cell. > > > > > > > > For example, I have a use case which plays a streaming role in our > > > > appliaction. It > > > > applies the CellBuilder(HBASE-18519) to build custom cells. These cells > > > > have many same fields so they are put in shared-memory for avoiding GC > > > > pause. Everything is wonderful. However, we have to access the > > IA.Private > > > > class - KeyValue#Type - to get the valid code of Put. > > > > > > > > I believe there are many use cases of custom cell, and consequently it > > is > > > > worth adding a way to get the valid type via IA.Public class. > > Otherwise, it > > > > may imply that the custom cell is based on a unstable way, because the > > > > related code can be changed at any time. > > > > -- > > > > Chia-Ping > > > > > > > > On 2017-09-29 00:49, Andrew Purtell <apurt...@apache.org> wrote: > > > > > I agree with Stack. Was typing up a reply to Anoop but let me move it > > > > down > > > > > here. > > > > > > > > > > The type code exposes some low level details of how our current > > stores > > > > are > > > > > architected. But what if in the future you could swap out HStore > > > > implements > > > > > Store with PStore implements Store, where HStore is backed by HFiles > > and > > > > > PStore is backed by Parquet? Just as a hypothetical example. I know > > there > > > > > would be larger issues if this were actually attempted. Bear with > > me. You > > > > > can imagine some different new Store implementation that has some > > > > > advantages but is not a design derived from the log structured merge > > tree > > > > > if you like. Most values from a new Cell.Type based on KeyValue.Type > > > > > wouldn't apply to cells from such a thing because they are > > particular to > > > > > how LSMs work. I'm sure such a project if attempted would make a > > number > > > > of > > > > > changes requiring a major version increment and low level details > > could > > > > be > > > > > unwound from Cell then, but if we could avoid doing it in the first > > > > place, > > > > > I think it would better for maintainability. > > > > > > > > > > > > > > > On Thu, Sep 28, 2017 at 9:39 AM, Stack <st...@duboce.net> wrote: > > > > > > > > > > > On Thu, Sep 28, 2017 at 2:25 AM, Chia-Ping Tsai < > > chia7...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > hi folks, > > > > > > > > > > > > > > User is allowed to create custom cell but the valid code of type > > - > > > > > > > KeyValue#Type - is declared as IA.Private. As i see it, we should > > > > expose > > > > > > > KeyValue#Type as Public Client. Three possible ways are shown > > below: > > > > > > > 1) Change declaration of KeyValue#Type from IA.Private to > > IA.Public > > > > > > > 2) Move KeyValue#Type into Cell. > > > > > > > 3) Move KeyValue#Type to upper level > > > > > > > > > > > > > > Any suggestions? > > > > > > > > > > > > > > > > > > > > What is the problem that we are trying to solve Chia-Ping? You > > want to > > > > make > > > > > > Cells of a new Type? > > > > > > > > > > > > My first reaction is that KV#Type is particular to the KV > > > > implementation. > > > > > > Any new Cell implementation should not have to adopt the KeyValue > > > > typing > > > > > > mechanism. > > > > > > > > > > > > S > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Chia-Ping > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > Andrew > > > > > > > > > > Words like orphans lost among the crosstalk, meaning torn from > > truth's > > > > > decrepit hands > > > > > - A23, Crosstalk > > > > > > > > > > > > > > > > > > > > > -- > > > Best regards, > > > Andrew > > > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > > decrepit hands > > > - A23, Crosstalk > > > > > > > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands > - A23, Crosstalk >
