I'd rather do that one layer up as I proposed in the PhoenixWriterCommiter - it helps separate concerns for cases where users want to use the indexing but don't want all the rest of the phoenix stuff.
------------------- Jesse Yates @jesse_yates jyates.github.com On Wed, Mar 5, 2014 at 6:09 PM, James Taylor (JIRA) <j...@apache.org> wrote: > > [ > https://issues.apache.org/jira/browse/PHOENIX-113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13921900#comment-13921900] > > James Taylor commented on PHOENIX-113: > -------------------------------------- > > My proposal for 3.0 is to just go through a new KeyValueBuilder that > potentially just makes a copy of the Mutation with a real KeyValue in the > case of ClientKeyValue. In ParallelWriterIndexCommitter:114: > {code} > final List<Mutation> mutations = (List<Mutation>) entry.getValue(); > // Pass the list through to the KeyValueBuilder to potentially make > copies of them, as > // the memstore requires the KeyValue to have a backing buffer > mutations = kvBuilder.copyMutationsIfNecessary(mutations); > {code} > > Seems worth the effort, given that we've done all this work for > KeyValueBuilder and we're getting very little benefit from it. > > > Enable usage of ClientKeyValue on for indexing on server > > -------------------------------------------------------- > > > > Key: PHOENIX-113 > > URL: https://issues.apache.org/jira/browse/PHOENIX-113 > > Project: Phoenix > > Issue Type: Bug > > Affects Versions: 3.0.0 > > Reporter: James Taylor > > Fix For: 3.0.0 > > > > Attachments: phoenix-113-sketch.txt > > > > > > Modify code to go through KeyValueBuilder where necessary. Adding > abstraction for having different KVComparator on each KeyValueBuilder impl. > > Still doesn't work on the server-side due to memstore needing a backing > buffer for the KeyValue. > > According to [~jesse_yates], the options are: > > 1. Update the ClientKeyValues on the way out via a new hook in the > codec, e.g. "preWriteToIndexTable(List<Mutation>, byte[] indexTable)", > > Wrap all the clientkvs in something like an ImmutableClientKeyValue that > does the right thing for getBuffer, getOffset, etc, but without doing all > the copying. This may be easier to just do as modifications to > ClientKeyValue, but seems easier to start as a new class first > > 2. Automatically check all the mutations being written in the > ParallelIndexWriter for being clientKeyvalues > > A bit more painful than the above, since you could short circuit there, > but easier in terms of code complexity > > 3. Check to see if the write is going to a local region and then > transform the ClientKeyValues when necessary > > Most difficult as it reaches down into the guts of the coprocessor logic > for determining region location. Unfortunately, it is already being done by > the CoprocessorHConnection, but we would need to recreate it for us before > it hits the HTable. > > You could do it by creating a CoprocessorHConnection and then wrapping > that in a ClientKeyValueCheckingHConnection and then passing that into the > constructor for HTable creation. > > This is even more onerous as that constructor is only available on an > HTable directly, not via the CoprocessorEnvironment, meaning we have to > manually close tables. We have pretty good exception wrapping, so its not > terrible to implement, but still a pain to do. > > > > -- > This message was sent by Atlassian JIRA > (v6.2#6252) >
