[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117428#comment-13117428 ] dhruba borthakur commented on HBASE-4477: - I ran all unit tests. All unit tests pass, except these: TestDistributedLogSplitting, TestRollingRestart, TestHTablePool. And these failures are not related to my patch. Andrew: will you be able to review this patch, especially because it changes the coprocessor interface? Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[
https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117441#comment-13117441
]
Todd Lipcon commented on HBASE-4477:
Just a thought while we're changing the APIs: might make sense to do something
like introduce the following interface:
{code}
interface CPPutInfo {
public Put getPutObject();
public boolean getWriteToWAL();
public WALEdit getWalEdit();
}
{code}
then, if we want to extend this API with more info about the put later, we can
simply add more methods and not break compatibility? We should probably do this
in another JIRA, and for all of the RegionObserver hooks - this JIRA just gave
me the idea.
Ability for an application to store metadata into the transaction log
-
Key: HBASE-4477
URL: https://issues.apache.org/jira/browse/HBASE-4477
Project: HBase
Issue Type: Improvement
Reporter: dhruba borthakur
Assignee: dhruba borthakur
Attachments: coprocessorPut1.txt, hlogMetadata1.txt
mySQL allows an application to store an arbitrary blob along with each
transaction in its transaction logs. This JIRA is to have a similar feature
request for HBASE.
The use case is as follows: An application on one data center A stores a blob
of data along with each transaction. A replication software picks up these
blobs from the transaction logs in A and hands it to another instance of the
same application running on a remote data center B. The application in B is
responsible for applying this to the remote Hbase cluster (and also handle
conflict resolution if any).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117446#comment-13117446 ] Ted Yu commented on HBASE-4477: --- I second Todd's idea. Should CPPutInfo be created in this JIRA and we can work on other parts in another JIRA ? Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117449#comment-13117449 ] Jonathan Gray commented on HBASE-4477: -- +1 on CPPutInfo, CPGetInfo, etc... Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117450#comment-13117450 ] Jonathan Gray commented on HBASE-4477: -- And yeah, maybe introduce CPPutInfo in this JIRA and open a follow-up to change the others Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117451#comment-13117451 ] dhruba borthakur commented on HBASE-4477: - Hi Todd, I like your idea, and I will create another jira for that one. It will have to touch each and every coprocessor api. (If you are ok with it, I would like this jira to make the prePut and preGet be similar in the sense that they both take the Put/get object). Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117459#comment-13117459 ] Todd Lipcon commented on HBASE-4477: Sure, I don't particularly care if it happens as part of this JIRA or separately. Just wanted to drop my 2 cents :) Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117472#comment-13117472 ] Gary Helmling commented on HBASE-4477: -- @Todd, I like the idea of using XXXInfo objects to insulate coprocessors from API changes they're not using. @Dhruba, Agree that it would be better to tackle that in a separate JIRA and keep all of the APIs in sync. Adopting just PutInfo here would leave us with an oddly inconsistent API after this patch. @All, Can we drop the CP prefix and just use PutInfo, GetInfo, DeleteInfo, etc? It looks ugly to me and unnecessary if we place these classes under o.a.h.h.coprocessor for namespacing. Also, I like that this makes the exposed Observer APIs less brittle, but it does add a fair amount of complexity to the APIs in new classes. To carry this through, we really need a XXXInfo class per operation... Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117479#comment-13117479 ] Ted Yu commented on HBASE-4477: --- Todd didn't mention which namespace CPPutInfo should be placed. If we cannot foresee that other components would utilize PutInfo, we should place PutInfo in o.a.h.h.coprocessor without the CP prefix. Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117496#comment-13117496 ] Jonathan Gray commented on HBASE-4477: -- PutInfo seems overly generic but I agree that CPPutInfo is straight ugly. And I keep thinking it says CPUInfo. So Dhruba should just extend the API for now and we can introduce these new classes in a follow-up jira. Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117497#comment-13117497 ] Jonathan Gray commented on HBASE-4477: -- And it looks like the patch from this morning does exactly that. I'm +1 on coprocessorPut1.txt. Someone else want to review? Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, coprocessorPut2.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117500#comment-13117500 ] Andrew Purtell commented on HBASE-4477: --- In my opinion prefixing anything regarding coprocessors with CP should be an anti-pattern. Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, coprocessorPut2.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117507#comment-13117507 ] Gary Helmling commented on HBASE-4477: -- +1 on coprocessorPut2.txt, which just removes the unused familyMap local vars in RegionCoprocessorHost. I'm curious about the cluster ID question as well though. Don't know enough about the replication stuff to understand the current usage. Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, coprocessorPut2.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117562#comment-13117562 ] dhruba borthakur commented on HBASE-4477: - I'm curious if passing the cluster ID to internalPut and internalDelete is necessary? I do not think that this is necessary. We can clean them up when we merge all the parameters to the coprocessor api. Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: coprocessorPut1.txt, coprocessorPut2.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117791#comment-13117791 ] Hudson commented on HBASE-4477: --- Integrated in HBase-0.92 #31 (See [https://builds.apache.org/job/HBase-0.92/31/]) HBASE-4477 Ability for an application to store metadata into the transaction log (dhruba via jgray) jgray : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/Mutation.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverBypass.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithAbort.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithRemove.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/replication/TestMasterReplication.java Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.92.0 Attachments: coprocessorPut1.txt, coprocessorPut2.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117840#comment-13117840 ] Hudson commented on HBASE-4477: --- Integrated in HBase-TRUNK #2270 (See [https://builds.apache.org/job/HBase-TRUNK/2270/]) HBASE-4477 Ability for an application to store metadata into the transaction log (dhruba via jgray) jgray : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Mutation.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverBypass.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithAbort.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithRemove.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestMasterReplication.java Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.92.0 Attachments: coprocessorPut1.txt, coprocessorPut2.txt, hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13116675#comment-13116675 ] Andrew Purtell commented on HBASE-4477: --- @dhruba bq. can I change the signature of RegionObserver.prePut() to take in two additional arguments: a WALEdit and Put object That sounds good. Since there is no release yet, no deprecation is necessary. Can be done simply by a patch on this issue I'd say. @Jon bq. If things are built only on Coprocessor interfaces, do people see us including these in some kind of coprocessor contrib or should they be out on github or something I think it depends. Some stuff like security we'd clearly want to bundle. And by that not a resurrection of contrib, instead as another package in main/. Random additions that people build for themselves should go up on GitHub. My opinion is anything that is core to a group of use cases is a candidate for bundling, if a contributor or committer wants to maintain it, and/or if people generally feel it is a good idea to bring the candidate in. Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13116106#comment-13116106 ] dhruba borthakur commented on HBASE-4477: - @Andrew: I would like to do this via a co-processor API, can I change the signature of RegionObserver.prePut() to take in two additional arguments: a WALEdit and Put object? If so, shall I mark the existing prePut precessor api as Deprecated? anybody else have any opinions on where to put the code that implements this functionality? One place is org.apache.hadoop.hbase.coprocessor.library.walMetadata. Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13115006#comment-13115006 ] Jonathan Gray commented on HBASE-4477: -- I think this will take a few changes to the Coprocessor API if we want to only use RegionObserver and WALObserver. Andy, what should one do if additional arguments are needed to, say, the prePut() call? Do we add multiple prePut() calls to the RegionObserver interface? Potentially deprecate the old ones? If things are built only on Coprocessor interfaces, do people see us including these in some kind of coprocessor contrib or should they be out on github or something? Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[
https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13113981#comment-13113981
]
Ted Yu commented on HBASE-4477:
---
{code}
+ public static final String WAL_META_ATTRIBUTE_NAME = _hbaseWalMetadata;
{code}
Maybe the 'hbase' is not needed in the WAL_META_ATTRIBUTE_NAME.
Ability for an application to store metadata into the transaction log
-
Key: HBASE-4477
URL: https://issues.apache.org/jira/browse/HBASE-4477
Project: HBase
Issue Type: Improvement
Reporter: dhruba borthakur
Assignee: dhruba borthakur
Attachments: hlogMetadata1.txt
mySQL allows an application to store an arbitrary blob along with each
transaction in its transaction logs. This JIRA is to have a similar feature
request for HBASE.
The use case is as follows: An application on one data center A stores a blob
of data along with each transaction. A replication software picks up these
blobs from the transaction logs in A and hands it to another instance of the
same application running on a remote data center B. The application in B is
responsible for applying this to the remote Hbase cluster (and also handle
conflict resolution if any).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13114003#comment-13114003 ] Andrew Purtell commented on HBASE-4477: --- We created the WALObserver coprocessor for this use case. You can build this simply and unobtrusively with a RegionObserver to get the attributes on the Put and so something with them, like add to the KV; and then a WALObserver can take application actions according to the added metadata or otherwise filter it out so it doesn't affect core. So my question is why does this need to be in core? Is the WALObserver insufficient? Is this for = 0.90? Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13114104#comment-13114104 ] dhruba borthakur commented on HBASE-4477: - This can be implemented by WALObserver coprocessor. But do you think we should keep this feature internal to our install or do you think we should contribute it back to Apache? If we can put it somewhere in open source (not necessarily in Apache, is github preferable?), we can also open source our cluster-replication software. Ability for an application to store metadata into the transaction log - Key: HBASE-4477 URL: https://issues.apache.org/jira/browse/HBASE-4477 Project: HBase Issue Type: Improvement Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: hlogMetadata1.txt mySQL allows an application to store an arbitrary blob along with each transaction in its transaction logs. This JIRA is to have a similar feature request for HBASE. The use case is as follows: An application on one data center A stores a blob of data along with each transaction. A replication software picks up these blobs from the transaction logs in A and hands it to another instance of the same application running on a remote data center B. The application in B is responsible for applying this to the remote Hbase cluster (and also handle conflict resolution if any). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
