[GitHub] incubator-omid pull request #19: [OMID-84] Today, all the writes done by a t...
Github user ohadshacham commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/19#discussion_r166572983
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/HBaseTransaction.java ---
@@ -31,25 +31,31 @@
public class HBaseTransaction extends AbstractTransaction {
private static final Logger LOG =
LoggerFactory.getLogger(HBaseTransaction.class);
-public HBaseTransaction(long transactionId, long epoch,
Set writeSet, AbstractTransactionManager tm) {
-super(transactionId, epoch, writeSet, tm);
+public HBaseTransaction(long transactionId, long epoch,
Set writeSet, Set conflictFreeWriteSet,
AbstractTransactionManager tm) {
+super(transactionId, epoch, writeSet, conflictFreeWriteSet, tm);
}
-public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
AbstractTransactionManager tm) {
-super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, tm);
+public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
Set conflictFreeWriteSet, AbstractTransactionManager tm) {
+super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, conflictFreeWriteSet, tm);
}
+private void cleanCell(HBaseCellId cell) {
+Delete delete = new Delete(cell.getRow());
+delete.deleteColumn(cell.getFamily(), cell.getQualifier(),
cell.getTimestamp());
+try {
+cell.getTable().delete(delete);
+} catch (IOException e) {
+LOG.warn("Failed cleanup cell {} for Tx {}. This issue has
been ignored", cell, getTransactionId(), e);
+}
+}
@Override
public void cleanup() {
-Set writeSet = getWriteSet();
-for (final HBaseCellId cell : writeSet) {
-Delete delete = new Delete(cell.getRow());
-delete.deleteColumn(cell.getFamily(), cell.getQualifier(),
cell.getTimestamp());
-try {
-cell.getTable().delete(delete);
-} catch (IOException e) {
-LOG.warn("Failed cleanup cell {} for Tx {}. This issue has
been ignored", cell, getTransactionId(), e);
-}
+for (final HBaseCellId cell : getWriteSet()) {
+cleanCell(cell);
+}
+
+for (final HBaseCellId cell : getConflictFreeWriteSet()) {
--- End diff --
Yes, deleting is actually adding a tombstone and after a rollback these
tombstones should be discarded. Naturally, these mutation are candidates for
conflict analysis unless someone asks differently.
---
[GitHub] incubator-omid pull request #17: [OMID-83] Attributes added to Put, Get, and...
Github user ohadshacham closed the pull request at: https://github.com/apache/incubator-omid/pull/17 ---
[GitHub] incubator-omid pull request #19: [OMID-84] Today, all the writes done by a t...
Github user ohadshacham closed the pull request at: https://github.com/apache/incubator-omid/pull/19 ---
[GitHub] incubator-omid pull request #20: [OMID-85] Writing directly to HBase using s...
Github user ohadshacham closed the pull request at: https://github.com/apache/incubator-omid/pull/20 ---
[jira] [Closed] (OMID-85) Autocommit
[ https://issues.apache.org/jira/browse/OMID-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Shacham closed OMID-85. > Autocommit > -- > > Key: OMID-85 > URL: https://issues.apache.org/jira/browse/OMID-85 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > Attachments: OMID-85.v1.patch > > > Writing directly to HBase using specific version marks the write as a write > that was done by a specific transaction. However, due to lack of shadow > cells, getting the commit timestamp of the transaction can be done only by > access the commit table. The motivation of this feature is to add the shadow > cells during the write and save the commit table access. > This feature is required by Apache Phoenix that during index creation adds > the data table's entries, appeared before creation, to the index. In this > case, the version and the commit timestamp should be the fence id and > therefore, a direct write to HBase with the addition of shadow cells is > required. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OMID-85) Autocommit
[ https://issues.apache.org/jira/browse/OMID-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Shacham resolved OMID-85. -- Resolution: Fixed > Autocommit > -- > > Key: OMID-85 > URL: https://issues.apache.org/jira/browse/OMID-85 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > Attachments: OMID-85.v1.patch > > > Writing directly to HBase using specific version marks the write as a write > that was done by a specific transaction. However, due to lack of shadow > cells, getting the commit timestamp of the transaction can be done only by > access the commit table. The motivation of this feature is to add the shadow > cells during the write and save the commit table access. > This feature is required by Apache Phoenix that during index creation adds > the data table's entries, appeared before creation, to the index. In this > case, the version and the commit timestamp should be the fence id and > therefore, a direct write to HBase with the addition of shadow cells is > required. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OMID-85) Autocommit
[ https://issues.apache.org/jira/browse/OMID-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Shacham updated OMID-85: - Attachment: OMID-85.v1.patch > Autocommit > -- > > Key: OMID-85 > URL: https://issues.apache.org/jira/browse/OMID-85 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > Attachments: OMID-85.v1.patch > > > Writing directly to HBase using specific version marks the write as a write > that was done by a specific transaction. However, due to lack of shadow > cells, getting the commit timestamp of the transaction can be done only by > access the commit table. The motivation of this feature is to add the shadow > cells during the write and save the commit table access. > This feature is required by Apache Phoenix that during index creation adds > the data table's entries, appeared before creation, to the index. In this > case, the version and the commit timestamp should be the fence id and > therefore, a direct write to HBase with the addition of shadow cells is > required. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OMID-85) Autocommit
[ https://issues.apache.org/jira/browse/OMID-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355527#comment-16355527 ] ASF GitHub Bot commented on OMID-85: Github user ohadshacham closed the pull request at: https://github.com/apache/incubator-omid/pull/20 > Autocommit > -- > > Key: OMID-85 > URL: https://issues.apache.org/jira/browse/OMID-85 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > > Writing directly to HBase using specific version marks the write as a write > that was done by a specific transaction. However, due to lack of shadow > cells, getting the commit timestamp of the transaction can be done only by > access the commit table. The motivation of this feature is to add the shadow > cells during the write and save the commit table access. > This feature is required by Apache Phoenix that during index creation adds > the data table's entries, appeared before creation, to the index. In this > case, the version and the commit timestamp should be the fence id and > therefore, a direct write to HBase with the addition of shadow cells is > required. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (OMID-84) Conflict free writes
[ https://issues.apache.org/jira/browse/OMID-84?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Shacham closed OMID-84. > Conflict free writes > > > Key: OMID-84 > URL: https://issues.apache.org/jira/browse/OMID-84 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > Attachments: OMID-84.v1.patch > > > Today, all the writes done by a transaction are taking part in conflict > analysis. The purpose of this feature is to let the user decide for each > write, whether it should take part in the conflict analysis. > The motivation infers from Apache Phoenix that utilizes this feature when > writing to the secondary index and also when writing to the data table for > immutable tables (each key is added once and is not modified). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OMID-84) Conflict free writes
[ https://issues.apache.org/jira/browse/OMID-84?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Shacham resolved OMID-84. -- Resolution: Fixed > Conflict free writes > > > Key: OMID-84 > URL: https://issues.apache.org/jira/browse/OMID-84 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > Attachments: OMID-84.v1.patch > > > Today, all the writes done by a transaction are taking part in conflict > analysis. The purpose of this feature is to let the user decide for each > write, whether it should take part in the conflict analysis. > The motivation infers from Apache Phoenix that utilizes this feature when > writing to the secondary index and also when writing to the data table for > immutable tables (each key is added once and is not modified). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OMID-84) Conflict free writes
[ https://issues.apache.org/jira/browse/OMID-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355371#comment-16355371 ] ASF GitHub Bot commented on OMID-84: Github user ohadshacham closed the pull request at: https://github.com/apache/incubator-omid/pull/19 > Conflict free writes > > > Key: OMID-84 > URL: https://issues.apache.org/jira/browse/OMID-84 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > Attachments: OMID-84.v1.patch > > > Today, all the writes done by a transaction are taking part in conflict > analysis. The purpose of this feature is to let the user decide for each > write, whether it should take part in the conflict analysis. > The motivation infers from Apache Phoenix that utilizes this feature when > writing to the secondary index and also when writing to the data table for > immutable tables (each key is added once and is not modified). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (OMID-83) Propagate Attributes to HBase
[ https://issues.apache.org/jira/browse/OMID-83?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Shacham closed OMID-83. > Propagate Attributes to HBase > - > > Key: OMID-83 > URL: https://issues.apache.org/jira/browse/OMID-83 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > Attachments: OMID-83.v1.patch > > > Attributes added to Put, Get, and Scan are not propagated to HBase. In many > cases, as in the Phoenix case, these attributes are required and should be > propagated to the server side. In Phoenix for example, attributes are used to > mark data as one that should propagate to the secondary index. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OMID-84) Conflict free writes
[ https://issues.apache.org/jira/browse/OMID-84?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Shacham updated OMID-84: - Attachment: OMID-84.v1.patch > Conflict free writes > > > Key: OMID-84 > URL: https://issues.apache.org/jira/browse/OMID-84 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > Attachments: OMID-84.v1.patch > > > Today, all the writes done by a transaction are taking part in conflict > analysis. The purpose of this feature is to let the user decide for each > write, whether it should take part in the conflict analysis. > The motivation infers from Apache Phoenix that utilizes this feature when > writing to the secondary index and also when writing to the data table for > immutable tables (each key is added once and is not modified). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OMID-83) Propagate Attributes to HBase
[ https://issues.apache.org/jira/browse/OMID-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355308#comment-16355308 ] ASF GitHub Bot commented on OMID-83: Github user ohadshacham closed the pull request at: https://github.com/apache/incubator-omid/pull/17 > Propagate Attributes to HBase > - > > Key: OMID-83 > URL: https://issues.apache.org/jira/browse/OMID-83 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > Attachments: OMID-83.v1.patch > > > Attributes added to Put, Get, and Scan are not propagated to HBase. In many > cases, as in the Phoenix case, these attributes are required and should be > propagated to the server side. In Phoenix for example, attributes are used to > mark data as one that should propagate to the secondary index. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OMID-83) Propagate Attributes to HBase
[ https://issues.apache.org/jira/browse/OMID-83?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Shacham resolved OMID-83. -- Resolution: Fixed > Propagate Attributes to HBase > - > > Key: OMID-83 > URL: https://issues.apache.org/jira/browse/OMID-83 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > Attachments: OMID-83.v1.patch > > > Attributes added to Put, Get, and Scan are not propagated to HBase. In many > cases, as in the Phoenix case, these attributes are required and should be > propagated to the server side. In Phoenix for example, attributes are used to > mark data as one that should propagate to the secondary index. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OMID-84) Conflict free writes
[
https://issues.apache.org/jira/browse/OMID-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355268#comment-16355268
]
ASF GitHub Bot commented on OMID-84:
Github user ohadshacham commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/19#discussion_r166572983
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/HBaseTransaction.java ---
@@ -31,25 +31,31 @@
public class HBaseTransaction extends AbstractTransaction {
private static final Logger LOG =
LoggerFactory.getLogger(HBaseTransaction.class);
-public HBaseTransaction(long transactionId, long epoch,
Set writeSet, AbstractTransactionManager tm) {
-super(transactionId, epoch, writeSet, tm);
+public HBaseTransaction(long transactionId, long epoch,
Set writeSet, Set conflictFreeWriteSet,
AbstractTransactionManager tm) {
+super(transactionId, epoch, writeSet, conflictFreeWriteSet, tm);
}
-public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
AbstractTransactionManager tm) {
-super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, tm);
+public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
Set conflictFreeWriteSet, AbstractTransactionManager tm) {
+super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, conflictFreeWriteSet, tm);
}
+private void cleanCell(HBaseCellId cell) {
+Delete delete = new Delete(cell.getRow());
+delete.deleteColumn(cell.getFamily(), cell.getQualifier(),
cell.getTimestamp());
+try {
+cell.getTable().delete(delete);
+} catch (IOException e) {
+LOG.warn("Failed cleanup cell {} for Tx {}. This issue has
been ignored", cell, getTransactionId(), e);
+}
+}
@Override
public void cleanup() {
-Set writeSet = getWriteSet();
-for (final HBaseCellId cell : writeSet) {
-Delete delete = new Delete(cell.getRow());
-delete.deleteColumn(cell.getFamily(), cell.getQualifier(),
cell.getTimestamp());
-try {
-cell.getTable().delete(delete);
-} catch (IOException e) {
-LOG.warn("Failed cleanup cell {} for Tx {}. This issue has
been ignored", cell, getTransactionId(), e);
-}
+for (final HBaseCellId cell : getWriteSet()) {
+cleanCell(cell);
+}
+
+for (final HBaseCellId cell : getConflictFreeWriteSet()) {
--- End diff --
Yes, deleting is actually adding a tombstone and after a rollback these
tombstones should be discarded. Naturally, these mutation are candidates for
conflict analysis unless someone asks differently.
> Conflict free writes
>
>
> Key: OMID-84
> URL: https://issues.apache.org/jira/browse/OMID-84
> Project: Apache Omid
> Issue Type: Sub-task
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>Priority: Major
>
> Today, all the writes done by a transaction are taking part in conflict
> analysis. The purpose of this feature is to let the user decide for each
> write, whether it should take part in the conflict analysis.
> The motivation infers from Apache Phoenix that utilizes this feature when
> writing to the secondary index and also when writing to the data table for
> immutable tables (each key is added once and is not modified).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OMID-83) Propagate Attributes to HBase
[ https://issues.apache.org/jira/browse/OMID-83?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Shacham updated OMID-83: - Attachment: OMID-83.v1.patch > Propagate Attributes to HBase > - > > Key: OMID-83 > URL: https://issues.apache.org/jira/browse/OMID-83 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > Attachments: OMID-83.v1.patch > > > Attributes added to Put, Get, and Scan are not propagated to HBase. In many > cases, as in the Phoenix case, these attributes are required and should be > propagated to the server side. In Phoenix for example, attributes are used to > mark data as one that should propagate to the secondary index. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
