[GitHub] incubator-omid pull request #20: [OMID-85] Writing directly to HBase using s...
Github user ebortnikov commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/20#discussion_r166407100
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/TTable.java ---
@@ -293,15 +293,11 @@ public void delete(Transaction tx, Delete delete)
throws IOException {
}
-/**
- * Transactional version of {@link HTableInterface#put(Put put)}
- *
- * @param put an instance of Put
- * @param tx an instance of transaction to be used
- * @throws IOException if a remote or network exception occurs.
- */
-public void put(Transaction tx, Put put) throws IOException {
+interface UpdateMetaData {
--- End diff --
What is the scope of this interface? Why is it package-level?
---
[GitHub] incubator-omid pull request #19: [OMID-84] Today, all the writes done by a t...
Github user ebortnikov commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/19#discussion_r166405171
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/HBaseTransaction.java ---
@@ -31,25 +31,31 @@
public class HBaseTransaction extends AbstractTransaction {
private static final Logger LOG =
LoggerFactory.getLogger(HBaseTransaction.class);
-public HBaseTransaction(long transactionId, long epoch,
Set writeSet, AbstractTransactionManager tm) {
-super(transactionId, epoch, writeSet, tm);
+public HBaseTransaction(long transactionId, long epoch,
Set writeSet, Set conflictFreeWriteSet,
AbstractTransactionManager tm) {
+super(transactionId, epoch, writeSet, conflictFreeWriteSet, tm);
}
-public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
AbstractTransactionManager tm) {
-super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, tm);
+public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
Set conflictFreeWriteSet, AbstractTransactionManager tm) {
+super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, conflictFreeWriteSet, tm);
}
+private void cleanCell(HBaseCellId cell) {
+Delete delete = new Delete(cell.getRow());
+delete.deleteColumn(cell.getFamily(), cell.getQualifier(),
cell.getTimestamp());
+try {
+cell.getTable().delete(delete);
+} catch (IOException e) {
+LOG.warn("Failed cleanup cell {} for Tx {}. This issue has
been ignored", cell, getTransactionId(), e);
+}
+}
@Override
public void cleanup() {
-Set writeSet = getWriteSet();
-for (final HBaseCellId cell : writeSet) {
-Delete delete = new Delete(cell.getRow());
-delete.deleteColumn(cell.getFamily(), cell.getQualifier(),
cell.getTimestamp());
-try {
-cell.getTable().delete(delete);
-} catch (IOException e) {
-LOG.warn("Failed cleanup cell {} for Tx {}. This issue has
been ignored", cell, getTransactionId(), e);
-}
+for (final HBaseCellId cell : getWriteSet()) {
+cleanCell(cell);
+}
+
+for (final HBaseCellId cell : getConflictFreeWriteSet()) {
--- End diff --
Are conflict free writes applicable to Delete?
---
[GitHub] incubator-omid pull request #20: [OMID-85] Writing directly to HBase using s...
Github user ebortnikov commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/20#discussion_r166408476
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/TTable.java ---
@@ -321,16 +317,51 @@ public void put(Transaction tx, Put put) throws
IOException {
Bytes.putLong(kv.getValueArray(), kv.getTimestampOffset(),
writeTimestamp);
tsput.add(kv);
-transaction.addWriteSetElement(
+updateMetaData.update(transaction, kv, tsput);
+}
+}
+
+table.put(tsput);
+}
+
+/**
+ * putWithAutocommit implementation. Similar to a transactional put
that includes the commit time stamp.
+ *
+ * @param put an instance of Put
+ * @param tx an instance of transaction to be used
+ * @throws IOException if a remote or network exception occurs.
+ */
+public void putWithAutocommit(Transaction tx, Put put) throws
IOException {
--- End diff --
Not sure it's a good idea to have a separate method for this. Is it painful
to add a parameter? Alternatively, maybe you could pass an attribute to
propagate (mechanism defined in OMID-83)?
---
[GitHub] incubator-omid pull request #19: [OMID-84] Today, all the writes done by a t...
Github user ebortnikov commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/19#discussion_r166405079
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/HBaseTransaction.java ---
@@ -31,25 +31,31 @@
public class HBaseTransaction extends AbstractTransaction {
private static final Logger LOG =
LoggerFactory.getLogger(HBaseTransaction.class);
-public HBaseTransaction(long transactionId, long epoch,
Set writeSet, AbstractTransactionManager tm) {
-super(transactionId, epoch, writeSet, tm);
+public HBaseTransaction(long transactionId, long epoch,
Set writeSet, Set conflictFreeWriteSet,
AbstractTransactionManager tm) {
+super(transactionId, epoch, writeSet, conflictFreeWriteSet, tm);
}
-public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
AbstractTransactionManager tm) {
-super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, tm);
+public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
Set conflictFreeWriteSet, AbstractTransactionManager tm) {
+super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, conflictFreeWriteSet, tm);
}
+private void cleanCell(HBaseCellId cell) {
--- End diff --
cleanCell --> deleteCell?
---
[GitHub] incubator-omid pull request #17: [OMID-83] Attributes added to Put, Get, and...
Github user ebortnikov commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/17#discussion_r166402848
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/TTable.java ---
@@ -186,6 +187,14 @@ public Result get(Transaction tx, final Get get)
throws IOException {
return snapshotFilter.get(this, tsget, transaction);
}
+private void updateAttributes(OperationWithAttributes from,
OperationWithAttributes to) {
--- End diff --
makes sense to call propagateAttributes()?
---
[jira] [Commented] (OMID-85) Autocommit
[
https://issues.apache.org/jira/browse/OMID-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354365#comment-16354365
]
ASF GitHub Bot commented on OMID-85:
Github user ebortnikov commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/20#discussion_r166407100
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/TTable.java ---
@@ -293,15 +293,11 @@ public void delete(Transaction tx, Delete delete)
throws IOException {
}
-/**
- * Transactional version of {@link HTableInterface#put(Put put)}
- *
- * @param put an instance of Put
- * @param tx an instance of transaction to be used
- * @throws IOException if a remote or network exception occurs.
- */
-public void put(Transaction tx, Put put) throws IOException {
+interface UpdateMetaData {
--- End diff --
What is the scope of this interface? Why is it package-level?
> Autocommit
> --
>
> Key: OMID-85
> URL: https://issues.apache.org/jira/browse/OMID-85
> Project: Apache Omid
> Issue Type: Sub-task
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>Priority: Major
>
> Writing directly to HBase using specific version marks the write as a write
> that was done by a specific transaction. However, due to lack of shadow
> cells, getting the commit timestamp of the transaction can be done only by
> access the commit table. The motivation of this feature is to add the shadow
> cells during the write and save the commit table access.
> This feature is required by Apache Phoenix that during index creation adds
> the data table's entries, appeared before creation, to the index. In this
> case, the version and the commit timestamp should be the fence id and
> therefore, a direct write to HBase with the addition of shadow cells is
> required.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OMID-85) Autocommit
[
https://issues.apache.org/jira/browse/OMID-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354364#comment-16354364
]
ASF GitHub Bot commented on OMID-85:
Github user ebortnikov commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/20#discussion_r166408476
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/TTable.java ---
@@ -321,16 +317,51 @@ public void put(Transaction tx, Put put) throws
IOException {
Bytes.putLong(kv.getValueArray(), kv.getTimestampOffset(),
writeTimestamp);
tsput.add(kv);
-transaction.addWriteSetElement(
+updateMetaData.update(transaction, kv, tsput);
+}
+}
+
+table.put(tsput);
+}
+
+/**
+ * putWithAutocommit implementation. Similar to a transactional put
that includes the commit time stamp.
+ *
+ * @param put an instance of Put
+ * @param tx an instance of transaction to be used
+ * @throws IOException if a remote or network exception occurs.
+ */
+public void putWithAutocommit(Transaction tx, Put put) throws
IOException {
--- End diff --
Not sure it's a good idea to have a separate method for this. Is it painful
to add a parameter? Alternatively, maybe you could pass an attribute to
propagate (mechanism defined in OMID-83)?
> Autocommit
> --
>
> Key: OMID-85
> URL: https://issues.apache.org/jira/browse/OMID-85
> Project: Apache Omid
> Issue Type: Sub-task
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>Priority: Major
>
> Writing directly to HBase using specific version marks the write as a write
> that was done by a specific transaction. However, due to lack of shadow
> cells, getting the commit timestamp of the transaction can be done only by
> access the commit table. The motivation of this feature is to add the shadow
> cells during the write and save the commit table access.
> This feature is required by Apache Phoenix that during index creation adds
> the data table's entries, appeared before creation, to the index. In this
> case, the version and the commit timestamp should be the fence id and
> therefore, a direct write to HBase with the addition of shadow cells is
> required.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OMID-84) Conflict free writes
[
https://issues.apache.org/jira/browse/OMID-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354343#comment-16354343
]
ASF GitHub Bot commented on OMID-84:
Github user ebortnikov commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/19#discussion_r166405171
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/HBaseTransaction.java ---
@@ -31,25 +31,31 @@
public class HBaseTransaction extends AbstractTransaction {
private static final Logger LOG =
LoggerFactory.getLogger(HBaseTransaction.class);
-public HBaseTransaction(long transactionId, long epoch,
Set writeSet, AbstractTransactionManager tm) {
-super(transactionId, epoch, writeSet, tm);
+public HBaseTransaction(long transactionId, long epoch,
Set writeSet, Set conflictFreeWriteSet,
AbstractTransactionManager tm) {
+super(transactionId, epoch, writeSet, conflictFreeWriteSet, tm);
}
-public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
AbstractTransactionManager tm) {
-super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, tm);
+public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
Set conflictFreeWriteSet, AbstractTransactionManager tm) {
+super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, conflictFreeWriteSet, tm);
}
+private void cleanCell(HBaseCellId cell) {
+Delete delete = new Delete(cell.getRow());
+delete.deleteColumn(cell.getFamily(), cell.getQualifier(),
cell.getTimestamp());
+try {
+cell.getTable().delete(delete);
+} catch (IOException e) {
+LOG.warn("Failed cleanup cell {} for Tx {}. This issue has
been ignored", cell, getTransactionId(), e);
+}
+}
@Override
public void cleanup() {
-Set writeSet = getWriteSet();
-for (final HBaseCellId cell : writeSet) {
-Delete delete = new Delete(cell.getRow());
-delete.deleteColumn(cell.getFamily(), cell.getQualifier(),
cell.getTimestamp());
-try {
-cell.getTable().delete(delete);
-} catch (IOException e) {
-LOG.warn("Failed cleanup cell {} for Tx {}. This issue has
been ignored", cell, getTransactionId(), e);
-}
+for (final HBaseCellId cell : getWriteSet()) {
+cleanCell(cell);
+}
+
+for (final HBaseCellId cell : getConflictFreeWriteSet()) {
--- End diff --
Are conflict free writes applicable to Delete?
> Conflict free writes
>
>
> Key: OMID-84
> URL: https://issues.apache.org/jira/browse/OMID-84
> Project: Apache Omid
> Issue Type: Sub-task
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>Priority: Major
>
> Today, all the writes done by a transaction are taking part in conflict
> analysis. The purpose of this feature is to let the user decide for each
> write, whether it should take part in the conflict analysis.
> The motivation infers from Apache Phoenix that utilizes this feature when
> writing to the secondary index and also when writing to the data table for
> immutable tables (each key is added once and is not modified).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OMID-84) Conflict free writes
[
https://issues.apache.org/jira/browse/OMID-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354342#comment-16354342
]
ASF GitHub Bot commented on OMID-84:
Github user ebortnikov commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/19#discussion_r166405079
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/HBaseTransaction.java ---
@@ -31,25 +31,31 @@
public class HBaseTransaction extends AbstractTransaction {
private static final Logger LOG =
LoggerFactory.getLogger(HBaseTransaction.class);
-public HBaseTransaction(long transactionId, long epoch,
Set writeSet, AbstractTransactionManager tm) {
-super(transactionId, epoch, writeSet, tm);
+public HBaseTransaction(long transactionId, long epoch,
Set writeSet, Set conflictFreeWriteSet,
AbstractTransactionManager tm) {
+super(transactionId, epoch, writeSet, conflictFreeWriteSet, tm);
}
-public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
AbstractTransactionManager tm) {
-super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, tm);
+public HBaseTransaction(long transactionId, long readTimestamp,
VisibilityLevel visibilityLevel, long epoch, Set writeSet,
Set conflictFreeWriteSet, AbstractTransactionManager tm) {
+super(transactionId, readTimestamp, visibilityLevel, epoch,
writeSet, conflictFreeWriteSet, tm);
}
+private void cleanCell(HBaseCellId cell) {
--- End diff --
cleanCell --> deleteCell?
> Conflict free writes
>
>
> Key: OMID-84
> URL: https://issues.apache.org/jira/browse/OMID-84
> Project: Apache Omid
> Issue Type: Sub-task
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>Priority: Major
>
> Today, all the writes done by a transaction are taking part in conflict
> analysis. The purpose of this feature is to let the user decide for each
> write, whether it should take part in the conflict analysis.
> The motivation infers from Apache Phoenix that utilizes this feature when
> writing to the secondary index and also when writing to the data table for
> immutable tables (each key is added once and is not modified).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OMID-83) Propagate Attributes to HBase
[ https://issues.apache.org/jira/browse/OMID-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354327#comment-16354327 ] Edward Bortnikov commented on OMID-83: -- +1 > Propagate Attributes to HBase > - > > Key: OMID-83 > URL: https://issues.apache.org/jira/browse/OMID-83 > Project: Apache Omid > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > > Attributes added to Put, Get, and Scan are not propagated to HBase. In many > cases, as in the Phoenix case, these attributes are required and should be > propagated to the server side. In Phoenix for example, attributes are used to > mark data as one that should propagate to the secondary index. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OMID-83) Propagate Attributes to HBase
[
https://issues.apache.org/jira/browse/OMID-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354325#comment-16354325
]
ASF GitHub Bot commented on OMID-83:
Github user ebortnikov commented on a diff in the pull request:
https://github.com/apache/incubator-omid/pull/17#discussion_r166402848
--- Diff:
hbase-client/src/main/java/org/apache/omid/transaction/TTable.java ---
@@ -186,6 +187,14 @@ public Result get(Transaction tx, final Get get)
throws IOException {
return snapshotFilter.get(this, tsget, transaction);
}
+private void updateAttributes(OperationWithAttributes from,
OperationWithAttributes to) {
--- End diff --
makes sense to call propagateAttributes()?
> Propagate Attributes to HBase
> -
>
> Key: OMID-83
> URL: https://issues.apache.org/jira/browse/OMID-83
> Project: Apache Omid
> Issue Type: Sub-task
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>Priority: Major
>
> Attributes added to Put, Get, and Scan are not propagated to HBase. In many
> cases, as in the Phoenix case, these attributes are required and should be
> propagated to the server side. In Phoenix for example, attributes are used to
> mark data as one that should propagate to the secondary index.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
