[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268396#comment-16268396
 ] 

ASF GitHub Bot commented on OMID-78:


Github user yonigottesman commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r153424921
  
--- Diff: 
hbase-client/src/main/java/org/apache/omid/transaction/SnapshotFilter.java ---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.omid.transaction;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.omid.committable.CommitTable.CommitTimestamp;
+
+import com.google.common.base.Optional;
+
+public interface SnapshotFilter {
+
+public Result get(TTable ttable, Get get, HBaseTransaction 
transaction) throws IOException;
+
+public ResultScanner getScanner(TTable ttable, Scan scan, 
HBaseTransaction transaction) throws IOException;
+
+public List filterCellsForSnapshot(List rawCells, 
HBaseTransaction transaction,
+int versionsToRequest, Map 
familyDeletionCache) throws IOException;
+
+public boolean isCommitted(HBaseCellId hBaseCellId, long epoch) throws 
TransactionException;
--- End diff --

Why do all other function throw ioexceptions and this one something else?


> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268400#comment-16268400
 ] 

ASF GitHub Bot commented on OMID-78:


Github user yonigottesman commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r153421050
  
--- Diff: 
hbase-client/src/main/java/org/apache/omid/transaction/TTable.java ---
@@ -76,6 +71,10 @@
 
 private HTableInterface table;
 
+private SnapshotFilter snapshotFilter;
+
+private boolean serverSideFilter;
--- End diff --

No need to be a class field, you use it once in a function so make it local 
there.


> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268399#comment-16268399
 ] 

ASF GitHub Bot commented on OMID-78:


Github user yonigottesman commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r153422231
  
--- Diff: 
hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/regionserver/OmidRegionScanner.java
 ---
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.regionserver.ScannerContext;
+import org.apache.omid.transaction.HBaseTransaction;
+import org.apache.omid.transaction.SnapshotFilterImpl;
+
+public class OmidRegionScanner implements RegionScanner {
+
+RegionScanner scanner;
--- End diff --

Why are these package-private and not private?



> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268398#comment-16268398
 ] 

ASF GitHub Bot commented on OMID-78:


Github user yonigottesman commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r153424483
  
--- Diff: 
hbase-client/src/main/java/org/apache/omid/transaction/HBaseTransactionManager.java
 ---
@@ -292,7 +268,7 @@ public ConflictDetectionLevel 
getConflictDetectionLevel() {
 get.addColumn(family, shadowCellQualifier);
 get.setMaxVersions(1);
 get.setTimeStamp(startTimestamp);
-Result result = hBaseCellId.getTable().get(get);
+Result result = (tableAccessWrapper != null) ? 
tableAccessWrapper.get(get) : hBaseCellId.getTable().get(get);
--- End diff --

why should this be null? Maybe its better to pass 
"hBaseCellId.getTable().get(get);" to the constructor code


> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251218#comment-16251218
 ] 

ASF GitHub Bot commented on OMID-78:


Github user ohadshacham commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r150795933
  
--- Diff: 
hbase-client/src/main/java/org/apache/omid/transaction/HTableAccessWrapper.java 
---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.omid.transaction;
+
+import java.io.IOException;
+import java.util.List;
+
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+
+public class HTableAccessWrapper implements TableAccessWrapper {
+
+private final HTableInterface healerTable;
--- End diff --

Done


> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251209#comment-16251209
 ] 

ASF GitHub Bot commented on OMID-78:


Github user ohadshacham commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r150795077
  
--- Diff: 
hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/regionserver/RegionAccessWrapper.java
 ---
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+import java.io.IOException;
+import java.util.List;
+
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.omid.transaction.TableAccessWrapper;
+
+public class RegionAccessWrapper implements TableAccessWrapper {
--- End diff --

done


> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251207#comment-16251207
 ] 

ASF GitHub Bot commented on OMID-78:


Github user ohadshacham commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r150794375
  
--- Diff: 
hbase-client/src/main/java/org/apache/omid/transaction/HTableAccessWrapper.java 
---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.omid.transaction;
+
+import java.io.IOException;
+import java.util.List;
+
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+
+public class HTableAccessWrapper implements TableAccessWrapper {
--- End diff --

RegionAccessWrapper


> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251205#comment-16251205
 ] 

ASF GitHub Bot commented on OMID-78:


Github user ohadshacham commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r150794291
  
--- Diff: 
hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/regionserver/OmidRegionScanner.java
 ---
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.regionserver.ScannerContext;
+import org.apache.omid.transaction.HBaseTransaction;
+import org.apache.omid.transaction.SnapshotFilter;
+
+public class OmidRegionScanner implements RegionScanner {
+
+RegionScanner scanner;
+SnapshotFilter snapshotFilter;
+HBaseTransaction transaction;
+int maxVersions;
+Map familyDeletionCache;
+
+public OmidRegionScanner(SnapshotFilter snapshotFilter,
+  RegionScanner s,
+  HBaseTransaction transaction,
+  int maxVersions) {
+this.snapshotFilter = snapshotFilter;
+this.scanner = s;
+this.transaction = transaction;
+this.maxVersions = maxVersions;
+this.familyDeletionCache = new HashMap();
+}
+
+@Override
+public boolean next(List results) throws IOException {
+   return next(results, Integer.MAX_VALUE);
+}
+
+public boolean next(List result, int limit) throws IOException {
--- End diff --

The function is defined in RegionScanner interface, we are just override it.


> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249572#comment-16249572
 ] 

ASF GitHub Bot commented on OMID-78:


Github user ebortnik commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r150533529
  
--- Diff: 
hbase-client/src/main/java/org/apache/omid/transaction/HTableAccessWrapper.java 
---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.omid.transaction;
+
+import java.io.IOException;
+import java.util.List;
+
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+
+public class HTableAccessWrapper implements TableAccessWrapper {
--- End diff --

What are other implementations of this interface? 


> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249573#comment-16249573
 ] 

ASF GitHub Bot commented on OMID-78:


Github user ebortnik commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r150533811
  
--- Diff: 
hbase-client/src/main/java/org/apache/omid/transaction/HTableAccessWrapper.java 
---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.omid.transaction;
+
+import java.io.IOException;
+import java.util.List;
+
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+
+public class HTableAccessWrapper implements TableAccessWrapper {
+
+private final HTableInterface healerTable;
--- End diff --

Consider other names, e.g., healerTable --> wTable, table --> rTable


> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249534#comment-16249534
 ] 

ASF GitHub Bot commented on OMID-78:


Github user ebortnik commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r150531254
  
--- Diff: 
hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/regionserver/OmidRegionScanner.java
 ---
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.regionserver.ScannerContext;
+import org.apache.omid.transaction.HBaseTransaction;
+import org.apache.omid.transaction.SnapshotFilter;
+
+public class OmidRegionScanner implements RegionScanner {
+
+RegionScanner scanner;
+SnapshotFilter snapshotFilter;
+HBaseTransaction transaction;
+int maxVersions;
+Map familyDeletionCache;
+
+public OmidRegionScanner(SnapshotFilter snapshotFilter,
+  RegionScanner s,
+  HBaseTransaction transaction,
+  int maxVersions) {
+this.snapshotFilter = snapshotFilter;
+this.scanner = s;
+this.transaction = transaction;
+this.maxVersions = maxVersions;
+this.familyDeletionCache = new HashMap();
+}
+
+@Override
+public boolean next(List results) throws IOException {
+   return next(results, Integer.MAX_VALUE);
+}
+
+public boolean next(List result, int limit) throws IOException {
--- End diff --

raw --> row


> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249533#comment-16249533
 ] 

ASF GitHub Bot commented on OMID-78:


Github user ebortnik commented on a diff in the pull request:

https://github.com/apache/incubator-omid/pull/16#discussion_r150531893
  
--- Diff: 
hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/regionserver/RegionAccessWrapper.java
 ---
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+import java.io.IOException;
+import java.util.List;
+
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.omid.transaction.TableAccessWrapper;
+
+public class RegionAccessWrapper implements TableAccessWrapper {
--- End diff --

High level comment please


> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.
> This feature is also needed to support the integration of Omid with Phoenix. 
> This is required since Phoenix has coprocessors that change and condence the 
> data returned to the client and therefore, filtering at the client side in 
> this case is impossible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OMID-78) Identify transaction snapshot at the server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/OMID-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16243805#comment-16243805
 ] 

ASF GitHub Bot commented on OMID-78:


GitHub user ohadshacham opened a pull request:

https://github.com/apache/incubator-omid/pull/16

[OMID-78] Identify transaction snapshot at the server

To identify a transaction's snapshot in scans and gets, Omid reads and 
filters cells at the client side. This filtering requires checking the shadow 
cells and accessing the commit table if needed.
Doing the filtering at the server side should be more efficient since less 
data is being transferred to the client and also, in some cases, RPCs are being 
saved.
This feature requires implementing a coprocessor that will do the filtering 
at the server side and returns only the relevant snapshot to the client side.

This feature is also needed to support the integration of Omid with 
Phoenix. This is required since Phoenix has coprocessors that change and 
condence the data returned to the client and therefore, filtering at the client 
side in this case is impossible. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ohadshacham/incubator-omid ServerFiltering

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-omid/pull/16.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #16


commit 83de11a7e64df7dd71c14f830fa96f8ec1646812
Author: Ohad Shacham 
Date:   2017-10-26T08:36:04Z

Refactor the code to extract the filtering to a different object.
The intention of the extraction is to share code between the client and 
server side filtering.

commit d49ff5b10fd4b35a740525e5ea9146e9cba48788
Author: Ohad Shacham 
Date:   2017-10-30T08:55:42Z

intermediate version

commit d5ea6a23e00b02260f8708172bc505ac4a5c0b1b
Author: Ohad Shacham 
Date:   2017-10-31T10:11:35Z

Implement get function at the coprocessor

commit 9aa6078a4821ea4e5a1e25e5a2bba2d894a3b055
Author: Ohad Shacham 
Date:   2017-10-31T10:53:33Z

addition

commit 11e90026749ad2ac667bcd27f3ca48878fc08ec4
Author: Ohad Shacham 
Date:   2017-11-01T13:04:53Z

Working get with tests

commit 51c887d4ab555318a3e70f83c2760fc0a7e5b1ad
Author: Ohad Shacham 
Date:   2017-11-01T14:32:50Z

Merge branch 'ServerFiltering' of 
https://github.com/ohadshacham/incubator-omid into ServerFiltering

commit 20077d4bacfb0ec6524c8e70e6355059be432d6f
Author: Ohad Shacham 
Date:   2017-11-07T15:18:10Z

Working version for hbase0

commit 8246c3c7e97474b7c7fc701c07d312093bb1cb05
Author: Ohad Shacham 
Date:   2017-11-08T09:50:55Z

Working version for hbase-0 and hbase-1

commit ab2bb7b22535b7fa165d2cc2c0dbacd4c2f28009
Author: Ohad Shacham 
Date:   2017-11-08T11:02:26Z

Let two mini clusters run concurrently.




> Identify transaction snapshot at the server
> ---
>
> Key: OMID-78
> URL: https://issues.apache.org/jira/browse/OMID-78
> Project: Apache Omid
>  Issue Type: New Feature
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>
> To identify a transaction's snapshot in scans and gets, Omid reads and filter 
> cells at the client side. This filtering requires checking the shadow cells 
> and accessing the commit table if needed.
> Doing the filtering at the server side should be more efficient since less 
> data is being transferred to the client and also, in some cases, RPCs are 
> being saved.
> This feature requires implementing a coprocessor that will do the filtering 
> at the server side and returns only the relevant snapshot to the client side.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)