svn commit: r1803949 - /phoenix/site/publish/contributing.html

2017-08-02 Thread jamestaylor 
Author: jamestaylor
Date: Thu Aug  3 04:44:53 2017
New Revision: 1803949

URL: http://svn.apache.org/viewvc?rev=1803949&view=rev
Log:
Update directions on generating patch

Modified:
phoenix/site/publish/contributing.html

Modified: phoenix/site/publish/contributing.html
URL: 
http://svn.apache.org/viewvc/phoenix/site/publish/contributing.html?rev=1803949&r1=1803948&r2=1803949&view=diff
==
--- phoenix/site/publish/contributing.html (original)
+++ phoenix/site/publish/contributing.html Thu Aug  3 04:44:53 2017
@@ -1,7 +1,7 @@
 
 
 
 
@@ -202,7 +202,7 @@
 
 Create a local branch git checkout -b  
 Make and commit changes 
- Generate a patch based on the name of the JIRA issue, as 
follows: git format-patch --stdout origin > 
PHOENIX-{NUMBER}.patch 
+ Generate a patch based on the name of the JIRA issue, as 
follows: git format-patch --stdout HEAD^ > 
PHOENIX-{NUMBER}.patch 
  Attach the created patch file to the jira ticket

svn commit: r1803948 - in /phoenix/site: publish/language/datatypes.html publish/language/functions.html publish/language/index.html source/src/site/markdown/contributing.md

2017-08-02 Thread jamestaylor 
Author: jamestaylor
Date: Thu Aug  3 04:42:57 2017
New Revision: 1803948

URL: http://svn.apache.org/viewvc?rev=1803948&view=rev
Log:
Update directions on generating patch

Modified:
phoenix/site/publish/language/datatypes.html
phoenix/site/publish/language/functions.html
phoenix/site/publish/language/index.html
phoenix/site/source/src/site/markdown/contributing.md

Modified: phoenix/site/publish/language/datatypes.html
URL: 
http://svn.apache.org/viewvc/phoenix/site/publish/language/datatypes.html?rev=1803948&r1=1803947&r2=1803948&view=diff
==
--- phoenix/site/publish/language/datatypes.html (original)
+++ phoenix/site/publish/language/datatypes.html Thu Aug  3 04:42:57 2017
@@ -1,7 +1,7 @@
 
 
 
 

Modified: phoenix/site/publish/language/functions.html
URL: 
http://svn.apache.org/viewvc/phoenix/site/publish/language/functions.html?rev=1803948&r1=1803947&r2=1803948&view=diff
==
--- phoenix/site/publish/language/functions.html (original)
+++ phoenix/site/publish/language/functions.html Thu Aug  3 04:42:57 2017
@@ -1,7 +1,7 @@
 
 
 
 

Modified: phoenix/site/publish/language/index.html
URL: 
http://svn.apache.org/viewvc/phoenix/site/publish/language/index.html?rev=1803948&r1=1803947&r2=1803948&view=diff
==
--- phoenix/site/publish/language/index.html (original)
+++ phoenix/site/publish/language/index.html Thu Aug  3 04:42:57 2017
@@ -1,7 +1,7 @@
 
 
 
 

Modified: phoenix/site/source/src/site/markdown/contributing.md
URL: 
http://svn.apache.org/viewvc/phoenix/site/source/src/site/markdown/contributing.md?rev=1803948&r1=1803947&r2=1803948&view=diff
==
--- phoenix/site/source/src/site/markdown/contributing.md (original)
+++ phoenix/site/source/src/site/markdown/contributing.md Thu Aug  3 04:42:57 
2017
@@ -49,7 +49,7 @@ The commit message should reference the
 2. Make and commit changes
 3. Generate a patch based on the name of the JIRA issue, as follows:
 
-`git format-patch --stdout origin > PHOENIX-{NUMBER}.patch`
+`git format-patch --stdout HEAD^ > PHOENIX-{NUMBER}.patch`
 
 4. Attach the created patch file to the jira ticket

Jenkins build is back to normal : Phoenix-4.x-HBase-1.1 #520

2017-08-02 Thread Apache Jenkins Server 
See

phoenix git commit: Fix compilation error; Fix a regression due to Calcite materialization changes

2017-08-02 Thread maryannxue 
Repository: phoenix
Updated Branches:
  refs/heads/calcite 17c206434 -> b7e5ae4b1


Fix compilation error; Fix a regression due to Calcite materialization changes


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/b7e5ae4b
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/b7e5ae4b
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/b7e5ae4b

Branch: refs/heads/calcite
Commit: b7e5ae4b17d99e13e6ff90242b8e39312d54edf5
Parents: 17c2064
Author: maryannxue 
Authored: Wed Aug 2 16:13:01 2017 -0700
Committer: maryannxue 
Committed: Wed Aug 2 16:13:01 2017 -0700

--
 .../apache/phoenix/calcite/PhoenixPrograms.java| 12 +---
 .../phoenix/calcite/PhoenixSqlConformance.java |  5 +
 .../apache/phoenix/compile/QueryOptimizerTest.java | 17 -
 3 files changed, 14 insertions(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/b7e5ae4b/phoenix-core/src/main/java/org/apache/phoenix/calcite/PhoenixPrograms.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/calcite/PhoenixPrograms.java 
b/phoenix-core/src/main/java/org/apache/phoenix/calcite/PhoenixPrograms.java
index 4c7454a..9e0412e 100644
--- a/phoenix-core/src/main/java/org/apache/phoenix/calcite/PhoenixPrograms.java
+++ b/phoenix-core/src/main/java/org/apache/phoenix/calcite/PhoenixPrograms.java
@@ -2,7 +2,6 @@ package org.apache.phoenix.calcite;
 
 import java.util.Collection;
 import java.util.Collections;
-import java.util.HashSet;
 import java.util.List;
 import java.util.Set;
 
@@ -144,24 +143,15 @@ public class PhoenixPrograms {
 List materializations,
 List lattices) {
 final List>> 
materializationUses;
-final Set applicableMaterializations;
+final Set applicableMaterializations = 
Collections.emptySet();
 final CalciteConnectionConfig config =
 
planner.getContext().unwrap(CalciteConnectionConfig.class);
 if (config != null && config.materializationsEnabled()) {
 // Transform rels using materialized views.
 materializationUses =
 RelOptMaterializations.useMaterializedViews(rel, 
materializations);
-
-// Add not used but potentially useful materialized views 
to the planner.
-applicableMaterializations = new HashSet<>(
-
RelOptMaterializations.getApplicableMaterializations(
-rel, materializations));
-for (Pair> use : 
materializationUses) {
-applicableMaterializations.removeAll(use.right);
-}
 } else {
 materializationUses = Collections.emptyList();
-applicableMaterializations = Collections.emptySet();
 }
 
 final List rules = planner.getRules();

http://git-wip-us.apache.org/repos/asf/phoenix/blob/b7e5ae4b/phoenix-core/src/main/java/org/apache/phoenix/calcite/PhoenixSqlConformance.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/calcite/PhoenixSqlConformance.java
 
b/phoenix-core/src/main/java/org/apache/phoenix/calcite/PhoenixSqlConformance.java
index 77bd75e..9e45198 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/calcite/PhoenixSqlConformance.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/calcite/PhoenixSqlConformance.java
@@ -90,4 +90,9 @@ public class PhoenixSqlConformance implements SqlConformance {
 public boolean allowExtend() {
 return true;
 }
+
+@Override
+public boolean isLimitStartCountAllowed() {
+return false;
+}
 }

http://git-wip-us.apache.org/repos/asf/phoenix/blob/b7e5ae4b/phoenix-core/src/test/java/org/apache/phoenix/compile/QueryOptimizerTest.java
--
diff --git 
a/phoenix-core/src/test/java/org/apache/phoenix/compile/QueryOptimizerTest.java 
b/phoenix-core/src/test/java/org/apache/phoenix/compile/QueryOptimizerTest.java
index 9a624be..7aff096 100644
--- 
a/phoenix-core/src/test/java/org/apache/phoenix/compile/QueryOptimizerTest.java
+++ 
b/phoenix-core/src/test/java/org/apache/phoenix/compile/QueryOptimizerTest.java
@@ -463,7 +463,7 @@ public class QueryOptimizerTest extends 
BaseConnectionlessQueryTest {
 strArray[1] = "CD";
 Array array = conn.createArrayOf("VARCHAR", strArray);
 stmt.setArray(counter++, array);
-assertPlanDetails(stmt, expectedColNames, 
expect

[1/2] phoenix git commit: PHOENIX-4057 Do not issue index updates for out of order mutation

2017-08-02 Thread jamestaylor 
Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-1.2 d55bc82a0 -> 03152d03e


PHOENIX-4057 Do not issue index updates for out of order mutation


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/31edca2c
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/31edca2c
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/31edca2c

Branch: refs/heads/4.x-HBase-1.2
Commit: 31edca2c15fddc42df32ae1a7d7c4736a29e7924
Parents: d55bc82
Author: James Taylor 
Authored: Tue Aug 1 16:41:01 2017 -0700
Committer: James Taylor 
Committed: Wed Aug 2 16:04:23 2017 -0700

--
 .../phoenix/end2end/ConcurrentMutationsIT.java  |   3 -
 .../phoenix/end2end/OutOfOrderMutationsIT.java  | 561 +++
 .../example/EndToEndCoveredIndexingIT.java  |   6 +-
 .../hbase/index/covered/NonTxIndexBuilder.java  |  75 ++-
 4 files changed, 613 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/31edca2c/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
index 19cb70e..9ed5174 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
@@ -52,7 +52,6 @@ import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.ReadOnlyProps;
 import org.apache.phoenix.util.TestUtil;
 import org.junit.BeforeClass;
-import org.junit.Ignore;
 import org.junit.Test;
 
 import com.google.common.collect.Maps;
@@ -80,7 +79,6 @@ public class ConcurrentMutationsIT extends 
BaseUniqueNamesOwnClusterIT {
 }
 
 @Test
-@Ignore
 public void testSynchronousDeletesAndUpsertValues() throws Exception {
 final String tableName = generateUniqueName();
 final String indexName = generateUniqueName();
@@ -164,7 +162,6 @@ public class ConcurrentMutationsIT extends 
BaseUniqueNamesOwnClusterIT {
 }
 
 @Test
-@Ignore
 public void testConcurrentDeletesAndUpsertValues() throws Exception {
 final String tableName = generateUniqueName();
 final String indexName = generateUniqueName();

http://git-wip-us.apache.org/repos/asf/phoenix/blob/31edca2c/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
new file mode 100644
index 000..3cf7336
--- /dev/null
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
@@ -0,0 +1,561 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Timestamp;
+import java.util.Properties;
+
+import org.apache.hadoop.hbase.CellScanner;
+import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
+import org.apache.phoenix.util.TestUtil;
+import org.junit.Ignore;
+import org.junit.Test;
+
+public class OutOfOrderMutation

[2/2] phoenix git commit: PHOENIX-4059 Index maintenance incorrect when indexed column updated from null to null with STORE_NULLS=true

2017-08-02 Thread jamestaylor 
PHOENIX-4059 Index maintenance incorrect when indexed column updated from null 
to null with STORE_NULLS=true


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/03152d03
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/03152d03
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/03152d03

Branch: refs/heads/4.x-HBase-1.2
Commit: 03152d03e7b9fe0b539a3b8c66cb2c44a6badda5
Parents: 31edca2
Author: James Taylor 
Authored: Wed Aug 2 14:27:42 2017 -0700
Committer: James Taylor 
Committed: Wed Aug 2 16:04:28 2017 -0700

--
 .../apache/phoenix/end2end/StoreNullsIT.java| 88 
 .../apache/phoenix/index/IndexMaintainer.java   |  5 +-
 2 files changed, 91 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/03152d03/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
index a37903f..120e25d 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
@@ -24,8 +24,11 @@ import static org.junit.Assert.assertTrue;
 
 import java.sql.Connection;
 import java.sql.DriverManager;
+import java.sql.PreparedStatement;
 import java.sql.ResultSet;
+import java.sql.SQLException;
 import java.sql.Statement;
+import java.sql.Timestamp;
 import java.util.Arrays;
 import java.util.Collection;
 import java.util.Properties;
@@ -34,6 +37,7 @@ import org.apache.hadoop.hbase.client.HTable;
 import org.apache.hadoop.hbase.client.Result;
 import org.apache.hadoop.hbase.client.ResultScanner;
 import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.phoenix.expression.KeyValueColumnExpression;
 import org.apache.phoenix.expression.SingleCellColumnExpression;
 import org.apache.phoenix.hbase.index.util.ImmutableBytesPtr;
@@ -48,6 +52,7 @@ import org.apache.phoenix.schema.tuple.ResultTuple;
 import org.apache.phoenix.schema.types.PVarchar;
 import org.apache.phoenix.util.ByteUtil;
 import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.TestUtil;
 import org.junit.Before;
 import org.junit.Test;
@@ -242,4 +247,87 @@ public class StoreNullsIT extends ParallelStatsDisabledIT {
 }
 }
 
+
+private static long getRowCount(Connection conn, String tableName) throws 
SQLException {
+ResultSet rs = conn.createStatement().executeQuery("SELECT /*+ 
NO_INDEX */ count(*) FROM " + tableName);
+assertTrue(rs.next());
+return rs.getLong(1);
+}
+
+@Test
+public void testSetIndexedColumnToNullTwiceWithStoreNulls() throws 
Exception {
+if (!mutable) {
+return;
+}
+
+String tableName = generateUniqueName();
+String indexName = generateUniqueName();
+Properties props = PropertiesUtil.deepCopy(TestUtil.TEST_PROPERTIES);
+long ts = 1000;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+Connection conn = DriverManager.getConnection(getUrl(), props); 
+conn.createStatement().execute("CREATE TABLE " + tableName + "(k1 
CHAR(2) NOT NULL, k2 CHAR(2) NOT NULL, ts TIMESTAMP, V VARCHAR, V2 VARCHAR, "
++ "CONSTRAINT pk PRIMARY KEY (k1,k2)) STORE_NULLS=" + 
storeNulls + (columnEncoded ? "" : ",COLUMN_ENCODED_BYTES=0"));
+conn.close();
+
+ts = 1010;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+conn.createStatement().execute("CREATE INDEX " + indexName + " ON " + 
tableName + "(k2,k1,ts) INCLUDE (V, v2)");
+conn.close();
+
+ts = 1020;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+PreparedStatement stmt = conn.prepareStatement("UPSERT INTO " + 
tableName + " VALUES('aa','aa',?, '0')");
+stmt.setTimestamp(1, new Timestamp(1000L));
+stmt.executeUpdate();
+conn.commit();
+conn.close();
+
+Timestamp expectedTimestamp;
+ts = 1030;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+stmt = conn.prepareStatement("UPSERT INTO " + tableName + " 
VALUES('aa','aa',?, null)");
+expectedTimestamp = null;
+stmt.setTimestamp(1, expectedTimesta

[2/2] phoenix git commit: PHOENIX-4059 Index maintenance incorrect when indexed column updated from null to null with STORE_NULLS=true

2017-08-02 Thread jamestaylor 
PHOENIX-4059 Index maintenance incorrect when indexed column updated from null 
to null with STORE_NULLS=true


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/a0772f52
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/a0772f52
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/a0772f52

Branch: refs/heads/4.x-HBase-1.1
Commit: a0772f522a4984e29bdfd8f67b509a179ac59f5a
Parents: defec53
Author: James Taylor 
Authored: Wed Aug 2 14:27:42 2017 -0700
Committer: James Taylor 
Committed: Wed Aug 2 16:02:39 2017 -0700

--
 .../apache/phoenix/end2end/StoreNullsIT.java| 88 
 .../apache/phoenix/index/IndexMaintainer.java   |  5 +-
 2 files changed, 91 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/a0772f52/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
index a37903f..120e25d 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
@@ -24,8 +24,11 @@ import static org.junit.Assert.assertTrue;
 
 import java.sql.Connection;
 import java.sql.DriverManager;
+import java.sql.PreparedStatement;
 import java.sql.ResultSet;
+import java.sql.SQLException;
 import java.sql.Statement;
+import java.sql.Timestamp;
 import java.util.Arrays;
 import java.util.Collection;
 import java.util.Properties;
@@ -34,6 +37,7 @@ import org.apache.hadoop.hbase.client.HTable;
 import org.apache.hadoop.hbase.client.Result;
 import org.apache.hadoop.hbase.client.ResultScanner;
 import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.phoenix.expression.KeyValueColumnExpression;
 import org.apache.phoenix.expression.SingleCellColumnExpression;
 import org.apache.phoenix.hbase.index.util.ImmutableBytesPtr;
@@ -48,6 +52,7 @@ import org.apache.phoenix.schema.tuple.ResultTuple;
 import org.apache.phoenix.schema.types.PVarchar;
 import org.apache.phoenix.util.ByteUtil;
 import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.TestUtil;
 import org.junit.Before;
 import org.junit.Test;
@@ -242,4 +247,87 @@ public class StoreNullsIT extends ParallelStatsDisabledIT {
 }
 }
 
+
+private static long getRowCount(Connection conn, String tableName) throws 
SQLException {
+ResultSet rs = conn.createStatement().executeQuery("SELECT /*+ 
NO_INDEX */ count(*) FROM " + tableName);
+assertTrue(rs.next());
+return rs.getLong(1);
+}
+
+@Test
+public void testSetIndexedColumnToNullTwiceWithStoreNulls() throws 
Exception {
+if (!mutable) {
+return;
+}
+
+String tableName = generateUniqueName();
+String indexName = generateUniqueName();
+Properties props = PropertiesUtil.deepCopy(TestUtil.TEST_PROPERTIES);
+long ts = 1000;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+Connection conn = DriverManager.getConnection(getUrl(), props); 
+conn.createStatement().execute("CREATE TABLE " + tableName + "(k1 
CHAR(2) NOT NULL, k2 CHAR(2) NOT NULL, ts TIMESTAMP, V VARCHAR, V2 VARCHAR, "
++ "CONSTRAINT pk PRIMARY KEY (k1,k2)) STORE_NULLS=" + 
storeNulls + (columnEncoded ? "" : ",COLUMN_ENCODED_BYTES=0"));
+conn.close();
+
+ts = 1010;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+conn.createStatement().execute("CREATE INDEX " + indexName + " ON " + 
tableName + "(k2,k1,ts) INCLUDE (V, v2)");
+conn.close();
+
+ts = 1020;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+PreparedStatement stmt = conn.prepareStatement("UPSERT INTO " + 
tableName + " VALUES('aa','aa',?, '0')");
+stmt.setTimestamp(1, new Timestamp(1000L));
+stmt.executeUpdate();
+conn.commit();
+conn.close();
+
+Timestamp expectedTimestamp;
+ts = 1030;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+stmt = conn.prepareStatement("UPSERT INTO " + tableName + " 
VALUES('aa','aa',?, null)");
+expectedTimestamp = null;
+stmt.setTimestamp(1, expectedTimesta

[1/2] phoenix git commit: PHOENIX-4057 Do not issue index updates for out of order mutation

2017-08-02 Thread jamestaylor 
Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-1.1 ef1e30e5a -> a0772f522


PHOENIX-4057 Do not issue index updates for out of order mutation


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/defec53e
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/defec53e
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/defec53e

Branch: refs/heads/4.x-HBase-1.1
Commit: defec53ebccdee7dae2a401bb0e09a00ff84eee5
Parents: ef1e30e
Author: James Taylor 
Authored: Tue Aug 1 16:41:01 2017 -0700
Committer: James Taylor 
Committed: Wed Aug 2 16:02:31 2017 -0700

--
 .../phoenix/end2end/ConcurrentMutationsIT.java  |   3 -
 .../phoenix/end2end/OutOfOrderMutationsIT.java  | 561 +++
 .../example/EndToEndCoveredIndexingIT.java  |   6 +-
 .../hbase/index/covered/NonTxIndexBuilder.java  |  75 ++-
 4 files changed, 613 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/defec53e/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
index 19cb70e..9ed5174 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
@@ -52,7 +52,6 @@ import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.ReadOnlyProps;
 import org.apache.phoenix.util.TestUtil;
 import org.junit.BeforeClass;
-import org.junit.Ignore;
 import org.junit.Test;
 
 import com.google.common.collect.Maps;
@@ -80,7 +79,6 @@ public class ConcurrentMutationsIT extends 
BaseUniqueNamesOwnClusterIT {
 }
 
 @Test
-@Ignore
 public void testSynchronousDeletesAndUpsertValues() throws Exception {
 final String tableName = generateUniqueName();
 final String indexName = generateUniqueName();
@@ -164,7 +162,6 @@ public class ConcurrentMutationsIT extends 
BaseUniqueNamesOwnClusterIT {
 }
 
 @Test
-@Ignore
 public void testConcurrentDeletesAndUpsertValues() throws Exception {
 final String tableName = generateUniqueName();
 final String indexName = generateUniqueName();

http://git-wip-us.apache.org/repos/asf/phoenix/blob/defec53e/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
new file mode 100644
index 000..3cf7336
--- /dev/null
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
@@ -0,0 +1,561 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Timestamp;
+import java.util.Properties;
+
+import org.apache.hadoop.hbase.CellScanner;
+import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
+import org.apache.phoenix.util.TestUtil;
+import org.junit.Ignore;
+import org.junit.Test;
+
+public class OutOfOrderMutation

[1/2] phoenix git commit: PHOENIX-4057 Do not issue index updates for out of order mutation

2017-08-02 Thread jamestaylor 
Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-0.98 0135c1075 -> 24a75045f


PHOENIX-4057 Do not issue index updates for out of order mutation


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/2448a95e
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/2448a95e
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/2448a95e

Branch: refs/heads/4.x-HBase-0.98
Commit: 2448a95e018b9baf30b85474b459fec788b7cd23
Parents: 0135c10
Author: James Taylor 
Authored: Tue Aug 1 16:41:01 2017 -0700
Committer: James Taylor 
Committed: Wed Aug 2 16:00:59 2017 -0700

--
 .../phoenix/end2end/ConcurrentMutationsIT.java  |   3 -
 .../phoenix/end2end/OutOfOrderMutationsIT.java  | 561 +++
 .../example/EndToEndCoveredIndexingIT.java  |   6 +-
 .../hbase/index/covered/NonTxIndexBuilder.java  |  75 ++-
 4 files changed, 613 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/2448a95e/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
index 19cb70e..9ed5174 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
@@ -52,7 +52,6 @@ import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.ReadOnlyProps;
 import org.apache.phoenix.util.TestUtil;
 import org.junit.BeforeClass;
-import org.junit.Ignore;
 import org.junit.Test;
 
 import com.google.common.collect.Maps;
@@ -80,7 +79,6 @@ public class ConcurrentMutationsIT extends 
BaseUniqueNamesOwnClusterIT {
 }
 
 @Test
-@Ignore
 public void testSynchronousDeletesAndUpsertValues() throws Exception {
 final String tableName = generateUniqueName();
 final String indexName = generateUniqueName();
@@ -164,7 +162,6 @@ public class ConcurrentMutationsIT extends 
BaseUniqueNamesOwnClusterIT {
 }
 
 @Test
-@Ignore
 public void testConcurrentDeletesAndUpsertValues() throws Exception {
 final String tableName = generateUniqueName();
 final String indexName = generateUniqueName();

http://git-wip-us.apache.org/repos/asf/phoenix/blob/2448a95e/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
new file mode 100644
index 000..3cf7336
--- /dev/null
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
@@ -0,0 +1,561 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Timestamp;
+import java.util.Properties;
+
+import org.apache.hadoop.hbase.CellScanner;
+import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
+import org.apache.phoenix.util.TestUtil;
+import org.junit.Ignore;
+import org.junit.Test;
+
+public class OutOfOrderMutati

[2/2] phoenix git commit: PHOENIX-4059 Index maintenance incorrect when indexed column updated from null to null with STORE_NULLS=true

2017-08-02 Thread jamestaylor 
PHOENIX-4059 Index maintenance incorrect when indexed column updated from null 
to null with STORE_NULLS=true


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/24a75045
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/24a75045
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/24a75045

Branch: refs/heads/4.x-HBase-0.98
Commit: 24a75045fe72c4d665fd47ada19dfce0a910ee2b
Parents: 2448a95
Author: James Taylor 
Authored: Wed Aug 2 14:27:42 2017 -0700
Committer: James Taylor 
Committed: Wed Aug 2 16:01:19 2017 -0700

--
 .../apache/phoenix/end2end/StoreNullsIT.java| 88 
 .../apache/phoenix/index/IndexMaintainer.java   |  5 +-
 2 files changed, 91 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/24a75045/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
index a37903f..120e25d 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
@@ -24,8 +24,11 @@ import static org.junit.Assert.assertTrue;
 
 import java.sql.Connection;
 import java.sql.DriverManager;
+import java.sql.PreparedStatement;
 import java.sql.ResultSet;
+import java.sql.SQLException;
 import java.sql.Statement;
+import java.sql.Timestamp;
 import java.util.Arrays;
 import java.util.Collection;
 import java.util.Properties;
@@ -34,6 +37,7 @@ import org.apache.hadoop.hbase.client.HTable;
 import org.apache.hadoop.hbase.client.Result;
 import org.apache.hadoop.hbase.client.ResultScanner;
 import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.phoenix.expression.KeyValueColumnExpression;
 import org.apache.phoenix.expression.SingleCellColumnExpression;
 import org.apache.phoenix.hbase.index.util.ImmutableBytesPtr;
@@ -48,6 +52,7 @@ import org.apache.phoenix.schema.tuple.ResultTuple;
 import org.apache.phoenix.schema.types.PVarchar;
 import org.apache.phoenix.util.ByteUtil;
 import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.TestUtil;
 import org.junit.Before;
 import org.junit.Test;
@@ -242,4 +247,87 @@ public class StoreNullsIT extends ParallelStatsDisabledIT {
 }
 }
 
+
+private static long getRowCount(Connection conn, String tableName) throws 
SQLException {
+ResultSet rs = conn.createStatement().executeQuery("SELECT /*+ 
NO_INDEX */ count(*) FROM " + tableName);
+assertTrue(rs.next());
+return rs.getLong(1);
+}
+
+@Test
+public void testSetIndexedColumnToNullTwiceWithStoreNulls() throws 
Exception {
+if (!mutable) {
+return;
+}
+
+String tableName = generateUniqueName();
+String indexName = generateUniqueName();
+Properties props = PropertiesUtil.deepCopy(TestUtil.TEST_PROPERTIES);
+long ts = 1000;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+Connection conn = DriverManager.getConnection(getUrl(), props); 
+conn.createStatement().execute("CREATE TABLE " + tableName + "(k1 
CHAR(2) NOT NULL, k2 CHAR(2) NOT NULL, ts TIMESTAMP, V VARCHAR, V2 VARCHAR, "
++ "CONSTRAINT pk PRIMARY KEY (k1,k2)) STORE_NULLS=" + 
storeNulls + (columnEncoded ? "" : ",COLUMN_ENCODED_BYTES=0"));
+conn.close();
+
+ts = 1010;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+conn.createStatement().execute("CREATE INDEX " + indexName + " ON " + 
tableName + "(k2,k1,ts) INCLUDE (V, v2)");
+conn.close();
+
+ts = 1020;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+PreparedStatement stmt = conn.prepareStatement("UPSERT INTO " + 
tableName + " VALUES('aa','aa',?, '0')");
+stmt.setTimestamp(1, new Timestamp(1000L));
+stmt.executeUpdate();
+conn.commit();
+conn.close();
+
+Timestamp expectedTimestamp;
+ts = 1030;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+stmt = conn.prepareStatement("UPSERT INTO " + tableName + " 
VALUES('aa','aa',?, null)");
+expectedTimestamp = null;
+stmt.setTimestamp(1, expectedTimest

[2/2] phoenix git commit: PHOENIX-4059 Index maintenance incorrect when indexed column updated from null to null with STORE_NULLS=true

2017-08-02 Thread jamestaylor 
PHOENIX-4059 Index maintenance incorrect when indexed column updated from null 
to null with STORE_NULLS=true


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/0bd43f53
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/0bd43f53
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/0bd43f53

Branch: refs/heads/master
Commit: 0bd43f536a13609b35629308c415aebc800d6799
Parents: e494fe9
Author: James Taylor 
Authored: Wed Aug 2 14:27:42 2017 -0700
Committer: James Taylor 
Committed: Wed Aug 2 14:40:31 2017 -0700

--
 .../apache/phoenix/end2end/StoreNullsIT.java| 88 
 .../apache/phoenix/index/IndexMaintainer.java   |  5 +-
 2 files changed, 91 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/0bd43f53/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
index a37903f..120e25d 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/StoreNullsIT.java
@@ -24,8 +24,11 @@ import static org.junit.Assert.assertTrue;
 
 import java.sql.Connection;
 import java.sql.DriverManager;
+import java.sql.PreparedStatement;
 import java.sql.ResultSet;
+import java.sql.SQLException;
 import java.sql.Statement;
+import java.sql.Timestamp;
 import java.util.Arrays;
 import java.util.Collection;
 import java.util.Properties;
@@ -34,6 +37,7 @@ import org.apache.hadoop.hbase.client.HTable;
 import org.apache.hadoop.hbase.client.Result;
 import org.apache.hadoop.hbase.client.ResultScanner;
 import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.phoenix.expression.KeyValueColumnExpression;
 import org.apache.phoenix.expression.SingleCellColumnExpression;
 import org.apache.phoenix.hbase.index.util.ImmutableBytesPtr;
@@ -48,6 +52,7 @@ import org.apache.phoenix.schema.tuple.ResultTuple;
 import org.apache.phoenix.schema.types.PVarchar;
 import org.apache.phoenix.util.ByteUtil;
 import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.TestUtil;
 import org.junit.Before;
 import org.junit.Test;
@@ -242,4 +247,87 @@ public class StoreNullsIT extends ParallelStatsDisabledIT {
 }
 }
 
+
+private static long getRowCount(Connection conn, String tableName) throws 
SQLException {
+ResultSet rs = conn.createStatement().executeQuery("SELECT /*+ 
NO_INDEX */ count(*) FROM " + tableName);
+assertTrue(rs.next());
+return rs.getLong(1);
+}
+
+@Test
+public void testSetIndexedColumnToNullTwiceWithStoreNulls() throws 
Exception {
+if (!mutable) {
+return;
+}
+
+String tableName = generateUniqueName();
+String indexName = generateUniqueName();
+Properties props = PropertiesUtil.deepCopy(TestUtil.TEST_PROPERTIES);
+long ts = 1000;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+Connection conn = DriverManager.getConnection(getUrl(), props); 
+conn.createStatement().execute("CREATE TABLE " + tableName + "(k1 
CHAR(2) NOT NULL, k2 CHAR(2) NOT NULL, ts TIMESTAMP, V VARCHAR, V2 VARCHAR, "
++ "CONSTRAINT pk PRIMARY KEY (k1,k2)) STORE_NULLS=" + 
storeNulls + (columnEncoded ? "" : ",COLUMN_ENCODED_BYTES=0"));
+conn.close();
+
+ts = 1010;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+conn.createStatement().execute("CREATE INDEX " + indexName + " ON " + 
tableName + "(k2,k1,ts) INCLUDE (V, v2)");
+conn.close();
+
+ts = 1020;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+PreparedStatement stmt = conn.prepareStatement("UPSERT INTO " + 
tableName + " VALUES('aa','aa',?, '0')");
+stmt.setTimestamp(1, new Timestamp(1000L));
+stmt.executeUpdate();
+conn.commit();
+conn.close();
+
+Timestamp expectedTimestamp;
+ts = 1030;
+props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, 
Long.toString(ts));
+conn = DriverManager.getConnection(getUrl(), props);
+stmt = conn.prepareStatement("UPSERT INTO " + tableName + " 
VALUES('aa','aa',?, null)");
+expectedTimestamp = null;
+stmt.setTimestamp(1, expectedTimestamp);
+

[1/2] phoenix git commit: PHOENIX-4057 Do not issue index updates for out of order mutation

2017-08-02 Thread jamestaylor 
Repository: phoenix
Updated Branches:
  refs/heads/master 5e33dc12b -> 0bd43f536


PHOENIX-4057 Do not issue index updates for out of order mutation


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/e494fe9f
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/e494fe9f
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/e494fe9f

Branch: refs/heads/master
Commit: e494fe9fa744ec2948cf75b007e92fef0b1ba829
Parents: 5e33dc1
Author: James Taylor 
Authored: Tue Aug 1 16:41:01 2017 -0700
Committer: James Taylor 
Committed: Wed Aug 2 14:40:26 2017 -0700

--
 .../phoenix/end2end/ConcurrentMutationsIT.java  |   3 -
 .../phoenix/end2end/OutOfOrderMutationsIT.java  | 561 +++
 .../example/EndToEndCoveredIndexingIT.java  |   6 +-
 .../hbase/index/covered/NonTxIndexBuilder.java  |  75 ++-
 4 files changed, 613 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/e494fe9f/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
index 19cb70e..9ed5174 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentMutationsIT.java
@@ -52,7 +52,6 @@ import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.ReadOnlyProps;
 import org.apache.phoenix.util.TestUtil;
 import org.junit.BeforeClass;
-import org.junit.Ignore;
 import org.junit.Test;
 
 import com.google.common.collect.Maps;
@@ -80,7 +79,6 @@ public class ConcurrentMutationsIT extends 
BaseUniqueNamesOwnClusterIT {
 }
 
 @Test
-@Ignore
 public void testSynchronousDeletesAndUpsertValues() throws Exception {
 final String tableName = generateUniqueName();
 final String indexName = generateUniqueName();
@@ -164,7 +162,6 @@ public class ConcurrentMutationsIT extends 
BaseUniqueNamesOwnClusterIT {
 }
 
 @Test
-@Ignore
 public void testConcurrentDeletesAndUpsertValues() throws Exception {
 final String tableName = generateUniqueName();
 final String indexName = generateUniqueName();

http://git-wip-us.apache.org/repos/asf/phoenix/blob/e494fe9f/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
new file mode 100644
index 000..3cf7336
--- /dev/null
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/OutOfOrderMutationsIT.java
@@ -0,0 +1,561 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Timestamp;
+import java.util.Properties;
+
+import org.apache.hadoop.hbase.CellScanner;
+import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.PropertiesUtil;
+import org.apache.phoenix.util.TestUtil;
+import org.junit.Ignore;
+import org.junit.Test;
+
+public class OutOfOrderMutationsIT extends Pa

Build failed in Jenkins: Phoenix Compile Compatibility with HBase #367

2017-08-02 Thread Apache Jenkins Server 
See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H25 (couchdbtest ubuntu xenial) in workspace 

[Phoenix_Compile_Compat_wHBase] $ /bin/bash /tmp/jenkins7307118796213527258.sh
core file size  (blocks, -c) 0
data seg size   (kbytes, -d) unlimited
scheduling priority (-e) 0
file size   (blocks, -f) unlimited
pending signals (-i) 386417
max locked memory   (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files  (-n) 6
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority  (-r) 0
stack size  (kbytes, -s) 8192
cpu time   (seconds, -t) unlimited
max user processes  (-u) 10240
virtual memory  (kbytes, -v) unlimited
file locks  (-x) unlimited
core id : 0
core id : 1
core id : 2
core id : 3
core id : 4
core id : 5
physical id : 0
physical id : 1
MemTotal:   98958120 kB
MemFree:37039496 kB
Filesystem  Size  Used Avail Use% Mounted on
udev 48G 0   48G   0% /dev
tmpfs   9.5G  962M  8.5G  10% /run
/dev/sda1   364G  136G  211G  40% /
tmpfs48G  516K   48G   1% /dev/shm
tmpfs   5.0M 0  5.0M   0% /run/lock
tmpfs48G 0   48G   0% /sys/fs/cgroup
tmpfs   9.5G 0  9.5G   0% /run/user/10025
apache-maven-2.2.1
apache-maven-3.0.4
apache-maven-3.0.5
apache-maven-3.2.1
apache-maven-3.2.5
apache-maven-3.3.3
apache-maven-3.3.9
apache-maven-3.5.0
latest
latest2
latest3


===
Verifying compile level compatibility with HBase 0.98 with Phoenix 
4.x-HBase-0.98
===

Cloning into 'hbase'...
Switched to a new branch '0.98'
Branch 0.98 set up to track remote branch 0.98 from origin.

main:
 [exec] 
~/jenkins-slave/workspace/Phoenix_Compile_Compat_wHBase/hbase/hbase-common 
~/jenkins-slave/workspace/Phoenix_Compile_Compat_wHBase/hbase/hbase-common
 [exec] 
~/jenkins-slave/workspace/Phoenix_Compile_Compat_wHBase/hbase/hbase-common

main:
[mkdir] Created dir: 

 [exec] tar: hadoop-snappy-nativelibs.tar: Cannot open: No such file or 
directory
 [exec] tar: Error is not recoverable: exiting now
 [exec] Result: 2

main:
[mkdir] Created dir: 

 [copy] Copying 20 files to 

[mkdir] Created dir: 

[mkdir] Created dir: 


main:
[mkdir] Created dir: 

 [copy] Copying 17 files to 

[mkdir] Created dir: 


main:
[mkdir] Created dir: 

 [copy] Copying 1 file to 

[mkdir] Created dir: 


HBase pom.xml:

Got HBase version as 0.98.25-SNAPSHOT
Cloning into 'phoenix'...
Switched to a new branch '4.x-HBase-0.98'
Branch 4.x-HBase-0.98 set up to track remote branch 4.x-HBase-0.98 from origin.
ANTLR Parser Generator  Version 3.5.2
Output file 

 does not exist: must build 

PhoenixSQL.g


===
Verifying compile level compatibility with HBase branch-1.3 with Phoenix master
=