svn commit: r1779374 - in /phoenix/site: publish/pherf.html source/src/site/markdown/pherf.md

2017-01-18 Thread mujtaba 
Author: mujtaba
Date: Wed Jan 18 20:08:58 2017
New Revision: 1779374

URL: http://svn.apache.org/viewvc?rev=1779374&view=rev
Log:
Fix broken link and update startup text in pherf.md

Modified:
phoenix/site/publish/pherf.html
phoenix/site/source/src/site/markdown/pherf.md

Modified: phoenix/site/publish/pherf.html
URL: 
http://svn.apache.org/viewvc/phoenix/site/publish/pherf.html?rev=1779374&r1=1779373&r2=1779374&view=diff
==
--- phoenix/site/publish/pherf.html (original)
+++ phoenix/site/publish/pherf.html Wed Jan 18 20:08:58 2017
@@ -1,7 +1,7 @@
 
 
 
 
@@ -159,43 +159,29 @@
   Overview
   
  Pherf is a standalone tool that can perform performance and functional 
testing through Phoenix. Pherf can be used both generate highly customized data 
sets and to measure performance of SQL against that data. 
- 
- 
- Build 
- Building Pherf is done through Phoenix’s normal build Maven process. 
Pherf can be built using two profiles. 
-  
-  Cluster ( default ) - This profile builds Pherf such that it can 
run along side an existing cluster. The dependencies are pulled from the HBase 
classpath. 
-  Standalone - This profile builds all of Pherf’s dependencies into a 
single standalone jar. The deps will be pulled from the versions specified in 
Phoenix’s pom. 
-  
   
   Build 
all of Phoenix. This includes Pherf’s default profile 
   mvn clean package -DskipTests 
   
-  
-  Build Phoenix with 
Pherf’s standalone profile 
-  mvn clean package -P standalone -DskipTests 
-  
  
  
- Installing 
- When Pherf is built using the Maven commands specified above, it will 
produce a https://github.com/apache/phoenix/tree/master/phoenix-pherf";>zip file 
in the module’s target directory. 
-  
-  Simply unpack this zip into the desired location. 
-  Edit the env.sh to include the required property values. 
-  ./pherf.sh -h 
-  To test on a real cluster: ./pherf.sh -drop all -l -q -z localhost 
-schemaFile .*user_defined_schema.sql -scenarioFile 
.*user_defined_scenario.xml 
-  That’s it. 
-  
+ Running 
+  
+  Edit the config/env.sh to include the required property values. 
+  bin/pherf-standalone.py -h 
+  To use libraries included with HBase deployment on a cluster: 
bin/pherf-cluster.py -h 
+  Example: bin/pherf-cluster.py -drop all -l -q -z [zookeeper] 
-schemaFile .*user_defined_schema.sql -scenarioFile 
.*user_defined_scenario.xml HBASE_CONF_DIR, HBASE_DIR environment variable 
needs to be set to use against a cluster deployment 
+  
  
  
  Example run commands. 
   
   List all scenario files 
available to run. 
-  $./pherf.sh -listFiles 
+  $./pherf-standalone.py -listFiles 
   
   
   Drop
 all existing tables, load and query data specified in all scenario files. 
-  $./pherf.sh -drop all -l -q -z localhost  
+  $./pherf-standalone.py -drop all -l -q -z localhost  
   
  
  
@@ -219,7 +205,7 @@
  
  
  Adding Rules for Data Creation 
- Review test_scenario.xml for 
syntax examples. 
+ Review https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=blob;f=phoenix-pherf/src/test/resources/scenario/test_scenario.xml";>test_scenario.xml
 for syntax examples. 
   
   Rules are defined as  and are applied in the 
order they appear in file. 
   Rules of the same type override the values of a prior rule of the same 
type. If true is set, rule will 
only apply override when type and name match the column name in Phoenix. 
@@ -520,7 +506,7 @@


Back to 
top
-   Copyright ©2016 http://www.apache.org";>Apache Software Foundation. All Rights 
Reserved.
+   Copyright ©2017 http://www.apache.org";>Apache Software Foundation. All Rights 
Reserved.




Modified: phoenix/site/source/src/site/markdown/pherf.md
URL: 
http://svn.apache.org/viewvc/phoenix/site/source/src/site/markdown/pherf.md?rev=1779374&r1=1779373&r2=1779374&view=diff
==
--- phoenix/site/source/src/site/markdown/pherf.md (original)
+++ phoenix/site/source/src/site/markdown/pherf.md Wed Jan 18 20:08:58 2017
@@ -5,35 +5,24 @@
 
 Pherf is a standalone tool that can perform performance and functional testing 
through Phoenix. Pherf can be used both generate highly customized data sets 
and to measure performance of SQL against that data.
 
-## Build
-
-Building Pherf is done through Phoenix's normal build Maven process. Pherf can 
be built using two profiles.
-
-* Cluster ( **default** ) - This profile builds Pherf such that it can run 
along side an existing cluster. The dependencies are pulled from the HBase 
classpath.
-* Standalone  - This profile builds all of Pherf's dependencies 
into a single standalone jar. The deps will be pulled from the versions 
specified in Phoenix's

Apache-Phoenix | Master | Build Successful

2017-01-18 Thread Apache Jenkins Server 
Master branch build status Successful
Source repository https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=shortlog;h=refs/heads/master

Last Successful Compiled Artifacts https://builds.apache.org/job/Phoenix-master/lastSuccessfulBuild/artifact/

Last Complete Test Report https://builds.apache.org/job/Phoenix-master/lastCompletedBuild/testReport/

Changes
[ankitsinghal59] PHOENIX-3134 varbinary fields bulk load difference between MR/psql and



Build times for last couple of runsLatest build time is the right most | Legend blue: normal, red: test failure, gray: timeout

Apache-Phoenix | 4.x-HBase-0.98 | Build Successful

2017-01-18 Thread Apache Jenkins Server 
4.x-HBase-0.98 branch build status Successful

Source repository https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=shortlog;h=refs/heads/4.x-HBase-0.98

Compiled Artifacts https://builds.apache.org/job/Phoenix-4.x-HBase-0.98/lastSuccessfulBuild/artifact/

Test Report https://builds.apache.org/job/Phoenix-4.x-HBase-0.98/lastCompletedBuild/testReport/

Changes
[ankitsinghal59] PHOENIX-3134 varbinary fields bulk load difference between MR/psql and



Build times for last couple of runsLatest build time is the right most | Legend blue: normal, red: test failure, gray: timeout

Build failed in Jenkins: Phoenix-4.x-HBase-1.1 #311

2017-01-18 Thread Apache Jenkins Server 
See 

Changes:

[ankitsinghal59] PHOENIX-3134 varbinary fields bulk load difference between 
MR/psql and

--
[...truncated 411 lines...]
Running org.apache.phoenix.expression.function.BuiltinFunctionConstructorTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.101 sec - in 
org.apache.phoenix.expression.function.BuiltinFunctionConstructorTest
Running org.apache.phoenix.expression.function.InstrFunctionTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec - in 
org.apache.phoenix.expression.function.InstrFunctionTest
Running org.apache.phoenix.expression.ArrayFillFunctionTest
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 sec - in 
org.apache.phoenix.expression.ArrayFillFunctionTest
Running org.apache.phoenix.expression.RegexpReplaceFunctionTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 sec - in 
org.apache.phoenix.expression.RegexpReplaceFunctionTest
Running org.apache.phoenix.expression.SqrtFunctionTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.018 sec - in 
org.apache.phoenix.expression.SqrtFunctionTest
Running org.apache.phoenix.expression.CbrtFunctionTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec - in 
org.apache.phoenix.expression.CbrtFunctionTest
Running org.apache.phoenix.expression.LnLogFunctionTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec - in 
org.apache.phoenix.expression.LnLogFunctionTest
Running org.apache.phoenix.expression.ColumnExpressionTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec - in 
org.apache.phoenix.expression.ColumnExpressionTest
Running org.apache.phoenix.expression.AbsFunctionTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.066 sec - in 
org.apache.phoenix.expression.AbsFunctionTest
Running org.apache.phoenix.expression.StringToArrayFunctionTest
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.051 sec - in 
org.apache.phoenix.expression.StringToArrayFunctionTest
Running org.apache.phoenix.query.OrderByTest
Tests run: 24, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.586 sec - in 
org.apache.phoenix.index.IndexMaintainerTest
Running org.apache.phoenix.query.ConnectionlessTest
Tests run: 35, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.291 sec - in 
org.apache.phoenix.expression.ArrayConcatFunctionTest
Running org.apache.phoenix.query.KeyRangeIntersectTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.213 sec - in 
org.apache.phoenix.expression.GetSetByteBitFunctionTest
Running org.apache.phoenix.query.KeyRangeUnionTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.093 sec - in 
org.apache.phoenix.query.KeyRangeIntersectTest
Running org.apache.phoenix.query.HBaseFactoryProviderTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec - in 
org.apache.phoenix.query.HBaseFactoryProviderTest
Running org.apache.phoenix.query.ScannerLeaseRenewalTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.106 sec - in 
org.apache.phoenix.query.KeyRangeUnionTest
Running org.apache.phoenix.query.ParallelIteratorsSplitTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.424 sec - in 
org.apache.phoenix.query.OrderByTest
Running org.apache.phoenix.query.QueryPlanTest
Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.433 sec - in 
org.apache.phoenix.expression.RoundFloorCeilExpressionsTest
Running org.apache.phoenix.query.PhoenixStatsCacheRemovalListenerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.022 sec - in 
org.apache.phoenix.query.PhoenixStatsCacheRemovalListenerTest
Running org.apache.phoenix.query.KeyRangeCoalesceTest
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.093 sec - in 
org.apache.phoenix.query.KeyRangeCoalesceTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.802 sec - in 
org.apache.phoenix.query.ConnectionlessTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.744 sec - in 
org.apache.phoenix.query.ParallelIteratorsSplitTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.848 sec - in 
org.apache.phoenix.query.QueryPlanTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.579 sec - in 
org.apache.phoenix.jdbc.SecureUserConnectionsTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.75 sec - in 
org.apache.phoenix.query.ScannerLeaseRenewalTest

Results :

Tests run: 1448, Failures: 0, Errors: 0, Skipped: 6

[INFO] 
[INFO] --- maven-source-plugin:2.2.1:jar-no-fork (attach-sources) @ 
phoenix-core ---
[INFO] Building jar: 

[INFO] 
[

phoenix git commit: PHOENIX-3134 varbinary fields bulk load difference between MR/psql and upserts

2017-01-18 Thread ankit 
Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-0.98 4f97085ee -> c7bb3faff


PHOENIX-3134 varbinary fields bulk load difference between MR/psql and upserts


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/c7bb3faf
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/c7bb3faf
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/c7bb3faf

Branch: refs/heads/4.x-HBase-0.98
Commit: c7bb3faff5c2a3c19198e681e6989d25e57de2ba
Parents: 4f97085
Author: Ankit Singhal 
Authored: Wed Jan 18 13:35:43 2017 +0530
Committer: Ankit Singhal 
Committed: Wed Jan 18 13:35:43 2017 +0530

--
 .../expression/function/EncodeFormat.java   |  4 +-
 .../phoenix/mapreduce/CsvBulkImportUtil.java|  7 +-
 .../phoenix/mapreduce/CsvBulkLoadTool.java  | 12 ++-
 .../org/apache/phoenix/query/QueryServices.java |  3 +
 .../phoenix/query/QueryServicesOptions.java |  9 ++-
 .../apache/phoenix/schema/types/PBinary.java|  6 +-
 .../apache/phoenix/schema/types/PVarbinary.java |  5 +-
 .../org/apache/phoenix/util/PhoenixRuntime.java | 11 +++
 .../phoenix/util/csv/CsvUpsertExecutor.java | 25 ++
 .../phoenix/util/json/JsonUpsertExecutor.java   | 44 +++
 .../mapreduce/CsvBulkImportUtilTest.java|  8 +-
 .../util/AbstractUpsertExecutorTest.java| 82 +---
 .../phoenix/util/csv/CsvUpsertExecutorTest.java | 26 +++
 .../util/json/JsonUpsertExecutorTest.java   |  6 ++
 14 files changed, 207 insertions(+), 41 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/c7bb3faf/phoenix-core/src/main/java/org/apache/phoenix/expression/function/EncodeFormat.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/EncodeFormat.java
 
b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/EncodeFormat.java
index ca6cb66..8130228 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/EncodeFormat.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/EncodeFormat.java
@@ -20,5 +20,7 @@ package org.apache.phoenix.expression.function;
 public enum EncodeFormat {
 
HEX, //format for encoding HEX value to bytes
-   BASE62 //format for encoding a base 10 long value to base 62 string
+   BASE62, //format for encoding a base 10 long value to base 62 string
+   BASE64, //format for encoding a base 10 long value to base 64 string
+   ASCII // Plain Text
 };

http://git-wip-us.apache.org/repos/asf/phoenix/blob/c7bb3faf/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkImportUtil.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkImportUtil.java
 
b/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkImportUtil.java
index 9289dbf..ff9ff72 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkImportUtil.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkImportUtil.java
@@ -22,6 +22,7 @@ import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.hbase.util.Base64;
 import org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil;
 import org.apache.phoenix.query.QueryConstants;
+import org.apache.phoenix.query.QueryServices;
 
 import com.google.common.annotations.VisibleForTesting;
 
@@ -38,15 +39,19 @@ public class CsvBulkImportUtil {
  * @param quoteChar quote character for the CSV input
  * @param escapeChar escape character for the CSV input
  * @param arrayDelimiter array delimiter character, can be null
+ * @param binaryEncoding 
  */
 public static void initCsvImportJob(Configuration conf, char 
fieldDelimiter, char quoteChar,
-char escapeChar, String arrayDelimiter) {
+char escapeChar, String arrayDelimiter, String binaryEncoding) {
 setChar(conf, CsvToKeyValueMapper.FIELD_DELIMITER_CONFKEY, 
fieldDelimiter);
 setChar(conf, CsvToKeyValueMapper.QUOTE_CHAR_CONFKEY, quoteChar);
 setChar(conf, CsvToKeyValueMapper.ESCAPE_CHAR_CONFKEY, escapeChar);
 if (arrayDelimiter != null) {
 conf.set(CsvToKeyValueMapper.ARRAY_DELIMITER_CONFKEY, 
arrayDelimiter);
 }
+if(binaryEncoding!=null){
+conf.set(QueryServices.UPLOAD_BINARY_DATA_TYPE_ENCODING, 
binaryEncoding);
+}
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/phoenix/blob/c7bb3faf/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkLoadTool.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkLoadTool.java 
b/phoenix-cor

phoenix git commit: PHOENIX-3134 varbinary fields bulk load difference between MR/psql and upserts

2017-01-18 Thread ankit 
Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-1.1 5bb383a9f -> 5bf44a99b


PHOENIX-3134 varbinary fields bulk load difference between MR/psql and upserts


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/5bf44a99
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/5bf44a99
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/5bf44a99

Branch: refs/heads/4.x-HBase-1.1
Commit: 5bf44a99bc505dc5fabac0715b99d274a9b89d2b
Parents: 5bb383a
Author: Ankit Singhal 
Authored: Wed Jan 18 13:30:22 2017 +0530
Committer: Ankit Singhal 
Committed: Wed Jan 18 13:30:22 2017 +0530

--
 .../expression/function/EncodeFormat.java   |  4 +-
 .../phoenix/mapreduce/CsvBulkImportUtil.java|  7 +-
 .../phoenix/mapreduce/CsvBulkLoadTool.java  | 12 ++-
 .../org/apache/phoenix/query/QueryServices.java |  3 +
 .../phoenix/query/QueryServicesOptions.java |  9 ++-
 .../apache/phoenix/schema/types/PBinary.java|  6 +-
 .../apache/phoenix/schema/types/PVarbinary.java |  5 +-
 .../org/apache/phoenix/util/PhoenixRuntime.java | 11 +++
 .../phoenix/util/csv/CsvUpsertExecutor.java | 25 ++
 .../phoenix/util/json/JsonUpsertExecutor.java   | 44 +++
 .../mapreduce/CsvBulkImportUtilTest.java|  8 +-
 .../util/AbstractUpsertExecutorTest.java| 82 +---
 .../phoenix/util/csv/CsvUpsertExecutorTest.java | 26 +++
 .../util/json/JsonUpsertExecutorTest.java   |  6 ++
 14 files changed, 207 insertions(+), 41 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/5bf44a99/phoenix-core/src/main/java/org/apache/phoenix/expression/function/EncodeFormat.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/EncodeFormat.java
 
b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/EncodeFormat.java
index ca6cb66..8130228 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/EncodeFormat.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/EncodeFormat.java
@@ -20,5 +20,7 @@ package org.apache.phoenix.expression.function;
 public enum EncodeFormat {
 
HEX, //format for encoding HEX value to bytes
-   BASE62 //format for encoding a base 10 long value to base 62 string
+   BASE62, //format for encoding a base 10 long value to base 62 string
+   BASE64, //format for encoding a base 10 long value to base 64 string
+   ASCII // Plain Text
 };

http://git-wip-us.apache.org/repos/asf/phoenix/blob/5bf44a99/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkImportUtil.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkImportUtil.java
 
b/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkImportUtil.java
index 9289dbf..ff9ff72 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkImportUtil.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkImportUtil.java
@@ -22,6 +22,7 @@ import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.hbase.util.Base64;
 import org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil;
 import org.apache.phoenix.query.QueryConstants;
+import org.apache.phoenix.query.QueryServices;
 
 import com.google.common.annotations.VisibleForTesting;
 
@@ -38,15 +39,19 @@ public class CsvBulkImportUtil {
  * @param quoteChar quote character for the CSV input
  * @param escapeChar escape character for the CSV input
  * @param arrayDelimiter array delimiter character, can be null
+ * @param binaryEncoding 
  */
 public static void initCsvImportJob(Configuration conf, char 
fieldDelimiter, char quoteChar,
-char escapeChar, String arrayDelimiter) {
+char escapeChar, String arrayDelimiter, String binaryEncoding) {
 setChar(conf, CsvToKeyValueMapper.FIELD_DELIMITER_CONFKEY, 
fieldDelimiter);
 setChar(conf, CsvToKeyValueMapper.QUOTE_CHAR_CONFKEY, quoteChar);
 setChar(conf, CsvToKeyValueMapper.ESCAPE_CHAR_CONFKEY, escapeChar);
 if (arrayDelimiter != null) {
 conf.set(CsvToKeyValueMapper.ARRAY_DELIMITER_CONFKEY, 
arrayDelimiter);
 }
+if(binaryEncoding!=null){
+conf.set(QueryServices.UPLOAD_BINARY_DATA_TYPE_ENCODING, 
binaryEncoding);
+}
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/phoenix/blob/5bf44a99/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkLoadTool.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkLoadTool.java 
b/phoenix-core/