[carbondata] branch branch-1.6 updated (79b533f -> a73cadd)

2019-08-12 Thread ravipesala 
This is an automated email from the ASF dual-hosted git repository.

ravipesala pushed a change to branch branch-1.6
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


 discard 79b533f  [maven-release-plugin] prepare release 
apache-CarbonData-1.6.0-rc2

This update removed existing revisions from the reference, leaving the
reference pointing at a previous point in the repository history.

 * -- * -- N   refs/heads/branch-1.6 (a73cadd)
\
 O -- O -- O   (79b533f)

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

No new revisions were added by this update.

Summary of changes:
 assembly/pom.xml  |  2 +-
 common/pom.xml|  2 +-
 core/pom.xml  |  2 +-
 datamap/bloom/pom.xml |  6 --
 datamap/examples/pom.xml  |  6 --
 datamap/lucene/pom.xml|  6 --
 datamap/mv/core/pom.xml   |  2 +-
 datamap/mv/plan/pom.xml   |  2 +-
 examples/flink/pom.xml|  2 +-
 examples/spark2/pom.xml   |  2 +-
 format/pom.xml|  2 +-
 hadoop/pom.xml|  2 +-
 integration/hive/pom.xml  |  2 +-
 integration/presto/pom.xml|  2 +-
 integration/spark-common-test/pom.xml | 14 +++---
 integration/spark-common/pom.xml  |  2 +-
 integration/spark-datasource/pom.xml  |  2 +-
 integration/spark2/pom.xml|  2 +-
 pom.xml   |  4 ++--
 processing/pom.xml|  2 +-
 store/sdk/pom.xml |  6 --
 streaming/pom.xml |  6 --
 tools/cli/pom.xml |  6 --
 23 files changed, 48 insertions(+), 36 deletions(-)

[carbondata] branch branch-1.6 updated: [maven-release-plugin] prepare release apache-CarbonData-1.6.0-rc2

2019-08-12 Thread raghunandan 
This is an automated email from the ASF dual-hosted git repository.

raghunandan pushed a commit to branch branch-1.6
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/branch-1.6 by this push:
 new 79b533f  [maven-release-plugin] prepare release 
apache-CarbonData-1.6.0-rc2
79b533f is described below

commit 79b533f4228614b7ef5a126a79885df8cf24203e
Author: Raghunandan S 
AuthorDate: Tue Aug 13 10:41:49 2019 +0530

[maven-release-plugin] prepare release apache-CarbonData-1.6.0-rc2
---
 assembly/pom.xml  |  2 +-
 common/pom.xml|  2 +-
 core/pom.xml  |  2 +-
 datamap/bloom/pom.xml |  6 ++
 datamap/examples/pom.xml  |  6 ++
 datamap/lucene/pom.xml|  6 ++
 datamap/mv/core/pom.xml   |  2 +-
 datamap/mv/plan/pom.xml   |  2 +-
 examples/flink/pom.xml|  2 +-
 examples/spark2/pom.xml   |  2 +-
 format/pom.xml|  2 +-
 hadoop/pom.xml|  2 +-
 integration/hive/pom.xml  |  2 +-
 integration/presto/pom.xml|  2 +-
 integration/spark-common-test/pom.xml | 14 +++---
 integration/spark-common/pom.xml  |  2 +-
 integration/spark-datasource/pom.xml  |  2 +-
 integration/spark2/pom.xml|  2 +-
 pom.xml   |  4 ++--
 processing/pom.xml|  2 +-
 store/sdk/pom.xml |  6 ++
 streaming/pom.xml |  6 ++
 tools/cli/pom.xml |  6 ++
 23 files changed, 36 insertions(+), 48 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index 9ac2e78..004788c 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-1.6.0-SNAPSHOT
+1.6.0
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index 14cd52f..8e5ddaa 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-1.6.0-SNAPSHOT
+1.6.0
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index 41481af..b39a42e 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-1.6.0-SNAPSHOT
+1.6.0
 ../pom.xml
   
 
diff --git a/datamap/bloom/pom.xml b/datamap/bloom/pom.xml
index 1e8c382..a29f77b 100644
--- a/datamap/bloom/pom.xml
+++ b/datamap/bloom/pom.xml
@@ -1,12 +1,10 @@
-http://maven.apache.org/POM/4.0.0;
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
   4.0.0
 
   
 org.apache.carbondata
 carbondata-parent
-1.6.0-SNAPSHOT
+1.6.0
 ../../pom.xml
   
 
diff --git a/datamap/examples/pom.xml b/datamap/examples/pom.xml
index 3720a1c..a9c179d 100644
--- a/datamap/examples/pom.xml
+++ b/datamap/examples/pom.xml
@@ -15,16 +15,14 @@
 See the License for the specific language governing permissions and
 limitations under the License.
 -->
-http://maven.apache.org/POM/4.0.0;
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
 
   4.0.0
 
   
 org.apache.carbondata
 carbondata-parent
-1.6.0-SNAPSHOT
+1.6.0
 ../../pom.xml
   
 
diff --git a/datamap/lucene/pom.xml b/datamap/lucene/pom.xml
index 3e93a83..1a23a52 100644
--- a/datamap/lucene/pom.xml
+++ b/datamap/lucene/pom.xml
@@ -1,12 +1,10 @@
-http://maven.apache.org/POM/4.0.0;
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
   4.0.0
 
   
 org.apache.carbondata
 carbondata-parent
-1.6.0-SNAPSHOT
+1.6.0
 ../../pom.xml
   
 
diff --git a/datamap/mv/core/pom.xml b/datamap/mv/core/pom.xml
index 0a1f0e2..5cb284d 100644
--- a/datamap/mv/core/pom.xml
+++ b/datamap/mv/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-1.6.0-SNAPSHOT
+1.6.0
 ../../../pom.xml
   
 
diff --git a/datamap/mv/plan/pom.xml

Jenkins build is still unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #1893

2019-08-12 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Store SDK #1893

2019-08-12 Thread Apache Jenkins Server 
See

Build failed in Jenkins: carbondata-master-spark-2.1 #3688

2019-08-12 Thread Apache Jenkins Server 
See 


Changes:

[ravipesala] [CARBONDATA-3490] Fix concurrent data load failure with carbondata

--
[...truncated 56.69 MB...]
|robot2|  2|   1.0|
|robot3|  3|   1.5|
|robot4|  4|   2.0|
|robot5|  5|   2.5|
|robot6|  6|   3.0|
|robot7|  7|   3.5|
|robot8|  8|   4.0|
|robot9|  9|   4.5|
+--+---+--+

- DirectSQLExample
2019-08-12 16:49:05 AUDIT audit:72 - {"time":"August 12, 2019 9:49:05 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2535918165841192","opStatus":"START"}
2019-08-12 16:49:05 AUDIT audit:93 - {"time":"August 12, 2019 9:49:05 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2535918165841192","opStatus":"SUCCESS","opTime":"109 
ms","table":"default.hive_carbon_example","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":""}}
2019-08-12 16:49:05 AUDIT audit:72 - {"time":"August 12, 2019 9:49:05 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2535918301556327","opStatus":"START"}
2019-08-12 16:49:05 AUDIT audit:93 - {"time":"August 12, 2019 9:49:05 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2535918301556327","opStatus":"SUCCESS","opTime":"186 
ms","table":"default.hive_carbon_example","extraInfo":{"SegmentId":"0","DataSize":"924.0B","IndexSize":"551.0B"}}
2019-08-12 16:49:05 AUDIT audit:72 - {"time":"August 12, 2019 9:49:05 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2535918497807509","opStatus":"START"}
2019-08-12 16:49:05 AUDIT audit:93 - {"time":"August 12, 2019 9:49:05 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2535918497807509","opStatus":"SUCCESS","opTime":"161 
ms","table":"default.hive_carbon_example","extraInfo":{"SegmentId":"1","DataSize":"924.0B","IndexSize":"551.0B"}}
+---+-++
| id| name|  salary|
+---+-++
|  1|  'liang'|20.0|
|  2|'anubhav'| 2.0|
|  1|  'liang'|20.0|
|  2|'anubhav'| 2.0|
+---+-++

2019-08-12 16:49:05 AUDIT audit:72 - {"time":"August 12, 2019 9:49:05 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2535918802303890","opStatus":"START"}
2019-08-12 16:49:05 AUDIT audit:93 - {"time":"August 12, 2019 9:49:05 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2535918802303890","opStatus":"SUCCESS","opTime":"73 
ms","table":"default.test_boundary","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":"","dictionary_include":"c6_timestamp"}}
2019-08-12 16:49:05 AUDIT audit:72 - {"time":"August 12, 2019 9:49:05 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2535918881208709","opStatus":"START"}
2019-08-12 16:49:05 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table test_boundary
2019-08-12 16:49:05 AUDIT audit:93 - {"time":"August 12, 2019 9:49:05 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2535918881208709","opStatus":"SUCCESS","opTime":"202 
ms","table":"default.test_boundary","extraInfo":{"SegmentId":"0","DataSize":"4.03KB","IndexSize":"1.06KB"}}
OK
**Total Number Of Rows Fetched ** 0
- HiveExample *** FAILED ***
  java.lang.AssertionError: assertion failed
  at scala.Predef$.assert(Predef.scala:156)
  at 
org.apache.carbondata.examples.HiveExample$.readFromHive(HiveExample.scala:171)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply$mcV$sp(RunExamples.scala:135)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply(RunExamples.scala:133)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply(RunExamples.scala:133)
  at 
org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  ...
Exception encountered when invoking run on a nested suite - Cannot call 
methods on a stopped SparkContext.
This stopped SparkContext was created at:

org.apache.spark.sql.test.util.QueryTest.(QueryTest.scala:115)
org.apache.carbondata.examplesCI.RunExamples.(RunExamples.scala:35)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
java.lang.Class.newInstance(Class.java:442)

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark2 #3688

2019-08-12 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark Common Test #3688

2019-08-12 Thread Apache Jenkins Server 
See

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Examples #3688

2019-08-12 Thread Apache Jenkins Server 
See 


--
[...truncated 262.21 KB...]
- TimeSeriesPreAggregateTableExample
2019-08-12 16:48:57 AUDIT audit:72 - {"time":"August 12, 2019 9:48:57 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2535910583259725","opStatus":"START"}
2019-08-12 16:48:57 AUDIT audit:93 - {"time":"August 12, 2019 9:48:57 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2535910583259725","opStatus":"SUCCESS","opTime":"93 
ms","table":"default.persontable","extraInfo":{"bad_record_path":"","streaming":"false","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":""}}
2019-08-12 16:48:57 AUDIT audit:72 - {"time":"August 12, 2019 9:48:57 AM 
PDT","username":"jenkins","opName":"LOAD DATA 
OVERWRITE","opId":"2535910679823574","opStatus":"START"}
2019-08-12 16:48:58 AUDIT audit:93 - {"time":"August 12, 2019 9:48:58 AM 
PDT","username":"jenkins","opName":"LOAD DATA 
OVERWRITE","opId":"2535910679823574","opStatus":"SUCCESS","opTime":"775 
ms","table":"default.personTable","extraInfo":{"SegmentId":"0","DataSize":"771.31KB","IndexSize":"720.0B"}}
2019-08-12 16:48:58 AUDIT audit:72 - {"time":"August 12, 2019 9:48:58 AM 
PDT","username":"jenkins","opName":"CREATE 
DATAMAP","opId":"2535911463224845","opStatus":"START"}
2019-08-12 16:49:01 AUDIT audit:93 - {"time":"August 12, 2019 9:49:01 AM 
PDT","username":"jenkins","opName":"CREATE 
DATAMAP","opId":"2535911463224845","opStatus":"SUCCESS","opTime":"2922 
ms","table":"default.persontable","extraInfo":{"provider":"lucene","dmName":"dm","index_columns":"id
 , name"}}
++
|count(1)|
++
|   6|
++

++
|count(1)|
++
|   6|
++

time for query on table with lucene datamap table:0.384
time for query on table without lucene datamap table:0.282
+-+-+
|   id| name|
+-+-+
|which test1 good0|who and name4|
|which test1 good6|who and name3|
|which test1 good4|who and name1|
|which test1 good0|who and name4|
|which test1 good6|who and name3|
|which test1 good4|who and name1|
+-+-+

+-+-+
|   id| name|
+-+-+
|which test1 good0|who and name4|
|which test1 good6|who and name3|
|which test1 good4|who and name1|
|which test1 good0|who and name4|
|which test1 good6|who and name3|
|which test1 good4|who and name1|
+-+-+

2019-08-12 16:49:02 AUDIT audit:72 - {"time":"August 12, 2019 9:49:02 AM 
PDT","username":"jenkins","opName":"DROP 
TABLE","opId":"2535915541426316","opStatus":"START"}
2019-08-12 16:49:02 AUDIT audit:93 - {"time":"August 12, 2019 9:49:02 AM 
PDT","username":"jenkins","opName":"DROP 
TABLE","opId":"2535915541426316","opStatus":"SUCCESS","opTime":"160 
ms","table":"default.persontable","extraInfo":{}}
- LuceneDataMapExample
2019-08-12 16:49:02 AUDIT audit:72 - {"time":"August 12, 2019 9:49:02 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2535915719550716","opStatus":"START"}
2019-08-12 16:49:02 AUDIT audit:93 - {"time":"August 12, 2019 9:49:02 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2535915719550716","opStatus":"SUCCESS","opTime":"80 
ms","table":"default.origin_table","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":""}}
2019-08-12 16:49:02 AUDIT audit:72 - {"time":"August 12, 2019 9:49:02 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2535915806321698","opStatus":"START"}
2019-08-12 16:49:02 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-12 16:49:02 AUDIT audit:93 - {"time":"August 12, 2019 9:49:02 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2535915806321698","opStatus":"SUCCESS","opTime":"189 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"0","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-12 16:49:02 AUDIT audit:72 - {"time":"August 12, 2019 9:49:02 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2535916001505705","opStatus":"START"}
2019-08-12 16:49:02 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-12 16:49:03 AUDIT audit:93 - {"time":"August 12, 2019 9:49:03 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2535916001505705","opStatus":"SUCCESS","opTime":"174 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"1","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-12 16:49:03 AUDIT audit:72 - {"time":"August 12, 2019 9:49:03 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2535916182268094","opStatus":"START"}
2019-08-12 16:49:03 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-12

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #3688

2019-08-12 Thread Apache Jenkins Server 
See

[carbondata] branch branch-1.6 updated (61a5bd3 -> a73cadd)

2019-08-12 Thread ravipesala 
This is an automated email from the ASF dual-hosted git repository.

ravipesala pushed a change to branch branch-1.6
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


from 61a5bd3  [HOTFIX] Fix dictionary include issue with codegen failure
 add d5db3f4  [CARBONDATA-3488] Check the file size after move local file 
to carbon path
 add a73cadd  [CARBONDATA-3490] Fix concurrent data load failure with 
carbondata FileNotFound exception

No new revisions were added by this update.

Summary of changes:
 .../apache/carbondata/core/util/CarbonUtil.java| 22 +++---
 .../apache/carbondata/spark/util/CommonUtil.scala  |  7 +--
 2 files changed, 24 insertions(+), 5 deletions(-)

[carbondata] branch master updated: [CARBONDATA-3490] Fix concurrent data load failure with carbondata FileNotFound exception

2019-08-12 Thread ravipesala 
This is an automated email from the ASF dual-hosted git repository.

ravipesala pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
 new a73cadd  [CARBONDATA-3490] Fix concurrent data load failure with 
carbondata FileNotFound exception
a73cadd is described below

commit a73cadda438de57713ffc5fd85a86b4fdb5442c7
Author: ajantha-bhat 
AuthorDate: Fri Aug 9 10:19:32 2019 +0530

[CARBONDATA-3490] Fix concurrent data load failure with carbondata 
FileNotFound exception

problem: When two load is happening concurrently, one load is cleaning the 
temp directory of the concurrent load

cause: temp directory to store the carbon files is created using system.get 
nano time, due to this two load have same store location. when one load is 
completed, it cleaned the temp directory. causing dataload failure for other 
load.

solution:
use UUID instead of nano time while creating the temp directory to have 
each load a unique directory.

This closes #3352
---
 .../main/scala/org/apache/carbondata/spark/util/CommonUtil.scala   | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git 
a/integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
 
b/integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
index 7015279..8d6cdfb 100644
--- 
a/integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
+++ 
b/integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
@@ -21,6 +21,7 @@ import java.io.File
 import java.math.BigDecimal
 import java.text.SimpleDateFormat
 import java.util
+import java.util.UUID
 import java.util.regex.{Matcher, Pattern}
 
 import scala.collection.JavaConverters._
@@ -777,8 +778,10 @@ object CommonUtil {
 val isCarbonUseYarnLocalDir = CarbonProperties.getInstance().getProperty(
   CarbonCommonConstants.CARBON_LOADING_USE_YARN_LOCAL_DIR,
   
CarbonCommonConstants.CARBON_LOADING_USE_YARN_LOCAL_DIR_DEFAULT).equalsIgnoreCase("true")
-val tmpLocationSuffix =
-  
s"${File.separator}carbon${System.nanoTime()}${CarbonCommonConstants.UNDERSCORE}$index"
+val tmpLocationSuffix = s"${ File.separator }carbon${
+  UUID.randomUUID().toString
+.replace("-", "")
+}${ CarbonCommonConstants.UNDERSCORE }$index"
 if (isCarbonUseYarnLocalDir) {
   val yarnStoreLocations = Util.getConfiguredLocalDirs(SparkEnv.get.conf)

[carbondata] branch branch-1.6 updated (9724fd4 -> 61a5bd3)

2019-08-12 Thread ravipesala 
This is an automated email from the ASF dual-hosted git repository.

ravipesala pushed a change to branch branch-1.6
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


 discard 9724fd4  [maven-release-plugin] prepare for next development iteration
omit 9ca7891  [maven-release-plugin] prepare release 
apache-CarbonData-1.6.0-rc2
omit 80438f7  [HOTFIX] Removed the hive-exec and commons dependency from 
hive module
omit 575b711  [CARBONDATA-3481] Multi-thread pruning fails when datamaps 
count is just near numOfThreadsForPruning
omit 2ebc041  [CARBONDATA-3478]Fix ArrayIndexOutOfBound Exception on 
compaction after alter operation
omit 917e041  [HOTFIX] CLI test case failed during release because of space 
differences
 add c8cc92b  [CARBONDATA-3478]Fix ArrayIndexOutOfBound Exception on 
compaction after alter operation
 add 10f3747  [HOTFIX] CLI test case failed during release because of space 
differences
 add 765712a  [CARBONDATA-3481] Multi-thread pruning fails when datamaps 
count is just near numOfThreadsForPruning
 add d7d70a8  [HOTFIX] Removed the hive-exec and commons dependency from 
hive module
 add f005fd4  [CARBONDATA-3477] deal line break chars correctly after 
'select' in 'update ... select columns' sql
 add 35f1501  [CARBONDATA-3483] don't require update.lock and 
compaction.lock again when execute 'IUD_UPDDEL_DELTA' compaction
 add e14c817  [CARBONDATA-3485] Data loading is failed from S3 to hdfs 
table having ~2K carbonfiles
 add 88ec830  [CARBONDATA-3476] Fix Read time and scan time stats in 
executor log for filter query
 add ebe4057  [CARBONDATA-3452] Fix select query failure when substring on 
dictionary column with join
 add bbeb974  [CARBONDATA-3487] wrong Input metrics (size/record) displayed 
in spark UI during insert into
 add aa67a99  [CARBONDATA-3486] Fix Serialization/Deserialization issue 
with DataType
 add 8f0724e  [CARBONDATA-3482] Fixed NPE in Concurrent query
 add 61a5bd3  [HOTFIX] Fix dictionary include issue with codegen failure

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (9724fd4)
\
 N -- N -- N   refs/heads/branch-1.6 (61a5bd3)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

No new revisions were added by this update.

Summary of changes:
 assembly/pom.xml   |  2 +-
 common/pom.xml |  2 +-
 core/pom.xml   |  2 +-
 .../core/constants/CarbonCommonConstants.java  | 10 
 .../block/SegmentPropertiesAndSchemaHolder.java| 21 ---
 .../core/indexstore/BlockletDataMapIndexStore.java |  2 +-
 .../indexstore/blockletindex/BlockDataMap.java | 36 
 .../indexstore/blockletindex/BlockletDataMap.java  |  7 +--
 .../carbondata/core/scan/filter/FilterUtil.java|  5 +-
 .../MeasureColumnResolvedFilterInfo.java   |  3 +-
 .../AbstractDetailQueryResultIterator.java | 10 ++--
 .../scan/scanner/impl/BlockletFilterScanner.java   | 34 
 .../carbondata/core/util/CarbonProperties.java | 24 
 .../carbondata/core/util/TaskMetricsMap.java   | 21 ++-
 datamap/bloom/pom.xml  |  6 +-
 datamap/examples/pom.xml   |  6 +-
 datamap/lucene/pom.xml |  6 +-
 datamap/mv/core/pom.xml|  2 +-
 datamap/mv/plan/pom.xml|  2 +-
 examples/flink/pom.xml |  2 +-
 examples/spark2/pom.xml|  2 +-
 format/pom.xml |  2 +-
 hadoop/pom.xml |  2 +-
 integration/hive/pom.xml   |  2 +-
 integration/presto/pom.xml |  2 +-
 integration/spark-common-test/pom.xml  | 14 ++---
 ...ryWithColumnMetCacheAndCacheLevelProperty.scala | 27 +
 .../iud/HorizontalCompactionTestCase.scala | 64 +-
 .../testsuite/iud/UpdateCarbonTableTestCase.scala  | 42 +-
 integration/spark-common/pom.xml   |  2 +-
 .../apache/carbondata/spark/InitInputMetrics.java  |  2 +-
 .../spark/load/DataLoadProcessBuilderOnSpark.scala |  2 +-
 .../apache/carbondata/spark/rdd/CarbonRDD.scala|  4 +-
 .../carbondata/spark/rdd/CarbonScanRDD.scala   |  2 +-

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark Common Test #3687

2019-08-12 Thread Apache Jenkins Server 
See

Build failed in Jenkins: carbondata-master-spark-2.1 #3687

2019-08-12 Thread Apache Jenkins Server 
See 


--
[...truncated 12.55 MB...]
+---+-++
| id| name|  salary|
+---+-++
|  1|  'liang'|20.0|
|  2|'anubhav'| 2.0|
|  1|  'liang'|20.0|
|  2|'anubhav'| 2.0|
+---+-++

2019-08-12 09:56:13 AUDIT audit:72 - {"time":"August 12, 2019 2:56:13 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2166994781004282","opStatus":"START"}
2019-08-12 09:56:13 AUDIT audit:93 - {"time":"August 12, 2019 2:56:13 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2166994781004282","opStatus":"SUCCESS","opTime":"58 
ms","table":"default.test_boundary","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":"","dictionary_include":"c6_timestamp"}}
2019-08-12 09:56:13 AUDIT audit:72 - {"time":"August 12, 2019 2:56:13 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2166994843546202","opStatus":"START"}
2019-08-12 09:56:13 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table test_boundary
2019-08-12 09:56:13 AUDIT audit:93 - {"time":"August 12, 2019 2:56:13 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2166994843546202","opStatus":"SUCCESS","opTime":"155 
ms","table":"default.test_boundary","extraInfo":{"SegmentId":"0","DataSize":"4.03KB","IndexSize":"1.06KB"}}
OK
**Total Number Of Rows Fetched ** 0
- HiveExample *** FAILED ***
  java.lang.AssertionError: assertion failed
  at scala.Predef$.assert(Predef.scala:156)
  at 
org.apache.carbondata.examples.HiveExample$.readFromHive(HiveExample.scala:171)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply$mcV$sp(RunExamples.scala:135)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply(RunExamples.scala:133)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply(RunExamples.scala:133)
  at 
org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  ...
Exception encountered when invoking run on a nested suite - Cannot call 
methods on a stopped SparkContext.
This stopped SparkContext was created at:

org.apache.spark.sql.test.util.QueryTest.(QueryTest.scala:115)
org.apache.carbondata.examplesCI.RunExamples.(RunExamples.scala:35)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
java.lang.Class.newInstance(Class.java:442)
org.scalatest.tools.DiscoverySuite$.getSuiteInstance(DiscoverySuite.scala:69)
org.scalatest.tools.DiscoverySuite$$anonfun$1.apply(DiscoverySuite.scala:38)
org.scalatest.tools.DiscoverySuite$$anonfun$1.apply(DiscoverySuite.scala:37)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
scala.collection.Iterator$class.foreach(Iterator.scala:893)
scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
scala.collection.AbstractIterable.foreach(Iterable.scala:54)
scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
scala.collection.AbstractTraversable.map(Traversable.scala:104)
org.scalatest.tools.DiscoverySuite.(DiscoverySuite.scala:37)
org.scalatest.tools.Runner$.genDiscoSuites$1(Runner.scala:2390)

The currently active SparkContext was created at:

(No active SparkContext.)
  *** ABORTED ***
  java.lang.IllegalStateException: Cannot call methods on a stopped 
SparkContext.
This stopped SparkContext was created at:

org.apache.spark.sql.test.util.QueryTest.(QueryTest.scala:115)
org.apache.carbondata.examplesCI.RunExamples.(RunExamples.scala:35)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark2 #3687

2019-08-12 Thread Apache Jenkins Server 
See

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Examples #3687

2019-08-12 Thread Apache Jenkins Server 
See 


--
[...truncated 264.34 KB...]
|count(1)|
++
|  16|
++

time for query on table with lucene datamap table:0.245
time for query on table without lucene datamap table:0.167
+-+-+
|   id| name|
+-+-+
|which test1 good6|who and name7|
|which test1 good0|who and name0|
|which test1 good9|who and name0|
|which test1 good5|who and name3|
|which test1 good7|who and name0|
|which test1 good6|who and name4|
|which test1 good5|who and name3|
|which test1 good6|who and name1|
|which test1 good6|who and name7|
|which test1 good0|who and name0|
|which test1 good9|who and name0|
|which test1 good5|who and name3|
|which test1 good7|who and name0|
|which test1 good6|who and name4|
|which test1 good5|who and name3|
|which test1 good6|who and name1|
+-+-+

+-+-+
|   id| name|
+-+-+
|which test1 good6|who and name7|
|which test1 good0|who and name0|
|which test1 good9|who and name0|
|which test1 good5|who and name3|
|which test1 good7|who and name0|
|which test1 good6|who and name4|
|which test1 good5|who and name3|
|which test1 good6|who and name1|
|which test1 good6|who and name7|
|which test1 good0|who and name0|
+-+-+

2019-08-12 09:56:10 AUDIT audit:72 - {"time":"August 12, 2019 2:56:10 AM 
PDT","username":"jenkins","opName":"DROP 
TABLE","opId":"2166991929450767","opStatus":"START"}
2019-08-12 09:56:10 AUDIT audit:93 - {"time":"August 12, 2019 2:56:10 AM 
PDT","username":"jenkins","opName":"DROP 
TABLE","opId":"2166991929450767","opStatus":"SUCCESS","opTime":"119 
ms","table":"default.persontable","extraInfo":{}}
- LuceneDataMapExample
2019-08-12 09:56:10 AUDIT audit:72 - {"time":"August 12, 2019 2:56:10 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2166992060235173","opStatus":"START"}
2019-08-12 09:56:10 AUDIT audit:93 - {"time":"August 12, 2019 2:56:10 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2166992060235173","opStatus":"SUCCESS","opTime":"61 
ms","table":"default.origin_table","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":""}}
2019-08-12 09:56:10 AUDIT audit:72 - {"time":"August 12, 2019 2:56:10 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2166992125933480","opStatus":"START"}
2019-08-12 09:56:10 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-12 09:56:10 AUDIT audit:93 - {"time":"August 12, 2019 2:56:10 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2166992125933480","opStatus":"SUCCESS","opTime":"150 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"0","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-12 09:56:10 AUDIT audit:72 - {"time":"August 12, 2019 2:56:10 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2166992280413350","opStatus":"START"}
2019-08-12 09:56:10 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-12 09:56:11 AUDIT audit:93 - {"time":"August 12, 2019 2:56:11 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2166992280413350","opStatus":"SUCCESS","opTime":"157 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"1","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-12 09:56:11 AUDIT audit:72 - {"time":"August 12, 2019 2:56:11 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2166992442545718","opStatus":"START"}
2019-08-12 09:56:11 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-12 09:56:11 AUDIT audit:93 - {"time":"August 12, 2019 2:56:11 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2166992442545718","opStatus":"SUCCESS","opTime":"161 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"2","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-12 09:56:11 AUDIT audit:72 - {"time":"August 12, 2019 2:56:11 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2166992610683369","opStatus":"START"}
2019-08-12 09:56:11 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-12 09:56:11 AUDIT audit:93 - {"time":"August 12, 2019 2:56:11 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"2166992610683369","opStatus":"SUCCESS","opTime":"166 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"3","DataSize":"2.85KB","IndexSize":"1.43KB"}}
++
|count(1)|
++
|  40|
++

2019-08-12 09:56:11 AUDIT audit:72 - {"time":"August 12, 2019 2:56:11 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"2166992858157588","opStatus":"START"}
2019-08-12 09:56:11 AUDIT audit:93 - {"time":"August 12,

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #3687

2019-08-12 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Store SDK #1892

2019-08-12 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #1892

2019-08-12 Thread Apache Jenkins Server 
See