Jenkins build is still unstable: carbondata-master-spark-2.2 #1871

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Store SDK #1871

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #1871

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark2 #3667

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #3667

2019-08-01 Thread Apache Jenkins Server 
See

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Examples #3667

2019-08-01 Thread Apache Jenkins Server 
See 


--
[...truncated 279.98 KB...]
2019-08-01 16:32:12 AUDIT audit:93 - {"time":"August 1, 2019 9:32:12 AM 
PDT","username":"jenkins","opName":"CREATE 
DATAMAP","opId":"1584502622256881","opStatus":"SUCCESS","opTime":"2601 
ms","table":"default.persontable","extraInfo":{"provider":"lucene","dmName":"dm","index_columns":"id
 , name"}}
++
|count(1)|
++
|  10|
++

++
|count(1)|
++
|  10|
++

time for query on table with lucene datamap table:0.268
time for query on table without lucene datamap table:0.22
+-+-+
|   id| name|
+-+-+
|which test1 good0|who and name5|
|which test1 good5|who and name7|
|which test1 good6|who and name4|
|which test1 good3|who and name6|
|which test1 good0|who and name5|
|which test1 good5|who and name7|
|which test1 good6|who and name4|
|which test1 good3|who and name6|
+-+-+

+-+-+
|   id| name|
+-+-+
|which test1 good1|who and name7|
|which test1 good0|who and name5|
|which test1 good5|who and name7|
|which test1 good6|who and name4|
|which test1 good3|who and name6|
|which test1 good1|who and name7|
|which test1 good0|who and name5|
|which test1 good5|who and name7|
|which test1 good6|who and name4|
|which test1 good3|who and name6|
+-+-+

2019-08-01 16:32:12 AUDIT audit:72 - {"time":"August 1, 2019 9:32:12 AM 
PDT","username":"jenkins","opName":"DROP 
TABLE","opId":"1584506085631049","opStatus":"START"}
2019-08-01 16:32:13 AUDIT audit:93 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"DROP 
TABLE","opId":"1584506085631049","opStatus":"SUCCESS","opTime":"122 
ms","table":"default.persontable","extraInfo":{}}
- LuceneDataMapExample
2019-08-01 16:32:13 AUDIT audit:72 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"1584506221377824","opStatus":"START"}
2019-08-01 16:32:13 AUDIT audit:93 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"1584506221377824","opStatus":"SUCCESS","opTime":"58 
ms","table":"default.origin_table","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":""}}
2019-08-01 16:32:13 AUDIT audit:72 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584506285240206","opStatus":"START"}
2019-08-01 16:32:13 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 16:32:13 AUDIT audit:93 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584506285240206","opStatus":"SUCCESS","opTime":"168 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"0","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-01 16:32:13 AUDIT audit:72 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584506458824114","opStatus":"START"}
2019-08-01 16:32:13 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 16:32:13 AUDIT audit:93 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584506458824114","opStatus":"SUCCESS","opTime":"149 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"1","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-01 16:32:13 AUDIT audit:72 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584506612146585","opStatus":"START"}
2019-08-01 16:32:13 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 16:32:13 AUDIT audit:93 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584506612146585","opStatus":"SUCCESS","opTime":"157 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"2","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-01 16:32:13 AUDIT audit:72 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584506773645013","opStatus":"START"}
2019-08-01 16:32:13 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 16:32:13 AUDIT audit:93 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584506773645013","opStatus":"SUCCESS","opTime":"171 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"3","DataSize":"2.85KB","IndexSize":"1.43KB"}}
++
|count(1)|
++
|  40|
++

2019-08-01 16:32:13 AUDIT audit:72 - {"time":"August 1, 2019 9:32:13 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"15845070107670

Build failed in Jenkins: carbondata-master-spark-2.1 #3667

2019-08-01 Thread Apache Jenkins Server 
See 


Changes:

[kunalkapoor642] [HOTFIX] CLI test case failed during release because of space

[kunalkapoor642] [CARBONDATA-3481] Multi-thread pruning fails when datamaps 
count is just

--
[...truncated 7.44 MB...]
|robot1|  1|   0.5|
|robot2|  2|   1.0|
|robot3|  3|   1.5|
|robot4|  4|   2.0|
|robot5|  5|   2.5|
|robot6|  6|   3.0|
|robot7|  7|   3.5|
|robot8|  8|   4.0|
|robot9|  9|   4.5|
+--+---+--+

- DirectSQLExample
2019-08-01 16:32:15 AUDIT audit:72 - {"time":"August 1, 2019 9:32:15 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"1584508646881260","opStatus":"START"}
2019-08-01 16:32:15 AUDIT audit:93 - {"time":"August 1, 2019 9:32:15 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"1584508646881260","opStatus":"SUCCESS","opTime":"109 
ms","table":"default.hive_carbon_example","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":""}}
2019-08-01 16:32:15 AUDIT audit:72 - {"time":"August 1, 2019 9:32:15 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584508787172793","opStatus":"START"}
2019-08-01 16:32:15 AUDIT audit:93 - {"time":"August 1, 2019 9:32:15 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584508787172793","opStatus":"SUCCESS","opTime":"237 
ms","table":"default.hive_carbon_example","extraInfo":{"SegmentId":"0","DataSize":"924.0B","IndexSize":"551.0B"}}
2019-08-01 16:32:15 AUDIT audit:72 - {"time":"August 1, 2019 9:32:15 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584509037339195","opStatus":"START"}
2019-08-01 16:32:16 AUDIT audit:93 - {"time":"August 1, 2019 9:32:16 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584509037339195","opStatus":"SUCCESS","opTime":"160 
ms","table":"default.hive_carbon_example","extraInfo":{"SegmentId":"1","DataSize":"924.0B","IndexSize":"551.0B"}}
+---+-++
| id| name|  salary|
+---+-++
|  1|  'liang'|20.0|
|  2|'anubhav'| 2.0|
|  1|  'liang'|20.0|
|  2|'anubhav'| 2.0|
+---+-++

2019-08-01 16:32:16 AUDIT audit:72 - {"time":"August 1, 2019 9:32:16 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"1584509348920961","opStatus":"START"}
2019-08-01 16:32:16 AUDIT audit:93 - {"time":"August 1, 2019 9:32:16 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"1584509348920961","opStatus":"SUCCESS","opTime":"81 
ms","table":"default.test_boundary","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":"","dictionary_include":"c6_timestamp"}}
2019-08-01 16:32:16 AUDIT audit:72 - {"time":"August 1, 2019 9:32:16 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584509436756498","opStatus":"START"}
2019-08-01 16:32:16 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table test_boundary
2019-08-01 16:32:16 AUDIT audit:93 - {"time":"August 1, 2019 9:32:16 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"1584509436756498","opStatus":"SUCCESS","opTime":"250 
ms","table":"default.test_boundary","extraInfo":{"SegmentId":"0","DataSize":"4.03KB","IndexSize":"1.06KB"}}
OK
**Total Number Of Rows Fetched ** 0
- HiveExample *** FAILED ***
  java.lang.AssertionError: assertion failed
  at scala.Predef$.assert(Predef.scala:156)
  at 
org.apache.carbondata.examples.HiveExample$.readFromHive(HiveExample.scala:171)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply$mcV$sp(RunExamples.scala:135)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply(RunExamples.scala:133)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply(RunExamples.scala:133)
  at 
org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  ...
Exception encountered when invoking run on a nested suite - Cannot call 
methods on a stopped SparkContext.
This stopped SparkContext was created at:

org.apache.spark.sql.test.util.QueryTest.(QueryTest.scala:115)
org.apache.carbondata.examplesCI.RunExamples.(RunExamples.scala:35)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.ja

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark Common Test #3667

2019-08-01 Thread Apache Jenkins Server 
See

[carbondata] branch branch-1.6 updated: [HOTFIX] CLI test case failed during release because of space differences

2019-08-01 Thread ravipesala 
This is an automated email from the ASF dual-hosted git repository.

ravipesala pushed a commit to branch branch-1.6
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/branch-1.6 by this push:
 new 917e041  [HOTFIX] CLI test case failed during release because of space 
differences
917e041 is described below

commit 917e041439282985ec28ff89249db6088e6771df
Author: ravipesala 
AuthorDate: Thu Aug 1 18:12:50 2019 +0530

[HOTFIX] CLI test case failed during release because of space differences

CLI test case is failed if the release name is short and without snapshot, 
it adds more space.
That's why changed test check the individual contains instead of a batch of 
lines.

This closes #3344
---
 .../src/test/java/org/apache/carbondata/tool/CarbonCliTest.java  | 9 +++--
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git 
a/tools/cli/src/test/java/org/apache/carbondata/tool/CarbonCliTest.java 
b/tools/cli/src/test/java/org/apache/carbondata/tool/CarbonCliTest.java
index 26901f8..af8d51d 100644
--- a/tools/cli/src/test/java/org/apache/carbondata/tool/CarbonCliTest.java
+++ b/tools/cli/src/test/java/org/apache/carbondata/tool/CarbonCliTest.java
@@ -241,12 +241,9 @@ public class CarbonCliTest {
 "20  3.36KB 4.06MB false  00.0B  
93.76KB  0.0   100.0  7298  " ,
 "21  2.04KB 1.49MB false  00.0B  
89.62KB  0.0   100.0  9299  ");
 Assert.assertTrue(output.contains(expectedOutput));
-
-expectedOutput = buildLines(
-"## version Details",
-"written_by  Version ",
-"TestUtil"+ CarbonVersionConstants.CARBONDATA_VERSION+"  ");
-Assert.assertTrue(output.contains(expectedOutput));
+Assert.assertTrue(output.contains("## version Details"));
+Assert.assertTrue(output.contains("written_by  Version"));
+Assert.assertTrue(output.contains("TestUtil"+ 
CarbonVersionConstants.CARBONDATA_VERSION));
   }
 
   @Test

[carbondata] branch master updated: [CARBONDATA-3481] Multi-thread pruning fails when datamaps count is just near numOfThreadsForPruning

2019-08-01 Thread kunalkapoor 
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
 new 765712a  [CARBONDATA-3481] Multi-thread pruning fails when datamaps 
count is just near numOfThreadsForPruning
765712a is described below

commit 765712aad649625fc3c8815cff82996b32d51422
Author: ajantha-bhat 
AuthorDate: Thu Jul 25 18:50:19 2019 +0530

[CARBONDATA-3481] Multi-thread pruning fails when datamaps count is just 
near numOfThreadsForPruning

Cause : When the datamaps count is just near numOfThreadsForPruning,
As code is checking '>= ', last thread may not get the datamaps for prune.
Hence array out of index exception is thrown in this scenario.
There is no issues with higher number of datamaps.

Solution: In this scenario launch threads based on the distribution value,
not on the hardcoded value

This closes #3336
---
 .../org/apache/carbondata/core/datamap/TableDataMap.java | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git 
a/core/src/main/java/org/apache/carbondata/core/datamap/TableDataMap.java 
b/core/src/main/java/org/apache/carbondata/core/datamap/TableDataMap.java
index 33fc3b1..ecdd586 100644
--- a/core/src/main/java/org/apache/carbondata/core/datamap/TableDataMap.java
+++ b/core/src/main/java/org/apache/carbondata/core/datamap/TableDataMap.java
@@ -207,9 +207,6 @@ public final class TableDataMap extends 
OperationEventListener {
  */
 
 int numOfThreadsForPruning = CarbonProperties.getNumOfThreadsForPruning();
-LOG.info(
-"Number of threads selected for multi-thread block pruning is " + 
numOfThreadsForPruning
-+ ". total files: " + totalFiles + ". total segments: " + 
segments.size());
 int filesPerEachThread = totalFiles / numOfThreadsForPruning;
 int prev;
 int filesCount = 0;
@@ -254,6 +251,15 @@ public final class TableDataMap extends 
OperationEventListener {
   // this should not happen
   throw new RuntimeException(" not all the files processed ");
 }
+if (datamapListForEachThread.size() < numOfThreadsForPruning) {
+  // If the total datamaps fitted in lesser number of threads than 
numOfThreadsForPruning.
+  // Launch only that many threads where datamaps are fitted while 
grouping.
+  LOG.info("Datamaps is distributed in " + datamapListForEachThread.size() 
+ " threads");
+  numOfThreadsForPruning = datamapListForEachThread.size();
+}
+LOG.info(
+"Number of threads selected for multi-thread block pruning is " + 
numOfThreadsForPruning
++ ". total files: " + totalFiles + ". total segments: " + 
segments.size());
 List> results = new ArrayList<>(numOfThreadsForPruning);
 final Map> prunedBlockletMap =
 new ConcurrentHashMap<>(segments.size());

[carbondata] branch master updated: [HOTFIX] CLI test case failed during release because of space differences

2019-08-01 Thread kunalkapoor 
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
 new 10f3747  [HOTFIX] CLI test case failed during release because of space 
differences
10f3747 is described below

commit 10f3747d69ce166cfa85638191b58f0dcc3bcbab
Author: ravipesala 
AuthorDate: Thu Aug 1 18:12:50 2019 +0530

[HOTFIX] CLI test case failed during release because of space differences

CLI test case is failed if the release name is short and without snapshot, 
it adds more space.
That's why changed test check the individual contains instead of a batch of 
lines.

This closes #3344
---
 .../src/test/java/org/apache/carbondata/tool/CarbonCliTest.java  | 9 +++--
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git 
a/tools/cli/src/test/java/org/apache/carbondata/tool/CarbonCliTest.java 
b/tools/cli/src/test/java/org/apache/carbondata/tool/CarbonCliTest.java
index 26901f8..af8d51d 100644
--- a/tools/cli/src/test/java/org/apache/carbondata/tool/CarbonCliTest.java
+++ b/tools/cli/src/test/java/org/apache/carbondata/tool/CarbonCliTest.java
@@ -241,12 +241,9 @@ public class CarbonCliTest {
 "20  3.36KB 4.06MB false  00.0B  
93.76KB  0.0   100.0  7298  " ,
 "21  2.04KB 1.49MB false  00.0B  
89.62KB  0.0   100.0  9299  ");
 Assert.assertTrue(output.contains(expectedOutput));
-
-expectedOutput = buildLines(
-"## version Details",
-"written_by  Version ",
-"TestUtil"+ CarbonVersionConstants.CARBONDATA_VERSION+"  ");
-Assert.assertTrue(output.contains(expectedOutput));
+Assert.assertTrue(output.contains("## version Details"));
+Assert.assertTrue(output.contains("written_by  Version"));
+Assert.assertTrue(output.contains("TestUtil"+ 
CarbonVersionConstants.CARBONDATA_VERSION));
   }
 
   @Test

Jenkins build is still unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Store SDK #1870

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #1870

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.2 #1870

2019-08-01 Thread Apache Jenkins Server 
See

Build failed in Jenkins: carbondata-master-spark-2.1 #3666

2019-08-01 Thread Apache Jenkins Server 
See 


Changes:

[kunalkapoor642] [CARBONDATA-3478]Fix ArrayIndexOutOfBound Exception on 
compaction after

--
[...truncated 12.80 MB...]
|robot1|  1|   0.5|
|robot2|  2|   1.0|
|robot3|  3|   1.5|
|robot4|  4|   2.0|
|robot5|  5|   2.5|
|robot6|  6|   3.0|
|robot7|  7|   3.5|
|robot8|  8|   4.0|
|robot9|  9|   4.5|
+--+---+--+

- DirectSQLExample
2019-08-01 12:06:56 AUDIT audit:72 - {"time":"August 1, 2019 5:06:56 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"24348943275566962","opStatus":"START"}
2019-08-01 12:06:56 AUDIT audit:93 - {"time":"August 1, 2019 5:06:56 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"24348943275566962","opStatus":"SUCCESS","opTime":"77 
ms","table":"default.hive_carbon_example","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":""}}
2019-08-01 12:06:56 AUDIT audit:72 - {"time":"August 1, 2019 5:06:56 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348943371025288","opStatus":"START"}
2019-08-01 12:06:56 AUDIT audit:93 - {"time":"August 1, 2019 5:06:56 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348943371025288","opStatus":"SUCCESS","opTime":"138 
ms","table":"default.hive_carbon_example","extraInfo":{"SegmentId":"0","DataSize":"924.0B","IndexSize":"551.0B"}}
2019-08-01 12:06:56 AUDIT audit:72 - {"time":"August 1, 2019 5:06:56 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348943517827214","opStatus":"START"}
2019-08-01 12:06:56 AUDIT audit:93 - {"time":"August 1, 2019 5:06:56 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348943517827214","opStatus":"SUCCESS","opTime":"141 
ms","table":"default.hive_carbon_example","extraInfo":{"SegmentId":"1","DataSize":"924.0B","IndexSize":"551.0B"}}
+---+-++
| id| name|  salary|
+---+-++
|  1|  'liang'|20.0|
|  2|'anubhav'| 2.0|
|  1|  'liang'|20.0|
|  2|'anubhav'| 2.0|
+---+-++

2019-08-01 12:06:56 AUDIT audit:72 - {"time":"August 1, 2019 5:06:56 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"24348943754554089","opStatus":"START"}
2019-08-01 12:06:56 AUDIT audit:93 - {"time":"August 1, 2019 5:06:56 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"24348943754554089","opStatus":"SUCCESS","opTime":"48 
ms","table":"default.test_boundary","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":"","dictionary_include":"c6_timestamp"}}
2019-08-01 12:06:56 AUDIT audit:72 - {"time":"August 1, 2019 5:06:56 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348943807475591","opStatus":"START"}
2019-08-01 12:06:57 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table test_boundary
2019-08-01 12:06:57 AUDIT audit:93 - {"time":"August 1, 2019 5:06:57 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348943807475591","opStatus":"SUCCESS","opTime":"153 
ms","table":"default.test_boundary","extraInfo":{"SegmentId":"0","DataSize":"4.03KB","IndexSize":"1.06KB"}}
OK
**Total Number Of Rows Fetched ** 0
- HiveExample *** FAILED ***
  java.lang.AssertionError: assertion failed
  at scala.Predef$.assert(Predef.scala:156)
  at 
org.apache.carbondata.examples.HiveExample$.readFromHive(HiveExample.scala:171)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply$mcV$sp(RunExamples.scala:135)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply(RunExamples.scala:133)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply(RunExamples.scala:133)
  at 
org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  ...
Exception encountered when invoking run on a nested suite - Cannot call 
methods on a stopped SparkContext.
This stopped SparkContext was created at:

org.apache.spark.sql.test.util.QueryTest.(QueryTest.scala:115)
org.apache.carbondata.examplesCI.RunExamples.(RunExamples.scala:35)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
java.lang.Class.newInstance(Class.java:442)
o

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark2 #3666

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #3666

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark Common Test #3666

2019-08-01 Thread Apache Jenkins Server 
See

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Examples #3666

2019-08-01 Thread Apache Jenkins Server 
See 


--
[...truncated 287.10 KB...]
|count(1)|
++
|  12|
++

++
|count(1)|
++
|  12|
++

time for query on table with lucene datamap table:0.258
time for query on table without lucene datamap table:0.156
+-+-+
|   id| name|
+-+-+
|which test1 good9|who and name6|
|which test1 good3|who and name4|
|which test1 good3|who and name7|
|which test1 good4|who and name4|
|which test1 good7|who and name1|
|which test1 good9|who and name6|
|which test1 good3|who and name4|
|which test1 good3|who and name7|
|which test1 good4|who and name4|
|which test1 good7|who and name1|
+-+-+

+-+-+
|   id| name|
+-+-+
|which test1 good9|who and name6|
|which test1 good3|who and name4|
|which test1 good3|who and name7|
|which test1 good4|who and name4|
|which test1 good1|who and name3|
|which test1 good7|who and name1|
|which test1 good9|who and name6|
|which test1 good3|who and name4|
|which test1 good3|who and name7|
|which test1 good4|who and name4|
+-+-+

2019-08-01 12:06:54 AUDIT audit:72 - {"time":"August 1, 2019 5:06:54 AM 
PDT","username":"jenkins","opName":"DROP 
TABLE","opId":"24348941158679072","opStatus":"START"}
2019-08-01 12:06:54 AUDIT audit:93 - {"time":"August 1, 2019 5:06:54 AM 
PDT","username":"jenkins","opName":"DROP 
TABLE","opId":"24348941158679072","opStatus":"SUCCESS","opTime":"126 
ms","table":"default.persontable","extraInfo":{}}
- LuceneDataMapExample
2019-08-01 12:06:54 AUDIT audit:72 - {"time":"August 1, 2019 5:06:54 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"24348941297640938","opStatus":"START"}
2019-08-01 12:06:54 AUDIT audit:93 - {"time":"August 1, 2019 5:06:54 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"24348941297640938","opStatus":"SUCCESS","opTime":"63 
ms","table":"default.origin_table","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":""}}
2019-08-01 12:06:54 AUDIT audit:72 - {"time":"August 1, 2019 5:06:54 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348941364823643","opStatus":"START"}
2019-08-01 12:06:54 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 12:06:54 AUDIT audit:93 - {"time":"August 1, 2019 5:06:54 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348941364823643","opStatus":"SUCCESS","opTime":"160 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"0","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-01 12:06:54 AUDIT audit:72 - {"time":"August 1, 2019 5:06:54 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348941529284648","opStatus":"START"}
2019-08-01 12:06:54 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 12:06:54 AUDIT audit:93 - {"time":"August 1, 2019 5:06:54 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348941529284648","opStatus":"SUCCESS","opTime":"146 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"1","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-01 12:06:54 AUDIT audit:72 - {"time":"August 1, 2019 5:06:54 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348941680457872","opStatus":"START"}
2019-08-01 12:06:54 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 12:06:54 AUDIT audit:93 - {"time":"August 1, 2019 5:06:54 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348941680457872","opStatus":"SUCCESS","opTime":"152 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"2","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-01 12:06:54 AUDIT audit:72 - {"time":"August 1, 2019 5:06:54 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348941836752152","opStatus":"START"}
2019-08-01 12:06:55 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 12:06:55 AUDIT audit:93 - {"time":"August 1, 2019 5:06:55 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"24348941836752152","opStatus":"SUCCESS","opTime":"152 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"3","DataSize":"2.85KB","IndexSize":"1.43KB"}}
++
|count(1)|
++
|  40|
++

2019-08-01 12:06:55 AUDIT audit:72 - {"time":"August 1, 2019 5:06:55 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"24348942043332741","opStatus":"START"}
2019-08-01 12:06:55 AUDIT audit:93 - {"time":"August 1, 2019 5:06:55 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"24348942043332741","opStatus":"SUCCESS","opTime":"50 
ms","table":"default

Build failed in Jenkins: carbondata-master-spark-2.1 #3665

2019-08-01 Thread Apache Jenkins Server 
See 


--
[...truncated 12.95 MB...]
|robot1|  1|   0.5|
|robot2|  2|   1.0|
|robot3|  3|   1.5|
|robot4|  4|   2.0|
|robot5|  5|   2.5|
|robot6|  6|   3.0|
|robot7|  7|   3.5|
|robot8|  8|   4.0|
|robot9|  9|   4.5|
+--+---+--+

- DirectSQLExample
2019-08-01 09:46:13 AUDIT audit:72 - {"time":"August 1, 2019 2:46:13 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"167717679469109","opStatus":"START"}
2019-08-01 09:46:14 AUDIT audit:93 - {"time":"August 1, 2019 2:46:14 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"167717679469109","opStatus":"SUCCESS","opTime":"56 
ms","table":"default.hive_carbon_example","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":""}}
2019-08-01 09:46:14 AUDIT audit:72 - {"time":"August 1, 2019 2:46:14 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167717750924785","opStatus":"START"}
2019-08-01 09:46:14 AUDIT audit:93 - {"time":"August 1, 2019 2:46:14 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167717750924785","opStatus":"SUCCESS","opTime":"132 
ms","table":"default.hive_carbon_example","extraInfo":{"SegmentId":"0","DataSize":"924.0B","IndexSize":"551.0B"}}
2019-08-01 09:46:14 AUDIT audit:72 - {"time":"August 1, 2019 2:46:14 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167717891629402","opStatus":"START"}
2019-08-01 09:46:14 AUDIT audit:93 - {"time":"August 1, 2019 2:46:14 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167717891629402","opStatus":"SUCCESS","opTime":"122 
ms","table":"default.hive_carbon_example","extraInfo":{"SegmentId":"1","DataSize":"924.0B","IndexSize":"551.0B"}}
+---+-++
| id| name|  salary|
+---+-++
|  1|  'liang'|20.0|
|  2|'anubhav'| 2.0|
|  1|  'liang'|20.0|
|  2|'anubhav'| 2.0|
+---+-++

2019-08-01 09:46:14 AUDIT audit:72 - {"time":"August 1, 2019 2:46:14 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"167718099956195","opStatus":"START"}
2019-08-01 09:46:14 AUDIT audit:93 - {"time":"August 1, 2019 2:46:14 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"167718099956195","opStatus":"SUCCESS","opTime":"51 
ms","table":"default.test_boundary","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":"","dictionary_include":"c6_timestamp"}}
2019-08-01 09:46:14 AUDIT audit:72 - {"time":"August 1, 2019 2:46:14 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167718154966310","opStatus":"START"}
2019-08-01 09:46:14 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table test_boundary
2019-08-01 09:46:14 AUDIT audit:93 - {"time":"August 1, 2019 2:46:14 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167718154966310","opStatus":"SUCCESS","opTime":"149 
ms","table":"default.test_boundary","extraInfo":{"SegmentId":"0","DataSize":"4.03KB","IndexSize":"1.06KB"}}
OK
**Total Number Of Rows Fetched ** 0
- HiveExample *** FAILED ***
  java.lang.AssertionError: assertion failed
  at scala.Predef$.assert(Predef.scala:156)
  at 
org.apache.carbondata.examples.HiveExample$.readFromHive(HiveExample.scala:171)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply$mcV$sp(RunExamples.scala:135)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply(RunExamples.scala:133)
  at 
org.apache.carbondata.examplesCI.RunExamples$$anonfun$21.apply(RunExamples.scala:133)
  at 
org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  ...
Exception encountered when invoking run on a nested suite - Cannot call 
methods on a stopped SparkContext.
This stopped SparkContext was created at:

org.apache.spark.sql.test.util.QueryTest.(QueryTest.scala:115)
org.apache.carbondata.examplesCI.RunExamples.(RunExamples.scala:35)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
java.lang.Class.newInstance(Class.java:442)
org.scalatest.tools.DiscoverySuite$.getSuiteInstance(DiscoverySuite.scala:69)
org.scalatest.tools.DiscoverySuite$$anonfun$1.ap

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark Common Test #3665

2019-08-01 Thread Apache Jenkins Server 
See

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Examples #3665

2019-08-01 Thread Apache Jenkins Server 
See 


--
[...truncated 285.76 KB...]
|count(1)|
++
|  16|
++

time for query on table with lucene datamap table:0.222
time for query on table without lucene datamap table:0.13
+-+-+
|   id| name|
+-+-+
|which test1 good9|who and name2|
|which test1 good2|who and name4|
|which test1 good6|who and name7|
|which test1 good5|who and name6|
|which test1 good6|who and name7|
|which test1 good8|who and name6|
|which test1 good7|who and name5|
|which test1 good2|who and name5|
|which test1 good9|who and name2|
|which test1 good2|who and name4|
|which test1 good6|who and name7|
|which test1 good5|who and name6|
|which test1 good6|who and name7|
|which test1 good8|who and name6|
|which test1 good7|who and name5|
|which test1 good2|who and name5|
+-+-+

+-+-+
|   id| name|
+-+-+
|which test1 good9|who and name2|
|which test1 good2|who and name4|
|which test1 good6|who and name7|
|which test1 good5|who and name6|
|which test1 good6|who and name7|
|which test1 good8|who and name6|
|which test1 good7|who and name5|
|which test1 good2|who and name5|
|which test1 good9|who and name2|
|which test1 good2|who and name4|
+-+-+

2019-08-01 09:46:12 AUDIT audit:72 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"DROP 
TABLE","opId":"167715962930869","opStatus":"START"}
2019-08-01 09:46:12 AUDIT audit:93 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"DROP 
TABLE","opId":"167715962930869","opStatus":"SUCCESS","opTime":"95 
ms","table":"default.persontable","extraInfo":{}}
- LuceneDataMapExample
2019-08-01 09:46:12 AUDIT audit:72 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"167716068691196","opStatus":"START"}
2019-08-01 09:46:12 AUDIT audit:93 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"167716068691196","opStatus":"SUCCESS","opTime":"49 
ms","table":"default.origin_table","extraInfo":{"bad_record_path":"","local_dictionary_enable":"true","external":"false","sort_columns":"","comment":""}}
2019-08-01 09:46:12 AUDIT audit:72 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167716122231469","opStatus":"START"}
2019-08-01 09:46:12 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 09:46:12 AUDIT audit:93 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167716122231469","opStatus":"SUCCESS","opTime":"143 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"0","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-01 09:46:12 AUDIT audit:72 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167716270385556","opStatus":"START"}
2019-08-01 09:46:12 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 09:46:12 AUDIT audit:93 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167716270385556","opStatus":"SUCCESS","opTime":"128 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"1","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-01 09:46:12 AUDIT audit:72 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167716402033620","opStatus":"START"}
2019-08-01 09:46:12 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 09:46:12 AUDIT audit:93 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167716402033620","opStatus":"SUCCESS","opTime":"132 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"2","DataSize":"2.85KB","IndexSize":"1.43KB"}}
2019-08-01 09:46:12 AUDIT audit:72 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167716537357455","opStatus":"START"}
2019-08-01 09:46:12 ERROR DataLoadExecutor:55 - Data Load is partially success 
for table origin_table
2019-08-01 09:46:12 AUDIT audit:93 - {"time":"August 1, 2019 2:46:12 AM 
PDT","username":"jenkins","opName":"LOAD 
DATA","opId":"167716537357455","opStatus":"SUCCESS","opTime":"126 
ms","table":"default.origin_table","extraInfo":{"SegmentId":"3","DataSize":"2.85KB","IndexSize":"1.43KB"}}
++
|count(1)|
++
|  40|
++

2019-08-01 09:46:13 AUDIT audit:72 - {"time":"August 1, 2019 2:46:13 AM 
PDT","username":"jenkins","opName":"CREATE 
TABLE","opId":"167716711564472","opStatus":"START"}
2019-08-01 09:46:13 AUDIT audit:93 - {"time":"August 1, 2019 2:46:13 AM 
PDT","usernam

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #3665

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark2 #3665

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.2 #1869

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Store SDK #1869

2019-08-01 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #1869

2019-08-01 Thread Apache Jenkins Server 
See

[carbondata] branch master updated: [CARBONDATA-3478]Fix ArrayIndexOutOfBound Exception on compaction after alter operation

2019-08-01 Thread kunalkapoor 
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
 new c8cc92b  [CARBONDATA-3478]Fix ArrayIndexOutOfBound Exception on 
compaction after alter operation
c8cc92b is described below

commit c8cc92b6db80e7b6d220901e65ddf1f5beb557fb
Author: Indhumathi27 
AuthorDate: Fri Jul 26 16:51:32 2019 +0530

[CARBONDATA-3478]Fix ArrayIndexOutOfBound Exception on compaction after 
alter operation

Problem:
In case of alter add, drop, rename operation, restructuredBlockExists will 
be true.
Currently, to get RawResultIterator for a block, we check if block has 
ColumnDrift
or not, by comparing SegmentProperties and columndrift columns.
SegmentProperties will be formed based on restructuredBlockExists.
if restructuredBlockExists is true, we will take current column schema to 
form SegmentProperties,
else, we will use datafilefooter columnschema to form SegmentProperties.

In the example given in CARBONDATA-3478 for both blocks, we use current 
column
schema to form SegmentProperties, as restructuredBlockExists will be true.
Hence, while iterating block 1, it throws ArrayIndexOutOfBound exception,
as it uses RawResultIterator instead of ColumnDriftRawResultIterator

Solution:
Use schema from datafilefooter of each block to check if it has columndrift 
or not

This closes #3337
---
 .../AlterTableColumnRenameTestCase.scala   | 54 ++
 .../merger/CarbonCompactionExecutor.java   |  9 +++-
 2 files changed, 61 insertions(+), 2 deletions(-)

diff --git 
a/integration/spark2/src/test/scala/org/apache/spark/carbondata/restructure/vectorreader/AlterTableColumnRenameTestCase.scala
 
b/integration/spark2/src/test/scala/org/apache/spark/carbondata/restructure/vectorreader/AlterTableColumnRenameTestCase.scala
index d927724..dd1fa0f 100644
--- 
a/integration/spark2/src/test/scala/org/apache/spark/carbondata/restructure/vectorreader/AlterTableColumnRenameTestCase.scala
+++ 
b/integration/spark2/src/test/scala/org/apache/spark/carbondata/restructure/vectorreader/AlterTableColumnRenameTestCase.scala
@@ -320,12 +320,66 @@ class AlterTableColumnRenameTestCase extends 
Spark2QueryTest with BeforeAndAfter
 }
   }
 
+  test("test compaction after table rename and alter set tblproerties") {
+sql("DROP TABLE IF EXISTS test_rename")
+sql("DROP TABLE IF EXISTS test_rename_compact")
+sql(
+  "CREATE TABLE test_rename (empno int, empname String, designation 
String, doj Timestamp, " +
+  "workgroupcategory int, workgroupcategoryname String, deptno int, 
deptname String, " +
+  "projectcode int, projectjoindate Timestamp, projectenddate 
Timestamp,attendance int," +
+  "utilization int,salary int) STORED BY 'org.apache.carbondata.format'")
+sql(
+  s"""LOAD DATA LOCAL INPATH '$resourcesPath/data.csv' INTO TABLE 
test_rename OPTIONS
+ |('DELIMITER'= ',', 'QUOTECHAR'= '\"')""".stripMargin)
+sql("alter table test_rename rename to test_rename_compact")
+sql("alter table test_rename_compact set 
tblproperties('sort_columns'='deptno,projectcode', 'sort_scope'='local_sort')")
+sql(
+  s"""LOAD DATA LOCAL INPATH '$resourcesPath/data.csv' INTO TABLE 
test_rename_compact OPTIONS
+ |('DELIMITER'= ',', 'QUOTECHAR'= '\"')""".stripMargin)
+val res1 = sql("select * from test_rename_compact")
+sql("alter table test_rename_compact compact 'major'")
+val res2 = sql("select * from test_rename_compact")
+assert(res1.collectAsList().containsAll(res2.collectAsList()))
+checkExistence(sql("show segments for table test_rename_compact"), true, 
"Compacted")
+sql("DROP TABLE IF EXISTS test_rename")
+sql("DROP TABLE IF EXISTS test_rename_compact")
+  }
+
+  test("test compaction after alter set tblproerties- add and drop") {
+sql("DROP TABLE IF EXISTS test_alter")
+sql(
+  "CREATE TABLE test_alter (empno int, empname String, designation String, 
doj Timestamp, " +
+  "workgroupcategory int, workgroupcategoryname String, deptno int, 
deptname String, " +
+  "projectcode int, projectjoindate Timestamp, projectenddate 
Timestamp,attendance int," +
+  "utilization int,salary int) STORED BY 'org.apache.carbondata.format'")
+sql(
+  s"""LOAD DATA LOCAL INPATH '$resourcesPath/data.csv' INTO TABLE 
test_alter OPTIONS
+ |('DELIMITER'= ',', 'QUOTECHAR'= '\"')""".stripMargin)
+sql("alter table test_alter set 
tblproperties('sort_columns'='deptno,projectcode', 'sort_scope'='local_sort')")
+sql("alter table test_alter drop columns(deptno)")
+sql(
+  s"""LOAD DATA LOCAL INPATH '$resourcesPath/data.csv' INTO TABLE 
test_alter OPTIONS
+ |('DELIMITER'= ',', 'QUOTECHAR'= '\"')""".stripMargin)
+sql("alter table test_alter