[GitHub] carbondata pull request #1575: [CARBONDATA-1698]Adding support for table lev...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/1575


---

[GitHub] carbondata pull request #1575: [CARBONDATA-1698]Adding support for table lev...

[chenliang613]

Github user chenliang613 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1575#discussion_r158188263
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/carbondata/spark/rdd/CarbonDataRDDFactory.scala
 ---
@@ -205,7 +205,8 @@ object CarbonDataRDDFactory {
 
   val newCarbonLoadModel = prepareCarbonLoadModel(table)
 
-  val compactionSize = 
CarbonDataMergerUtil.getCompactionSize(CompactionType.MAJOR)
+  val compactionSize = CarbonDataMergerUtil
+.getCompactionSize(CompactionType.MAJOR, carbonLoadModel)
--- End diff --

ok


---

[GitHub] carbondata pull request #1575: [CARBONDATA-1698]Adding support for table lev...

[Xaprice]

Github user Xaprice commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1575#discussion_r158186274
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/carbondata/spark/rdd/CarbonDataRDDFactory.scala
 ---
@@ -205,7 +205,8 @@ object CarbonDataRDDFactory {
 
   val newCarbonLoadModel = prepareCarbonLoadModel(table)
 
-  val compactionSize = 
CarbonDataMergerUtil.getCompactionSize(CompactionType.MAJOR)
+  val compactionSize = CarbonDataMergerUtil
+.getCompactionSize(CompactionType.MAJOR, carbonLoadModel)
--- End diff --

carbonLoadModel may contain table-level major compaction size if it is 
specified in create table SQL, so the purpose for adding parameter 
'carbonLoadModel' is to get the table-level major compaction size.


---

[GitHub] carbondata pull request #1575: [CARBONDATA-1698]Adding support for table lev...

[chenliang613]

Github user chenliang613 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1575#discussion_r158056650
  
--- Diff: 
integration/spark2/src/main/scala/org/apache/carbondata/spark/rdd/CarbonDataRDDFactory.scala
 ---
@@ -205,7 +205,8 @@ object CarbonDataRDDFactory {
 
   val newCarbonLoadModel = prepareCarbonLoadModel(table)
 
-  val compactionSize = 
CarbonDataMergerUtil.getCompactionSize(CompactionType.MAJOR)
+  val compactionSize = CarbonDataMergerUtil
+.getCompactionSize(CompactionType.MAJOR, carbonLoadModel)
--- End diff --

why need add one more input parameter(carbonLoadModel) for 
getCompactionSize()?


---

[GitHub] carbondata pull request #1575: [CARBONDATA-1698]Adding support for table lev...

[chenliang613]

Github user chenliang613 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1575#discussion_r158054020
  
--- Diff: 
integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/datacompaction/TableLevelCompactionParameterTest.scala
 ---
@@ -0,0 +1,183 @@
+/*
--- End diff --

This test case is same as TableLevelCompactionOptionTest.scala, please 
remove this one(TableLevelCompactionParameterTest.scala)


---

[GitHub] carbondata pull request #1575: [CARBONDATA-1698]Adding support for table lev...

[chenliang613]

Github user chenliang613 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1575#discussion_r157361376
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
 ---
@@ -873,6 +873,16 @@
   public static final String TABLE_BLOCKSIZE = "table_blocksize";
   // set in column level to disable inverted index
   public static final String NO_INVERTED_INDEX = "no_inverted_index";
+  // table property name of major compaction size
+  public static final String MAJOR_COMPACTION_SIZE = 
"major_compaction_size";
--- End diff --

For these compaction properties of Table level , suggest adding "TABLE", 
such as : TABLE_MAJOR_COMPACTION_SIZE,TABLE_AUTO_LOAD_MERGE...


---

[GitHub] carbondata pull request #1575: [CARBONDATA-1698]Adding support for table lev...

[Xaprice]

Github user Xaprice commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1575#discussion_r154612395
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
 ---
@@ -863,6 +863,16 @@
   public static final String TABLE_BLOCKSIZE = "table_blocksize";
   // set in column level to disable inverted index
   public static final String NO_INVERTED_INDEX = "no_inverted_index";
+  // table property name of major compaction size
+  public static final String TBL_PROP_MAJOR_COMPACTION_SIZE = 
"major_compaction_size";
--- End diff --

TBL_PROPs removed


---

[GitHub] carbondata pull request #1575: [CARBONDATA-1698]Adding support for table lev...

[chenliang613]

Github user chenliang613 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1575#discussion_r154353706
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
 ---
@@ -863,6 +863,16 @@
   public static final String TABLE_BLOCKSIZE = "table_blocksize";
   // set in column level to disable inverted index
   public static final String NO_INVERTED_INDEX = "no_inverted_index";
+  // table property name of major compaction size
+  public static final String TBL_PROP_MAJOR_COMPACTION_SIZE = 
"major_compaction_size";
--- End diff --

suggest keeping the consistent name format , how about 
MAJOR_COMPACTION_SIZE,  remove "TBL_PROP"


---

[GitHub] carbondata pull request #1575: [CARBONDATA-1698]Adding support for table lev...

[Xaprice]

GitHub user Xaprice opened a pull request:

https://github.com/apache/carbondata/pull/1575

[CARBONDATA-1698]Adding support for table level compaction configuration

Adding support for table level compaction configuration

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Xaprice/carbondata master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1575.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1575


commit 0a6ba166795872b41c8fa3fa8a5e1a2e5faa81b0
Author: 周瑾 
Date:   2017-11-27T02:30:17Z

add support for table level compaction properties

commit fa8e847cf26f3b8daa067af792b84f5666ad3920
Author: Jin Zhou 
Date:   2017-11-27T09:05:08Z

[CARBONDATA-1698]Adding support for table level compaction configuration

commit f50cd67caf4ea9280d16f34dfe984e218634824c
Author: Jin Zhou 
Date:   2017-11-27T09:45:53Z

[CARBONDATA-1698]Adding table level compaction configuration

commit 763e22ce95b829f6a5cb43fa92a523137807a7db
Author: Jin Zhou 
Date:   2017-11-27T09:46:04Z

Merge branch 'master' of https://github.com/apache/carbondata




---