[GitHub] carbondata pull request #3023: [CARBONDATA-3197][BloomDataMap] Include bloom...

2019-01-10 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3023#discussion_r246663278 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala --- @@ -184,6 +184,9 @@ object CarbonEnv { .addListener

[GitHub] carbondata pull request #3059: [HOTFIX][DataLoad]fix task assignment issue u...

2019-01-09 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3059#discussion_r246376652 --- Diff: processing/src/main/java/org/apache/carbondata/processing/util/CarbonLoaderUtil.java --- @@ -609,6 +609,14 @@ public static Dictionary

[GitHub] carbondata pull request #3054: [CARBONDATA-3232] Add example and doc for all...

2019-01-07 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3054#discussion_r245902021 --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/util/ExampleUtils.scala --- @@ -30,13 +30,20 @@ object ExampleUtils

[GitHub] carbondata pull request #3054: [CARBONDATA-3232] Optimize carbonData using a...

2019-01-07 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3054#discussion_r245641130 --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/util/ExampleUtils.scala --- @@ -30,13 +30,17 @@ object ExampleUtils

[GitHub] carbondata pull request #3054: [CARBONDATA-3232] Optimize carbonData using a...

2019-01-07 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3054#discussion_r245639998 --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/AlluxioExample.scala --- @@ -26,48 +30,88 @@ import

[GitHub] carbondata pull request #3054: [CARBONDATA-3232] Optimize carbonData using a...

2019-01-07 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3054#discussion_r245639533 --- Diff: docs/Integration/alluxio-guide.md --- @@ -0,0 +1,44 @@ + + + +# Presto guide +This tutorial provides a quick introduction

[GitHub] carbondata pull request #3045: [CARBONDATA-3222]Fix dataload failure after c...

2019-01-06 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3045#discussion_r245511058 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala --- @@ -110,7 +110,29

[GitHub] carbondata pull request #3045: [CARBONDATA-3222]Fix dataload failure after c...

2019-01-06 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3045#discussion_r245511008 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala --- @@ -110,7 +110,29

[GitHub] carbondata pull request #3023: [CARBONDATA-3197][BloomDataMap] Include bloom...

2019-01-05 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3023#discussion_r245480571 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/events/MergeBloomIndexEventListener.scala --- @@ -24,59 +24,96 @@ import

[GitHub] carbondata pull request #3023: [CARBONDATA-3197][BloomDataMap] Merge bloom i...

2019-01-05 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3023#discussion_r245480089 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/events/MergeBloomIndexEventListener.scala --- @@ -24,59 +24,96 @@ import

[GitHub] carbondata pull request #3023: [CARBONDATA-3197][BloomDataMap] Merge bloom i...

2019-01-05 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3023#discussion_r245480061 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala --- @@ -184,6 +184,9 @@ object CarbonEnv { .addListener

[GitHub] carbondata issue #2973: [WIP][CARBONDATA-3144] CarbonData support spark-2.4....

2019-01-04 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2973 I test this PR by running `CarbonSessionExample`. Please check following exception: ``` java.lang.ClassCastException: org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener

[GitHub] carbondata pull request #2996: [WIP] Fix Rename-Fail & Datamap-creation-Fail

2019-01-02 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2996#discussion_r244906592 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableRenameCommand.scala --- @@ -165,15 +167,22

[GitHub] carbondata pull request #2996: [WIP] Fix Rename-Fail & Datamap-creation-Fail

2019-01-02 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2996#discussion_r244906489 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableRenameCommand.scala --- @@ -165,15 +167,22

[GitHub] carbondata pull request #3017: [HOTFIX] remove this useless assignment

2018-12-26 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3017#discussion_r244096878 --- Diff: core/src/main/java/org/apache/carbondata/core/locks/LocalFileLock.java --- @@ -112,7 +112,7 @@ public LocalFileLock(String lockFileLocation

[GitHub] carbondata pull request #3023: [CARBONDATA-3197][BloomDataMap] Merge bloom i...

2018-12-25 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3023#discussion_r243928456 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/events/MergeBloomIndexEventListener.scala --- @@ -24,59 +24,88 @@ import

[GitHub] carbondata pull request #3023: [CARBONDATA-3197][BloomDataMap] Merge bloom i...

2018-12-25 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3023#discussion_r243928449 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/events/MergeBloomIndexEventListener.scala --- @@ -24,59 +24,88 @@ import

[GitHub] carbondata pull request #3023: [CARBONDATA-3197][BloomDataMap] Merge bloom i...

2018-12-25 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3023#discussion_r243905862 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/events/MergeBloomIndexEventListener.scala --- @@ -24,59 +24,88 @@ import

[GitHub] carbondata pull request #3023: [CARBONDATA-3197][BloomDataMap] Merge bloom i...

2018-12-25 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3023#discussion_r243905231 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/events/MergeBloomIndexEventListener.scala --- @@ -24,59 +24,88 @@ import

[GitHub] carbondata pull request #3023: [CARBONDATA-3197][BloomDataMap] Merge bloom i...

2018-12-25 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/3023 [CARBONDATA-3197][BloomDataMap] Merge bloom index before accessible **Problem** Currently carbon allows to query when bloom index files are merging, but this will cause problems when

[GitHub] carbondata pull request #2161: [CARBONDATA-2218] AlluxioCarbonFile while try...

2018-12-24 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2161#discussion_r243819959 --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java --- @@ -960,7 +960,7 @@ private CarbonCommonConstants

[GitHub] carbondata pull request #2161: [CARBONDATA-2218] AlluxioCarbonFile while try...

2018-12-22 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2161#discussion_r243732085 --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java --- @@ -960,7 +960,7 @@ private CarbonCommonConstants

[GitHub] carbondata issue #2161: [CARBONDATA-2218] AlluxioCarbonFile while trying to ...

2018-12-20 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2161 > Since user normally uses Alluxio as a read cache, I think we can firstly verify carbon on alluxio for the query scenario. As I am still not very sure what is the correct way to implem

[GitHub] carbondata pull request #2161: [CARBONDATA-2218] AlluxioCarbonFile while try...

2018-12-20 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2161#discussion_r243254949 --- Diff: core/src/main/java/org/apache/carbondata/core/locks/AlluxioFileLock.java --- @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache

[GitHub] carbondata pull request #2161: [CARBONDATA-2218] AlluxioCarbonFile while try...

2018-12-19 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2161#discussion_r243168473 --- Diff: core/src/main/java/org/apache/carbondata/core/locks/AlluxioFileLock.java --- @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache

[GitHub] carbondata pull request #2161: [CARBONDATA-2218] AlluxioCarbonFile while try...

2018-12-19 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2161#discussion_r243167033 --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/filesystem/AlluxioCarbonFile.java --- @@ -94,14 +93,9 @@ public CarbonFile

[GitHub] carbondata pull request #3005: [CARBONDATA-3185] Fix alluxio file rename

2018-12-19 Thread kevinjmh
Github user kevinjmh closed the pull request at: https://github.com/apache/carbondata/pull/3005 ---

[GitHub] carbondata issue #3005: [CARBONDATA-3185] Fix alluxio file rename

2018-12-19 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/3005 > There is a PR to fix the same issue: #2161 I see. It fixed LockFile problem I faced now too. Hope it got fix soon. And I am closing this PR now ---

[GitHub] carbondata pull request #3005: [CARBONDATA-3185] Fix alluxio file rename

2018-12-19 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3005#discussion_r243150578 --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/filesystem/AlluxioCarbonFile.java --- @@ -95,15 +94,13 @@ public boolean

[GitHub] carbondata pull request #3005: [CARBONDATA-3185] Fix alluxio file rename

2018-12-19 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/3005 [CARBONDATA-3185] Fix alluxio file rename **Problem** Exception thrown when create table on alluxio because rename schema file failed **Analyse** Re-run the command after

[GitHub] carbondata pull request #3000: [CARBONDATA-3181][BloomDataMap] Fix access fi...

2018-12-19 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/3000#discussion_r243130074 --- Diff: datamap/bloom/src/main/java/org/apache/hadoop/util/bloom/CarbonBloomFilter.java --- @@ -49,27 +49,23 @@ public CarbonBloomFilter(int

[GitHub] carbondata pull request #3000: [CARBONDATA-3181][BloomDataMap] Fix access fi...

2018-12-18 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/3000 [CARBONDATA-3181][BloomDataMap] Fix access field error for BitSet in bloom filter **Problem** java.lang.IllegalAccessError is thrown when query on bloom filter without compress

[GitHub] carbondata pull request #2713: [WIP][CARBONDATA-2931][BloomDataMap] Optimize...

2018-12-18 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2713#discussion_r242507682 --- Diff: datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMapFactory.java --- @@ -218,56 +218,46 @@ public

[GitHub] carbondata pull request #2917: [WIP]Show load/insert/update/delete row numbe...

2018-12-18 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2917#discussion_r242500505 --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/iud/TestShowIUDRowCount.scala --- @@ -0,0 +1,60

[GitHub] carbondata pull request #2713: [WIP][CARBONDATA-2931][BloomDataMap] Optimize...

2018-12-18 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2713#discussion_r242454233 --- Diff: datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMapFactory.java --- @@ -218,56 +218,46 @@ public

[GitHub] carbondata pull request #2713: [WIP][CARBONDATA-2931][BloomDataMap] Optimize...

2018-12-17 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2713#discussion_r242394373 --- Diff: datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMap.java --- @@ -178,15 +178,9 @@ private String

[GitHub] carbondata issue #2878: [CARBONDATA-3107] Optimize error/exception coding fo...

2018-12-02 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2878 conflicts fixed. ---

[GitHub] carbondata pull request #2917: [WIP]Show load/insert/update/delete row numbe...

2018-11-22 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2917#discussion_r235654816 --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/iud/TestShowIUDRowCount.scala --- @@ -0,0 +1,60

[GitHub] carbondata pull request #2917: [WIP]Show load/insert/update/delete row numbe...

2018-11-13 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2917 [WIP]Show load/insert/update/delete row number Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces

[GitHub] carbondata pull request #2900: [CARBONDATA-3078] Disable explain collector f...

2018-11-05 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2900#discussion_r231014871 --- Diff: hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonTableInputFormat.java --- @@ -575,6 +576,8 @@ private BitSet

[GitHub] carbondata pull request #2900: [CARBONDATA-3078] Disable explain collector f...

2018-11-05 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2900#discussion_r231010531 --- Diff: hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonTableInputFormat.java --- @@ -575,6 +576,8 @@ private BitSet

[GitHub] carbondata pull request #2900: [CARBONDATA-3078] Disable explain collector f...

2018-11-05 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2900#discussion_r230972236 --- Diff: hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonTableInputFormat.java --- @@ -575,6 +576,8 @@ private BitSet

[GitHub] carbondata issue #2900: [CARBONDATA-3078] Disable explain collector for coun...

2018-11-05 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2900 retest this please ---

[GitHub] carbondata pull request #2900: [CARBONDATA-3078] Disable explain collector f...

2018-11-05 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2900 [CARBONDATA-3078] Disable explain collector for count star query without filter An issue is found about count star query without filter in explain command. It is a special case. It uses

[GitHub] carbondata issue #2894: [CARBONDATA-3074] Change default sort temp compresso...

2018-11-05 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2894 retest this please ---

[GitHub] carbondata pull request #2894: [CARBONDATA-3074] Change default sort temp co...

2018-11-05 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2894 [CARBONDATA-3074] Change default sort temp compressor to SNAPPY Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily

[GitHub] carbondata issue #2886: [CARBONDATA-3065]make inverted index false by defaut

2018-11-02 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2886 @akashrn5 thanks for reply. 1. Let's take a detail case. you can check whether it is right. In `DictDimensionIndexCodec#createEncoder`, as the setting I said above

[GitHub] carbondata issue #2886: [CARBONDATA-3065]make inverted index false by defaut

2018-11-01 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2886 The InvertedIndex/NoInvertedIndex setting is confusing. 1. the value `isInvertedIndex` assigned to different IndexCodec in `createEncoderForDimensionLegacy` requires us to set the column

[GitHub] carbondata pull request #2886: [CARBONDATA-3065]make inverted index false by...

2018-11-01 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2886#discussion_r230260481 --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java --- @@ -359,8 +359,13 @@ private CarbonCommonConstants

[GitHub] carbondata issue #2862: [HOTFIX] Enable Local dictionary by default

2018-11-01 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2862 please remember to update the doc too. ---

[GitHub] carbondata pull request #2879: [CARBONDATA-3058] Fix some exception coding i...

2018-10-30 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2879#discussion_r229287888 --- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/steps/CarbonRowDataWriterProcessorStepImpl.java --- @@ -308,7 +312,7

[GitHub] carbondata pull request #2879: [CARBONDATA-3058] Fix some exception coding i...

2018-10-30 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2879#discussion_r229285339 --- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/steps/CarbonRowDataWriterProcessorStepImpl.java --- @@ -212,7 +212,11

[GitHub] carbondata issue #2879: [CARBONDATA-3058] Fix some exception coding in data ...

2018-10-30 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2879 retest this please ---

[GitHub] carbondata pull request #2879: [CARBONDATA-3058] Fix some exception coding i...

2018-10-30 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2879 [CARBONDATA-3058] Fix some exception coding in data loading 1. when exception occur in `dataHandler.finish();`, carbon does not proceed it immediately. Carbon keeps the exception and calls

[GitHub] carbondata pull request #2878: [WIP] Modification of error/exception for bet...

2018-10-30 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2878 [WIP] Modification of error/exception for better debugging Changes in this PR follows these rules: | Code Case| Problem | Suggest Modification

[GitHub] carbondata issue #2732: [CARBONDATA-3020] support lz4 as column compressor

2018-10-28 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2732 retest this please ---

[GitHub] carbondata pull request #2866: [CARBONDATA-3050][Doc] Remove unused paramete...

2018-10-28 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2866 [CARBONDATA-3050][Doc] Remove unused parameter doc Remove documentation of parameter 'carbon.use.multiple.temp.dir' *Related PR:* > #2824 - removed the parame

[GitHub] carbondata pull request #2732: [CARBONDATA-3020] support lz4 as column compr...

2018-10-28 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2732#discussion_r228778578 --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/compression/Lz4Compressor.java --- @@ -0,0 +1,199 @@ +/* + * Licensed

[GitHub] carbondata issue #2851: [CARBONDATA-3040][BloomDataMap] Add checking before ...

2018-10-24 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2851 description updated ---

[GitHub] carbondata pull request #2851: [CARBONDATA-3040][BloomDataMap] Add checking ...

2018-10-24 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2851 [CARBONDATA-3040][BloomDataMap] Add checking before merging bloom index *Scene* There is a bug which causes query failure when we create two bloom datamaps on same table with data

[GitHub] carbondata pull request #2665: [CARBONDATA-2897][DataMap] Assign to datamap ...

2018-10-07 Thread kevinjmh
Github user kevinjmh closed the pull request at: https://github.com/apache/carbondata/pull/2665 ---

[GitHub] carbondata issue #2781: [CARBONDATA-2983][BloomDataMap] Change bloom query m...

2018-09-28 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2781 retest this please ---

[GitHub] carbondata pull request #2781: [CARBONDATA-2983][BloomDataMap] Change bloom ...

2018-09-28 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2781 [CARBONDATA-2983][BloomDataMap] Change bloom query model to proceed multiple filter values currently, bloom generates multiple query model for each value of InExpression. This PR changes

[GitHub] carbondata pull request #2767: [CARBONDATA-2974] Fixed multiple expressions ...

2018-09-27 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2767#discussion_r221123795 --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/DataMapChooser.java --- @@ -282,7 +280,9 @@ private void extractColumnExpression

[GitHub] carbondata pull request #2767: [CARBONDATA-2974] Fixed multiple expressions ...

2018-09-27 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2767#discussion_r221117943 --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/DataMapChooser.java --- @@ -177,34 +180,35 @@ private ExpressionTuple selectDataMap

[GitHub] carbondata issue #2767: [CARBONDATA-2974] Fixed multiple expressions issue o...

2018-09-27 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2767 Hi @ravipesala , I have some question about this PR 1. It is a little confusing to only deal with AND case and ignoring OR case if we want the framework do expression merging in general

[GitHub] carbondata issue #2765: [CARBONDATA-2971] Add shard info of blocklet for deb...

2018-09-26 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2765 retest this please ---

[GitHub] carbondata pull request #2765: [CARBONDATA-2971] Add shard info of blocklet ...

2018-09-26 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2765#discussion_r220490591 --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/Blocklet.java --- @@ -92,7 +95,13 @@ public String getFilePath

[GitHub] carbondata pull request #2765: [CARBONDATA-2971] Add shard info of blocklet ...

2018-09-26 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2765#discussion_r220455196 --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/Blocklet.java --- @@ -92,7 +95,13 @@ public String getFilePath

[GitHub] carbondata pull request #2765: [CARBONDATA-2971] Add shard info of blocklet ...

2018-09-25 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2765 [CARBONDATA-2971] Add shard info of blocklet for debugging add `toString` method to print both shard name and blocklet id for debugging. Be sure to do all of the following

[GitHub] carbondata pull request #2665: [CARBONDATA-2897][DataMap] Optimize datamap c...

2018-09-20 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2665#discussion_r219378413 --- Diff: datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMapFactory.java --- @@ -123,6 +129,42 @@ public

[GitHub] carbondata pull request #2665: [CARBONDATA-2897][DataMap] Optimize datamap c...

2018-09-20 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2665#discussion_r219378217 --- Diff: datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMapFactory.java --- @@ -123,6 +129,42 @@ public

[GitHub] carbondata pull request #2665: [CARBONDATA-2897][DataMap] Optimize datamap c...

2018-09-20 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2665#discussion_r219376540 --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockletDataMapFactory.java --- @@ -364,6 +365,13 @@ public

[GitHub] carbondata issue #2734: [CARBONDATA-2946] Bloom filter backward compatibilit...

2018-09-20 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2734 Test case which found this bug pass. Please refer last PR #2654 in description for tracking ---

[GitHub] carbondata issue #2732: [WIP] lz4 as column compressor in final store

2018-09-19 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2732 retest this please ---

[GitHub] carbondata issue #2665: [CARBONDATA-2897][DataMap] Optimize datamap chooser

2018-09-19 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2665 retest this please ---

[GitHub] carbondata issue #2654: [CARBONDATA-2896] Adaptive Encoding for Primitive da...

2018-09-19 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2654 OK ---

[GitHub] carbondata pull request #2665: [CARBONDATA-2897][DataMap] Optimize datamap c...

2018-09-19 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2665#discussion_r218679293 --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/DataMapChooser.java --- @@ -268,23 +238,38 @@ private ExpressionTuple selectDataMap

[GitHub] carbondata issue #2654: [CARBONDATA-2896] Adaptive Encoding for Primitive da...

2018-09-18 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2654 I ran a test on table with bloom datamap created before applying this PR, and query it after this PR merged, but the answer is not correct. Can you check it? Procedure to reproduce

[GitHub] carbondata pull request #2732: [WIP] lz4 as column compressor in final store

2018-09-18 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2732#discussion_r218645316 --- Diff: core/src/main/java/net/jpountz/lz4/LZ4DecompressorWithLength.java --- @@ -0,0 +1,191 @@ +/* + * Licensed to the Apache Software

[GitHub] carbondata pull request #2732: [WIP] lz4 as column compressor in final store

2018-09-18 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2732#discussion_r218635891 --- Diff: core/src/main/java/net/jpountz/lz4/LZ4CompressorWithLength.java --- @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software

[GitHub] carbondata pull request #2732: [WIP] lz4 as column compressor in final store

2018-09-18 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2732 [WIP] lz4 as column compressor in final store Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces

[GitHub] carbondata pull request #2706: [CARBONDATA-2927] multiple issue fixes for va...

2018-09-18 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2706#discussion_r218403711 --- Diff: processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java --- @@ -239,6 +239,7 @@ public void

[GitHub] carbondata pull request #2723: [CARBONDATA-2938][DataMap] Update comment of ...

2018-09-14 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2723 [CARBONDATA-2938][DataMap] Update comment of blockletId in IndexDataMapRebuildRDD **Background**: #2539 Tried to make use of blocklet id information from query when rebuilding datamap

[GitHub] carbondata issue #2713: [CARBONDATA-2931][BloomDataMap] Optimize bloom datam...

2018-09-12 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2713 retest this please ---

[GitHub] carbondata pull request #2713: [CARBONDATA-2931][BloomDataMap] Optimize bloo...

2018-09-12 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2713 [CARBONDATA-2931][BloomDataMap] Optimize bloom datamap pruning 1. re-use shard pruning info from default datamap 2. create one BloomCoarseGrainDataMap object per segment instead of per

[GitHub] carbondata pull request #2696: [CARBONDATA-2902][DataMap] Fix showing negati...

2018-09-12 Thread kevinjmh
Github user kevinjmh closed the pull request at: https://github.com/apache/carbondata/pull/2696 ---

[GitHub] carbondata pull request #2711: [CARBONDATA-2929][DataMap] Add block skipped ...

2018-09-11 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2711 [CARBONDATA-2929][DataMap] Add block skipped info for explain command This pr will add block skipped info by counting distinct file path from hit blocklet. It shows like below

[GitHub] carbondata issue #2696: [CARBONDATA-2902][DataMap] Fix showing negative prun...

2018-09-10 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2696 I found some material which may be helpful. > A PR can be closed without committing it like so: https://mahout.apache.org/developers/github.html#closing-a-pr-without-committ

[GitHub] carbondata issue #2696: [CARBONDATA-2902][DataMap] Fix showing negative prun...

2018-09-10 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2696 This PR is already merged in this [commit](https://github.com/apache/carbondata/commit/f04850f39d8c42b96ee419140c9506f7df988075), but asfgit did not close this. Is it OK if I close this PR

[GitHub] carbondata pull request #2654: [CARBONDATA-2896] Adaptive Encoding for Primi...

2018-09-06 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2654#discussion_r215829032 --- Diff: datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMap.java --- @@ -331,8 +332,18 @@ private

[GitHub] carbondata pull request #2654: [CARBONDATA-2896] Adaptive Encoding for Primi...

2018-09-06 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2654#discussion_r215827023 --- Diff: datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMap.java --- @@ -331,8 +332,18 @@ private

[GitHub] carbondata pull request #2696: [CARBONDATA-2902][DataMap] Fix showing negati...

2018-09-06 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2696 [CARBONDATA-2902][DataMap] Fix showing negative pruning result for explain command #2676 used method `ByteBuffer.getShort(int index)` to get number of blocklets in block, but it used wrong

[GitHub] carbondata pull request #2654: [CARBONDATA-2896] Adaptive Encoding for Primi...

2018-09-05 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2654#discussion_r215153728 --- Diff: integration/spark2/src/main/scala/org/apache/carbondata/datamap/IndexDataMapRebuildRDD.scala --- @@ -264,8 +264,17 @@ class

[GitHub] carbondata pull request #2685: [CARBONDATA-2910] Support backward compatabil...

2018-09-04 Thread kevinjmh
Github user kevinjmh commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2685#discussion_r214886972 --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockDataMap.java --- @@ -630,6 +638,24 @@ private boolean

[GitHub] carbondata pull request #2688: [CARBONDATA-2911] Remove unused BTree related...

2018-09-03 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2688 [CARBONDATA-2911] Remove unused BTree related code 1. BTree related code is only used by a test class called`BTreeBlockFinderTest`. 2. BTreeDataRefNodeFinder

[GitHub] carbondata pull request #2676: [CARBONDATA-2902][DataMap] Fix showing negati...

2018-08-30 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2676 [CARBONDATA-2902][DataMap] Fix showing negative pruning result for explain command Be sure to do all of the following checklist to help us incorporate your contribution quickly

[GitHub] carbondata issue #2598: [CARBONDATA-2811][BloomDataMap] Add query test case ...

2018-08-29 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2598 retest this please ---

[GitHub] carbondata issue #2665: [CARBONDATA-2897][DataMap] Optimize datamap chooser

2018-08-28 Thread kevinjmh
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2665 retest this please ---

[GitHub] carbondata pull request #2665: [CARBONDATA-2897][DataMap] Optimize datamap c...

2018-08-28 Thread kevinjmh
GitHub user kevinjmh opened a pull request: https://github.com/apache/carbondata/pull/2665 [CARBONDATA-2897][DataMap] Optimize datamap chooser In this PR, 1. Remove code for merging into one datamap when some datamap hits both child nodes of And/Or expression in DataMapChooser

  1   2   >