[GitHub] carbondata issue #3046: [CARBONDATA-3231] Fix OOM exception when dictionary ...

2019-01-09 Thread xuchuanyin
Github user xuchuanyin commented on the issue: https://github.com/apache/carbondata/pull/3046 We do not need to expose this threshold to the user. Instead, we can judge ourselves in carbondata. Step1. We can get the size of non-dictionary-encoded page (say M) and the size of

[GitHub] carbondata issue #3046: [CARBONDATA-3231] Fix OOM exception when dictionary ...

2019-01-09 Thread kumarvishal09
Github user kumarvishal09 commented on the issue: https://github.com/apache/carbondata/pull/3046 @xuchuanyin In near future we are planing to change threshold(currently based on number) to size based local dictionary. Size based threshold will give more control. Current changes

[GitHub] carbondata issue #3046: [CARBONDATA-3231] Fix OOM exception when dictionary ...

2019-01-08 Thread xuchuanyin
Github user xuchuanyin commented on the issue: https://github.com/apache/carbondata/pull/3046 > This PR is just to add the size based limitation so that the map size can be controlled. @kunal642 Yeah, I noticed that. So my proposal is that please make a reservation for minimal

[GitHub] carbondata issue #3046: [CARBONDATA-3231] Fix OOM exception when dictionary ...

2019-01-07 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3046 Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10461/ ---

[GitHub] carbondata issue #3046: [CARBONDATA-3231] Fix OOM exception when dictionary ...

2019-01-07 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3046 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2421/ ---

[GitHub] carbondata issue #3046: [CARBONDATA-3231] Fix OOM exception when dictionary ...

2019-01-07 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3046 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2204/ ---

[GitHub] carbondata issue #3046: [CARBONDATA-3231] Fix OOM exception when dictionary ...

2019-01-07 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3046 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2404/ ---

[GitHub] carbondata issue #3046: [CARBONDATA-3231] Fix OOM exception when dictionary ...

2019-01-07 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3046 Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10444/ ---

[GitHub] carbondata issue #3046: [CARBONDATA-3231] Fix OOM exception when dictionary ...

2019-01-07 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3046 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2188/ ---