Re: Carbon over-use cluster resources

2020-04-20 Thread Manhua Jiang
2020 at 6:06 PM Liang Chen wrote: > > > OK, thank you feedbacked this issue, let us look into it. > > > > Regards > > Liang > > > > > > Manhua Jiang wrote > > > Hi All, > > > Recently, I found carbon over-use cluster resources. Generally

Re: Carbon over-use cluster resources

2020-04-20 Thread Manhua Jiang
> > Ajantha > > > > On Tue, Apr 14, 2020 at 6:06 PM Liang Chen > > wrote: > > > > > OK, thank you feedbacked this issue, let us look into it. > > > > > > Regards > > > Liang > > > > > > > > > Manhua Jian

Carbon over-use cluster resources

2020-04-02 Thread Manhua Jiang
Hi All, Recently, I found carbon over-use cluster resources. Generally the design of carbon work flow does not act as common spark task which only do one small work in one thread, but the task has its mind/logic. For example, 1.launch carbon with --num-executors=1 but set carbon.number.of.cores

Re: [ANNOUNCE] Kunal Kapoor as new PMC for Apache CarbonData

2020-03-31 Thread Manhua Jiang
Congratulations Kunal ! Regards, Manhua On 2020/03/30 09:31:33, Indhumathi wrote: > Congratulations Kunal! > > Regards, > Indhumathi > > > > -- > Sent from: > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ >

Re: Apply to open 'Issues' tab in Apache CarbonData github

2019-12-18 Thread Manhua Jiang
+1 Issues tab is easier to reach than JIRA too On 2019/12/19 03:06:58, "恩爸" <441586...@qq.com> wrote: > Hi community: >   I suggest community to open 'Issues' tab in carbondata github page, we can > use this feature to collect the information of carbondata users, like this: > https://github.com

Re: [DISCUSSION] Page Level Bloom Filter

2019-11-04 Thread Manhua Jiang
Hi Jacky, If we create bloom filter in blocklet level, maybe too similar to bloom datamap and have to face the same problems bloom datamap facing, except the pruning is running in executor side. Page level is preferred since page size is KNOWN and this let us get rid of considering how many

[DISCUSSION] Page Level Bloom Filter

2019-10-31 Thread Manhua Jiang
Hi Community, Bloom datamap has been implemented for a while at blocklet level. One problem of bloom datamap is that the pruning process is done in driver side and caching the bloom index data is expensive. So here we are proposing to build bloom filter inside the carbon data file at page lev