[jira] [Created] (CARBONDATA-820) Redundant BitSet created in data load

2017-03-25 Thread Jacky Li (JIRA)
Jacky Li created CARBONDATA-820: --- Summary: Redundant BitSet created in data load Key: CARBONDATA-820 URL: https://issues.apache.org/jira/browse/CARBONDATA-820 Project: CarbonData Issue Type: Bu

[DISCUSSION] Initiating Apache CarbonData-1.1.0 incubating Release

2017-03-25 Thread Ravindra Pesala
Hi All, As planned we are going to release Apache CarbonData-1.1.0. Please discuss and vote for it to initiate 1.1.0 release, i will start to prepare the release after 3-days of discussion. It will have following features. 1. Introduced new data format called V3(version 3). Improves the seque

Re: Questions about dictionary-encoded column and MDK

2017-03-25 Thread Liang Chen
Hi william Exactly! your understanding is pretty correct. And currently community is developing sort_columns feature, user can specify columns to make MDK. the PR number is 635. Invite all of you to review this pr code. Regards Liang 2017-03-26 9:15 GMT+05:30 william : > 1. Dictionary encodin

Re: Questions about dictionary-encoded column and MDK

2017-03-25 Thread william
1. Dictionary encoding make column storage more efficient with small size and improved search performance。 2. when search,MDK/Min-Max can be used to do block/blocklet prunning in oder to reduce IO. For now ,MDK is composed by dimensions with the oder of declared in create table statement On Th

Re:Re: insert into carbon table failed

2017-03-25 Thread a
Thank you Ravindra! Version: My carbondata version is 1.0,spark version is 1.6.3,hadoop version is 2.7.1,hive version is 1.1.0 one of the containers log: 17/03/25 22:07:09 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL 15: SIGTERM 17/03/25 22:07:09 INFO storage.DiskBlockManager: Sh

Re: insert into carbon table failed

2017-03-25 Thread Ravindra Pesala
Hi, Carbodata launches one job per each node to sort the data at node level and avoid shuffling. Internally it uses threads to use parallel load. Please use carbon.number.of.cores.while.loading property in carbon.properties file and set the number of cores it should use per machine while loading.

Re:Re: insert into carbon table failed

2017-03-25 Thread a
| col_name | data_type | 基数数量 | | dt | string | date | | pt | string | 3 | | lst | string | 1 | | plat | string | 1 | | sty | string | 2 | | is_pay | string | 2 | | is_vip | string | 2 | | is_mpack | string | 2 | | scene| string | 3 | | status | string | 4 | | nw

Re:Re: insert into carbon table failed

2017-03-25 Thread a
At 2017-03-25 18:52:07, "Liang Chen" wrote: >Hi > >Please provide all columns' cardinality info(distinct value). > >Regards >Liang > > >ww...@163.com wrote >> Hello! >> >> 0、The failure >> When i insert into carbon table，i encounter failure。The failure is as >> follow: >> Job aborted du

Re: insert into carbon table failed

2017-03-25 Thread Liang Chen
Hi Please provide all columns' cardinality info(distinct value). Regards Liang ww...@163.com wrote > Hello! > > 0、The failure > When i insert into carbon table，i encounter failure。The failure is as > follow: > Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most > recent

insert into carbon table failed

2017-03-25 Thread ww...@163.com
Hello! 0、The failure When i insert into carbon table，i encounter failure。The failure is as follow: Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 1007, hd26): ExecutorLostFailure (executor 1 exited caused by one of the