how to distribute cabon.properties file

2017-04-10 Thread ZhuWilliam
As we know, cabon.properties is the configuration file of carbondata. However ,for now, the driver and executors load this file from local disk which means we should distribute this file to all yarn nodes. At the same time ,we also should configure some items like follows: --conf "spark.driver.ext

CarbonLock Exception refractor

2017-04-10 Thread ZhuWilliam
Carbondata will try to lock table/column/dict when loading data. However ,f

Re: [DISCUSSION]: (New Feature) Streaming Ingestion into CarbonData

2017-03-30 Thread ZhuWilliam
The Design is too complex and we may spend to much time and people to develop it. Can we simplify it and just support streaming first? -- View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/DISCUSSION-New-Feature-Streaming-Ingestion-into-CarbonData

Re: Apache CarbonData online meetup on 13th Mar,2017

2017-03-06 Thread ZhuWilliam
Carbondata's reversed index,BTree implementation and how carbondata do the spark integration -- View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Apache-CarbonData-online-meetup-on-13th-Mar-2017-tp8339p8343.html Sent from the Apache CarbonData

Re: [Improvement] Carbon query gc problem

2016-12-19 Thread ZhuWilliam
+1 Heap should not store data ,it should be used to store runtime temp data. -- View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Improvement-Carbon-query-gc-problem-tp4322p4718.html Sent from the Apache CarbonData Mailing List archive mailing

Re: Using DataFrame to write carbondata file cause no table found error

2016-11-25 Thread ZhuWilliam
When I change the SaveMode.Append to Override,then the error is more weird: INFO 25-11 20:19:46,572 - streaming-job-executor-0 Query [ CREATE TABLE IF NOT EXISTS DEFAULT.CARBON2 (A STRING, B STRING) STORED BY 'ORG.APACHE.CARBONDATA.FORMAT' ] INFO 25-11 20:19

Re: Using DataFrame to write carbondata file cause no table found error

2016-11-25 Thread ZhuWilliam
Here is the error: ERROR 25-11 18:13:40,116 - Data loading failed. table not found: default.carbon1 AUDIT 25-11 18:13:40,118 - [allwefantasy][allwefantasy][Thread-98]Data loading failed. table not found: default.carbon1 INFO 25-11 18:13:40,119 - Finished job streaming job 148006882 ms.0 from

Re: Please vote and advise on building thrift files

2016-11-16 Thread ZhuWilliam
+1 for proposal 1 . Auto generated code should not be added to project. Also most the of time ,people who dive into carbondata may not touch format code. -- View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Please-vote-and-advise-on-building-th