Re: [Discussion] Implement Partition Table Feature

1. carbon use different sql parser in spark1.6 and 2.1, need to change CarbonSQLParser for 1.6 2. for interval range partition, no fixed partition name is defined in DDL, but need to keep partition name in schema and update when new partition is added. 3. one btree for one partition and one

bucket table

Hi Dev, I created a bucket table and loaded 100 million rows of data. But there is only one data file and one index file in path. It returned a "partially success" error when loading was done. I debugged into the program and it shows bad record is not null. Any one can explain why bad record will

Re: Getting Error in Cloudera Distribution

HOT.jar > > > On 07-Apr-2017 1:36 pm, "Lu Cao" <whuca...@gmail.com> wrote: > > What's the spark version you're using? > Did you add "--jars " when you start the spark shell? > > On Fri, Apr 7, 2017 at 3:38 PM, Srigopal Mohanty < > srigopalmoha...@

Re: Getting Error in Cloudera Distribution

reconfigured. > > CarbonData is cloned it from git URL specified. And maven build was done > as per the steps mentioned. > > Thanks, > Srigopal > > On 07-Apr-2017 1:05 pm, "Lu Cao" <whuca...@gmail.com> wrote: > > Hi Srigopal, > You can follow this: >

Re: Getting Error in Cloudera Distribution

Hi Srigopal, You can follow this: https://github.com/apache/incubator-carbondata/blob/master/docs/quick-start-guide.md Make sure you have correctly configured carbon and spark. Thanks, Lionel On Fri, Apr 7, 2017 at 3:16 PM, Srigopal Mohanty wrote: > Hi Team, > >

Re: Re:Re: Re: Optimize Order By + Limit Query

@Liang, Yes, actually I'm currently working on the limit query optimization. I get the limited dictionary value and convert to the filter condition in CarbonOptimizer step. It would definitely improve the query performance in some scenario. On Thu, Mar 30, 2017 at 2:07 PM, Liang Chen

A question about sort in carbon

Hi dev team, I have a question about the sort in carbon data. When we have following query: select country, area, name, salary from table_a order by country; It seems carbon will decode the country column from dictionary value to original value first, and then sort by original value. My question

Re: [jira] [Created] (CARBONDATA-559) Job failed at last step

Hi team, Could you help look into this issue? I have attached the log in the Jira ticket. Thanks & Best Regards, Lionel On Fri, Dec 23, 2016 at 5:47 PM, Cao, Lionel (JIRA) wrote: > Cao, Lionel created CARBONDATA-559: > -- > >

Re: carbondata-0.2 load data failed in yarn molde

upt...@gmail.com> wrote: > Hi Lu Cao, > > The problem you are facing "Dictionary file is locked for updation" can > also come when the path formation is incorrect for the dictionary files. > > You have to set carbon.properties file path both in driver and executor > side.

Re: carbondata-0.2 load data failed in yarn molde

Hi team, Looks like I've met the same problem about dictionary file is locked. Could you share what changes you made about the configuration? ERROR 23-12 09:55:26,222 - Executor task launch worker-0 java.lang.RuntimeException: Dictionary file vehsyspwrmod is locked for updation. Please try after

Re: ［Carbondata-0.2.0-incubating][Issue Report] -- Select statement return error when add String column in where clause

Executor$TaskRunner.run(Executor.scala:213) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) On Wed, Dec 14, 2016 at 10:18 AM, Lu Cao <whuca...@gmail.com> wrote: >

Re: ［Carbondata-0.2.0-incubating][Issue Report] -- Select statement return error when add String column in where clause

Hi, I just uploaded the data file to Baidu: 链接: https://pan.baidu.com/s/1slERWL3 密码: m7kj Thanks, Lionel On Wed, Dec 14, 2016 at 10:12 AM, Lu Cao <whuca...@gmail.com> wrote: > Hi Dev team, > As discussed this afternoon, I've changed back to 0.2.0 version for the > testing

［Carbondata-0.2.0-incubating][Issue Report] -- Select statement return error when add String column in where clause

Hi Dev team, As discussed this afternoon, I've changed back to 0.2.0 version for the testing. Please ignore the former email about "error when save DF to carbondata file", that's on master branch. Spark version: 1.6.0 System: Mac OS X EI Capitan(10.11.6) [lucao]$ spark-shell --master local[*]

error when save DF to carbondata file

Hi Dev team, I run spark-shell in my local spark standalone mode. It returned error java.io.IOException: No input paths specified in job when I was trying to save the df to carbondata file. Do I miss any settings about the path??

Re: select return error when filter string column in where clause

olExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) On Tue, Dec 6, 2016 at 9:35 AM, Lu Cao <whuca...@gmail.com> wrote: > Hi Dev team, > I have loaded some data into carbondata table

select return error when filter string column in where clause

Hi Dev team, I have loaded some data into carbondata table. But when I put the id column(String type) in where clause it always return error as below: cc.sql("select to_date(data_date),count(*) from default.carbontest_001 where id='LSJW26762FS044062' group by to_date(data_date)").show

carbondata loading

Hi dev team, I'm loading data from parquet file to carbondata file(DF read parquet and save to csv then load into carbondata file). The job is blocked at "collect at CarbonDataRDDFactory.scala:963" *Job Id* *Description* *Submitted* *Duration* *Stages: Succeeded/Total* *Tasks (for all

Re: carbon data

Thank you for the response Liang. I think I have followed the example but it still returns error: Data loading failed. table not found: default.carbontest attached my code below: I read data from a hive table with HiveContext and convert it to CarbonContext then generate the df and save to

carbon data

Hi team, I'm trying to save spark dataframe to carbondata file. I see the example in your wiki option("tableName", "carbontable"). Does that mean I have to create a carbondata table first and then save data into the table? Can I save it directly without creating the carbondata table? the code is