[jira] [Created] (CARBONDATA-941) 7. Compaction of Partition Table
QiangCai created CARBONDATA-941: --- Summary: 7. Compaction of Partition Table Key: CARBONDATA-941 URL: https://issues.apache.org/jira/browse/CARBONDATA-941 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai compaction same partition of segments -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-939) 5. Partition tables join on partition column
QiangCai created CARBONDATA-939: --- Summary: 5. Partition tables join on partition column Key: CARBONDATA-939 URL: https://issues.apache.org/jira/browse/CARBONDATA-939 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-940) 6. Alter table add/drop partition
QiangCai created CARBONDATA-940: --- Summary: 6. Alter table add/drop partition Key: CARBONDATA-940 URL: https://issues.apache.org/jira/browse/CARBONDATA-940 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-938) 4. Detail filter query on partition column
QiangCai created CARBONDATA-938: --- Summary: 4. Detail filter query on partition column Key: CARBONDATA-938 URL: https://issues.apache.org/jira/browse/CARBONDATA-938 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai use filter(equal,range, in etc.) to get partition id list, use this partition id list to filter BTree. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-937) 3. Data loading of partition table
QiangCai created CARBONDATA-937: --- Summary: 3. Data loading of partition table Key: CARBONDATA-937 URL: https://issues.apache.org/jira/browse/CARBONDATA-937 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai use PartitionInfo to generate Partitioner (hash, list, range) use Partitioner to repartition input data file, reuse loadDataFrame flow use partition id to replace task no in carbondata/index file name -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-936) 2. Create Table with Partition
QiangCai created CARBONDATA-936: --- Summary: 2. Create Table with Partition Key: CARBONDATA-936 URL: https://issues.apache.org/jira/browse/CARBONDATA-936 Project: CarbonData Issue Type: Sub-task Environment: CarbonSparkSqlParser parse partition part to generate PartitionInfo, add PartitionInfo to TableModel. CreateTable add PartitionInfo to TableInfo, store PartitionInfo in TableSchema support spark 2.1 at first. Reporter: QiangCai -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-935) 1. Define PartitionInfo model
QiangCai created CARBONDATA-935: --- Summary: 1. Define PartitionInfo model Key: CARBONDATA-935 URL: https://issues.apache.org/jira/browse/CARBONDATA-935 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai modify schema.thrift to define PartitionInfo, add PartitionInfo to TableSchema -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-923) InserInto read from one row not working
QiangCai created CARBONDATA-923: --- Summary: InserInto read from one row not working Key: CARBONDATA-923 URL: https://issues.apache.org/jira/browse/CARBONDATA-923 Project: CarbonData Issue Type: Bug Reporter: QiangCai Assignee: QiangCai Reproduce: create table OneRowTable(col1 string, col2 string, col3 int, col4 double) stored by 'carbondata' insert into OneRowTable select '0.1', 'a.b', 1, 1.2 Exception: org.apache.spark.sql.AnalysisException: cannot resolve '`0.1`' given input columns: [0.1, a.b, 1, 1.2];; 'Project ['0.1, 'a.b] +- Project [0.1 AS 0.1#11, a.b AS a.b#12, 1 AS 1#13, 1.2 AS 1.2#14] +- OneRowRelation$ -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-887) lazy rdd iterator for InsertInto
QiangCai created CARBONDATA-887: --- Summary: lazy rdd iterator for InsertInto Key: CARBONDATA-887 URL: https://issues.apache.org/jira/browse/CARBONDATA-887 Project: CarbonData Issue Type: Improvement Reporter: QiangCai Assignee: QiangCai -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-886) remove all redundant local variable
QiangCai created CARBONDATA-886: --- Summary: remove all redundant local variable Key: CARBONDATA-886 URL: https://issues.apache.org/jira/browse/CARBONDATA-886 Project: CarbonData Issue Type: Improvement Reporter: QiangCai Assignee: QiangCai Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-841) improve the compress encoding for numeric type column to give good performance
[ https://issues.apache.org/jira/browse/CARBONDATA-841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai reassigned CARBONDATA-841: --- Assignee: QiangCai > improve the compress encoding for numeric type column to give good performance > -- > > Key: CARBONDATA-841 > URL: https://issues.apache.org/jira/browse/CARBONDATA-841 > Project: CarbonData > Issue Type: Sub-task >Reporter: QiangCai >Assignee: QiangCai > > Now no-dictionary column use LV(length-value) encoding. It isn't the best > choice for numeric type column. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-842) when SORT_COLUMN is empty, no need to sort data.
[ https://issues.apache.org/jira/browse/CARBONDATA-842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai reassigned CARBONDATA-842: --- Assignee: QiangCai > when SORT_COLUMN is empty, no need to sort data. > > > Key: CARBONDATA-842 > URL: https://issues.apache.org/jira/browse/CARBONDATA-842 > Project: CarbonData > Issue Type: Sub-task >Reporter: QiangCai >Assignee: QiangCai > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-842) when SORT_COLUMN is empty, no need to sort data.
QiangCai created CARBONDATA-842: --- Summary: when SORT_COLUMN is empty, no need to sort data. Key: CARBONDATA-842 URL: https://issues.apache.org/jira/browse/CARBONDATA-842 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-841) improve the compress encoding for numeric type column to give good performance
QiangCai created CARBONDATA-841: --- Summary: improve the compress encoding for numeric type column to give good performance Key: CARBONDATA-841 URL: https://issues.apache.org/jira/browse/CARBONDATA-841 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai Now no-dictionary column use LV(length-value) encoding. It isn't the best choice for numeric type column. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-782) Support SORT_COLUMNS
QiangCai created CARBONDATA-782:
---
Summary: Support SORT_COLUMNS
Key: CARBONDATA-782
URL: https://issues.apache.org/jira/browse/CARBONDATA-782
Project: CarbonData
Issue Type: New Feature
Reporter: QiangCai
Assignee: QiangCai
The tasks of SORT_COLUMNS:
1.Support create table with sort_columns property.
e.g. tblproperties('sort_columns' = 'col7,col3')
The table with SORT_COLUMNS property will be sorted by SORT_COLUMNS. The order
of columns is decided by SORT_COLUMNS.
2.Change the encoding rule of SORT_COLUMNS
Firstly, the rule of column encoding will keep consistent with previous.
Secondly, if a column of SORT_COLUMNS is a measure before, now this column will
be created as a dimension. And this dimension is a no-dicitonary column(Better
to use other direct-dictionary).
Thirdly, the dimension of SORT_COLUMNS have RLE and ROWID page, other dimension
have only RLE(not sorted).
3.The start/end key should be composed of SORT_COLUMNS.
Using SORT_COLUMNS to build start/end key during data loading and select query.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CARBONDATA-763) Add L5 loading support, global sorting like HBase
[ https://issues.apache.org/jira/browse/CARBONDATA-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai updated CARBONDATA-763: Issue Type: New Feature (was: Bug) > Add L5 loading support, global sorting like HBase > - > > Key: CARBONDATA-763 > URL: https://issues.apache.org/jira/browse/CARBONDATA-763 > Project: CarbonData > Issue Type: New Feature >Reporter: QiangCai > > Add L5 loading support, global sorting like HBase -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-765) dataframe wirter need to first drop table unless loading said table not found
QiangCai created CARBONDATA-765: --- Summary: dataframe wirter need to first drop table unless loading said table not found Key: CARBONDATA-765 URL: https://issues.apache.org/jira/browse/CARBONDATA-765 Project: CarbonData Issue Type: Bug Reporter: QiangCai Assignee: QiangCai dataframe wirter need to first drop table unless loading said table not found -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-764) Improving Non-dictionary storage & performance
QiangCai created CARBONDATA-764: --- Summary: Improving Non-dictionary storage & performance Key: CARBONDATA-764 URL: https://issues.apache.org/jira/browse/CARBONDATA-764 Project: CarbonData Issue Type: Improvement Reporter: QiangCai mail list: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Improving-Non-dictionary-storage-amp-performance-td8146.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-763) Add L5 loading support, global sorting like HBase
QiangCai created CARBONDATA-763: --- Summary: Add L5 loading support, global sorting like HBase Key: CARBONDATA-763 URL: https://issues.apache.org/jira/browse/CARBONDATA-763 Project: CarbonData Issue Type: Bug Reporter: QiangCai Add L5 loading support, global sorting like HBase -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-762) modify all schemaName->databaseName, cubeName->tableName
QiangCai created CARBONDATA-762: --- Summary: modify all schemaName->databaseName, cubeName->tableName Key: CARBONDATA-762 URL: https://issues.apache.org/jira/browse/CARBONDATA-762 Project: CarbonData Issue Type: Bug Reporter: QiangCai Assignee: QiangCai Priority: Minor modify all schemaName->databaseName, cubeName->tableName -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-761) Dictionary server should not be shutdown after loading
QiangCai created CARBONDATA-761: --- Summary: Dictionary server should not be shutdown after loading Key: CARBONDATA-761 URL: https://issues.apache.org/jira/browse/CARBONDATA-761 Project: CarbonData Issue Type: Bug Components: data-load Reporter: QiangCai Assignee: QiangCai Priority: Minor Code: CarbonTableSchema/LoadTable -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-760) Should to avoid ERROR log for successful select query
QiangCai created CARBONDATA-760: --- Summary: Should to avoid ERROR log for successful select query Key: CARBONDATA-760 URL: https://issues.apache.org/jira/browse/CARBONDATA-760 Project: CarbonData Issue Type: Bug Components: data-query Reporter: QiangCai Assignee: QiangCai Priority: Minor Some table without delete or update operator maybe not have delta files. Select query shouldn't record error log. Code: SegmentUpdateStatusManager.getDeltaFiles Log detail: ERROR 06-03 19:21:37,531 - pool-475-thread-1 Invalid tuple id arbonstore/default/comparetest_carbon/Fact/0/0/0-0-0-1488799238178/0 ERROR 06-03 19:21:37,948 - pool-475-thread-1 Invalid tuple id arbonstore/default/comparetest_carbon/Fact/0/0/0-0-0-1488799238178/1 ERROR 06-03 19:21:38,517 - pool-475-thread-1 Invalid tuple id arbonstore/default/comparetest_carbon/Fact/0/0/0-0-0-1488799238178/2 ERROR 06-03 19:21:38,909 - pool-475-thread-1 Invalid tuple id arbonstore/default/comparetest_carbon/Fact/0/0/0-0-0-1488799238178/3 ERROR 06-03 19:21:39,292 - pool-475-thread-1 Invalid tuple id arbonstore/default/comparetest_carbon/Fact/0/0/0-0-0-1488799238178/4 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-701) There is a memory leak issue in no kettle loading flow
QiangCai created CARBONDATA-701: --- Summary: There is a memory leak issue in no kettle loading flow Key: CARBONDATA-701 URL: https://issues.apache.org/jira/browse/CARBONDATA-701 Project: CarbonData Issue Type: Improvement Components: data-load Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.1-incubating When loading more data, throw OOM exception. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-659) Should add WhitespaceAround and ParenPad to javastyle
QiangCai created CARBONDATA-659: --- Summary: Should add WhitespaceAround and ParenPad to javastyle Key: CARBONDATA-659 URL: https://issues.apache.org/jira/browse/CARBONDATA-659 Project: CarbonData Issue Type: Improvement Reporter: QiangCai Assignee: QiangCai Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-238) CarbonOptimizer shouldn't add CarbonDictionaryCatalystDecoder for HiveTable
[ https://issues.apache.org/jira/browse/CARBONDATA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai closed CARBONDATA-238. --- Resolution: Fixed Fix Version/s: 0.2.0-incubating > CarbonOptimizer shouldn't add CarbonDictionaryCatalystDecoder for HiveTable > --- > > Key: CARBONDATA-238 > URL: https://issues.apache.org/jira/browse/CARBONDATA-238 > Project: CarbonData > Issue Type: Bug >Reporter: QiangCai >Assignee: QiangCai >Priority: Minor > Fix For: 0.2.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-651) The license header of java file should be same with scala's
QiangCai created CARBONDATA-651: --- Summary: The license header of java file should be same with scala's Key: CARBONDATA-651 URL: https://issues.apache.org/jira/browse/CARBONDATA-651 Project: CarbonData Issue Type: Improvement Reporter: QiangCai Assignee: QiangCai Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CARBONDATA-602) When we are loading data 3 or 4 time using 'USE_KETTLE' ='false' with 'SINGLE_PASS'='true', It is throwing an error
[
https://issues.apache.org/jira/browse/CARBONDATA-602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823833#comment-15823833
]
QiangCai commented on CARBONDATA-602:
-
can you provide sample data? 10 rows maybe enough.
> When we are loading data 3 or 4 time using 'USE_KETTLE' ='false' with
> 'SINGLE_PASS'='true', It is throwing an error
>
>
> Key: CARBONDATA-602
> URL: https://issues.apache.org/jira/browse/CARBONDATA-602
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
>Reporter: Payal
>Assignee: QiangCai
>
> When we are Loading data using 'USE_KETTLE' ='false' with
> 'SINGLE_PASS'='true' ,It is Throwing an error -- Error: java.lang.Exception:
> Data load failed due to error while write dictionary file! (state=,code=0)
> and without 'USE_KETTLE' ='false' Data load is successful
> For Example:
> CREATE TABLE uniqdata_INCLUDEDICTIONARY (CUST_ID int,CUST_NAME
> String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp,
> BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
> DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2
> double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format'
> TBLPROPERTIES('DICTIONARY_INCLUDE'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> 0: jdbc:hive2://192.168.2.126:1> LOAD DATA INPATH
> 'hdfs://localhost:54311/payal/7000_UniqData.csv' into table
> uniqdata_INCLUDEDICTIONARY OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
> 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='true','USE_KETTLE'
> ='fail');
> Error: java.lang.IllegalArgumentException: For input string: "fail"
> (state=,code=0)
> LOAD DATA INPATH 'hdfs://hadoop-master:54311/data/uniqdata/7000_UniqData.csv'
> into table uniqdata_INCLUDEDICTIONARY OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
> 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='true');
> +-+--+
> | Result |
> +-+--+
> +-+--+
> LOGS.
> INFO 06-01 13:31:54,820 - Running query 'LOAD DATA INPATH
> 'hdfs://hadoop-master:54311/data/uniqdata/7000_UniqData.csv' into table
> uniqdata_INCLUDEDICTIONARY OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
> 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='true','USE_KETTLE'
> ='false')' with 2e6007f7-946d-4071-a73f-30d90538ebd6
> INFO 06-01 13:31:54,820 - pool-26-thread-58 Query [LOAD DATA INPATH
> 'HDFS://HADOOP-MASTER:54311/DATA/UNIQDATA/7000_UNIQDATA.CSV' INTO TABLE
> UNIQDATA_INCLUDEDICTIONARY OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
> 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,DOUBLE_COLUMN1,DOUBLE_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='TRUE','USE_KETTLE'
> ='FALSE')]
> INFO 06-01 13:31:54,831 - Successfully able to get the table metadata file
> lock
> INFO 06-01 13:31:54,834 - pool-26-thread-58 Initiating Direct Load for the
> Table : (meradb.uniqdata_includedictionary)
> AUDIT 06-01 13:31:54,838 - [deepak-Vostro-3546][hduser][Thread-494]Data load
> request has been received for table meradb.uniqdata_includedictionary
> AUDIT 06-01 13:31:54,838 - [deepak-Vostro-3546][hduser][Thread-494]Data is
> loading with New Data Flow for table meradb.uniqdata_includedictionary
> INFO 06-01 13:31:54,891 - pool-26-thread-58 [Block Distribution]
> INFO 06-01 13:31:54,891 - pool-26-thread-58 totalInputSpaceConsumed: 1505367
> , defaultParallelism: 8
> INFO 06-01 13:31:54,891 - pool-26-thread-58
> mapreduce.input.fileinputformat.split.maxsize: 16777216
> INFO 06-01 13:31:54,891 - Total input paths to process : 1
> INFO 06-01 13:31:54,892 - pool-26-thread-58 Executors configured : 1
> INFO 06-01 13:31:54,893 - pool-26-thread-58 Requesting total executors: 1
> INFO 06-01 13:31:54,897 - pool-26-thread-58 Total Time taken to ensure the
> required executors : 3
> INFO 06-01 13:31:54,897 - pool-26-thread-58 Time elapsed to allocate the
> required executors: 0
> INFO
[jira] [Commented] (CARBONDATA-617) Insert query not working with UNION
[ https://issues.apache.org/jira/browse/CARBONDATA-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15820052#comment-15820052 ] QiangCai commented on CARBONDATA-617: - I am working for this issue > Insert query not working with UNION > --- > > Key: CARBONDATA-617 > URL: https://issues.apache.org/jira/browse/CARBONDATA-617 > Project: CarbonData > Issue Type: Bug > Components: data-query >Affects Versions: 1.0.0-incubating > Environment: Spark 1.6 > Hadoop 2.6 >Reporter: Deepti Bhardwaj >Assignee: QiangCai >Priority: Minor > Attachments: 2000_UniqData.csv, > thrift-error-log-during-insert-with-union > > > I created 3 table all having same schema > Create table commands: > CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION > string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 > bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 > decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 > int) STORED BY 'org.apache.carbondata.format'; > CREATE TABLE student (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION > string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 > bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 > decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 > int) STORED BY 'org.apache.carbondata.format'; > CREATE TABLE department (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION > string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 > bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 > decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 > int) STORED BY 'org.apache.carbondata.format'; > and I loaded the uniqdata and department table with the attached > CSV(2000_UniqData.csv) > and the insert query used to load data in student table was: > insert into student select > CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1 > from uniqdata UNION select > CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1 > from department; > When I try to insert data into student with union operation, it gives > java.lang.Exception: DataLoad failure.(attached below) > The Union query works well when used alone but when insert is used with Union > it fails. > Also, if I used hive tables instead of carbon tables insert does not work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CARBONDATA-617) Insert query not working with UNION
[ https://issues.apache.org/jira/browse/CARBONDATA-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai reassigned CARBONDATA-617: --- Assignee: QiangCai > Insert query not working with UNION > --- > > Key: CARBONDATA-617 > URL: https://issues.apache.org/jira/browse/CARBONDATA-617 > Project: CarbonData > Issue Type: Bug > Components: data-query >Affects Versions: 1.0.0-incubating > Environment: Spark 1.6 > Hadoop 2.6 >Reporter: Deepti Bhardwaj >Assignee: QiangCai >Priority: Minor > Attachments: 2000_UniqData.csv, > thrift-error-log-during-insert-with-union > > > I created 3 table all having same schema > Create table commands: > CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION > string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 > bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 > decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 > int) STORED BY 'org.apache.carbondata.format'; > CREATE TABLE student (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION > string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 > bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 > decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 > int) STORED BY 'org.apache.carbondata.format'; > CREATE TABLE department (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION > string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 > bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 > decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 > int) STORED BY 'org.apache.carbondata.format'; > and I loaded the uniqdata and department table with the attached > CSV(2000_UniqData.csv) > and the insert query used to load data in student table was: > insert into student select > CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1 > from uniqdata UNION select > CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1 > from department; > When I try to insert data into student with union operation, it gives > java.lang.Exception: DataLoad failure.(attached below) > The Union query works well when used alone but when insert is used with Union > it fails. > Also, if I used hive tables instead of carbon tables insert does not work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CARBONDATA-626) [Dataload] Dataloading is not working with delimiter set as "|"
[
https://issues.apache.org/jira/browse/CARBONDATA-626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
QiangCai reassigned CARBONDATA-626:
---
Assignee: QiangCai
> [Dataload] Dataloading is not working with delimiter set as "|"
> ---
>
> Key: CARBONDATA-626
> URL: https://issues.apache.org/jira/browse/CARBONDATA-626
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
>Affects Versions: 1.0.0-incubating
> Environment: 3 node cluster
>Reporter: SOURYAKANTA DWIVEDY
>Assignee: QiangCai
>
> Description : Data loading fail with delimiter as "|" .
> Steps:
> > 1. Create table
> > 2. Load data into table
> Log :-
> -
> - create table DIM_TERMINAL
> (
> ID int,
> TAC String,
> TER_BRAND_NAME String,
> TER_MODEL_NAME String,
> TER_MODENAME String,
> TER_TYPE_ID String,
> TER_TYPE_NAME_EN String,
> TER_TYPE_NAME_CHN String,
> TER_OSTYPE String,
> TER_OS_TYPE_NAME String,
> HSPASPEED String,
> LTESPEED String,
> VOLTE_FLAG String,
> flag String
> ) stored by 'org.apache.carbondata.format' TBLPROPERTIES
> ('DICTIONARY_INCLUDE'='TAC,TER_BRAND_NAME,TER_MODEL_NAME,TER_MODENAME,TER_TYPE_ID,TER_TYPE_NAME_EN,TER_TYPE_NAME_CHN,TER_OSTYPE,TER_OS_TYPE_NAME,HSPASPEED,LTESPEED,VOLTE_FLAG,flag');
> - jdbc:hive2://172.168.100.212:23040> LOAD DATA inpath
> 'hdfs://hacluster/SEQIQ/IQ_DIM_TERMINAL.csv' INTO table DIM_TERMINAL1
> OPTIONS('DELIMITER'='|','USE_KETTLE'='false','QUOTECHAR'='','FILEHEADER'=
> 'ID,TAC,TER_BRAND_NAME,TER_MODEL_NAME,TER_MODENAME,TER_TYPE_ID,TER_TYPE_NAME_EN,TER_TYPE_NAME_CHN,TER_OSTYPE,TER_OS_TYPE_NAME,HSPASPEED,LTESPEED,VOLTE_FLAG,flag');
> Error: java.lang.RuntimeException: Data loading failed. table not found:
> default.dim_terminal1 (state=,code=0)
> 0: jdbc:hive2://172.168.100.212:23040> LOAD DATA inpath
> 'hdfs://hacluster/SEQIQ/IQ_DIM_TERMINAL1.csv' INTO table DIM_TERMINAL
> OPTIONS('DELIMITER'='|','USE_KETTLE'='false','QUOTECHAR'='','FILEHEADER'=
> 'ID,TAC,TER_BRAND_NAME,TER_MODEL_NAME,TER_MODENAME,TER_TYPE_ID,TER_TYPE_NAME_EN,TER_TYPE_NAME_CHN,TER_OSTYPE,TER_OS_TYPE_NAME,HSPASPEED,LTESPEED,VOLTE_FLAG,flag');
> Error: org.apache.spark.sql.AnalysisException: Reference 'D' is ambiguous,
> could be: D#4893, D#4907, D#4920, D#4935, D#4952, D#5025, D#5034.;
> (state=,code=0)
> - csv raw details :
> 103880|99000537|MI|2S H1SC 3C|2G/3G|0|SmartPhone|SmartPhone|4|Android|||1|
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CARBONDATA-626) [Dataload] Dataloading is not working with delimiter set as "|"
[
https://issues.apache.org/jira/browse/CARBONDATA-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819937#comment-15819937
]
QiangCai commented on CARBONDATA-626:
-
PR518 has fixed this issue.
https://github.com/apache/incubator-carbondata/pull/518
> [Dataload] Dataloading is not working with delimiter set as "|"
> ---
>
> Key: CARBONDATA-626
> URL: https://issues.apache.org/jira/browse/CARBONDATA-626
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
>Affects Versions: 1.0.0-incubating
> Environment: 3 node cluster
>Reporter: SOURYAKANTA DWIVEDY
>
> Description : Data loading fail with delimiter as "|" .
> Steps:
> > 1. Create table
> > 2. Load data into table
> Log :-
> -
> - create table DIM_TERMINAL
> (
> ID int,
> TAC String,
> TER_BRAND_NAME String,
> TER_MODEL_NAME String,
> TER_MODENAME String,
> TER_TYPE_ID String,
> TER_TYPE_NAME_EN String,
> TER_TYPE_NAME_CHN String,
> TER_OSTYPE String,
> TER_OS_TYPE_NAME String,
> HSPASPEED String,
> LTESPEED String,
> VOLTE_FLAG String,
> flag String
> ) stored by 'org.apache.carbondata.format' TBLPROPERTIES
> ('DICTIONARY_INCLUDE'='TAC,TER_BRAND_NAME,TER_MODEL_NAME,TER_MODENAME,TER_TYPE_ID,TER_TYPE_NAME_EN,TER_TYPE_NAME_CHN,TER_OSTYPE,TER_OS_TYPE_NAME,HSPASPEED,LTESPEED,VOLTE_FLAG,flag');
> - jdbc:hive2://172.168.100.212:23040> LOAD DATA inpath
> 'hdfs://hacluster/SEQIQ/IQ_DIM_TERMINAL.csv' INTO table DIM_TERMINAL1
> OPTIONS('DELIMITER'='|','USE_KETTLE'='false','QUOTECHAR'='','FILEHEADER'=
> 'ID,TAC,TER_BRAND_NAME,TER_MODEL_NAME,TER_MODENAME,TER_TYPE_ID,TER_TYPE_NAME_EN,TER_TYPE_NAME_CHN,TER_OSTYPE,TER_OS_TYPE_NAME,HSPASPEED,LTESPEED,VOLTE_FLAG,flag');
> Error: java.lang.RuntimeException: Data loading failed. table not found:
> default.dim_terminal1 (state=,code=0)
> 0: jdbc:hive2://172.168.100.212:23040> LOAD DATA inpath
> 'hdfs://hacluster/SEQIQ/IQ_DIM_TERMINAL1.csv' INTO table DIM_TERMINAL
> OPTIONS('DELIMITER'='|','USE_KETTLE'='false','QUOTECHAR'='','FILEHEADER'=
> 'ID,TAC,TER_BRAND_NAME,TER_MODEL_NAME,TER_MODENAME,TER_TYPE_ID,TER_TYPE_NAME_EN,TER_TYPE_NAME_CHN,TER_OSTYPE,TER_OS_TYPE_NAME,HSPASPEED,LTESPEED,VOLTE_FLAG,flag');
> Error: org.apache.spark.sql.AnalysisException: Reference 'D' is ambiguous,
> could be: D#4893, D#4907, D#4920, D#4935, D#4952, D#5025, D#5034.;
> (state=,code=0)
> - csv raw details :
> 103880|99000537|MI|2S H1SC 3C|2G/3G|0|SmartPhone|SmartPhone|4|Android|||1|
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CARBONDATA-627) Fix Union unit test case for spark2
QiangCai created CARBONDATA-627: --- Summary: Fix Union unit test case for spark2 Key: CARBONDATA-627 URL: https://issues.apache.org/jira/browse/CARBONDATA-627 Project: CarbonData Issue Type: Bug Components: data-query Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 1.0.0-incubating UnionTestCase failed in spark2, We should fix it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-622) Should use the same fileheader reader for dict generation and data loading
QiangCai created CARBONDATA-622: --- Summary: Should use the same fileheader reader for dict generation and data loading Key: CARBONDATA-622 URL: https://issues.apache.org/jira/browse/CARBONDATA-622 Project: CarbonData Issue Type: Bug Components: data-load Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 1.0.0-incubating We can get file header from DDL command and CSV file. 1. If the file header comes from DDL command, separate this file header by comma "," 2. if the file header comes from CSV file, sparate this file header by specify delimiter in DDL command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-614) Should fix dictionary locked issue
QiangCai created CARBONDATA-614: --- Summary: Should fix dictionary locked issue Key: CARBONDATA-614 URL: https://issues.apache.org/jira/browse/CARBONDATA-614 Project: CarbonData Issue Type: Bug Components: data-load Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating When carbon.properties.filepath is configured exactly, still show the following exception. Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 8, hadoop-slave-2): java.lang.RuntimeException: Dictionary file name is locked for updation. Please try after some time at scala.sys.package$.error(package.scala:27) at org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD$$anon$1.(CarbonGlobalDictionaryRDD.scala:364) at org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD.compute(CarbonGlobalDictionaryRDD.scala:302) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306) at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) at org.apache.spark.scheduler.Task.run(Task.scala:89) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-601) Should reuse unit test case for integration module
QiangCai created CARBONDATA-601: --- Summary: Should reuse unit test case for integration module Key: CARBONDATA-601 URL: https://issues.apache.org/jira/browse/CARBONDATA-601 Project: CarbonData Issue Type: Test Components: spark-integration Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 1.0.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-600) Should reuse unit test case for integration module
QiangCai created CARBONDATA-600: --- Summary: Should reuse unit test case for integration module Key: CARBONDATA-600 URL: https://issues.apache.org/jira/browse/CARBONDATA-600 Project: CarbonData Issue Type: Bug Components: spark-integration Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 1.0.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-540) Support insertInto without kettle for spark2
[ https://issues.apache.org/jira/browse/CARBONDATA-540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai updated CARBONDATA-540: Summary: Support insertInto without kettle for spark2 (was: Support inserInto without kettle for spark2) > Support insertInto without kettle for spark2 > > > Key: CARBONDATA-540 > URL: https://issues.apache.org/jira/browse/CARBONDATA-540 > Project: CarbonData > Issue Type: Improvement > Components: data-load >Affects Versions: 1.0.0-incubating >Reporter: QiangCai >Assignee: QiangCai > Fix For: 1.0.0-incubating > > > Support inserInto without kettle for spark2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-540) Support inserInto without kettle for spark2
QiangCai created CARBONDATA-540: --- Summary: Support inserInto without kettle for spark2 Key: CARBONDATA-540 URL: https://issues.apache.org/jira/browse/CARBONDATA-540 Project: CarbonData Issue Type: Improvement Components: data-load Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating Support inserInto without kettle for spark2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-536) For spark2, GlobalDictionaryUtil.updateTableMetadataFunc should been initialized
QiangCai created CARBONDATA-536: --- Summary: For spark2, GlobalDictionaryUtil.updateTableMetadataFunc should been initialized Key: CARBONDATA-536 URL: https://issues.apache.org/jira/browse/CARBONDATA-536 Project: CarbonData Issue Type: Bug Components: data-load Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating For spark2, GlobalDictionaryUtil.updateTableMetadataFunc should been initialized -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-535) carbondata should support datatype: Date and Char
QiangCai created CARBONDATA-535: --- Summary: carbondata should support datatype: Date and Char Key: CARBONDATA-535 URL: https://issues.apache.org/jira/browse/CARBONDATA-535 Project: CarbonData Issue Type: Improvement Components: file-format Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating carbondata should support datatype: Date and Char -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-516) [SPARK2]fix union issue in CarbonLateDecoderRule
QiangCai created CARBONDATA-516: --- Summary: [SPARK2]fix union issue in CarbonLateDecoderRule Key: CARBONDATA-516 URL: https://issues.apache.org/jira/browse/CARBONDATA-516 Project: CarbonData Issue Type: Bug Reporter: QiangCai Assignee: QiangCai In spark2, Union class is no longer the sub-class of BinaryNode. We need fix union issue in CarbonLateDecoderRule for spark2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-497) [Spark2] fix datatype issue of CarbonLateDecoderRule
QiangCai created CARBONDATA-497: --- Summary: [Spark2] fix datatype issue of CarbonLateDecoderRule Key: CARBONDATA-497 URL: https://issues.apache.org/jira/browse/CARBONDATA-497 Project: CarbonData Issue Type: Bug Components: data-query Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating In spark2, LogicalPlan resolve method need to check input data type. If the data type is wrong, the logical plan will be unresolved. CarbonLateDecoderRule should correct the datatype of dictionary dimension to resolve the logical plan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-493) Insertinto sql can not select from a empty table
[ https://issues.apache.org/jira/browse/CARBONDATA-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai updated CARBONDATA-493: Description: example: Insert into target_table select * from source_table if source table is empty, data loading step will throw exception > Insertinto sql can not select from a empty table > > > Key: CARBONDATA-493 > URL: https://issues.apache.org/jira/browse/CARBONDATA-493 > Project: CarbonData > Issue Type: Bug >Affects Versions: 1.0.0-incubating >Reporter: QiangCai >Assignee: QiangCai > Fix For: 1.0.0-incubating > > > example: > Insert into target_table select * from source_table > if source table is empty, data loading step will throw exception -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-493) Insertinto sql can not select from a empty table
QiangCai created CARBONDATA-493: --- Summary: Insertinto sql can not select from a empty table Key: CARBONDATA-493 URL: https://issues.apache.org/jira/browse/CARBONDATA-493 Project: CarbonData Issue Type: Bug Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-492) When profile spark-2.0 is avtive , CarbonExample have error in intellij idea
QiangCai created CARBONDATA-492: --- Summary: When profile spark-2.0 is avtive , CarbonExample have error in intellij idea Key: CARBONDATA-492 URL: https://issues.apache.org/jira/browse/CARBONDATA-492 Project: CarbonData Issue Type: Bug Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Critical Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-488) add InsertInto feature for spark2
QiangCai created CARBONDATA-488: --- Summary: add InsertInto feature for spark2 Key: CARBONDATA-488 URL: https://issues.apache.org/jira/browse/CARBONDATA-488 Project: CarbonData Issue Type: New Feature Components: data-load Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-486) Reading dataframe concurrently will lead to wrong data
[ https://issues.apache.org/jira/browse/CARBONDATA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai updated CARBONDATA-486: Summary: Reading dataframe concurrently will lead to wrong data (was: Rreading dataframe concurrently will lead to wrong data) > Reading dataframe concurrently will lead to wrong data > -- > > Key: CARBONDATA-486 > URL: https://issues.apache.org/jira/browse/CARBONDATA-486 > Project: CarbonData > Issue Type: Bug > Components: data-load >Affects Versions: 0.3.0-incubating >Reporter: QiangCai >Assignee: QiangCai > Fix For: 0.3.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-486) Rreading dataframe concurrently will lead to wrong data
QiangCai created CARBONDATA-486: --- Summary: Rreading dataframe concurrently will lead to wrong data Key: CARBONDATA-486 URL: https://issues.apache.org/jira/browse/CARBONDATA-486 Project: CarbonData Issue Type: Bug Components: data-load Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-481) [SPARK2]fix late decoder and support whole stage code gen
QiangCai created CARBONDATA-481: --- Summary: [SPARK2]fix late decoder and support whole stage code gen Key: CARBONDATA-481 URL: https://issues.apache.org/jira/browse/CARBONDATA-481 Project: CarbonData Issue Type: Bug Components: data-query Affects Versions: 0.2.0-incubating Reporter: QiangCai Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-478) Spark2 module should have different SparkRowReadSupportImpl with spark1
QiangCai created CARBONDATA-478: --- Summary: Spark2 module should have different SparkRowReadSupportImpl with spark1 Key: CARBONDATA-478 URL: https://issues.apache.org/jira/browse/CARBONDATA-478 Project: CarbonData Issue Type: Bug Components: data-query Affects Versions: 0.2.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-443) Implement nosort dataloading
QiangCai created CARBONDATA-443: --- Summary: Implement nosort dataloading Key: CARBONDATA-443 URL: https://issues.apache.org/jira/browse/CARBONDATA-443 Project: CarbonData Issue Type: Improvement Components: data-load Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 0.3.0-incubating Improve dataloading performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-368) Should improve performance of DataFrame loading
QiangCai created CARBONDATA-368: --- Summary: Should improve performance of DataFrame loading Key: CARBONDATA-368 URL: https://issues.apache.org/jira/browse/CARBONDATA-368 Project: CarbonData Issue Type: Improvement Components: data-load Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-287) Save the sorted temp files to multi local dirs to improve dataloading perfomance
QiangCai created CARBONDATA-287: --- Summary: Save the sorted temp files to multi local dirs to improve dataloading perfomance Key: CARBONDATA-287 URL: https://issues.apache.org/jira/browse/CARBONDATA-287 Project: CarbonData Issue Type: Improvement Components: data-load Affects Versions: 0.2.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 0.2.0-incubating Now for each dataloading, we use only a different local dir to save the sorted temp files. I think it is neccessary to use multi local dirs for each dataloading to improve dataloading performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-279) [DataLoading]Save a DataFrame to CarbonData file without writing CSV file
QiangCai created CARBONDATA-279: --- Summary: [DataLoading]Save a DataFrame to CarbonData file without writing CSV file Key: CARBONDATA-279 URL: https://issues.apache.org/jira/browse/CARBONDATA-279 Project: CarbonData Issue Type: Improvement Affects Versions: 0.1.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 0.2.0-incubating Directly save a DataFrame to CarbonData file without writing CSV file -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-268) CarbonOptimizer has performance problem
[ https://issues.apache.org/jira/browse/CARBONDATA-268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai updated CARBONDATA-268: Description: 1. invoke relation lookup many times 2. lookup list many times > CarbonOptimizer has performance problem > --- > > Key: CARBONDATA-268 > URL: https://issues.apache.org/jira/browse/CARBONDATA-268 > Project: CarbonData > Issue Type: Bug >Affects Versions: 0.1.0-incubating >Reporter: QiangCai > > 1. invoke relation lookup many times > 2. lookup list many times -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-268) CarbonOptimizer has performance problem
QiangCai created CARBONDATA-268: --- Summary: CarbonOptimizer has performance problem Key: CARBONDATA-268 URL: https://issues.apache.org/jira/browse/CARBONDATA-268 Project: CarbonData Issue Type: Bug Affects Versions: 0.1.0-incubating Reporter: QiangCai -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-238) CarbonOptimizer shouldn't add CarbonDictionaryCatalystDecoder for HiveTable
QiangCai created CARBONDATA-238: --- Summary: CarbonOptimizer shouldn't add CarbonDictionaryCatalystDecoder for HiveTable Key: CARBONDATA-238 URL: https://issues.apache.org/jira/browse/CARBONDATA-238 Project: CarbonData Issue Type: Bug Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 0.2.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CARBONDATA-218) Remove Dependency: spark-csv and Unify CSV Reader for dataloading
[ https://issues.apache.org/jira/browse/CARBONDATA-218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai reassigned CARBONDATA-218: --- Assignee: QiangCai > Remove Dependency: spark-csv and Unify CSV Reader for dataloading > - > > Key: CARBONDATA-218 > URL: https://issues.apache.org/jira/browse/CARBONDATA-218 > Project: CarbonData > Issue Type: Improvement >Reporter: QiangCai >Assignee: QiangCai >Priority: Minor > Fix For: 0.2.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CARBONDATA-213) Remove thrift complier dependency
[ https://issues.apache.org/jira/browse/CARBONDATA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai reassigned CARBONDATA-213: --- Assignee: QiangCai > Remove thrift complier dependency > - > > Key: CARBONDATA-213 > URL: https://issues.apache.org/jira/browse/CARBONDATA-213 > Project: CarbonData > Issue Type: Bug >Reporter: QiangCai >Assignee: QiangCai >Priority: Minor > Fix For: 0.2.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-218) Remove Dependency: spark-csv and Unify CSV Reader for dataloading
QiangCai created CARBONDATA-218: --- Summary: Remove Dependency: spark-csv and Unify CSV Reader for dataloading Key: CARBONDATA-218 URL: https://issues.apache.org/jira/browse/CARBONDATA-218 Project: CarbonData Issue Type: Improvement Reporter: QiangCai Priority: Minor Fix For: 0.2.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-213) Remove thrift complier dependency
QiangCai created CARBONDATA-213: --- Summary: Remove thrift complier dependency Key: CARBONDATA-213 URL: https://issues.apache.org/jira/browse/CARBONDATA-213 Project: CarbonData Issue Type: Bug Reporter: QiangCai Priority: Minor Fix For: 0.2.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
