[jira] [Created] (CARBONDATA-941) 7. Compaction of Partition Table
QiangCai created CARBONDATA-941: --- Summary: 7. Compaction of Partition Table Key: CARBONDATA-941 URL: https://issues.apache.org/jira/browse/CARBONDATA-941 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai compaction same partition of segments -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-940) 6. Alter table add/drop partition
QiangCai created CARBONDATA-940: --- Summary: 6. Alter table add/drop partition Key: CARBONDATA-940 URL: https://issues.apache.org/jira/browse/CARBONDATA-940 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-939) 5. Partition tables join on partition column
QiangCai created CARBONDATA-939: --- Summary: 5. Partition tables join on partition column Key: CARBONDATA-939 URL: https://issues.apache.org/jira/browse/CARBONDATA-939 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-938) 4. Detail filter query on partition column
QiangCai created CARBONDATA-938: --- Summary: 4. Detail filter query on partition column Key: CARBONDATA-938 URL: https://issues.apache.org/jira/browse/CARBONDATA-938 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai use filter(equal,range, in etc.) to get partition id list, use this partition id list to filter BTree. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-937) 3. Data loading of partition table
QiangCai created CARBONDATA-937: --- Summary: 3. Data loading of partition table Key: CARBONDATA-937 URL: https://issues.apache.org/jira/browse/CARBONDATA-937 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai use PartitionInfo to generate Partitioner (hash, list, range) use Partitioner to repartition input data file, reuse loadDataFrame flow use partition id to replace task no in carbondata/index file name -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-936) 2. Create Table with Partition
QiangCai created CARBONDATA-936: --- Summary: 2. Create Table with Partition Key: CARBONDATA-936 URL: https://issues.apache.org/jira/browse/CARBONDATA-936 Project: CarbonData Issue Type: Sub-task Environment: CarbonSparkSqlParser parse partition part to generate PartitionInfo, add PartitionInfo to TableModel. CreateTable add PartitionInfo to TableInfo, store PartitionInfo in TableSchema support spark 2.1 at first. Reporter: QiangCai -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-935) 1. Define PartitionInfo model
QiangCai created CARBONDATA-935: --- Summary: 1. Define PartitionInfo model Key: CARBONDATA-935 URL: https://issues.apache.org/jira/browse/CARBONDATA-935 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai modify schema.thrift to define PartitionInfo, add PartitionInfo to TableSchema -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-923) InserInto read from one row not working
QiangCai created CARBONDATA-923: --- Summary: InserInto read from one row not working Key: CARBONDATA-923 URL: https://issues.apache.org/jira/browse/CARBONDATA-923 Project: CarbonData Issue Type: Bug Reporter: QiangCai Assignee: QiangCai Reproduce: create table OneRowTable(col1 string, col2 string, col3 int, col4 double) stored by 'carbondata' insert into OneRowTable select '0.1', 'a.b', 1, 1.2 Exception: org.apache.spark.sql.AnalysisException: cannot resolve '`0.1`' given input columns: [0.1, a.b, 1, 1.2];; 'Project ['0.1, 'a.b] +- Project [0.1 AS 0.1#11, a.b AS a.b#12, 1 AS 1#13, 1.2 AS 1.2#14] +- OneRowRelation$ -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-887) lazy rdd iterator for InsertInto
QiangCai created CARBONDATA-887: --- Summary: lazy rdd iterator for InsertInto Key: CARBONDATA-887 URL: https://issues.apache.org/jira/browse/CARBONDATA-887 Project: CarbonData Issue Type: Improvement Reporter: QiangCai Assignee: QiangCai -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-886) remove all redundant local variable
QiangCai created CARBONDATA-886: --- Summary: remove all redundant local variable Key: CARBONDATA-886 URL: https://issues.apache.org/jira/browse/CARBONDATA-886 Project: CarbonData Issue Type: Improvement Reporter: QiangCai Assignee: QiangCai Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-842) when SORT_COLUMN is empty, no need to sort data.
QiangCai created CARBONDATA-842: --- Summary: when SORT_COLUMN is empty, no need to sort data. Key: CARBONDATA-842 URL: https://issues.apache.org/jira/browse/CARBONDATA-842 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-841) improve the compress encoding for numeric type column to give good performance
QiangCai created CARBONDATA-841: --- Summary: improve the compress encoding for numeric type column to give good performance Key: CARBONDATA-841 URL: https://issues.apache.org/jira/browse/CARBONDATA-841 Project: CarbonData Issue Type: Sub-task Reporter: QiangCai Now no-dictionary column use LV(length-value) encoding. It isn't the best choice for numeric type column. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-782) Support SORT_COLUMNS
QiangCai created CARBONDATA-782:
---
Summary: Support SORT_COLUMNS
Key: CARBONDATA-782
URL: https://issues.apache.org/jira/browse/CARBONDATA-782
Project: CarbonData
Issue Type: New Feature
Reporter: QiangCai
Assignee: QiangCai
The tasks of SORT_COLUMNS:
1.Support create table with sort_columns property.
e.g. tblproperties('sort_columns' = 'col7,col3')
The table with SORT_COLUMNS property will be sorted by SORT_COLUMNS. The order
of columns is decided by SORT_COLUMNS.
2.Change the encoding rule of SORT_COLUMNS
Firstly, the rule of column encoding will keep consistent with previous.
Secondly, if a column of SORT_COLUMNS is a measure before, now this column will
be created as a dimension. And this dimension is a no-dicitonary column(Better
to use other direct-dictionary).
Thirdly, the dimension of SORT_COLUMNS have RLE and ROWID page, other dimension
have only RLE(not sorted).
3.The start/end key should be composed of SORT_COLUMNS.
Using SORT_COLUMNS to build start/end key during data loading and select query.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CARBONDATA-765) dataframe wirter need to first drop table unless loading said table not found
QiangCai created CARBONDATA-765: --- Summary: dataframe wirter need to first drop table unless loading said table not found Key: CARBONDATA-765 URL: https://issues.apache.org/jira/browse/CARBONDATA-765 Project: CarbonData Issue Type: Bug Reporter: QiangCai Assignee: QiangCai dataframe wirter need to first drop table unless loading said table not found -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-764) Improving Non-dictionary storage & performance
QiangCai created CARBONDATA-764: --- Summary: Improving Non-dictionary storage & performance Key: CARBONDATA-764 URL: https://issues.apache.org/jira/browse/CARBONDATA-764 Project: CarbonData Issue Type: Improvement Reporter: QiangCai mail list: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Improving-Non-dictionary-storage-amp-performance-td8146.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-762) modify all schemaName->databaseName, cubeName->tableName
QiangCai created CARBONDATA-762: --- Summary: modify all schemaName->databaseName, cubeName->tableName Key: CARBONDATA-762 URL: https://issues.apache.org/jira/browse/CARBONDATA-762 Project: CarbonData Issue Type: Bug Reporter: QiangCai Assignee: QiangCai Priority: Minor modify all schemaName->databaseName, cubeName->tableName -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-763) Add L5 loading support, global sorting like HBase
QiangCai created CARBONDATA-763: --- Summary: Add L5 loading support, global sorting like HBase Key: CARBONDATA-763 URL: https://issues.apache.org/jira/browse/CARBONDATA-763 Project: CarbonData Issue Type: Bug Reporter: QiangCai Add L5 loading support, global sorting like HBase -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-761) Dictionary server should not be shutdown after loading
QiangCai created CARBONDATA-761: --- Summary: Dictionary server should not be shutdown after loading Key: CARBONDATA-761 URL: https://issues.apache.org/jira/browse/CARBONDATA-761 Project: CarbonData Issue Type: Bug Components: data-load Reporter: QiangCai Assignee: QiangCai Priority: Minor Code: CarbonTableSchema/LoadTable -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-760) Should to avoid ERROR log for successful select query
QiangCai created CARBONDATA-760: --- Summary: Should to avoid ERROR log for successful select query Key: CARBONDATA-760 URL: https://issues.apache.org/jira/browse/CARBONDATA-760 Project: CarbonData Issue Type: Bug Components: data-query Reporter: QiangCai Assignee: QiangCai Priority: Minor Some table without delete or update operator maybe not have delta files. Select query shouldn't record error log. Code: SegmentUpdateStatusManager.getDeltaFiles Log detail: ERROR 06-03 19:21:37,531 - pool-475-thread-1 Invalid tuple id arbonstore/default/comparetest_carbon/Fact/0/0/0-0-0-1488799238178/0 ERROR 06-03 19:21:37,948 - pool-475-thread-1 Invalid tuple id arbonstore/default/comparetest_carbon/Fact/0/0/0-0-0-1488799238178/1 ERROR 06-03 19:21:38,517 - pool-475-thread-1 Invalid tuple id arbonstore/default/comparetest_carbon/Fact/0/0/0-0-0-1488799238178/2 ERROR 06-03 19:21:38,909 - pool-475-thread-1 Invalid tuple id arbonstore/default/comparetest_carbon/Fact/0/0/0-0-0-1488799238178/3 ERROR 06-03 19:21:39,292 - pool-475-thread-1 Invalid tuple id arbonstore/default/comparetest_carbon/Fact/0/0/0-0-0-1488799238178/4 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-701) There is a memory leak issue in no kettle loading flow
QiangCai created CARBONDATA-701: --- Summary: There is a memory leak issue in no kettle loading flow Key: CARBONDATA-701 URL: https://issues.apache.org/jira/browse/CARBONDATA-701 Project: CarbonData Issue Type: Improvement Components: data-load Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.1-incubating When loading more data, throw OOM exception. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-659) Should add WhitespaceAround and ParenPad to javastyle
QiangCai created CARBONDATA-659: --- Summary: Should add WhitespaceAround and ParenPad to javastyle Key: CARBONDATA-659 URL: https://issues.apache.org/jira/browse/CARBONDATA-659 Project: CarbonData Issue Type: Improvement Reporter: QiangCai Assignee: QiangCai Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-651) The license header of java file should be same with scala's
QiangCai created CARBONDATA-651: --- Summary: The license header of java file should be same with scala's Key: CARBONDATA-651 URL: https://issues.apache.org/jira/browse/CARBONDATA-651 Project: CarbonData Issue Type: Improvement Reporter: QiangCai Assignee: QiangCai Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-627) Fix Union unit test case for spark2
QiangCai created CARBONDATA-627: --- Summary: Fix Union unit test case for spark2 Key: CARBONDATA-627 URL: https://issues.apache.org/jira/browse/CARBONDATA-627 Project: CarbonData Issue Type: Bug Components: data-query Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 1.0.0-incubating UnionTestCase failed in spark2, We should fix it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-622) Should use the same fileheader reader for dict generation and data loading
QiangCai created CARBONDATA-622: --- Summary: Should use the same fileheader reader for dict generation and data loading Key: CARBONDATA-622 URL: https://issues.apache.org/jira/browse/CARBONDATA-622 Project: CarbonData Issue Type: Bug Components: data-load Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 1.0.0-incubating We can get file header from DDL command and CSV file. 1. If the file header comes from DDL command, separate this file header by comma "," 2. if the file header comes from CSV file, sparate this file header by specify delimiter in DDL command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-614) Should fix dictionary locked issue
QiangCai created CARBONDATA-614: --- Summary: Should fix dictionary locked issue Key: CARBONDATA-614 URL: https://issues.apache.org/jira/browse/CARBONDATA-614 Project: CarbonData Issue Type: Bug Components: data-load Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating When carbon.properties.filepath is configured exactly, still show the following exception. Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 8, hadoop-slave-2): java.lang.RuntimeException: Dictionary file name is locked for updation. Please try after some time at scala.sys.package$.error(package.scala:27) at org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD$$anon$1.(CarbonGlobalDictionaryRDD.scala:364) at org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD.compute(CarbonGlobalDictionaryRDD.scala:302) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306) at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) at org.apache.spark.scheduler.Task.run(Task.scala:89) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-600) Should reuse unit test case for integration module
QiangCai created CARBONDATA-600: --- Summary: Should reuse unit test case for integration module Key: CARBONDATA-600 URL: https://issues.apache.org/jira/browse/CARBONDATA-600 Project: CarbonData Issue Type: Bug Components: spark-integration Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 1.0.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-601) Should reuse unit test case for integration module
QiangCai created CARBONDATA-601: --- Summary: Should reuse unit test case for integration module Key: CARBONDATA-601 URL: https://issues.apache.org/jira/browse/CARBONDATA-601 Project: CarbonData Issue Type: Test Components: spark-integration Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 1.0.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-540) Support inserInto without kettle for spark2
QiangCai created CARBONDATA-540: --- Summary: Support inserInto without kettle for spark2 Key: CARBONDATA-540 URL: https://issues.apache.org/jira/browse/CARBONDATA-540 Project: CarbonData Issue Type: Improvement Components: data-load Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating Support inserInto without kettle for spark2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-536) For spark2, GlobalDictionaryUtil.updateTableMetadataFunc should been initialized
QiangCai created CARBONDATA-536: --- Summary: For spark2, GlobalDictionaryUtil.updateTableMetadataFunc should been initialized Key: CARBONDATA-536 URL: https://issues.apache.org/jira/browse/CARBONDATA-536 Project: CarbonData Issue Type: Bug Components: data-load Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating For spark2, GlobalDictionaryUtil.updateTableMetadataFunc should been initialized -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-535) carbondata should support datatype: Date and Char
QiangCai created CARBONDATA-535: --- Summary: carbondata should support datatype: Date and Char Key: CARBONDATA-535 URL: https://issues.apache.org/jira/browse/CARBONDATA-535 Project: CarbonData Issue Type: Improvement Components: file-format Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating carbondata should support datatype: Date and Char -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-516) [SPARK2]fix union issue in CarbonLateDecoderRule
QiangCai created CARBONDATA-516: --- Summary: [SPARK2]fix union issue in CarbonLateDecoderRule Key: CARBONDATA-516 URL: https://issues.apache.org/jira/browse/CARBONDATA-516 Project: CarbonData Issue Type: Bug Reporter: QiangCai Assignee: QiangCai In spark2, Union class is no longer the sub-class of BinaryNode. We need fix union issue in CarbonLateDecoderRule for spark2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-497) [Spark2] fix datatype issue of CarbonLateDecoderRule
QiangCai created CARBONDATA-497: --- Summary: [Spark2] fix datatype issue of CarbonLateDecoderRule Key: CARBONDATA-497 URL: https://issues.apache.org/jira/browse/CARBONDATA-497 Project: CarbonData Issue Type: Bug Components: data-query Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating In spark2, LogicalPlan resolve method need to check input data type. If the data type is wrong, the logical plan will be unresolved. CarbonLateDecoderRule should correct the datatype of dictionary dimension to resolve the logical plan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-493) Insertinto sql can not select from a empty table
QiangCai created CARBONDATA-493: --- Summary: Insertinto sql can not select from a empty table Key: CARBONDATA-493 URL: https://issues.apache.org/jira/browse/CARBONDATA-493 Project: CarbonData Issue Type: Bug Affects Versions: 1.0.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 1.0.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-492) When profile spark-2.0 is avtive , CarbonExample have error in intellij idea
QiangCai created CARBONDATA-492: --- Summary: When profile spark-2.0 is avtive , CarbonExample have error in intellij idea Key: CARBONDATA-492 URL: https://issues.apache.org/jira/browse/CARBONDATA-492 Project: CarbonData Issue Type: Bug Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Critical Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-488) add InsertInto feature for spark2
QiangCai created CARBONDATA-488: --- Summary: add InsertInto feature for spark2 Key: CARBONDATA-488 URL: https://issues.apache.org/jira/browse/CARBONDATA-488 Project: CarbonData Issue Type: New Feature Components: data-load Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-486) Rreading dataframe concurrently will lead to wrong data
QiangCai created CARBONDATA-486: --- Summary: Rreading dataframe concurrently will lead to wrong data Key: CARBONDATA-486 URL: https://issues.apache.org/jira/browse/CARBONDATA-486 Project: CarbonData Issue Type: Bug Components: data-load Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-481) [SPARK2]fix late decoder and support whole stage code gen
QiangCai created CARBONDATA-481: --- Summary: [SPARK2]fix late decoder and support whole stage code gen Key: CARBONDATA-481 URL: https://issues.apache.org/jira/browse/CARBONDATA-481 Project: CarbonData Issue Type: Bug Components: data-query Affects Versions: 0.2.0-incubating Reporter: QiangCai Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-478) Spark2 module should have different SparkRowReadSupportImpl with spark1
QiangCai created CARBONDATA-478: --- Summary: Spark2 module should have different SparkRowReadSupportImpl with spark1 Key: CARBONDATA-478 URL: https://issues.apache.org/jira/browse/CARBONDATA-478 Project: CarbonData Issue Type: Bug Components: data-query Affects Versions: 0.2.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 0.3.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-443) Implement nosort dataloading
QiangCai created CARBONDATA-443: --- Summary: Implement nosort dataloading Key: CARBONDATA-443 URL: https://issues.apache.org/jira/browse/CARBONDATA-443 Project: CarbonData Issue Type: Improvement Components: data-load Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai Fix For: 0.3.0-incubating Improve dataloading performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-368) Should improve performance of DataFrame loading
QiangCai created CARBONDATA-368: --- Summary: Should improve performance of DataFrame loading Key: CARBONDATA-368 URL: https://issues.apache.org/jira/browse/CARBONDATA-368 Project: CarbonData Issue Type: Improvement Components: data-load Affects Versions: 0.3.0-incubating Reporter: QiangCai Assignee: QiangCai -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-287) Save the sorted temp files to multi local dirs to improve dataloading perfomance
QiangCai created CARBONDATA-287: --- Summary: Save the sorted temp files to multi local dirs to improve dataloading perfomance Key: CARBONDATA-287 URL: https://issues.apache.org/jira/browse/CARBONDATA-287 Project: CarbonData Issue Type: Improvement Components: data-load Affects Versions: 0.2.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 0.2.0-incubating Now for each dataloading, we use only a different local dir to save the sorted temp files. I think it is neccessary to use multi local dirs for each dataloading to improve dataloading performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-279) [DataLoading]Save a DataFrame to CarbonData file without writing CSV file
QiangCai created CARBONDATA-279: --- Summary: [DataLoading]Save a DataFrame to CarbonData file without writing CSV file Key: CARBONDATA-279 URL: https://issues.apache.org/jira/browse/CARBONDATA-279 Project: CarbonData Issue Type: Improvement Affects Versions: 0.1.0-incubating Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 0.2.0-incubating Directly save a DataFrame to CarbonData file without writing CSV file -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-268) CarbonOptimizer has performance problem
QiangCai created CARBONDATA-268: --- Summary: CarbonOptimizer has performance problem Key: CARBONDATA-268 URL: https://issues.apache.org/jira/browse/CARBONDATA-268 Project: CarbonData Issue Type: Bug Affects Versions: 0.1.0-incubating Reporter: QiangCai -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-238) CarbonOptimizer shouldn't add CarbonDictionaryCatalystDecoder for HiveTable
QiangCai created CARBONDATA-238: --- Summary: CarbonOptimizer shouldn't add CarbonDictionaryCatalystDecoder for HiveTable Key: CARBONDATA-238 URL: https://issues.apache.org/jira/browse/CARBONDATA-238 Project: CarbonData Issue Type: Bug Reporter: QiangCai Assignee: QiangCai Priority: Minor Fix For: 0.2.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-218) Remove Dependency: spark-csv and Unify CSV Reader for dataloading
QiangCai created CARBONDATA-218: --- Summary: Remove Dependency: spark-csv and Unify CSV Reader for dataloading Key: CARBONDATA-218 URL: https://issues.apache.org/jira/browse/CARBONDATA-218 Project: CarbonData Issue Type: Improvement Reporter: QiangCai Priority: Minor Fix For: 0.2.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-213) Remove thrift complier dependency
QiangCai created CARBONDATA-213: --- Summary: Remove thrift complier dependency Key: CARBONDATA-213 URL: https://issues.apache.org/jira/browse/CARBONDATA-213 Project: CarbonData Issue Type: Bug Reporter: QiangCai Priority: Minor Fix For: 0.2.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-180) give proper error message when dataloading with wrong delimiter value
QiangCai created CARBONDATA-180:
---
Summary: give proper error message when dataloading with wrong
delimiter value
Key: CARBONDATA-180
URL: https://issues.apache.org/jira/browse/CARBONDATA-180
Project: CarbonData
Issue Type: Bug
Reporter: QiangCai
Assignee: QiangCai
Priority: Minor
Reproduce:
CREATE TABLE IF NOT EXISTS t3
(ID Int, date Timestamp, country String, name String, phonetype String,
serialname String, salary Int)
STORED BY 'carbondata'
LOAD DATA LOCAL INPATH 'data.csv' into table t3 options('DELIMITER'='#',
'FILEHEADER'='ID,date,country,name,phonetype,serialname,salary')
File name:data.csv
1,2015/7/23,china,aaa1,phone197,ASD69643,15000
2,2015/7/24,china,aaa2,phone756,ASD42892,15001
3,2015/7/25,china,aaa3,phone1904,ASD37014,15002
Exception:
ERROR 26-08 11:44:12,881 - generate global dictionary failed
java.lang.ArrayIndexOutOfBoundsException: 2
at
org.apache.carbondata.spark.util.GlobalDictionaryUtil$$anonfun$pruneDimensions$1$$anonfun$apply$1$$anonfun$apply$mcV$sp$1.apply(GlobalDictionaryUtil.scala:79)
at
org.apache.carbondata.spark.util.GlobalDictionaryUtil$$anonfun$pruneDimensions$1$$anonfun$apply$1$$anonfun$apply$mcV$sp$1.apply(GlobalDictionaryUtil.scala:76)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
at
org.apache.carbondata.spark.util.GlobalDictionaryUtil$$anonfun$pruneDimensions$1$$anonfun$apply$1.apply$mcV$sp(GlobalDictionaryUtil.scala:76)
at scala.util.control.Breaks.breakable(Breaks.scala:37)
at
org.apache.carbondata.spark.util.GlobalDictionaryUtil$$anonfun$pruneDimensions$1.apply(GlobalDictionaryUtil.scala:75)
at
org.apache.carbondata.spark.util.GlobalDictionaryUtil$$anonfun$pruneDimensions$1.apply(GlobalDictionaryUtil.scala:74)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
at
org.apache.carbondata.spark.util.GlobalDictionaryUtil$.pruneDimensions(GlobalDictionaryUtil.scala:74)
at
org.apache.carbondata.spark.util.GlobalDictionaryUtil$.generateGlobalDictionary(GlobalDictionaryUtil.scala:697)
at
org.apache.spark.sql.execution.command.LoadTable.run(carbonTableSchema.scala:1159)
at
org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57)
at
org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57)
at
org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:69)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:140)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:138)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
at
org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:138)
at
org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:933)
at
org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:933)
at org.apache.spark.sql.DataFrame.(DataFrame.scala:144)
at org.apache.spark.sql.DataFrame.(DataFrame.scala:129)
at
org.apache.carbondata.spark.rdd.CarbonDataFrameRDD.(CarbonDataFrameRDD.scala:23)
at org.apache.spark.sql.CarbonContext.sql(CarbonContext.scala:130)
at
org.apache.carbondata.examples.CarbonExample$.main(CarbonExample.scala:42)
at
org.apache.carbondata.examples.CarbonExample.main(CarbonExample.scala)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CARBONDATA-1) CarbonData supports docker for simplifying deployment
[ https://issues.apache.org/jira/browse/CARBONDATA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QiangCai reassigned CARBONDATA-1: - Assignee: QiangCai > CarbonData supports docker for simplifying deployment > - > > Key: CARBONDATA-1 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1 > Project: CarbonData > Issue Type: New Feature > Environment: the latest docker stable version >Reporter: ChenLiang >Assignee: QiangCai > > CarbonData supports docker for simplifying deployment -- This message was sent by Atlassian JIRA (v6.3.4#6332)
