[jira] [Commented] (CARBONDATA-2447) Range Partition Table。When the update operation is performed, the data will be lost.
[ https://issues.apache.org/jira/browse/CARBONDATA-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480136#comment-16480136 ] Cao, Lionel commented on CARBONDATA-2447: - Hi， Update operation is not supported for carbon partition table, please try "standard partition". > Range Partition Table。When the update operation is performed, the data will > be lost. > > > Key: CARBONDATA-2447 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2447 > Project: CarbonData > Issue Type: Bug > Components: spark-integration >Affects Versions: 1.3.1 > Environment: centos6.5 > java8 > Spark2.1.0 > CarbonData1.3.1 >Reporter: duweike >Priority: Blocker > Fix For: NONE > > Attachments: 微信图片_20180507113738.jpg, 微信图片_20180507113748.jpg > > Original Estimate: 72h > Remaining Estimate: 72h > > Range Partition Table。When the update operation is performed, the data will > be lost. > As shown in the picture。 > 如下面图片所示，数据丢失必现。 > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CARBONDATA-1648) Change alter table drop partition to support two level partitions
Cao, Lionel created CARBONDATA-1648: --- Summary: Change alter table drop partition to support two level partitions Key: CARBONDATA-1648 URL: https://issues.apache.org/jira/browse/CARBONDATA-1648 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1647) Change show partition to support two level partitions
Cao, Lionel created CARBONDATA-1647: --- Summary: Change show partition to support two level partitions Key: CARBONDATA-1647 URL: https://issues.apache.org/jira/browse/CARBONDATA-1647 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1646) Concurrent performance testing of partition tables
Cao, Lionel created CARBONDATA-1646: --- Summary: Concurrent performance testing of partition tables Key: CARBONDATA-1646 URL: https://issues.apache.org/jira/browse/CARBONDATA-1646 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1645) Change alter table add/split partition to support two level partitions
Cao, Lionel created CARBONDATA-1645: --- Summary: Change alter table add/split partition to support two level partitions Key: CARBONDATA-1645 URL: https://issues.apache.org/jira/browse/CARBONDATA-1645 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1644) Change query process to support two level partitions
Cao, Lionel created CARBONDATA-1644: --- Summary: Change query process to support two level partitions Key: CARBONDATA-1644 URL: https://issues.apache.org/jira/browse/CARBONDATA-1644 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1643) Change load process to support two level partitions
Cao, Lionel created CARBONDATA-1643: --- Summary: Change load process to support two level partitions Key: CARBONDATA-1643 URL: https://issues.apache.org/jira/browse/CARBONDATA-1643 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1642) Implement Hash-Hash partitioner
Cao, Lionel created CARBONDATA-1642: --- Summary: Implement Hash-Hash partitioner Key: CARBONDATA-1642 URL: https://issues.apache.org/jira/browse/CARBONDATA-1642 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1641) Implement Hash-List partitioner
Cao, Lionel created CARBONDATA-1641: --- Summary: Implement Hash-List partitioner Key: CARBONDATA-1641 URL: https://issues.apache.org/jira/browse/CARBONDATA-1641 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1640) Implement Hash-Range partitioner
Cao, Lionel created CARBONDATA-1640: --- Summary: Implement Hash-Range partitioner Key: CARBONDATA-1640 URL: https://issues.apache.org/jira/browse/CARBONDATA-1640 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1639) Implement List-Hash partitioner
Cao, Lionel created CARBONDATA-1639: --- Summary: Implement List-Hash partitioner Key: CARBONDATA-1639 URL: https://issues.apache.org/jira/browse/CARBONDATA-1639 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1638) Implement List-List partitioner
Cao, Lionel created CARBONDATA-1638: --- Summary: Implement List-List partitioner Key: CARBONDATA-1638 URL: https://issues.apache.org/jira/browse/CARBONDATA-1638 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1637) Implement List-Range partitioner
Cao, Lionel created CARBONDATA-1637: --- Summary: Implement List-Range partitioner Key: CARBONDATA-1637 URL: https://issues.apache.org/jira/browse/CARBONDATA-1637 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1636) Implement Range-Hash partitioner
Cao, Lionel created CARBONDATA-1636: --- Summary: Implement Range-Hash partitioner Key: CARBONDATA-1636 URL: https://issues.apache.org/jira/browse/CARBONDATA-1636 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1635) Implement Range-List partitioner
Cao, Lionel created CARBONDATA-1635: --- Summary: Implement Range-List partitioner Key: CARBONDATA-1635 URL: https://issues.apache.org/jira/browse/CARBONDATA-1635 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1634) Implement Range-Range partitioner
Cao, Lionel created CARBONDATA-1634: --- Summary: Implement Range-Range partitioner Key: CARBONDATA-1634 URL: https://issues.apache.org/jira/browse/CARBONDATA-1634 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1633) Change parser to support two level partitions
Cao, Lionel created CARBONDATA-1633: --- Summary: Change parser to support two level partitions Key: CARBONDATA-1633 URL: https://issues.apache.org/jira/browse/CARBONDATA-1633 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (CARBONDATA-1631) Implement Range Interval Partition
[ https://issues.apache.org/jira/browse/CARBONDATA-1631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel reassigned CARBONDATA-1631: --- Assignee: (was: Cao, Lionel) > Implement Range Interval Partition > -- > > Key: CARBONDATA-1631 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1631 > Project: CarbonData > Issue Type: Sub-task > Components: core, spark-integration, sql >Reporter: Cao, Lionel > Fix For: 1.3.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1632) Change PartitionInfo and related model to support two level partitions
Cao, Lionel created CARBONDATA-1632: --- Summary: Change PartitionInfo and related model to support two level partitions Key: CARBONDATA-1632 URL: https://issues.apache.org/jira/browse/CARBONDATA-1632 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1631) Implement Range Interval Partition
Cao, Lionel created CARBONDATA-1631: --- Summary: Implement Range Interval Partition Key: CARBONDATA-1631 URL: https://issues.apache.org/jira/browse/CARBONDATA-1631 Project: CarbonData Issue Type: Sub-task Reporter: Cao, Lionel Assignee: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1629) Partition Function Enhancement
Cao, Lionel created CARBONDATA-1629: --- Summary: Partition Function Enhancement Key: CARBONDATA-1629 URL: https://issues.apache.org/jira/browse/CARBONDATA-1629 Project: CarbonData Issue Type: New Feature Components: core, spark-integration, sql Affects Versions: 1.3.0 Reporter: Cao, Lionel Assignee: Cao, Lionel Fix For: 1.3.0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (CARBONDATA-1599) Optimize pull request template for reminding contributors to provide full info.
[ https://issues.apache.org/jira/browse/CARBONDATA-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel resolved CARBONDATA-1599. - Resolution: Fixed > Optimize pull request template for reminding contributors to provide full > info. > --- > > Key: CARBONDATA-1599 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1599 > Project: CarbonData > Issue Type: Improvement > Components: other >Reporter: Liang Chen >Assignee: Liang Chen >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (CARBONDATA-1401) List Info validate Issue
[ https://issues.apache.org/jira/browse/CARBONDATA-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel resolved CARBONDATA-1401. - Resolution: Fixed > List Info validate Issue > > > Key: CARBONDATA-1401 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1401 > Project: CarbonData > Issue Type: Bug > Components: spark-integration, sql >Reporter: Cao, Lionel >Assignee: Cao, Lionel > Time Spent: 1h 40m > Remaining Estimate: 0h > > fix duplicate issue in list info -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (CARBONDATA-1427) After Splitting Partition, Data doesn't get Divided to Different Partitions.
[
https://issues.apache.org/jira/browse/CARBONDATA-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cao, Lionel reassigned CARBONDATA-1427:
---
Assignee: Cao, Lionel (was: Pallavi Singh)
> After Splitting Partition, Data doesn't get Divided to Different Partitions.
>
>
> Key: CARBONDATA-1427
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1427
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.2.0
> Environment: spark 2.1
>Reporter: Neha Bhardwaj
>Assignee: Cao, Lionel
>Priority: Minor
> Attachments: list_partition_table.csv, screenshot-1.png
>
>
> When Performing a Split Partition Query on a Partitioned Table, The data
> doesn't get affected at all, however, we can see the updated Partitions using
> the show Partitions Query and the old partition as deleted.
> But the data still remains in that partition, Ideally, the data should be
> divided as per the new partitions, Which happens after the subsequent loads,
> the data then gets to the latest partitions.
> Example :
> 1. Create Table :
> DROP TABLE IF EXISTS list_partition_table;
> CREATE TABLE list_partition_table(shortField SHORT, intField INT, bigintField
> LONG, doubleField DOUBLE, timestampField TIMESTAMP, decimalField
> DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT,
> complexData ARRAY ) PARTITIONED BY (stringField STRING) STORED BY
> 'carbondata' TBLPROPERTIES('PARTITION_TYPE'='LIST', 'LIST_INFO'='Asia,
> (China, Europe, NoPartition)');
> 2. Load Data :
> load data inpath 'hdfs://localhost:54310/CSV/list_partition_table.csv' into
> table list_partition_table
> options('FILEHEADER'='shortfield,intfield,bigintfield,doublefield,stringfield,timestampfield,decimalfield,datefield,charfield,floatfield,complexdata',
> 'COMPLEX_DELIMITER_LEVEL_1'='$','COMPLEX_DELIMITER_LEVEL_2'='#');
> 3. Show Partitions :
> show partitions list_partition_table;
> +--+--+
> | partition |
> +--+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia|
> | 2, stringfield = China, Europe, NoPartition |
> +--+--+
> 3 rows selected (0.09 seconds)
> 4. Split Partition :
> ALTER TABLE list_partition_table SPLIT PARTITION(2) INTO('China', '(Europe,
> NoPartition)' );
> 5. Show Partition :
> show partitions list_partition_table;
> +---+--+
> | partition |
> +---+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia |
> | 3, stringfield = China|
> | 4, stringfield = Europe, NoPartition |
> +---+--+
> 4 rows selected (0.065 seconds)
> The partitions get updated , but still the data remains the
> same(UNPARTITIONED), in the same partition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CARBONDATA-1427) After Splitting Partition, Data doesn't get Divided to Different Partitions.
[
https://issues.apache.org/jira/browse/CARBONDATA-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153261#comment-16153261
]
Cao, Lionel commented on CARBONDATA-1427:
-
Hi [~pallavisingh_09],
Thanks for your information.
Yes, I've located the root cause. May I re-assign this ticket to myself? :)
> After Splitting Partition, Data doesn't get Divided to Different Partitions.
>
>
> Key: CARBONDATA-1427
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1427
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.2.0
> Environment: spark 2.1
>Reporter: Neha Bhardwaj
>Assignee: Pallavi Singh
>Priority: Minor
> Attachments: list_partition_table.csv, screenshot-1.png
>
>
> When Performing a Split Partition Query on a Partitioned Table, The data
> doesn't get affected at all, however, we can see the updated Partitions using
> the show Partitions Query and the old partition as deleted.
> But the data still remains in that partition, Ideally, the data should be
> divided as per the new partitions, Which happens after the subsequent loads,
> the data then gets to the latest partitions.
> Example :
> 1. Create Table :
> DROP TABLE IF EXISTS list_partition_table;
> CREATE TABLE list_partition_table(shortField SHORT, intField INT, bigintField
> LONG, doubleField DOUBLE, timestampField TIMESTAMP, decimalField
> DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT,
> complexData ARRAY ) PARTITIONED BY (stringField STRING) STORED BY
> 'carbondata' TBLPROPERTIES('PARTITION_TYPE'='LIST', 'LIST_INFO'='Asia,
> (China, Europe, NoPartition)');
> 2. Load Data :
> load data inpath 'hdfs://localhost:54310/CSV/list_partition_table.csv' into
> table list_partition_table
> options('FILEHEADER'='shortfield,intfield,bigintfield,doublefield,stringfield,timestampfield,decimalfield,datefield,charfield,floatfield,complexdata',
> 'COMPLEX_DELIMITER_LEVEL_1'='$','COMPLEX_DELIMITER_LEVEL_2'='#');
> 3. Show Partitions :
> show partitions list_partition_table;
> +--+--+
> | partition |
> +--+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia|
> | 2, stringfield = China, Europe, NoPartition |
> +--+--+
> 3 rows selected (0.09 seconds)
> 4. Split Partition :
> ALTER TABLE list_partition_table SPLIT PARTITION(2) INTO('China', '(Europe,
> NoPartition)' );
> 5. Show Partition :
> show partitions list_partition_table;
> +---+--+
> | partition |
> +---+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia |
> | 3, stringfield = China|
> | 4, stringfield = Europe, NoPartition |
> +---+--+
> 4 rows selected (0.065 seconds)
> The partitions get updated , but still the data remains the
> same(UNPARTITIONED), in the same partition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (CARBONDATA-1427) After Splitting Partition, Data doesn't get Divided to Different Partitions.
[
https://issues.apache.org/jira/browse/CARBONDATA-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cao, Lionel updated CARBONDATA-1427:
Attachment: screenshot-1.png
> After Splitting Partition, Data doesn't get Divided to Different Partitions.
>
>
> Key: CARBONDATA-1427
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1427
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.2.0
> Environment: spark 2.1
>Reporter: Neha Bhardwaj
>Assignee: Pallavi Singh
>Priority: Minor
> Attachments: list_partition_table.csv, screenshot-1.png
>
>
> When Performing a Split Partition Query on a Partitioned Table, The data
> doesn't get affected at all, however, we can see the updated Partitions using
> the show Partitions Query and the old partition as deleted.
> But the data still remains in that partition, Ideally, the data should be
> divided as per the new partitions, Which happens after the subsequent loads,
> the data then gets to the latest partitions.
> Example :
> 1. Create Table :
> DROP TABLE IF EXISTS list_partition_table;
> CREATE TABLE list_partition_table(shortField SHORT, intField INT, bigintField
> LONG, doubleField DOUBLE, timestampField TIMESTAMP, decimalField
> DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT,
> complexData ARRAY ) PARTITIONED BY (stringField STRING) STORED BY
> 'carbondata' TBLPROPERTIES('PARTITION_TYPE'='LIST', 'LIST_INFO'='Asia,
> (China, Europe, NoPartition)');
> 2. Load Data :
> load data inpath 'hdfs://localhost:54310/CSV/list_partition_table.csv' into
> table list_partition_table
> options('FILEHEADER'='shortfield,intfield,bigintfield,doublefield,stringfield,timestampfield,decimalfield,datefield,charfield,floatfield,complexdata',
> 'COMPLEX_DELIMITER_LEVEL_1'='$','COMPLEX_DELIMITER_LEVEL_2'='#');
> 3. Show Partitions :
> show partitions list_partition_table;
> +--+--+
> | partition |
> +--+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia|
> | 2, stringfield = China, Europe, NoPartition |
> +--+--+
> 3 rows selected (0.09 seconds)
> 4. Split Partition :
> ALTER TABLE list_partition_table SPLIT PARTITION(2) INTO('China', '(Europe,
> NoPartition)' );
> 5. Show Partition :
> show partitions list_partition_table;
> +---+--+
> | partition |
> +---+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia |
> | 3, stringfield = China|
> | 4, stringfield = Europe, NoPartition |
> +---+--+
> 4 rows selected (0.065 seconds)
> The partitions get updated , but still the data remains the
> same(UNPARTITIONED), in the same partition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CARBONDATA-1427) After Splitting Partition, Data doesn't get Divided to Different Partitions.
[
https://issues.apache.org/jira/browse/CARBONDATA-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153209#comment-16153209
]
Cao, Lionel commented on CARBONDATA-1427:
-
Was the alter table split statement successful executed or any exceptions?
> After Splitting Partition, Data doesn't get Divided to Different Partitions.
>
>
> Key: CARBONDATA-1427
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1427
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.2.0
> Environment: spark 2.1
>Reporter: Neha Bhardwaj
>Assignee: Pallavi Singh
>Priority: Minor
> Attachments: list_partition_table.csv
>
>
> When Performing a Split Partition Query on a Partitioned Table, The data
> doesn't get affected at all, however, we can see the updated Partitions using
> the show Partitions Query and the old partition as deleted.
> But the data still remains in that partition, Ideally, the data should be
> divided as per the new partitions, Which happens after the subsequent loads,
> the data then gets to the latest partitions.
> Example :
> 1. Create Table :
> DROP TABLE IF EXISTS list_partition_table;
> CREATE TABLE list_partition_table(shortField SHORT, intField INT, bigintField
> LONG, doubleField DOUBLE, timestampField TIMESTAMP, decimalField
> DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT,
> complexData ARRAY ) PARTITIONED BY (stringField STRING) STORED BY
> 'carbondata' TBLPROPERTIES('PARTITION_TYPE'='LIST', 'LIST_INFO'='Asia,
> (China, Europe, NoPartition)');
> 2. Load Data :
> load data inpath 'hdfs://localhost:54310/CSV/list_partition_table.csv' into
> table list_partition_table
> options('FILEHEADER'='shortfield,intfield,bigintfield,doublefield,stringfield,timestampfield,decimalfield,datefield,charfield,floatfield,complexdata',
> 'COMPLEX_DELIMITER_LEVEL_1'='$','COMPLEX_DELIMITER_LEVEL_2'='#');
> 3. Show Partitions :
> show partitions list_partition_table;
> +--+--+
> | partition |
> +--+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia|
> | 2, stringfield = China, Europe, NoPartition |
> +--+--+
> 3 rows selected (0.09 seconds)
> 4. Split Partition :
> ALTER TABLE list_partition_table SPLIT PARTITION(2) INTO('China', '(Europe,
> NoPartition)' );
> 5. Show Partition :
> show partitions list_partition_table;
> +---+--+
> | partition |
> +---+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia |
> | 3, stringfield = China|
> | 4, stringfield = Europe, NoPartition |
> +---+--+
> 4 rows selected (0.065 seconds)
> The partitions get updated , but still the data remains the
> same(UNPARTITIONED), in the same partition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CARBONDATA-1448) PartitionInfo is null in CarbonTable
Cao, Lionel created CARBONDATA-1448: --- Summary: PartitionInfo is null in CarbonTable Key: CARBONDATA-1448 URL: https://issues.apache.org/jira/browse/CARBONDATA-1448 Project: CarbonData Issue Type: Bug Components: core Reporter: Cao, Lionel Assignee: Cao, Lionel PartitionInfo is null in CarbonTable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Issue Comment Deleted] (CARBONDATA-1427) After Splitting Partition, Data doesn't get Divided to Different Partitions.
[
https://issues.apache.org/jira/browse/CARBONDATA-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cao, Lionel updated CARBONDATA-1427:
Comment: was deleted
(was: Hi Neha,
Could you attach the data file you used so that I can try to reproduce the
issue?
Thanks,
Lionel)
> After Splitting Partition, Data doesn't get Divided to Different Partitions.
>
>
> Key: CARBONDATA-1427
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1427
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
> Environment: spark 2.1
>Reporter: Neha Bhardwaj
>Priority: Minor
>
> When Performing a Split Partition Query on a Partitioned Table, The data
> doesn't get affected at all, however, we can see the updated Partitions using
> the show Partitions Query and the old partition as deleted.
> But the data still remains in that partition, Ideally, the data should be
> divided as per the new partitions, Which happens after the subsequent loads,
> the data then gets to the latest partitions.
> Example :
> 1. Create Table :
> DROP TABLE IF EXISTS list_partition_table;
> CREATE TABLE list_partition_table(shortField SHORT, intField INT, bigintField
> LONG, doubleField DOUBLE, timestampField TIMESTAMP, decimalField
> DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT,
> complexData ARRAY ) PARTITIONED BY (stringField STRING) STORED BY
> 'carbondata' TBLPROPERTIES('PARTITION_TYPE'='LIST', 'LIST_INFO'='Asia,
> (China, Europe, NoPartition)');
> 2. Load Data :
> load data inpath 'hdfs://localhost:54310/CSV/list_partition_table.csv' into
> table list_partition_table
> options('FILEHEADER'='shortfield,intfield,bigintfield,doublefield,stringfield,timestampfield,decimalfield,datefield,charfield,floatfield,complexdata',
> 'COMPLEX_DELIMITER_LEVEL_1'='$','COMPLEX_DELIMITER_LEVEL_2'='#');
> 3. Show Partitions :
> show partitions list_partition_table;
> +--+--+
> | partition |
> +--+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia|
> | 2, stringfield = China, Europe, NoPartition |
> +--+--+
> 3 rows selected (0.09 seconds)
> 4. Split Partition :
> ALTER TABLE list_partition_table SPLIT PARTITION(2) INTO('China', '(Europe,
> NoPartition)' );
> 5. Show Partition :
> show partitions list_partition_table;
> +---+--+
> | partition |
> +---+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia |
> | 3, stringfield = China|
> | 4, stringfield = Europe, NoPartition |
> +---+--+
> 4 rows selected (0.065 seconds)
> The partitions get updated , but still the data remains the
> same(UNPARTITIONED), in the same partition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CARBONDATA-1427) After Splitting Partition, Data doesn't get Divided to Different Partitions.
[
https://issues.apache.org/jira/browse/CARBONDATA-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153044#comment-16153044
]
Cao, Lionel commented on CARBONDATA-1427:
-
Hi Neha,
Could you attach the data file you used so that I can try to reproduce the
issue?
Thanks,
Lionel
> After Splitting Partition, Data doesn't get Divided to Different Partitions.
>
>
> Key: CARBONDATA-1427
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1427
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
> Environment: spark 2.1
>Reporter: Neha Bhardwaj
>Priority: Minor
>
> When Performing a Split Partition Query on a Partitioned Table, The data
> doesn't get affected at all, however, we can see the updated Partitions using
> the show Partitions Query and the old partition as deleted.
> But the data still remains in that partition, Ideally, the data should be
> divided as per the new partitions, Which happens after the subsequent loads,
> the data then gets to the latest partitions.
> Example :
> 1. Create Table :
> DROP TABLE IF EXISTS list_partition_table;
> CREATE TABLE list_partition_table(shortField SHORT, intField INT, bigintField
> LONG, doubleField DOUBLE, timestampField TIMESTAMP, decimalField
> DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT,
> complexData ARRAY ) PARTITIONED BY (stringField STRING) STORED BY
> 'carbondata' TBLPROPERTIES('PARTITION_TYPE'='LIST', 'LIST_INFO'='Asia,
> (China, Europe, NoPartition)');
> 2. Load Data :
> load data inpath 'hdfs://localhost:54310/CSV/list_partition_table.csv' into
> table list_partition_table
> options('FILEHEADER'='shortfield,intfield,bigintfield,doublefield,stringfield,timestampfield,decimalfield,datefield,charfield,floatfield,complexdata',
> 'COMPLEX_DELIMITER_LEVEL_1'='$','COMPLEX_DELIMITER_LEVEL_2'='#');
> 3. Show Partitions :
> show partitions list_partition_table;
> +--+--+
> | partition |
> +--+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia|
> | 2, stringfield = China, Europe, NoPartition |
> +--+--+
> 3 rows selected (0.09 seconds)
> 4. Split Partition :
> ALTER TABLE list_partition_table SPLIT PARTITION(2) INTO('China', '(Europe,
> NoPartition)' );
> 5. Show Partition :
> show partitions list_partition_table;
> +---+--+
> | partition |
> +---+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia |
> | 3, stringfield = China|
> | 4, stringfield = Europe, NoPartition |
> +---+--+
> 4 rows selected (0.065 seconds)
> The partitions get updated , but still the data remains the
> same(UNPARTITIONED), in the same partition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CARBONDATA-1427) After Splitting Partition, Data doesn't get Divided to Different Partitions.
[
https://issues.apache.org/jira/browse/CARBONDATA-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153038#comment-16153038
]
Cao, Lionel commented on CARBONDATA-1427:
-
Hi Neha,
Could you attach the data file you used so that I can try to reproduce the
issue?
Thanks,
Lionel
> After Splitting Partition, Data doesn't get Divided to Different Partitions.
>
>
> Key: CARBONDATA-1427
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1427
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
> Environment: spark 2.1
>Reporter: Neha Bhardwaj
>Priority: Minor
>
> When Performing a Split Partition Query on a Partitioned Table, The data
> doesn't get affected at all, however, we can see the updated Partitions using
> the show Partitions Query and the old partition as deleted.
> But the data still remains in that partition, Ideally, the data should be
> divided as per the new partitions, Which happens after the subsequent loads,
> the data then gets to the latest partitions.
> Example :
> 1. Create Table :
> DROP TABLE IF EXISTS list_partition_table;
> CREATE TABLE list_partition_table(shortField SHORT, intField INT, bigintField
> LONG, doubleField DOUBLE, timestampField TIMESTAMP, decimalField
> DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT,
> complexData ARRAY ) PARTITIONED BY (stringField STRING) STORED BY
> 'carbondata' TBLPROPERTIES('PARTITION_TYPE'='LIST', 'LIST_INFO'='Asia,
> (China, Europe, NoPartition)');
> 2. Load Data :
> load data inpath 'hdfs://localhost:54310/CSV/list_partition_table.csv' into
> table list_partition_table
> options('FILEHEADER'='shortfield,intfield,bigintfield,doublefield,stringfield,timestampfield,decimalfield,datefield,charfield,floatfield,complexdata',
> 'COMPLEX_DELIMITER_LEVEL_1'='$','COMPLEX_DELIMITER_LEVEL_2'='#');
> 3. Show Partitions :
> show partitions list_partition_table;
> +--+--+
> | partition |
> +--+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia|
> | 2, stringfield = China, Europe, NoPartition |
> +--+--+
> 3 rows selected (0.09 seconds)
> 4. Split Partition :
> ALTER TABLE list_partition_table SPLIT PARTITION(2) INTO('China', '(Europe,
> NoPartition)' );
> 5. Show Partition :
> show partitions list_partition_table;
> +---+--+
> | partition |
> +---+--+
> | 0, stringfield = DEFAULT |
> | 1, stringfield = Asia |
> | 3, stringfield = China|
> | 4, stringfield = Europe, NoPartition |
> +---+--+
> 4 rows selected (0.065 seconds)
> The partitions get updated , but still the data remains the
> same(UNPARTITIONED), in the same partition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (CARBONDATA-1401) RangeInfo & List Info validate Issue
[ https://issues.apache.org/jira/browse/CARBONDATA-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel updated CARBONDATA-1401: Description: fix duplicate issue in list info(was: fix line break issue in range info & duplicate issue in list info ) > RangeInfo & List Info validate Issue > > > Key: CARBONDATA-1401 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1401 > Project: CarbonData > Issue Type: Bug > Components: spark-integration, sql >Reporter: Cao, Lionel >Assignee: Cao, Lionel > > fix duplicate issue in list info -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (CARBONDATA-1401) List Info validate Issue
[ https://issues.apache.org/jira/browse/CARBONDATA-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel updated CARBONDATA-1401: Summary: List Info validate Issue (was: RangeInfo & List Info validate Issue) > List Info validate Issue > > > Key: CARBONDATA-1401 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1401 > Project: CarbonData > Issue Type: Bug > Components: spark-integration, sql >Reporter: Cao, Lionel >Assignee: Cao, Lionel > > fix duplicate issue in list info -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1401) RangeInfo & List Info validate Issue
Cao, Lionel created CARBONDATA-1401: --- Summary: RangeInfo & List Info validate Issue Key: CARBONDATA-1401 URL: https://issues.apache.org/jira/browse/CARBONDATA-1401 Project: CarbonData Issue Type: Bug Components: spark-integration, sql Reporter: Cao, Lionel Assignee: Cao, Lionel fix line break issue in range info & duplicate issue in list info -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CARBONDATA-1369) timestamp type column in where clause cause empty result
[ https://issues.apache.org/jira/browse/CARBONDATA-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128363#comment-16128363 ] Cao, Lionel commented on CARBONDATA-1369: - Fixed in CarbonData-1379, cast function issue. > timestamp type column in where clause cause empty result > > > Key: CARBONDATA-1369 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1369 > Project: CarbonData > Issue Type: Bug > Components: sql >Reporter: Cao, Lionel >Assignee: Cao, Lionel > > if where clause contains column which is timestamp type, it will return empty > result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (CARBONDATA-1369) timestamp type column in where clause cause empty result
[ https://issues.apache.org/jira/browse/CARBONDATA-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel closed CARBONDATA-1369. --- Resolution: Fixed > timestamp type column in where clause cause empty result > > > Key: CARBONDATA-1369 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1369 > Project: CarbonData > Issue Type: Bug > Components: sql >Reporter: Cao, Lionel >Assignee: Cao, Lionel > > if where clause contains column which is timestamp type, it will return empty > result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1369) timestamp type column in where clause cause empty result
Cao, Lionel created CARBONDATA-1369: --- Summary: timestamp type column in where clause cause empty result Key: CARBONDATA-1369 URL: https://issues.apache.org/jira/browse/CARBONDATA-1369 Project: CarbonData Issue Type: Bug Components: sql Reporter: Cao, Lionel Assignee: Cao, Lionel if where clause contains column which is timestamp type, it will return empty result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1357) byte[] convert to UTF8String bug
Cao, Lionel created CARBONDATA-1357:
---
Summary: byte[] convert to UTF8String bug
Key: CARBONDATA-1357
URL: https://issues.apache.org/jira/browse/CARBONDATA-1357
Project: CarbonData
Issue Type: Bug
Components: core
Reporter: Cao, Lionel
Assignee: Cao, Lionel
public Object convertFromByteToUTF8String(Object data) {
return data.toString();
}
toString will get incorrect result like B[ should use new String
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CARBONDATA-1344) remove useless variables
Cao, Lionel created CARBONDATA-1344: --- Summary: remove useless variables Key: CARBONDATA-1344 URL: https://issues.apache.org/jira/browse/CARBONDATA-1344 Project: CarbonData Issue Type: Task Reporter: Cao, Lionel Assignee: Cao, Lionel remove aggTables -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1325) 16. create guidance documents for partition table
Cao, Lionel created CARBONDATA-1325: --- Summary: 16. create guidance documents for partition table Key: CARBONDATA-1325 URL: https://issues.apache.org/jira/browse/CARBONDATA-1325 Project: CarbonData Issue Type: Sub-task Components: docs Reporter: Cao, Lionel Assignee: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (CARBONDATA-1316) 15. alter table drop partition
[ https://issues.apache.org/jira/browse/CARBONDATA-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel updated CARBONDATA-1316: Summary: 15. alter table drop partition (was: 15. alter table drop/merge partition) > 15. alter table drop partition > -- > > Key: CARBONDATA-1316 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1316 > Project: CarbonData > Issue Type: Sub-task > Components: core, spark-integration, sql >Reporter: Cao, Lionel >Assignee: Cao, Lionel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1316) 6.1 alter table drop/merge partition
Cao, Lionel created CARBONDATA-1316: --- Summary: 6.1 alter table drop/merge partition Key: CARBONDATA-1316 URL: https://issues.apache.org/jira/browse/CARBONDATA-1316 Project: CarbonData Issue Type: Sub-task Components: core, spark-integration, sql Reporter: Cao, Lionel Assignee: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (CARBONDATA-1316) 15. alter table drop/merge partition
[ https://issues.apache.org/jira/browse/CARBONDATA-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel updated CARBONDATA-1316: Summary: 15. alter table drop/merge partition (was: 6.1 alter table drop/merge partition) > 15. alter table drop/merge partition > > > Key: CARBONDATA-1316 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1316 > Project: CarbonData > Issue Type: Sub-task > Components: core, spark-integration, sql >Reporter: Cao, Lionel >Assignee: Cao, Lionel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (CARBONDATA-940) 6. Alter table add/split partition
[ https://issues.apache.org/jira/browse/CARBONDATA-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel updated CARBONDATA-940: --- Summary: 6. Alter table add/split partition (was: 6. Alter table add/drop partition) > 6. Alter table add/split partition > -- > > Key: CARBONDATA-940 > URL: https://issues.apache.org/jira/browse/CARBONDATA-940 > Project: CarbonData > Issue Type: Sub-task > Components: core, data-load, data-query >Reporter: QiangCai >Assignee: Cao, Lionel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (CARBONDATA-1312) 14. Fix comparator bug
[ https://issues.apache.org/jira/browse/CARBONDATA-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel updated CARBONDATA-1312: Summary: 14. Fix comparator bug (was: Fix comparator bug) > 14. Fix comparator bug > -- > > Key: CARBONDATA-1312 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1312 > Project: CarbonData > Issue Type: Sub-task > Components: core >Reporter: Cao, Lionel >Assignee: Cao, Lionel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1312) Fix comparator bug
Cao, Lionel created CARBONDATA-1312: --- Summary: Fix comparator bug Key: CARBONDATA-1312 URL: https://issues.apache.org/jira/browse/CARBONDATA-1312 Project: CarbonData Issue Type: Sub-task Components: core Reporter: Cao, Lionel Assignee: Cao, Lionel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (CARBONDATA-1209) 12. Add partitionId in show partition result
[ https://issues.apache.org/jira/browse/CARBONDATA-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel reassigned CARBONDATA-1209: --- Assignee: Cao, Lionel Component/s: (was: data-load) (was: data-query) (was: core) spark-integration examples Summary: 12. Add partitionId in show partition result (was: 12. ) > 12. Add partitionId in show partition result > > > Key: CARBONDATA-1209 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1209 > Project: CarbonData > Issue Type: Sub-task > Components: examples, spark-integration >Reporter: QiangCai >Assignee: Cao, Lionel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CARBONDATA-1250) 13. Change default partition id from Max to 0
[ https://issues.apache.org/jira/browse/CARBONDATA-1250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081759#comment-16081759 ] Cao, Lionel commented on CARBONDATA-1250: - This change will be used in later alter table partition feature. > 13. Change default partition id from Max to 0 > - > > Key: CARBONDATA-1250 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1250 > Project: CarbonData > Issue Type: Sub-task > Components: data-load, sql >Reporter: Cao, Lionel >Assignee: Cao, Lionel > > Change default partition id from Max to 0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (CARBONDATA-1250) 13. Change default partition id from Max to 0
[ https://issues.apache.org/jira/browse/CARBONDATA-1250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel updated CARBONDATA-1250: Summary: 13. Change default partition id from Max to 0 (was: Change default partition id from Max to 0) > 13. Change default partition id from Max to 0 > - > > Key: CARBONDATA-1250 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1250 > Project: CarbonData > Issue Type: Sub-task > Components: data-load, sql >Reporter: Cao, Lionel >Assignee: Cao, Lionel > > Change default partition id from Max to 0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (CARBONDATA-1250) Change default partition id from Max to 0
[ https://issues.apache.org/jira/browse/CARBONDATA-1250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel updated CARBONDATA-1250: Issue Type: Sub-task (was: Improvement) Parent: CARBONDATA-910 > Change default partition id from Max to 0 > - > > Key: CARBONDATA-1250 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1250 > Project: CarbonData > Issue Type: Sub-task > Components: data-load, sql >Reporter: Cao, Lionel >Assignee: Cao, Lionel > > Change default partition id from Max to 0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1250) Change default partition id from Max to 0
Cao, Lionel created CARBONDATA-1250: --- Summary: Change default partition id from Max to 0 Key: CARBONDATA-1250 URL: https://issues.apache.org/jira/browse/CARBONDATA-1250 Project: CarbonData Issue Type: Improvement Components: data-load, sql Reporter: Cao, Lionel Assignee: Cao, Lionel Change default partition id from Max to 0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (CARBONDATA-936) 2. Create Table with Partition
[ https://issues.apache.org/jira/browse/CARBONDATA-936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao, Lionel reassigned CARBONDATA-936: -- Assignee: Cao, Lionel > 2. Create Table with Partition > -- > > Key: CARBONDATA-936 > URL: https://issues.apache.org/jira/browse/CARBONDATA-936 > Project: CarbonData > Issue Type: Sub-task > Components: core, data-load, data-query > Environment: CarbonSparkSqlParser parse partition part to generate > PartitionInfo, add PartitionInfo to TableModel. > CreateTable add PartitionInfo to TableInfo, store PartitionInfo in > TableSchema > support spark 2.1 at first. >Reporter: QiangCai >Assignee: Cao, Lionel > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
