[GitHub] carbondata pull request #1050: [CARBONDATA -1186]resolved bug for class cast...
Github user anubhav100 closed the pull request at: https://github.com/apache/carbondata/pull/1050 ---
[GitHub] carbondata issue #1050: [CARBONDATA -1186]resolved bug for class cast except...
Github user anubhav100 commented on the issue: https://github.com/apache/carbondata/pull/1050 @sraghunandan i am unable to reproduce this bug closing the ticket ---
[GitHub] carbondata issue #1419: [CARBONDATA-1601] Add carbondata-store module
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1419 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1425/ ---
[GitHub] carbondata issue #1419: [CARBONDATA-1601] Add carbondata-store module
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1419 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/792/ ---
[GitHub] carbondata issue #1460: [Docs] Fix partition-guide.md docs NUM_PARTITIONS wr...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1460 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1460: [Docs] Fix partition-guide.md docs NUM_PARTITIONS wr...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1460 Can one of the admins verify this patch? ---
[GitHub] carbondata pull request #1460: [Docs] Fix partition-guide.md docs NUM_PARTIT...
GitHub user LiShuMing opened a pull request: https://github.com/apache/carbondata/pull/1460 [Docs] Fix partition-guide.md docs NUM_PARTITIONS wrong keyword It's an obvious keyword spelling mistake, `PARTITION_NUM` should be `NUM_PARTITION `. No Tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/LiShuMing/carbondata fix-docs-partition-guide Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1460.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1460 commit 517a40e0707709ebf4b9812e4208717222ef2e06 Author: lishuming Date: 2017-11-02T03:55:02Z Fix partition-guide.md docs NUM_PARTITIONS wrong keyword ---
[GitHub] carbondata issue #1419: [CARBONDATA-1601] Add carbondata-store module
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/1419 retest this please ---
[GitHub] carbondata issue #1455: [CARBONDATA-1624]Set the default value of 'carbon.nu...
Github user zzcclp commented on the issue: https://github.com/apache/carbondata/pull/1455 @ravipesala @jackylk why it used 2 as the default value of 'carbon.number.of.cores.while.loading' before? ---
[GitHub] carbondata pull request #1452: [CARBONDATA-1593] Add partition to table caus...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/1452 ---
[GitHub] carbondata issue #1417: [CARBONDATA-1594] Add precision and scale to Decimal...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1417 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1424/ ---
[GitHub] carbondata issue #1104: [CARBONDATA-1239] Add validation for set command par...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/1104 please rebase ---
[jira] [Resolved] (CARBONDATA-1617) Merging carbonindex files for each segment.
[ https://issues.apache.org/jira/browse/CARBONDATA-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacky Li resolved CARBONDATA-1617. -- Resolution: Fixed Fix Version/s: 1.3.0 > Merging carbonindex files for each segment. > --- > > Key: CARBONDATA-1617 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1617 > Project: CarbonData > Issue Type: New Feature >Reporter: Ravindra Pesala >Priority: Major > Fix For: 1.3.0 > > Time Spent: 10h 20m > Remaining Estimate: 0h > > Hi, > Problem : > The first-time query of carbon becomes very slow. It is because of reading > many small carbonindex files and cache to the driver at the first time. > Many carbonindex files are created in below case > Loading data in large cluster >For example, if the cluster size is 100 nodes then for each load 100 index > files are created per segment. So after 100 loads, the number of carbonindex > files becomes 1. . > It will be slower to read all the files from the driver since a lot of > namenode calls and IO operations. > Solution : > Merge the carbonindex files in two levels.so that we can reduce the IO calls > to namenode and improves the read performance. > Merge within a segment. > Merge the carbonindex files to single file immediately after load completes > within the segment. It would be named as a .carbonindexmerge file. It is > actually not a true data merging but a simple file merge. So that the current > structure of carbonindex files does not change. While reading we just read > one file instead of many carbonindex files within the segment. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] carbondata pull request #1436: [CARBONDATA-1617] Merging carbonindex files w...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/1436 ---
[GitHub] carbondata issue #1417: [CARBONDATA-1594] Add precision and scale to Decimal...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1417 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/791/ ---
[GitHub] carbondata issue #1436: [CARBONDATA-1617] Merging carbonindex files within s...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/1436 LGTM ---
[GitHub] carbondata issue #1417: [CARBONDATA-1594] Add precision and scale to Decimal...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/1417 retest this please ---
[GitHub] carbondata issue #1437: [CARBONDATA-1618] Fix issue of not support table com...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1437 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1423/ ---
[GitHub] carbondata issue #1452: [CARBONDATA-1593] Add partition to table cause NoSuc...
Github user sounakr commented on the issue: https://github.com/apache/carbondata/pull/1452 LGTM ---
[GitHub] carbondata issue #1437: [CARBONDATA-1618] Fix issue of not support table com...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1437 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/790/ ---
[GitHub] carbondata issue #1437: [CARBONDATA-1618] Fix issue of not support table com...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1437 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1422/ ---
[GitHub] carbondata issue #1437: [CARBONDATA-1618] Fix issue of not support table com...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1437 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/789/ ---
[GitHub] carbondata issue #1437: [CARBONDATA-1618] Fix issue of not support table com...
Github user chenerlu commented on the issue: https://github.com/apache/carbondata/pull/1437 retest this please ---
[GitHub] carbondata issue #1459: [CARBONDATA-1661] Fixed bug related to display of ti...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1459 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1421/ ---
[GitHub] carbondata issue #1164: [CARBONDATA-1303] Update CarbonContext.scala
Github user sraghunandan commented on the issue: https://github.com/apache/carbondata/pull/1164 can you please add test case to reproduce this scenario ---
[GitHub] carbondata issue #1459: [CARBONDATA-1661] Fixed bug related to display of ti...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1459 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/788/ ---
[GitHub] carbondata issue #1452: [CARBONDATA-1593] Add partition to table cause NoSuc...
Github user lionelcao commented on the issue: https://github.com/apache/carbondata/pull/1452 LGTM ---
[jira] [Commented] (CARBONDATA-1657) Partition column is empty when insert from a hive table
[
https://issues.apache.org/jira/browse/CARBONDATA-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16233951#comment-16233951
]
cen yuhai commented on CARBONDATA-1657:
---
dt's datatype is string
> Partition column is empty when insert from a hive table
> ---
>
> Key: CARBONDATA-1657
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1657
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
>Affects Versions: 1.2.0
> Environment: carbonata1.2.0 spark 2.1.1
>Reporter: cen yuhai
>Priority: Critical
>
> I create table a carbon table, the schema is like a hive table(dt is the
> partition column).
> And then
> {code}
> insert overwrite table dm_test.dm_trd_wide_carbondata select * from
> hive_table where dt='2017-10-10';
> insert overwrite table dm_test.dm_trd_wide_parquet select * from hive_table
> where dt='2017-10-10';
> {code}
> {code}
> spark-sql> select dt from dm_test.dm_trd_wide_parquet limit 10;
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> Time taken: 1.259 seconds, Fetched 10 row(s)
> spark-sql> select dt from dm_test.dm_trd_wide_carbondata limit 10;
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (CARBONDATA-1657) Partition column is empty when insert from a hive table
[
https://issues.apache.org/jira/browse/CARBONDATA-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16233951#comment-16233951
]
cen yuhai edited comment on CARBONDATA-1657 at 11/1/17 11:36 AM:
-
the datatype of dt is string
was (Author: cenyuhai):
dt's datatype is string
> Partition column is empty when insert from a hive table
> ---
>
> Key: CARBONDATA-1657
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1657
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
>Affects Versions: 1.2.0
> Environment: carbonata1.2.0 spark 2.1.1
>Reporter: cen yuhai
>Priority: Critical
>
> I create table a carbon table, the schema is like a hive table(dt is the
> partition column).
> And then
> {code}
> insert overwrite table dm_test.dm_trd_wide_carbondata select * from
> hive_table where dt='2017-10-10';
> insert overwrite table dm_test.dm_trd_wide_parquet select * from hive_table
> where dt='2017-10-10';
> {code}
> {code}
> spark-sql> select dt from dm_test.dm_trd_wide_parquet limit 10;
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> Time taken: 1.259 seconds, Fetched 10 row(s)
> spark-sql> select dt from dm_test.dm_trd_wide_carbondata limit 10;
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CARBONDATA-1657) Partition column is empty when insert from a hive table
[
https://issues.apache.org/jira/browse/CARBONDATA-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16233948#comment-16233948
]
Babulal commented on CARBONDATA-1657:
-
Hi
if possible please provide create statement for hive_table and
dm_trd_wide_carbondata .
also please check whether date format in carbon.properties is in -MM-dd
format .
Thanks
Babu
> Partition column is empty when insert from a hive table
> ---
>
> Key: CARBONDATA-1657
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1657
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
>Affects Versions: 1.2.0
> Environment: carbonata1.2.0 spark 2.1.1
>Reporter: cen yuhai
>Priority: Critical
>
> I create table a carbon table, the schema is like a hive table(dt is the
> partition column).
> And then
> {code}
> insert overwrite table dm_test.dm_trd_wide_carbondata select * from
> hive_table where dt='2017-10-10';
> insert overwrite table dm_test.dm_trd_wide_parquet select * from hive_table
> where dt='2017-10-10';
> {code}
> {code}
> spark-sql> select dt from dm_test.dm_trd_wide_parquet limit 10;
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> 2017-10-10
> Time taken: 1.259 seconds, Fetched 10 row(s)
> spark-sql> select dt from dm_test.dm_trd_wide_carbondata limit 10;
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> NULL
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] carbondata issue #1050: [CARBONDATA -1186]resolved bug for class cast except...
Github user sraghunandan commented on the issue: https://github.com/apache/carbondata/pull/1050 can you please add test case to reproduce this scenario.Also pls rebase ---
[GitHub] carbondata pull request #1459: [CARBONDATA-1661] Fixed bug related to displa...
GitHub user geetikagupta16 opened a pull request: https://github.com/apache/carbondata/pull/1459 [CARBONDATA-1661] Fixed bug related to display of timestamp values in presto You can merge this pull request into a Git repository by running: $ git pull https://github.com/geetikagupta16/incubator-carbondata CARBONDATA-1661 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1459.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1459 commit fccfdc6be7be70467c31297ef318581d575c0a87 Author: Geetika Gupta Date: 2017-11-01T11:12:20Z Fixed bug related to display of timestamp values in presto ---
[jira] [Assigned] (CARBONDATA-1661) Incorrect output of select query with timestamp data type on presto CLI
[
https://issues.apache.org/jira/browse/CARBONDATA-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
anubhav tarar reassigned CARBONDATA-1661:
-
Assignee: anubhav tarar
> Incorrect output of select query with timestamp data type on presto CLI
> ---
>
> Key: CARBONDATA-1661
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1661
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.3.0
> Environment: spark 2.1
>Reporter: Vandana Yadav
>Assignee: anubhav tarar
>Priority: Minor
> Attachments: 2000_UniqData.csv
>
>
> Incorrect output of select query with timestamp data type on presto CLI
> Steps to Reproduce:
> On Beeline:
> 1) Create Table:
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB")
> 2)Load Data:
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')
> 3) Start presto server:
> bin/launcher run
> 4) run presto CLI:
> ./presto --server localhost:9000 --catalog carbondata --schema newpresto
> On presto CLI
> 1) Execute select Query:
> select cust_name from uniqdata where dob= cast('1970-01-11 01:00:03.000' as
> timestamp);
> 2)Expected Result: it should display correct output as on beeline:
> +--+--+
> |cust_name |
> +--+--+
> | CUST_NAME_00010 |
> +--+--+
> 3) Actual Result:
> cust_name
> ---
> (0 rows)
> Query 20171031_084306_00030_k9q68, FINISHED, 1 node
> Splits: 17 total, 17 done (100.00%)
> 0:00 [0 rows, 0B] [0 rows/s, 0B/s]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (CARBONDATA-1660) Incorrect result displays while executing select query with where clause for decimal data type
[
https://issues.apache.org/jira/browse/CARBONDATA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
anubhav tarar reassigned CARBONDATA-1660:
-
Assignee: anubhav tarar
> Incorrect result displays while executing select query with where clause for
> decimal data type
> --
>
> Key: CARBONDATA-1660
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1660
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.3.0
> Environment: spark 2.1
>Reporter: Vandana Yadav
>Assignee: anubhav tarar
>Priority: Minor
> Attachments: 2000_UniqData.csv
>
>
> Incorrect result displays while executing select query with where clause for
> decimal data type
> Steps to reproduce:
> On Beeline:
> 1) Create Table:
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB")
> 2)Load Data:
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')
> 3) Start presto server:
> bin/launcher run
> 4) run presto CLI:
> ./presto --server localhost:9000 --catalog carbondata --schema newpresto
> On presto CLI
> 1) Execute select Query:
> select cust_name from uniqdata where decimal_column1=12345678902.123400;
> Expected Result: it should display the cust_name as on beeline
> +--+--+
> |cust_name |
> +--+--+
> | CUST_NAME_1 |
> +--+--+
> Actual Result:
> it throws an error saying error while setting filter expression to job.
> presto:newpresto> select cust_name from uniqdata where
> decimal_column1=12345678902.123400;
> Query 20171031_074909_00013_k9q68 failed: Error while setting filter
> expression to Job
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
