date:20180101

[GitHub] carbondata issue #1732: [CARBONDATA-1946] Exception thrown after alter data ...

2018-01-01 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1732
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/2648/



---

[GitHub] carbondata issue #1082: [CARBONDATA-1218] [GLOBAL SORT] In case of data-load...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1082
  
Build Success with Spark 2.2.0, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1246/



---

[GitHub] carbondata issue #1082: [CARBONDATA-1218] [GLOBAL SORT] In case of data-load...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1082
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2470/



---

[jira] [Created] (CARBONDATA-1963) Support S3 table with dictionary

2018-01-01 Thread Sangeeta Gulia (JIRA)

Sangeeta Gulia created CARBONDATA-1963:
--

 Summary: Support S3 table with dictionary
 Key: CARBONDATA-1963
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1963
 Project: CarbonData
  Issue Type: Task
Reporter: Sangeeta Gulia
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (CARBONDATA-1960) Add example for creating a local table and load CSV data which is stored in S3.

2018-01-01 Thread Jatin (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jatin reassigned CARBONDATA-1960:
-

Assignee: Jatin

> Add example for creating a local table and load CSV data which is stored in 
> S3.
> ---
>
> Key: CARBONDATA-1960
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1960
> Project: CarbonData
>  Issue Type: Task
>Reporter: Sangeeta Gulia
>Assignee: Jatin
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (CARBONDATA-1962) Support alter table add columns/drop columns on S3 table

2018-01-01 Thread Jatin (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jatin reassigned CARBONDATA-1962:
-

Assignee: Jatin

> Support alter table add columns/drop columns on S3 table
> 
>
> Key: CARBONDATA-1962
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1962
> Project: CarbonData
>  Issue Type: Task
>Reporter: Sangeeta Gulia
>Assignee: Jatin
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (CARBONDATA-1961) Support data update/delete on S3 table

2018-01-01 Thread Jatin (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jatin reassigned CARBONDATA-1961:
-

Assignee: Jatin

> Support data update/delete on S3 table
> --
>
> Key: CARBONDATA-1961
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1961
> Project: CarbonData
>  Issue Type: Task
>Reporter: Sangeeta Gulia
>Assignee: Jatin
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (CARBONDATA-1959) Support compaction on S3 table

2018-01-01 Thread Jatin (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jatin reassigned CARBONDATA-1959:
-

Assignee: Jatin

> Support compaction on S3 table
> --
>
> Key: CARBONDATA-1959
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1959
> Project: CarbonData
>  Issue Type: Task
>Reporter: Sangeeta Gulia
>Assignee: Jatin
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] carbondata issue #1718: [CARBONDATA-1929][Validation]carbon property configu...

2018-01-01 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1718
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/2647/



---

[GitHub] carbondata issue #1732: [CARBONDATA-1946] Exception thrown after alter data ...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1732
  
Build Success with Spark 2.2.0, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1245/



---

[GitHub] carbondata issue #1732: [CARBONDATA-1946] Exception thrown after alter data ...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1732
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2469/



---

[jira] [Updated] (CARBONDATA-1827) Add Support to provide S3 Functionality in Carbondata

2018-01-01 Thread Sangeeta Gulia (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sangeeta Gulia updated CARBONDATA-1827:
---
Issue Type: Task  (was: New Feature)

> Add Support to provide S3 Functionality in Carbondata
> -
>
> Key: CARBONDATA-1827
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1827
> Project: CarbonData
>  Issue Type: Task
>  Components: core
>Reporter: Sangeeta Gulia
>Assignee: Jatin
>Priority: Minor
>  Time Spent: 9h
>  Remaining Estimate: 0h
>
> Added Support to provide S3 Functionality in Carbondata. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (CARBONDATA-1958) CarbonSqlCliDriver not show result in case of select query

2018-01-01 Thread anubhav tarar (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

anubhav tarar reassigned CARBONDATA-1958:
-

Assignee: anubhav tarar

> CarbonSqlCliDriver not show result in case of select query
> --
>
> Key: CARBONDATA-1958
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1958
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-query
>Affects Versions: 1.3.0
> Environment: sparksqlclidriver,hadoop2.7
>Reporter: anubhav tarar
>Assignee: anubhav tarar
>
> 1.start the carbonsparksqlcli driver
> go to bin folder of carbon project and execute command
> ./carbon-spark-sql
> 2.create carbon data table
> spark-sql> create table sparktable(id int)stored by 'carbondata';
> 3.load data in carbontable
> spark-sql> insert into sparktable values(1);
> 4.query the data
> 18/01/02 12:06:45 INFO TableInfo: main Table block size not specified for 
> default_sparktable. Therefore considering the default value 1024 MB
> 18/01/02 12:06:45 INFO BlockletDataMap: main Time taken to load blocklet 
> datamap from file : 
> /home/anubhav/Documents/carbondata/carbondata/bin/carbonsqlclistore/default/sparktable/Fact/Part0/Segment_0/0_batchno0-0-1514874975013.carbonindexis
>  19
> 18/01/02 12:06:45 INFO CarbonScanRDD: 
>  Identified no.of.blocks: 0,
>  no.of.tasks: 0,
>  no.of.nodes: 0,
>  parallelism: 4
>
> 18/01/02 12:06:45 INFO SparkContext: Starting job: processCmd at 
> CliDriver.java:376
> 18/01/02 12:06:45 INFO DAGScheduler: Job 4 finished: processCmd at 
> CliDriver.java:376, took 0.73 s
> Time taken: 0.378 seconds
> 18/01/02 12:06:45 INFO CliDriver: Time taken: 0.378 seconds
> here no records get displayed



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (CARBONDATA-1962) Support alter table add columns/drop columns on S3 table

2018-01-01 Thread Sangeeta Gulia (JIRA)

Sangeeta Gulia created CARBONDATA-1962:
--

 Summary: Support alter table add columns/drop columns on S3 table
 Key: CARBONDATA-1962
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1962
 Project: CarbonData
  Issue Type: Task
Reporter: Sangeeta Gulia
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (CARBONDATA-1952) Incorrect Result displays while applying delete query on table

2018-01-01 Thread Kunal Kapoor (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor updated CARBONDATA-1952:
-
Component/s: hive-integration

> Incorrect Result displays while applying delete query on table
> --
>
> Key: CARBONDATA-1952
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1952
> Project: CarbonData
>  Issue Type: Bug
>  Components: hive-integration
>Affects Versions: 1.3.0
> Environment: spark 2.1
>Reporter: Vandana Yadav
>Priority: Minor
> Attachments: 2000_UniqData.csv
>
>
> Incorrect Result while applying delete query on the table.
> Steps to reproduce:
> 1)Create table:
> CREATE TABLE uniqdata_delete (CUST_ID int,CUST_NAME 
> String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, 
> BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), 
> DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double, 
> INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES 
> ("TABLE_BLOCKSIZE"= "256 MB")
> 2)Load Data Into table:
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into 
> table uniqdata_delete 
> OPTIONS('FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')
> 3)Execute Query:
> a) select count(*) from uniqdata_delete;
> Output:
>  +---+--+
> | count(1)  |
> +---+--+
> | 2013  |
> +---+--+
> 1 row selected (0.203 seconds)
> b)delete from uniqdata_delete where CUST_ID NOT IN(9996,);
> c) select count(*) from uniqdata_delete;
> output: 
> +---+--+
> | count(1)  |
> +---+--+
> | 14|
> +---+--+
> 1 row selected (0.22 seconds)
> d) select * from uniqdata_delete;
> output:
> +--+--++++-+-+-+-+--+---+--+--+
> | cust_id  |cust_name |active_emui_version |  dob 
>   |  doj   | bigint_column1  | bigint_column2  | 
> decimal_column1 | decimal_column2 |double_column1|
> double_column2 | integer_column1  |
> +--+--++++-+-+-+-+--+---+--+--+
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| NULL 
>| NULL| NULL | NULL
>   | NULL |
> | NULL |  || NULL 
>   | NULL   | 1233720368578   | NULL| NULL 
>| NULL| NULL | NULL
>   | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| -223372036854   | NULL 
>| NULL| NULL | NULL
>   | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| 
> 12345678901.123400  | NULL| NULL | 
> NULL  | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| NULL 
>| 22345678901.123400  | NULL | NULL
>   | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| NULL 
>| NULL| 1.12345674897976E10  | NULL
>   | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| NULL 
>| NULL| NULL | 
> -1.12345674897976E10  | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| NULL 
>| NULL

[jira] [Updated] (CARBONDATA-1952) Incorrect Result displays while applying delete query on table

2018-01-01 Thread Kunal Kapoor (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Kapoor updated CARBONDATA-1952:
-
Component/s: (was: data-query)

> Incorrect Result displays while applying delete query on table
> --
>
> Key: CARBONDATA-1952
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1952
> Project: CarbonData
>  Issue Type: Bug
>Affects Versions: 1.3.0
> Environment: spark 2.1
>Reporter: Vandana Yadav
>Priority: Minor
> Attachments: 2000_UniqData.csv
>
>
> Incorrect Result while applying delete query on the table.
> Steps to reproduce:
> 1)Create table:
> CREATE TABLE uniqdata_delete (CUST_ID int,CUST_NAME 
> String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, 
> BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), 
> DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double, 
> INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES 
> ("TABLE_BLOCKSIZE"= "256 MB")
> 2)Load Data Into table:
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into 
> table uniqdata_delete 
> OPTIONS('FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')
> 3)Execute Query:
> a) select count(*) from uniqdata_delete;
> Output:
>  +---+--+
> | count(1)  |
> +---+--+
> | 2013  |
> +---+--+
> 1 row selected (0.203 seconds)
> b)delete from uniqdata_delete where CUST_ID NOT IN(9996,);
> c) select count(*) from uniqdata_delete;
> output: 
> +---+--+
> | count(1)  |
> +---+--+
> | 14|
> +---+--+
> 1 row selected (0.22 seconds)
> d) select * from uniqdata_delete;
> output:
> +--+--++++-+-+-+-+--+---+--+--+
> | cust_id  |cust_name |active_emui_version |  dob 
>   |  doj   | bigint_column1  | bigint_column2  | 
> decimal_column1 | decimal_column2 |double_column1|
> double_column2 | integer_column1  |
> +--+--++++-+-+-+-+--+---+--+--+
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| NULL 
>| NULL| NULL | NULL
>   | NULL |
> | NULL |  || NULL 
>   | NULL   | 1233720368578   | NULL| NULL 
>| NULL| NULL | NULL
>   | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| -223372036854   | NULL 
>| NULL| NULL | NULL
>   | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| 
> 12345678901.123400  | NULL| NULL | 
> NULL  | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| NULL 
>| 22345678901.123400  | NULL | NULL
>   | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| NULL 
>| NULL| 1.12345674897976E10  | NULL
>   | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| NULL 
>| NULL| NULL | 
> -1.12345674897976E10  | NULL |
> | NULL |  || NULL 
>   | NULL   | NULL| NULL| NULL 
>| NULL| NULL | NULL

[jira] [Updated] (CARBONDATA-1961) Support data update/delete on S3 table

2018-01-01 Thread Sangeeta Gulia (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sangeeta Gulia updated CARBONDATA-1961:
---
Priority: Minor  (was: Major)

> Support data update/delete on S3 table
> --
>
> Key: CARBONDATA-1961
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1961
> Project: CarbonData
>  Issue Type: Task
>Reporter: Sangeeta Gulia
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (CARBONDATA-1961) Support data update/delete on S3 table

2018-01-01 Thread Sangeeta Gulia (JIRA)

Sangeeta Gulia created CARBONDATA-1961:
--

 Summary: Support data update/delete on S3 table
 Key: CARBONDATA-1961
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1961
 Project: CarbonData
  Issue Type: Task
Reporter: Sangeeta Gulia






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (CARBONDATA-1960) Add example for creating a local table and load CSV data which is stored in S3.

2018-01-01 Thread Sangeeta Gulia (JIRA)

Sangeeta Gulia created CARBONDATA-1960:
--

 Summary: Add example for creating a local table and load CSV data 
which is stored in S3.
 Key: CARBONDATA-1960
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1960
 Project: CarbonData
  Issue Type: Task
Reporter: Sangeeta Gulia
Priority: Trivial






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (CARBONDATA-1959) Support compaction on S3 table

2018-01-01 Thread Sangeeta Gulia (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sangeeta Gulia updated CARBONDATA-1959:
---
Priority: Minor  (was: Major)

> Support compaction on S3 table
> --
>
> Key: CARBONDATA-1959
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1959
> Project: CarbonData
>  Issue Type: Task
>Reporter: Sangeeta Gulia
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (CARBONDATA-1959) Support compaction on S3 table

2018-01-01 Thread Sangeeta Gulia (JIRA)

Sangeeta Gulia created CARBONDATA-1959:
--

 Summary: Support compaction on S3 table
 Key: CARBONDATA-1959
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1959
 Project: CarbonData
  Issue Type: Task
Reporter: Sangeeta Gulia






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (CARBONDATA-1958) CarbonSqlCliDriver not show result in case of select query

2018-01-01 Thread anubhav tarar (JIRA)

anubhav tarar created CARBONDATA-1958:
-

 Summary: CarbonSqlCliDriver not show result in case of select query
 Key: CARBONDATA-1958
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1958
 Project: CarbonData
  Issue Type: Bug
  Components: data-query
Affects Versions: 1.3.0
 Environment: sparksqlclidriver,hadoop2.7
Reporter: anubhav tarar


1.start the carbonsparksqlcli driver
go to bin folder of carbon project and execute command
./carbon-spark-sql

2.create carbon data table
spark-sql> create table sparktable(id int)stored by 'carbondata';

3.load data in carbontable
spark-sql> insert into sparktable values(1);

4.query the data

18/01/02 12:06:45 INFO TableInfo: main Table block size not specified for 
default_sparktable. Therefore considering the default value 1024 MB
18/01/02 12:06:45 INFO BlockletDataMap: main Time taken to load blocklet 
datamap from file : 
/home/anubhav/Documents/carbondata/carbondata/bin/carbonsqlclistore/default/sparktable/Fact/Part0/Segment_0/0_batchno0-0-1514874975013.carbonindexis
 19
18/01/02 12:06:45 INFO CarbonScanRDD: 
 Identified no.of.blocks: 0,
 no.of.tasks: 0,
 no.of.nodes: 0,
 parallelism: 4
   
18/01/02 12:06:45 INFO SparkContext: Starting job: processCmd at 
CliDriver.java:376
18/01/02 12:06:45 INFO DAGScheduler: Job 4 finished: processCmd at 
CliDriver.java:376, took 0.73 s
Time taken: 0.378 seconds
18/01/02 12:06:45 INFO CliDriver: Time taken: 0.378 seconds

here no records get displayed




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (CARBONDATA-1957) create datamap query fails on table having dictionary_include

2018-01-01 Thread Geetika Gupta (JIRA)

Geetika Gupta created CARBONDATA-1957:
-

 Summary: create datamap query fails on table having 
dictionary_include
 Key: CARBONDATA-1957
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1957
 Project: CarbonData
  Issue Type: Bug
  Components: data-query
Affects Versions: 1.3.0
 Environment: spark2.1
Reporter: Geetika Gupta
 Fix For: 1.3.0
 Attachments: 2000_UniqData.csv

I created a datamap using the following command:

create datamap uniqdata_agg on table uniqdata using 'preaggregate' as select 
cust_id, cust_name,avg(decimal_column1) from uniqdata group by 
cust_id,cust_name;

It throws the following error:
Error: java.lang.Exception: DataLoad failure: (state=,code=0)

Steps to reproduce:
CREATE TABLE uniqdata(CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, 
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 
int) STORED BY 'org.apache.carbondata.format' 
TBLPROPERTIES('DICTIONARY_INCLUDE'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')

Load command:
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into 
table uniqdata OPTIONS('DELIMITER'=',', 
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')

Create datamap commad:
create datamap uniqdata_agg on table uniqdata using 'preaggregate' as select 
cust_id, cust_name,avg(decimal_column1) from uniqdata group by 
cust_id,cust_name;

The above command throws the following exception:
Error: java.lang.Exception: DataLoad failure: (state=,code=0)

Here are the logs:
18/01/02 11:46:58 ERROR ParallelReadMergeSorterImpl: 
SafeParallelSorterPool:uniqdata_uniqdata_agg 
java.lang.IllegalArgumentException: requirement failed: Decimal precision 2922 
exceeds max precision 38
at scala.Predef$.require(Predef.scala:224)
at org.apache.spark.sql.types.Decimal.set(Decimal.scala:113)
at org.apache.spark.sql.types.Decimal$.apply(Decimal.scala:426)
at org.apache.spark.sql.types.Decimal.apply(Decimal.scala)
at 
org.apache.spark.sql.catalyst.expressions.UnsafeRow.getDecimal(UnsafeRow.java:409)
at 
org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificSafeProjection.apply_0$(Unknown
 Source)
at 
org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificSafeProjection.apply(Unknown
 Source)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at 
org.apache.carbondata.spark.rdd.LazyRddIterator.next(NewCarbonDataLoadRDD.scala:514)
at 
org.apache.carbondata.spark.rdd.LazyRddIterator.next(NewCarbonDataLoadRDD.scala:477)
at 
org.apache.carbondata.processing.loading.steps.InputProcessorStepImpl$InputProcessorIterator.getBatch(InputProcessorStepImpl.java:239)
at 
org.apache.carbondata.processing.loading.steps.InputProcessorStepImpl$InputProcessorIterator.next(InputProcessorStepImpl.java:200)
at 
org.apache.carbondata.processing.loading.steps.InputProcessorStepImpl$InputProcessorIterator.next(InputProcessorStepImpl.java:129)
at 
org.apache.carbondata.processing.loading.steps.DataConverterProcessorStepImpl$1.next(DataConverterProcessorStepImpl.java:97)
at 
org.apache.carbondata.processing.loading.steps.DataConverterProcessorStepImpl$1.next(DataConverterProcessorStepImpl.java:83)
at 
org.apache.carbondata.processing.loading.sort.impl.ParallelReadMergeSorterImpl$SortIteratorThread.run(ParallelReadMergeSorterImpl.java:218)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18/01/02 11:46:58 ERROR ForwardDictionaryCache: 
SafeParallelSorterPool:uniqdata_uniqdata_agg Error loading the dictionary: null
18/01/02 11:46:58 ERROR ForwardDictionaryCache: 
SafeParallelSorterPool:uniqdata_uniqdata_agg Error loading the dictionary: null
18/01/02 11:46:58 ERROR ForwardDictionaryCache: 
SafeParallelSorterPool:uniqdata_uniqdata_agg Error loading the dictionary: null
18/01/02 11:46:58 ERROR ForwardDictionaryCache: 
SafeParallelSorterPool:uniqdata_uniqdata_agg Error loading the dictionary: null
18/01/02 11:46:58 ERROR ParallelReadMergeSorterImpl: 
SafeParallelSorterPool:uniqdata_uniqdata_agg 
java.lang.IllegalArgumentException: requirement failed: Decimal precision 3128 
exceeds max precision 38
at

[GitHub] carbondata pull request #1746: [TEST] Enable pre-aggregate tests as CI issue...

2018-01-01 Thread kunal642

GitHub user kunal642 opened a pull request:

https://github.com/apache/carbondata/pull/1746

[TEST] Enable pre-aggregate tests as CI issue is fixed

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kunal642/carbondata enable_preagg_tests

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1746.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1746


commit 6e47cb226d5eefe45b6a8ec1c01cf84922e611e6
Author: kunal642 
Date:   2018-01-02T06:30:09Z

Enable pre-aggregate tests as CI issue is fixed




---

[GitHub] carbondata pull request #1745: [CARBONDATA-1955] Delta DataType calculation ...

2018-01-01 Thread mohammadshahidkhan

Github user mohammadshahidkhan closed the pull request at:

https://github.com/apache/carbondata/pull/1745


---

[GitHub] carbondata issue #1745: [CARBONDATA-1955] Delta DataType calculation is inco...

2018-01-01 Thread mohammadshahidkhan

Github user mohammadshahidkhan commented on the issue:

https://github.com/apache/carbondata/pull/1745
  
Closed after discussion with Ravindra


---

[GitHub] carbondata issue #1702: [CARBONDATA-1896] Clean files operation improvement

2018-01-01 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1702
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/2646/



---

[GitHub] carbondata issue #1104: [CARBONDATA-1239] Add validation for set command par...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1104
  
Build Success with Spark 2.2.0, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1244/



---

[GitHub] carbondata issue #1104: [CARBONDATA-1239] Add validation for set command par...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1104
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2468/



---

[GitHub] carbondata issue #1718: [CARBONDATA-1929][Validation]carbon property configu...

2018-01-01 Thread manishgupta88

Github user manishgupta88 commented on the issue:

https://github.com/apache/carbondata/pull/1718
  
retest sdv please


---

[jira] [Created] (CARBONDATA-1956) Select query with sum, count and avg throws exception for pre aggregate table

2018-01-01 Thread Geetika Gupta (JIRA)

Geetika Gupta created CARBONDATA-1956:
-

 Summary: Select query with sum, count and avg throws exception for 
pre aggregate table
 Key: CARBONDATA-1956
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1956
 Project: CarbonData
  Issue Type: Bug
  Components: data-query
Affects Versions: 1.3.0
 Environment: spark2.1
Reporter: Geetika Gupta
 Fix For: 1.3.0
 Attachments: 2000_UniqData.csv

I create a datamap using the following command:

create datamap uniqdata_agg_d on table uniqdata_29 using 'preaggregate' as 
select sum(decimal_column1), count(cust_id), avg(bigint_column1) from 
uniqdata_29 group by cust_id;

The datamap creation was successfull, but when I tried the following query:
select sum(decimal_column1), count(cust_id), avg(bigint_column1) from 
uniqdata_29 group by cust_id;

It throws the following exception:
Error: org.apache.spark.sql.AnalysisException: cannot resolve 
'(sum(uniqdata_29_uniqdata_agg_d.`uniqdata_29_bigint_column1_sum`) / 
sum(uniqdata_29_uniqdata_agg_d.`uniqdata_29_bigint_column1_count`))' due to 
data type mismatch: 
'(sum(uniqdata_29_uniqdata_agg_d.`uniqdata_29_bigint_column1_sum`) / 
sum(uniqdata_29_uniqdata_agg_d.`uniqdata_29_bigint_column1_count`))' requires 
(double or decimal) type, not bigint;;
'Aggregate [uniqdata_29_cust_id_count#244], 
[sum(uniqdata_29_decimal_column1_sum#243) AS sum(decimal_column1)#274, 
sum(cast(uniqdata_29_cust_id_count#244 as bigint)) AS count(cust_id)#276L, 
(sum(uniqdata_29_bigint_column1_sum#245L) / 
sum(uniqdata_29_bigint_column1_count#246L)) AS avg(bigint_column1)#279]
+- 
Relation[uniqdata_29_decimal_column1_sum#243,uniqdata_29_cust_id_count#244,uniqdata_29_bigint_column1_sum#245L,uniqdata_29_bigint_column1_count#246L]
 CarbonDatasourceHadoopRelation [ Database name :28dec, Table name 
:uniqdata_29_uniqdata_agg_d, Schema 
:Some(StructType(StructField(uniqdata_29_decimal_column1_sum,DecimalType(30,10),true),
 StructField(uniqdata_29_cust_id_count,IntegerType,true), 
StructField(uniqdata_29_bigint_column1_sum,LongType,true), 
StructField(uniqdata_29_bigint_column1_count,LongType,true))) ] (state=,code=0)

Steps for creation of maintable:
CREATE TABLE uniqdata_29(CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION 
string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 
int) STORED BY 'org.apache.carbondata.format';

Load command:
LOAD DATA INPATH 'hdfs://localhost:54311/Files/2000_UniqData.csv' into table 
uniqdata_29 OPTIONS('DELIMITER'=',', 
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');

Datamap creation command:
create datamap uniqdata_agg_d on table uniqdata_29 using 'preaggregate' as 
select sum(decimal_column1), count(cust_id), avg(bigint_column1) from 
uniqdata_29 group by cust_id;

Note: sum(decimal_column1), count(cust_id), avg(bigint_column1) from 
uniqdata_29 group by cust_id; executed successfully on maintable





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] carbondata issue #1104: [CARBONDATA-1239] Add validation for set command par...

2018-01-01 Thread mohammadshahidkhan

Github user mohammadshahidkhan commented on the issue:

https://github.com/apache/carbondata/pull/1104
  
retest this please


---

[GitHub] carbondata issue #1702: [CARBONDATA-1896] Clean files operation improvement

2018-01-01 Thread dhatchayani

Github user dhatchayani commented on the issue:

https://github.com/apache/carbondata/pull/1702
  
Retest sdv please


---

[GitHub] carbondata issue #1720: [CARBONDATA-1935]fix the backword compatibility issu...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1720
  
Build Success with Spark 2.2.0, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1243/



---

[GitHub] carbondata issue #1720: [CARBONDATA-1935]fix the backword compatibility issu...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1720
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2467/



---

[jira] [Closed] (CARBONDATA-1790) (Carbon1.3.0 - Streaming) Data load in Stream Segment fails if batch load is performed in between the streaming

2018-01-01 Thread Ramakrishna S (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramakrishna S closed CARBONDATA-1790.
-
Resolution: Fixed

This is is resolved in the B007 release.

> (Carbon1.3.0 - Streaming) Data load in Stream Segment fails if batch load is 
> performed in between the streaming
> ---
>
> Key: CARBONDATA-1790
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1790
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-query
>Affects Versions: 1.3.0
> Environment: 3 node ant cluster
>Reporter: Ramakrishna S
>Assignee: Bhavya Aggarwal
>  Labels: DFX
>
> Steps :
> 1. Create a streaming table and do a batch load
> 2. Set up the Streaming , so that it does streaming in chunk of 1000 records 
> 20 times
> 3. Do another batch load on the table
> 4. Do one more time streaming
> +-++--+--+--++--+
> | Segment Id  |   Status   | Load Start Time  |  Load End Time
>| File Format  | Merged To  |
> +-++--+--+--++--+
> | 2   | Success| 2017-11-21 21:42:36.77   | 2017-11-21 
> 21:42:40.396  | COLUMNAR_V3  | NA |
> | 1   | Streaming  | 2017-11-21 21:40:46.2| NULL  
>| ROW_V1   | NA |
> | 0   | Success| 2017-11-21 21:40:39.782  | 2017-11-21 
> 21:40:43.168  | COLUMNAR_V3  | NA |
> +-++--+--+--++--+
> *+Expected:+* Data should be loaded
> *+Actual+* : Data load fiails
> 1. One addition offset file is created(marked in bold)
> -rw-r--r--   2 root users 62 2017-11-21 21:40 
> /user/hive/warehouse/Ram/default/stream_table5/.streaming/checkpoint/offsets/0
> -rw-r--r--   2 root users 63 2017-11-21 21:40 
> /user/hive/warehouse/Ram/default/stream_table5/.streaming/checkpoint/offsets/1
> -rw-r--r--   2 root users 63 2017-11-21 21:42 
> /user/hive/warehouse/Ram/default/stream_table5/.streaming/checkpoint/offsets/10
> -rw-r--r--   2 root users 63 2017-11-21 21:40 
> /user/hive/warehouse/Ram/default/stream_table5/.streaming/checkpoint/offsets/2
> -rw-r--r--   2 root users 63 2017-11-21 21:41 
> /user/hive/warehouse/Ram/default/stream_table5/.streaming/checkpoint/offsets/3
> -rw-r--r--   2 root users 64 2017-11-21 21:41 
> /user/hive/warehouse/Ram/default/stream_table5/.streaming/checkpoint/offsets/4
> -rw-r--r--   2 root users 64 2017-11-21 21:41 
> /user/hive/warehouse/Ram/default/stream_table5/.streaming/checkpoint/offsets/5
> -rw-r--r--   2 root users 64 2017-11-21 21:41 
> /user/hive/warehouse/Ram/default/stream_table5/.streaming/checkpoint/offsets/6
> -rw-r--r--   2 root users 64 2017-11-21 21:41 
> /user/hive/warehouse/Ram/default/stream_table5/.streaming/checkpoint/offsets/7
> -rw-r--r--   2 root users 64 2017-11-21 21:41 
> /user/hive/warehouse/Ram/default/stream_table5/.streaming/checkpoint/offsets/8
> *-rw-r--r--   2 root users 63 2017-11-21 21:42 
> /user/hive/warehouse/Ram/default/stream_table5/.streaming/checkpoint/offsets/9*
> 2. Following error thrown:
> === Streaming Query ===
> Identifier: [id = 3a5334bc-d471-4676-b6ce-f21105d491d1, runId = 
> b2be9f97-8141-46be-89db-9a0f98d13369]
> Current Offsets: 
> {org.apache.spark.sql.execution.streaming.TextSocketSource@14c45193: 1000}
> Current State: ACTIVE
> Thread State: RUNNABLE
> Logical Plan:
> org.apache.spark.sql.execution.streaming.TextSocketSource@14c45193
> at 
> org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runBatches(StreamExecution.scala:284)
> at 
> org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:177)
> Caused by: java.lang.RuntimeException: Offsets committed out of order: 20019 
> followed by 1000
> at scala.sys.package$.error(package.scala:27)
> at 
> org.apache.spark.sql.execution.streaming.TextSocketSource.commit(socket.scala:151)
> at 
> org.apache.spark.sql.execution.streaming.StreamExecution$$anonfun$org$apache$spark$sql$execution$streaming$StreamExecution$$constructNextBatch$2$$anonfun$apply$mcV$sp$4.apply(StreamExecution.scala:421)
> at 
> org.apache.spark.sql.execution.streaming.StreamExecution$$anonfun$org$apache$spark$sql$execution$streaming$StreamExecution$$constructNextBatch$2$$anonfun$apply$mcV$sp$4.apply(StreamExecution.scala:420)
> at scala.collection.Iterator$class.foreach(Itera

[GitHub] carbondata issue #1720: [CARBONDATA-1935]fix the backword compatibility issu...

2018-01-01 Thread akashrn5

Github user akashrn5 commented on the issue:

https://github.com/apache/carbondata/pull/1720
  
retest this please


---

[GitHub] carbondata issue #1743: [CARBONDATA-1954] HiveMetastore updated while droppi...

2018-01-01 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1743
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/2645/



---

[GitHub] carbondata issue #1743: [CARBONDATA-1954] HiveMetastore updated while droppi...

2018-01-01 Thread rahulforallp

Github user rahulforallp commented on the issue:

https://github.com/apache/carbondata/pull/1743
  
retest sdv please


---

[GitHub] carbondata issue #1738: [CARBONDATA-1932]Add version info for CarbonData

2018-01-01 Thread zzcclp

Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/1738
  
@xubo245 , we don't package pom.xml file into release jar, so can't 
directly read version info from pom.xml


---

[GitHub] carbondata issue #1738: [CARBONDATA-1932]Add version info for CarbonData

2018-01-01 Thread xubo245

Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/1738
  
Can we obtain directly the CarbonData version from pom.xml ? 


---

[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1707
  
Build Success with Spark 2.2.0, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1242/



---

[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1707
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2466/



---

[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...

2018-01-01 Thread xuchuanyin

Github user xuchuanyin commented on the issue:

https://github.com/apache/carbondata/pull/1707
  
retest this please


---

[GitHub] carbondata issue #1742: [CARBONDATA-1953]Pre-aggregate Should inherit sort c...

2018-01-01 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1742
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/2644/



---

[GitHub] carbondata issue #1742: [CARBONDATA-1953]Pre-aggregate Should inherit sort c...

2018-01-01 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1742
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/2643/



---

[GitHub] carbondata issue #1742: [CARBONDATA-1953]Pre-aggregate Should inherit sort c...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1742
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2465/



---

[GitHub] carbondata issue #1742: [CARBONDATA-1953]Pre-aggregate Should inherit sort c...

2018-01-01 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1742
  
Build Success with Spark 2.2.0, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1241/



---

48 matches

Mail list logo