[jira] [Created] (CARBONDATA-862) USE_KETTLE option described in dml-operation-on-carbondata.md document doesn't work

2017-04-05 Thread Divya Gupta (JIRA) 
Divya Gupta created CARBONDATA-862:
--

 Summary: USE_KETTLE option described in 
dml-operation-on-carbondata.md document doesn't work
 Key: CARBONDATA-862
 URL: https://issues.apache.org/jira/browse/CARBONDATA-862
 Project: CarbonData
  Issue Type: Bug
  Components: docs
Affects Versions: 1.0.0-incubating
Reporter: Divya Gupta
Assignee: Divya Gupta
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CARBONDATA-604) Use Kettle wrong option in Thrift server give wrong error message

2017-01-25 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-604:
---
Affects Version/s: 1.0.0-incubating

> Use Kettle wrong option in Thrift server give wrong error message
> -
>
> Key: CARBONDATA-604
> URL: https://issues.apache.org/jira/browse/CARBONDATA-604
> Project: CarbonData
>  Issue Type: Bug
>Affects Versions: 1.0.0-incubating
> Environment: Spark 1.6
>Reporter: Harmeet Singh
>Assignee: Ravindra Pesala
>Priority: Trivial
>
> I am using Thrift Server with following configuration: 
> -Duse.kettle=default
> Here kettle value is wrong and when I run the thrift server, the thrift 
> server is running successfully. After that, I am going to start beeline. In 
> beeline, I am executing load command without any kettle option as below: 
>  LOAD DATA INPATH 'hdfs://localhost:54310/csvs/3000_UniqData.csv' into table 
> uniqdata OPTIONS('DELIMITER'=',' , 
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> After executing above command, I am getting Error: 
> java.lang.IllegalArgumentException: For input string: "default" 
> (state=,code=0)
> This error, seems to be wrong, because I am not using any default value in 
> load query, and default is used with thrift.
> So I have following Suggestions: 
> 1. Change the error message, while load query is executing.
> 2. While starting thrift server with wrong value, thrift give us an error and 
> not start.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-665) Comparision Failure occurs when we execute the same query in hive and Carbondata

2017-01-25 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-665:
---
Environment: Spark 1.6

> Comparision Failure occurs when we execute the same query in hive and 
> Carbondata
> 
>
> Key: CARBONDATA-665
> URL: https://issues.apache.org/jira/browse/CARBONDATA-665
> Project: CarbonData
>  Issue Type: Bug
>Affects Versions: 1.0.0-incubating
> Environment: Spark 1.6
>Reporter: SWATI RAO
> Attachments: Test_Data1.csv
>
>
> Orderby is not working , so records are not coming in sequence as well there 
> is data difference and some values being stored as null
> Data itself is stored incorrectly and is different from Hive
> Spark version :1.6.2
> Create 1 query : create table Test_Boundary (c1_int int,c2_Bigint 
> Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string 
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY 
> 'org.apache.carbondata.format'
> Load 1 Query : LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO 
> table Test_Boundary 
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> Create 2 query : create table Test_Boundary1 (c1_int int,c2_Bigint 
> Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string 
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY 
> 'org.apache.carbondata.format'
> Load 2 query:  LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO 
> table Test_Boundary1 
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> Select Query : 
> select 
> c1_int,c2_Bigint,c3_Decimal,c4_double,c5_string,c6_Timestamp,c7_Datatype_Desc 
> from Test_Boundary where c2_bigint=c2_bigint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-631) Select,Delete and Insert Query Failing for table created in 0.2 with data loaded in 1.0

2017-01-25 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-631:
---
Affects Version/s: 1.0.0-incubating

> Select,Delete and Insert Query Failing for table created in 0.2 with data 
> loaded in 1.0
> ---
>
> Key: CARBONDATA-631
> URL: https://issues.apache.org/jira/browse/CARBONDATA-631
> Project: CarbonData
>  Issue Type: Bug
>Affects Versions: 1.0.0-incubating
> Environment: Spark 1.6 
>Reporter: Pallavi Singh
>Assignee: kumar vishal
> Fix For: NONE
>
>
> Created table  with the 0.2 jar:
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION 
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES 
> ("TABLE_BLOCKSIZE"= "256 MB");
> then 
> LOAD DATA INPATH 'hdfs://localhost:54310/csv/2000_UniqData.csv' into table 
> uniqdata OPTIONS('DELIMITER'=',' , 
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> Switched to 1.0 jar
> LOAD DATA INPATH 'hdfs://localhost:54310/csv/2000_UniqData.csv' into table 
> uniqdata OPTIONS('DELIMITER'=',' , 
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> After successful load :
> select count(*) from uniqdata;
> I get following error : 
> INFO  12-01 18:31:04,057 - Running query 'select count(*) from uniqdata' with 
> 81129cf3-fcd4-429d-9adf-d37d35cdf051
> INFO  12-01 18:31:04,058 - pool-27-thread-46 Query [SELECT COUNT(*) FROM 
> UNIQDATA]
> INFO  12-01 18:31:04,060 - Parsing command: select count(*) from uniqdata
> INFO  12-01 18:31:04,060 - Parse Completed
> INFO  12-01 18:31:04,061 - Parsing command: select count(*) from uniqdata
> INFO  12-01 18:31:04,061 - Parse Completed
> INFO  12-01 18:31:04,061 - 27: get_table : db=12jan17 tbl=uniqdata
> INFO  12-01 18:31:04,061 - ugi=pallaviip=unknown-ip-addr  
> cmd=get_table : db=12jan17 tbl=uniqdata 
> INFO  12-01 18:31:04,061 - 27: Opening raw store with implemenation 
> class:org.apache.hadoop.hive.metastore.ObjectStore
> INFO  12-01 18:31:04,063 - ObjectStore, initialize called
> INFO  12-01 18:31:04,068 - Reading in results for query 
> "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is 
> closing
> INFO  12-01 18:31:04,069 - Using direct SQL, underlying DB is DERBY
> INFO  12-01 18:31:04,069 - Initialized ObjectStore
> INFO  12-01 18:31:04,101 - pool-27-thread-46 Starting to optimize plan
> ERROR 12-01 18:31:04,168 - pool-27-thread-46 Cannot convert12-01-2017 
> 16:02:28 to Time/Long type valueUnparseable date: "12-01-2017 16:02:28"
> ERROR 12-01 18:31:04,185 - pool-27-thread-46 Cannot convert12-01-2017 
> 16:02:08 to Time/Long type valueUnparseable date: "12-01-2017 16:02:08"
> ERROR 12-01 18:31:04,185 - pool-27-thread-46 Cannot convert12-01-2017 
> 16:02:08 to Time/Long type valueUnparseable date: "12-01-2017 16:02:08"
> ERROR 12-01 18:31:04,204 - pool-27-thread-46 Cannot convert12-01-2017 
> 16:02:08 to Time/Long type valueUnparseable date: "12-01-2017 16:02:08"
> ERROR 12-01 18:31:04,210 - Error executing query, currentState RUNNING, 
> org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree:
> CarbonDictionaryDecoder [CarbonDecoderRelation(Map(dob#280 -> dob#280, 
> double_column1#287 -> double_column1#287, decimal_column1#285 -> 
> decimal_column1#285, cust_id#282L -> cust_id#282L, integer_column1#289L -> 
> integer_column1#289L, decimal_column2#286 -> decimal_column2#286, 
> cust_name#278 -> cust_name#278, double_column2#288 -> double_column2#288, 
> active_emui_version#279 -> active_emui_version#279, bigint_column1#283L -> 
> bigint_column1#283L, bigint_column2#284L -> bigint_column2#284L, doj#281 -> 
> doj#281),CarbonDatasourceRelation(`12jan17`.`uniqdata`,None))], 
> ExcludeProfile(ArrayBuffer()), CarbonAliasDecoderRelation()
> +- TungstenAggregate(key=[], 
> functions=[(count(1),mode=Final,isDistinct=false)], output=[_c0#750L])
>+- TungstenExchange SinglePartition, None
>   +- TungstenAggregate(key=[], 
> functions=[(count(1),mode=Partial,isDistinct=false)], output=[count#754L])
>  +- CarbonScan CarbonRelation 12jan17, uniqdata, 
> CarbonMetaData(ArrayBuffer(cust_name, active_emui_version, dob, 
> doj),ArrayBuffer(cust_id, bigint_column1,

[jira] [Updated] (CARBONDATA-678) Corr function is not working for double datatype.

2017-01-25 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-678:
---
Affects Version/s: 1.0.0-incubating

> Corr function is not working for double datatype.
> -
>
> Key: CARBONDATA-678
> URL: https://issues.apache.org/jira/browse/CARBONDATA-678
> Project: CarbonData
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 1.0.0-incubating
> Environment: Spark 1.6
>Reporter: SWATI RAO
>Priority: Trivial
> Fix For: 1.1.0-incubating
>
> Attachments: 2000_UniqData.csv, 3000_1_UniqData.csv, 
> 4000_UniqData.csv, 5000_UniqData.csv, 6000_UniqData.csv, 7000_UniqData.csv
>
>
> We have created a table uniqdata_h and then we upload data in this table then 
> we perform select query having data-type double then it displays null value 
> but when we perform select query with integer or bigint data-type it displays 
> a value.
> "Double" data-type is not working.
> CREATE QUERY :
> 0: jdbc:hive2://localhost:1> CREATE TABLE uniqdata_h (CUST_ID 
> int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ 
> timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 
> decimal(30,10), DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, 
> Double_COLUMN2 double,INTEGER_COLUMN1 int) ROW FORMAT DELIMITED FIELDS 
> TERMINATED BY ',';
> +-+--+
> | result  |
> +-+--+
> +-+--+
> No rows selected (4.249 seconds)
> LOAD 1 :
> 0: jdbc:hive2://localhost:1> load data local inpath 
> '/opt/Carbon/CarbonData/TestData/Data/uniqdata/2000_UniqData.csv' into table 
> uniqdata_h;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (1.176 seconds)
> LOAD 2 :
> 0: jdbc:hive2://localhost:1> load data local inpath 
> '/opt/Carbon/CarbonData/TestData/Data/uniqdata/4000_UniqData.csv' into table 
> uniqdata_h;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (0.485 seconds)
> LOAD 3 :
> 0: jdbc:hive2://localhost:1> load data local inpath 
> '/opt/Carbon/CarbonData/TestData/Data/uniqdata/5000_UniqData.csv' into table 
> uniqdata_h;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (0.365 seconds)
> LOAD 4 :
> 0: jdbc:hive2://localhost:1> load data local inpath 
> '/opt/Carbon/CarbonData/TestData/Data/uniqdata/6000_UniqData.csv' into table 
> uniqdata_h;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (0.594 seconds)
> LOAD 5 :
> 0: jdbc:hive2://localhost:1> load data local inpath 
> '/opt/Carbon/CarbonData/TestData/Data/uniqdata/7000_UniqData.csv' into table 
> uniqdata_h;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (0.753 seconds)
> LOAD 6 :
> 0: jdbc:hive2://localhost:1> load data local inpath 
> '/opt/Carbon/CarbonData/TestData/Data/uniqdata/3000_1_UniqData.csv' into 
> table uniqdata_h;
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (0.417 seconds)
> SELECT QUERY : double data-type displays NULL value
> 0: jdbc:hive2://localhost:1> select corr(Double_COLUMN1,Double_COLUMN1)  
> as a from uniqdata_h ;
> +---+--+
> |   a   |
> +---+--+
> | NULL  |
> +---+--+
> whereas select query with integer & bigdata type displays some value.
> 0: jdbc:hive2://localhost:1> select corr(integer_COLUMN1,bigint_COLUMN1)  
> as a from uniqdata_h ;
> +-+--+
> |  a  |
> +-+--+
> | 1.0002  |
> +-+--+
> 1 row selected (0.54 seconds)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-665) Comparision Failure occurs when we execute the same query in hive and Carbondata

2017-01-25 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-665:
---
Affects Version/s: 1.0.0-incubating

> Comparision Failure occurs when we execute the same query in hive and 
> Carbondata
> 
>
> Key: CARBONDATA-665
> URL: https://issues.apache.org/jira/browse/CARBONDATA-665
> Project: CarbonData
>  Issue Type: Bug
>Affects Versions: 1.0.0-incubating
>Reporter: SWATI RAO
> Attachments: Test_Data1.csv
>
>
> Orderby is not working , so records are not coming in sequence as well there 
> is data difference and some values being stored as null
> Data itself is stored incorrectly and is different from Hive
> Spark version :1.6.2
> Create 1 query : create table Test_Boundary (c1_int int,c2_Bigint 
> Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string 
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY 
> 'org.apache.carbondata.format'
> Load 1 Query : LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO 
> table Test_Boundary 
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> Create 2 query : create table Test_Boundary1 (c1_int int,c2_Bigint 
> Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string 
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY 
> 'org.apache.carbondata.format'
> Load 2 query:  LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO 
> table Test_Boundary1 
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> Select Query : 
> select 
> c1_int,c2_Bigint,c3_Decimal,c4_double,c5_string,c6_Timestamp,c7_Datatype_Desc 
> from Test_Boundary where c2_bigint=c2_bigint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-596) unable to crete filter resolver tree exception when using equals expression with carbon input format api

2017-01-16 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-596:
---
Environment: Spark 2.1 Spark 1.6

> unable to crete filter resolver tree exception when using equals expression 
> with carbon input format api
> 
>
> Key: CARBONDATA-596
> URL: https://issues.apache.org/jira/browse/CARBONDATA-596
> Project: CarbonData
>  Issue Type: Bug
>  Components: hadoop-integration
>Affects Versions: 1.0.0-incubating
> Environment: Spark 2.1 Spark 1.6
>Reporter: anubhav tarar
>Assignee: Manish Gupta
>Priority: Minor
>
> unable to crete filter resolver tree exception when using equals expression
>  Expression expression = new EqualToExpression(new ColumnExpression("c1", 
> DataType.STRING),
> new LiteralExpression("a", DataType.STRING));
> CarbonInputFormat.setFilterPredicates(job.getConfiguration(), 
> expression);
> List splits = carbonInputFormat.getSplits(job);
> it throws java.io.IOException: Error while resolving filter expression



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-590) unusual behaviour of using carbonthrift server with spark 2.0

2017-01-16 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-590:
---
Environment: Spark 2.0

> unusual behaviour of using carbonthrift server with spark 2.0
> -
>
> Key: CARBONDATA-590
> URL: https://issues.apache.org/jira/browse/CARBONDATA-590
> Project: CarbonData
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.0.0-incubating
> Environment: Spark 2.0
>Reporter: anubhav tarar
>
> have a look at these logs
> started thrift server 
> ./bin/spark-submit --conf spark.sql.hive.thriftServer.singleSession=true 
> --class org.apache.carbondata.spark.thriftserver.CarbonThriftServer 
> $SPARK_HOME/carbonlib/carbondata_2.11-1.0.0-incubating-SNAPSHOT-shade-hadoop2.2.0.jar
>  hdfs://localhost:54310/opt/carbonStore
> logs
> INFO  04-01 12:46:55,421 - Service:ThriftBinaryCLIService is started.
> INFO  04-01 12:46:55,421 - Service:HiveServer2 is started.
> started beeline
> connected to thrift server
>  !connect jdbc:hive2://localhost:1
> executed the query
> CREATE TABLE Bug212(int string)USING org.apache.spark.sql.CarbonSource
> OPTIONS("bucketnumber"="1", "bucketcolumns"="String","tableName"="t100");
> Error:
> org.apache.carbondata.spark.exception.MalformedCarbonCommandException: Table
> default.t 100 can not be created without key columns. Please use
> DICTIONARY_INCLUDE or DICTIONARY_EXCLUDE to set at least one key column if
> all specified columns are numeric types (state=,code=0)
> 2 minutes later
>  CREATE TABLE Bug211(int int)USING org.apache.spark.sql.CarbonSource
> OPTIONS("bucketnumber"="1", "bucketcolumns"="String","tableName"="t 100");
> +-+--+
> | Result  |
> +-+--+
> +-+--+
> No rows selected (0.212 seconds)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-589) carbon spark shell is not working with spark 2.0

2017-01-16 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-589:
---
Environment: Spark 2.0

> carbon spark shell is not working with spark 2.0
> 
>
> Key: CARBONDATA-589
> URL: https://issues.apache.org/jira/browse/CARBONDATA-589
> Project: CarbonData
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.0.0-incubating
> Environment: Spark 2.0
>Reporter: anubhav tarar
>Priority: Minor
>
> carbon shell is not working with spark 2.0 version 
> here are the logs
> ./carknoldus@knoldus:~/Desktop/open source/incubator-carbondata/bin$ 
> ./carbon-spark-shell
> java.lang.ClassNotFoundException: org.apache.spark.repl.carbon.Main
>   at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>   at java.lang.Class.forName0(Native Method)
>   at java.lang.Class.forName(Class.java:348)
>   at org.apache.spark.util.Utils$.classForName(Utils.scala:225)
>   at 
> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:686)
>   at 
> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
>   at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
>   at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
>   at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-615) Update query store wrong value for Date data type

2017-01-16 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-615:
---
Environment: Spark 1.6 Spark 2.1  (was: Spark 1.6)

> Update query store wrong value for Date data type
> -
>
> Key: CARBONDATA-615
> URL: https://issues.apache.org/jira/browse/CARBONDATA-615
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-query
>Affects Versions: 1.0.0-incubating
> Environment: Spark 1.6 Spark 2.1
>Reporter: Anurag Srivastava
>Assignee: ravikiran
>Priority: Minor
> Attachments: 2000_UniqData.csv, update_dob.png
>
>
> I am trying to update DOB column with Date Data Type. It is storing a day 
> before date which I have mentioned for updating in DOB column.
> *Create Table :* CREATE TABLE uniqdata (CUST_ID int,CUST_NAME 
> char(30),ACTIVE_EMUI_VERSION string, DOB Date, DOJ Date, BIGINT_COLUMN1 
> bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double, INTEGER_COLUMN1 
> int) STORED BY 'org.apache.carbondata.format';
> *Load Data :* LOAD DATA INPATH 'hdfs://localhost:54310/2000_UniqData.csv' 
> into table uniqdata OPTIONS ('DELIMITER'=',' 
> ,'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='true');
> *Update Query :*  update uniqdata set (dob)=(to_date('2016-12-01')) where 
> cust_name = 'CUST_NAME_01999';
> *Expected Result :* It should update DOB column with *2016-12-01*.
> *Actual Result :* It is updating DOB column with *2016-11-30*.
> !https://issues.apache.org/jira/secure/attachment/12846515/update_dob.png!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-615) Update query store wrong value for Date data type

2017-01-16 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-615:
---
Environment: Spark 1.6

> Update query store wrong value for Date data type
> -
>
> Key: CARBONDATA-615
> URL: https://issues.apache.org/jira/browse/CARBONDATA-615
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-query
>Affects Versions: 1.0.0-incubating
> Environment: Spark 1.6
>Reporter: Anurag Srivastava
>Assignee: ravikiran
>Priority: Minor
> Attachments: 2000_UniqData.csv, update_dob.png
>
>
> I am trying to update DOB column with Date Data Type. It is storing a day 
> before date which I have mentioned for updating in DOB column.
> *Create Table :* CREATE TABLE uniqdata (CUST_ID int,CUST_NAME 
> char(30),ACTIVE_EMUI_VERSION string, DOB Date, DOJ Date, BIGINT_COLUMN1 
> bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double, INTEGER_COLUMN1 
> int) STORED BY 'org.apache.carbondata.format';
> *Load Data :* LOAD DATA INPATH 'hdfs://localhost:54310/2000_UniqData.csv' 
> into table uniqdata OPTIONS ('DELIMITER'=',' 
> ,'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='true');
> *Update Query :*  update uniqdata set (dob)=(to_date('2016-12-01')) where 
> cust_name = 'CUST_NAME_01999';
> *Expected Result :* It should update DOB column with *2016-12-01*.
> *Actual Result :* It is updating DOB column with *2016-11-30*.
> !https://issues.apache.org/jira/secure/attachment/12846515/update_dob.png!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-593) Select command seems to be not working on carbon-spark-shell . It throws a runtime error on select query after show method is invoked

2017-01-16 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-593:
---
Environment: Spark 1.6

> Select command seems to be not working on carbon-spark-shell . It throws a 
> runtime error on select query after show method is invoked
> -
>
> Key: CARBONDATA-593
> URL: https://issues.apache.org/jira/browse/CARBONDATA-593
> Project: CarbonData
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 1.0.0-incubating
> Environment: Spark 1.6
>Reporter: DEEPAK MEHRA
>Priority: Minor
> Attachments: employee.csv
>
>
> Select command seems to be not working on carbon-spark-shell . It throws a 
> runtime error on select query after show method is invoked. It says 
> java.io.FileNotFoundException: File does not exist: 
> /home/hduser/software/spark-1.6.2-bin-hadoop2.6/carbon.store/mydb/demo/Fact/Part0/Segment_0
> Query Executed :
> scala> cc.sql("create table demo(id int,name string,age int,city string) 
> stored by 'carbondata'").show
> scala> cc.sql("LOAD DATA inpath 
> 'hdfs://hadoop-master:54311/data/employee.csv' INTO table demo");
>  cc.sql("select * from demo").show
> Result :It's throwing java.lang.RuntimeException : File Not Found
> scala> cc.sql("select * from mydb.demo").show
> INFO  04-01 16:29:22,157 - main Query [SELECT * FROM MYDB.DEMO]
> INFO  04-01 16:29:22,160 - Parsing command: select * from mydb.demo
> INFO  04-01 16:29:22,161 - Parse Completed
> INFO  04-01 16:29:22,162 - Parsing command: select * from mydb.demo
> INFO  04-01 16:29:22,163 - Parse Completed
> INFO  04-01 16:29:22,163 - 0: get_table : db=mydb tbl=demo
> INFO  04-01 16:29:22,163 - ugi=hduser ip=unknown-ip-addr  cmd=get_table : 
> db=mydb tbl=demo
> INFO  04-01 16:29:22,195 - main Starting to optimize plan
> java.io.FileNotFoundException: File does not exist: 
> /home/hduser/software/spark-1.6.2-bin-hadoop2.6/carbon.store/mydb/demo/Fact/Part0/Segment_0
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1110)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1102)
>   at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1102)
>   at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:747)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$15.(DistributedFileSystem.java:726)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus(DistributedFileSystem.java:717)
>   at 
> org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1780)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getFileStatusOfSegments(CarbonInputFormat.java:559)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.listStatus(CarbonInputFormat.java:519)
>   at 
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:340)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getSplitsInternal(CarbonInputFormat.java:251)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getTableBlockInfo(CarbonInputFormat.java:372)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getSegmentAbstractIndexs(CarbonInputFormat.java:402)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getDataBlocksOfSegment(CarbonInputFormat.java:325)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getSplits(CarbonInputFormat.java:288)
>   at 
> org.apache.carbondata.hadoop.CarbonInputFormat.getSplits(CarbonInputFormat.java:237)
>   at 
> org.apache.carbondata.spark.rdd.CarbonScanRDD.getPartitions(CarbonScanRDD.scala:82)
>   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
>   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
>   at scala.Option.getOrElse(Option.scala:120)
>   at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
>   at 
> org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
>   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
>   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
>   at scala.Option.getOrElse(Option.scala:120)
>   at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
>   at 
> org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
>   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
>   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
>   at

[jira] [Updated] (CARBONDATA-604) Use Kettle wrong option in Thrift server give wrong error message

2017-01-16 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-604:
---
Environment: Spark 1.6

> Use Kettle wrong option in Thrift server give wrong error message
> -
>
> Key: CARBONDATA-604
> URL: https://issues.apache.org/jira/browse/CARBONDATA-604
> Project: CarbonData
>  Issue Type: Bug
> Environment: Spark 1.6
>Reporter: Harmeet Singh
>Priority: Trivial
>
> I am using Thrift Server with following configuration: 
> -Duse.kettle=default
> Here kettle value is wrong and when I run the thrift server, the thrift 
> server is running successfully. After that, I am going to start beeline. In 
> beeline, I am executing load command without any kettle option as below: 
>  LOAD DATA INPATH 'hdfs://localhost:54310/csvs/3000_UniqData.csv' into table 
> uniqdata OPTIONS('DELIMITER'=',' , 
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> After executing above command, I am getting Error: 
> java.lang.IllegalArgumentException: For input string: "default" 
> (state=,code=0)
> This error, seems to be wrong, because I am not using any default value in 
> load query, and default is used with thrift.
> So I have following Suggestions: 
> 1. Change the error message, while load query is executing.
> 2. While starting thrift server with wrong value, thrift give us an error and 
> not start.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-619) Compaction API for Spark 2.1 : Issue in compaction type

2017-01-16 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-619:
---
Labels: 2.1.0 Spark  (was: )

> Compaction API for Spark 2.1 : Issue in compaction type
> ---
>
> Key: CARBONDATA-619
> URL: https://issues.apache.org/jira/browse/CARBONDATA-619
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-query
>Affects Versions: 1.0.0-incubating
>Reporter: DEEPAK MEHRA
>Assignee: ravikiran
>Priority: Minor
> Attachments: compaction_result.png, employee.csv
>
>
> Compaction API for Spark 2.1 : Compaction is successful on the segment ids 
> irrespective of the compaction types which are( Minor and Major) i.e The 
> compaction was successful for type other than Minor/Major 
> I created a table with name carbon_table and made 5 loads into the table with 
> the threshold of (4,3)
> cc.sql("create table carbon_table(id int,name string) stored by 'carbondata'")
> cc.sql("LOAD DATA LOCAL INPATH 'hdfs://hadoop-master:54311/data/employee.csv'
> INTO TABLE carbon_table options('FILEHEADER'='id,name')")
> For instance 
> Compaction.compaction(spark,"default","carbon_table","xyz")
> here 
> default is the database name
> carbon_table is the table name 
> xyz is the compaction type
> I invoked the above method to compact the segments and it was executed 
> successfully and compacted the segment ids. On show segment It produced the 
> result with success for compaction . I have attached the screen shot for the 
> same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-619) Compaction API for Spark 2.1 : Issue in compaction type

2017-01-16 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-619:
---
Labels:   (was: 2.1.0 Spark)

> Compaction API for Spark 2.1 : Issue in compaction type
> ---
>
> Key: CARBONDATA-619
> URL: https://issues.apache.org/jira/browse/CARBONDATA-619
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-query
>Affects Versions: 1.0.0-incubating
>Reporter: DEEPAK MEHRA
>Assignee: ravikiran
>Priority: Minor
> Attachments: compaction_result.png, employee.csv
>
>
> Compaction API for Spark 2.1 : Compaction is successful on the segment ids 
> irrespective of the compaction types which are( Minor and Major) i.e The 
> compaction was successful for type other than Minor/Major 
> I created a table with name carbon_table and made 5 loads into the table with 
> the threshold of (4,3)
> cc.sql("create table carbon_table(id int,name string) stored by 'carbondata'")
> cc.sql("LOAD DATA LOCAL INPATH 'hdfs://hadoop-master:54311/data/employee.csv'
> INTO TABLE carbon_table options('FILEHEADER'='id,name')")
> For instance 
> Compaction.compaction(spark,"default","carbon_table","xyz")
> here 
> default is the database name
> carbon_table is the table name 
> xyz is the compaction type
> I invoked the above method to compact the segments and it was executed 
> successfully and compacted the segment ids. On show segment It produced the 
> result with success for compaction . I have attached the screen shot for the 
> same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CARBONDATA-619) Compaction API for Spark 2.1 : Issue in compaction type

2017-01-16 Thread Divya Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Divya Gupta updated CARBONDATA-619:
---
Environment: Spark 2.1

> Compaction API for Spark 2.1 : Issue in compaction type
> ---
>
> Key: CARBONDATA-619
> URL: https://issues.apache.org/jira/browse/CARBONDATA-619
> Project: CarbonData
>  Issue Type: Bug
>  Components: data-query
>Affects Versions: 1.0.0-incubating
> Environment: Spark 2.1
>Reporter: DEEPAK MEHRA
>Assignee: ravikiran
>Priority: Minor
> Attachments: compaction_result.png, employee.csv
>
>
> Compaction API for Spark 2.1 : Compaction is successful on the segment ids 
> irrespective of the compaction types which are( Minor and Major) i.e The 
> compaction was successful for type other than Minor/Major 
> I created a table with name carbon_table and made 5 loads into the table with 
> the threshold of (4,3)
> cc.sql("create table carbon_table(id int,name string) stored by 'carbondata'")
> cc.sql("LOAD DATA LOCAL INPATH 'hdfs://hadoop-master:54311/data/employee.csv'
> INTO TABLE carbon_table options('FILEHEADER'='id,name')")
> For instance 
> Compaction.compaction(spark,"default","carbon_table","xyz")
> here 
> default is the database name
> carbon_table is the table name 
> xyz is the compaction type
> I invoked the above method to compact the segments and it was executed 
> successfully and compacted the segment ids. On show segment It produced the 
> result with success for compaction . I have attached the screen shot for the 
> same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)