[jira] [Reopened] (CARBONDATA-867) Shows an error on select query with date & timestamp datatype after altering the table.
[
https://issues.apache.org/jira/browse/CARBONDATA-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO reopened CARBONDATA-867:
--
select distinct(date1) from uniqdata1 ;
it is displaying NULL value instead of date.
> Shows an error on select query with date & timestamp datatype after altering
> the table.
> ---
>
> Key: CARBONDATA-867
> URL: https://issues.apache.org/jira/browse/CARBONDATA-867
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(tmpstmp date)
> TBLPROPERTIES('DEFAULT.VALUE.tmpstmp'= '2017-01-01');
> 0: jdbc:hive2://192.168.2.126:1> select distinct(tmpstmp) from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 44.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 44.0 (TID 1038, localhost, executor driver):
> java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Long
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDirectDictionaryData(RestructureBasedVectorResultCollector.java:151)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingDimensions(RestructureBasedVectorResultCollector.java:114)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:97)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Driver stacktrace: (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Closed] (CARBONDATA-864) After adding column using alter query, when we put any column in "Dictionary Exclude" then perform select query on that column then it will throws an exception.
[
https://issues.apache.org/jira/browse/CARBONDATA-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO closed CARBONDATA-864.
Resolution: Fixed
> After adding column using alter query, when we put any column in "Dictionary
> Exclude" then perform select query on that column then it will throws an
> exception.
>
>
> Key: CARBONDATA-864
> URL: https://issues.apache.org/jira/browse/CARBONDATA-864
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv, error.png
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(dict int)
> TBLPROPERTIES('DICTIONARY_INCLUDE'='dict','DEFAULT.VALUE.dict'= '');
> select distinct(dict) from uniqdata1 ;
> when we perform select query on "DICTIONARY_INCLUDE" it is working
> but when we perform select query on "DICTIONARY_EXCLUDE" then it will throws
> an exception:
> alter table uniqdata1 add columns(nodict string)
> TBLPROPERTIES('DICTIONARY_EXCLUDE'='nodict', 'DEFAULT.VALUE.NoDict'= 'abcd');
> select distinct(nodict) from uniqdata1 ;
> 0: jdbc:hive2://192.168.2.126:1> select distinct(nodict) from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 21.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 21.0 (TID 419, localhost, executor driver):
> java.lang.ArrayIndexOutOfBoundsException: 4186
> at
> org.apache.spark.sql.execution.vectorized.OnHeapColumnVector.putByteArray(OnHeapColumnVector.java:401)
> at
> org.apache.spark.sql.execution.vectorized.ColumnVector.putByteArray(ColumnVector.java:569)
> at
> org.apache.carbondata.spark.vectorreader.ColumnarVectorWrapper.putBytes(ColumnarVectorWrapper.java:77)
> at
> org.apache.carbondata.spark.vectorreader.ColumnarVectorWrapper.putBytes(ColumnarVectorWrapper.java:83)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillNoDictionaryData(RestructureBasedVectorResultCollector.java:167)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingDimensions(RestructureBasedVectorResultCollector.java:122)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:97)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
> at
>
[jira] [Closed] (CARBONDATA-867) Shows an error on select query with date & timestamp datatype after altering the table.
[
https://issues.apache.org/jira/browse/CARBONDATA-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO closed CARBONDATA-867.
Resolution: Fixed
> Shows an error on select query with date & timestamp datatype after altering
> the table.
> ---
>
> Key: CARBONDATA-867
> URL: https://issues.apache.org/jira/browse/CARBONDATA-867
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(tmpstmp date)
> TBLPROPERTIES('DEFAULT.VALUE.tmpstmp'= '2017-01-01');
> 0: jdbc:hive2://192.168.2.126:1> select distinct(tmpstmp) from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 44.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 44.0 (TID 1038, localhost, executor driver):
> java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Long
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDirectDictionaryData(RestructureBasedVectorResultCollector.java:151)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingDimensions(RestructureBasedVectorResultCollector.java:114)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:97)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Driver stacktrace: (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Closed] (CARBONDATA-877) String datatype is throwing an error when included in DIctionary_Exclude in a alter query
[
https://issues.apache.org/jira/browse/CARBONDATA-877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO closed CARBONDATA-877.
> String datatype is throwing an error when included in DIctionary_Exclude in a
> alter query
> -
>
> Key: CARBONDATA-877
> URL: https://issues.apache.org/jira/browse/CARBONDATA-877
> Project: CarbonData
> Issue Type: Bug
>Reporter: SWATI RAO
>Assignee: Kunal Kapoor
> Fix For: 1.1.0-incubating
>
> Attachments: 2000_UniqData.csv
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 drop columns(CUST_NAME);
> alter table uniqdata1 add columns(CUST_NAME string)
> TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME',
> 'DEFAULT.VALUE.CUST_NAME'='testuser') ;
> Column added successfully. But when we execute:
> select distinct(CUST_NAME) from uniqdata1 ;
> &
> select count(CUST_NAME) from uniqdata1 ;
> it throws an error :
> "Job aborted due to stage failure: Task 0 in stage 9.0 failed 1 times, most
> recent failure: Lost task 0.0 in stage 9.0 (TID 206, localhost, executor
> driver): java.lang.ArrayIndexOutOfBoundsException: 4186"
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CARBONDATA-868) Select query on decimal datatype is not working fine after adding decimal column using alter
[
https://issues.apache.org/jira/browse/CARBONDATA-868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974139#comment-15974139
]
SWATI RAO commented on CARBONDATA-868:
--
@Srigopal Mohanty : Okay I will check it once more and will get back to you.
> Select query on decimal datatype is not working fine after adding decimal
> column using alter
>
>
> Key: CARBONDATA-868
> URL: https://issues.apache.org/jira/browse/CARBONDATA-868
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(msrField
> decimal(5,2))TBLPROPERTIES('DEFAULT.VALUE.msrfield'= '123.45');
> 0: jdbc:hive2://192.168.2.126:1> select msrField from uniqdata1;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 48.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 48.0 (TID 1041, localhost, executor driver):
> java.lang.ArrayIndexOutOfBoundsException: 4186
> at
> org.apache.spark.sql.execution.vectorized.OnHeapColumnVector.putInt(OnHeapColumnVector.java:202)
> at
> org.apache.spark.sql.execution.vectorized.ColumnVector.putDecimal(ColumnVector.java:608)
> at
> org.apache.carbondata.spark.vectorreader.ColumnarVectorWrapper.putDecimal(ColumnarVectorWrapper.java:58)
> at
> org.apache.carbondata.spark.vectorreader.ColumnarVectorWrapper.putDecimals(ColumnarVectorWrapper.java:64)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingMeasures(RestructureBasedVectorResultCollector.java:202)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:98)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:231)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:225)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:826)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:826)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
[jira] [Updated] (CARBONDATA-918) Select query is not working for Complex datatype
[
https://issues.apache.org/jira/browse/CARBONDATA-918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-918:
-
Environment: Spark 2.1 (was: Spark 1.6)
> Select query is not working for Complex datatype
>
>
> Key: CARBONDATA-918
> URL: https://issues.apache.org/jira/browse/CARBONDATA-918
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.0.0-incubating
> Environment: Spark 2.1
>Reporter: SWATI RAO
> Attachments: Array.csv
>
>
> Select Query is not working in carbondata for complex datatype:
> CARBONDATA:
> 0: jdbc:hive2://hadoop-master:1> create table Array_com (CUST_ID string,
> YEAR int, MONTH int, AGE int, GENDER string, EDUCATED string, IS_MARRIED
> string, ARRAY_INT array,ARRAY_STRING array,ARRAY_DATE
> array,CARD_COUNT int,DEBIT_COUNT int, CREDIT_COUNT int, DEPOSIT
> double, HQ_DEPOSIT double) STORED BY 'org.apache.carbondata.format' ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (3.567 seconds)
> 0: jdbc:hive2://hadoop-master:1> LOAD DATA INPATH
> 'hdfs://192.168.2.145:54310/BabuStore/Data/complex/Array.csv' INTO table
> Array_com options ('DELIMITER'=',', 'QUOTECHAR'='"',
> 'FILEHEADER'='CUST_ID,YEAR,MONTH,AGE,GENDER,EDUCATED,IS_MARRIED,ARRAY_INT,ARRAY_STRING,ARRAY_DATE,CARD_COUNT,DEBIT_COUNT,CREDIT_COUNT,DEPOSIT,HQ_DEPOSIT','COMPLEX_DELIMITER_LEVEL_1'='$')
> ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (6.541 seconds)
> 0: jdbc:hive2://hadoop-master:1> select array_int[0], array_int[0]+ 10 as
> a from array_com ;
> 0: jdbc:hive2://192.168.2.126:1> select * from array_com ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 1 in stage 254.0 failed 1 times, most recent failure: Lost task 1.0 in
> stage 254.0 (TID 8631, localhost, executor driver):
> java.lang.NullPointerException
> at
> org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getLastModifiedTime(AbstractDFSCarbonFile.java:135)
> at
> org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.isFileModified(AbstractDFSCarbonFile.java:210)
> at
> org.apache.carbondata.core.cache.dictionary.AbstractDictionaryCache.isDictionaryMetaFileModified(AbstractDictionaryCache.java:119)
> at
> org.apache.carbondata.core.cache.dictionary.AbstractDictionaryCache.checkAndLoadDictionaryData(AbstractDictionaryCache.java:159)
> at
> org.apache.carbondata.core.cache.dictionary.ForwardDictionaryCache.getDictionary(ForwardDictionaryCache.java:195)
> at
> org.apache.carbondata.core.cache.dictionary.ForwardDictionaryCache.get(ForwardDictionaryCache.java:71)
> at
> org.apache.carbondata.core.cache.dictionary.ForwardDictionaryCache.get(ForwardDictionaryCache.java:40)
> at
> org.apache.carbondata.hadoop.readsupport.impl.DictionaryDecodeReadSupport.initialize(DictionaryDecodeReadSupport.java:65)
> at
> org.apache.carbondata.spark.readsupport.SparkRowReadSupportImpl.initialize(SparkRowReadSupportImpl.java:33)
> at
> org.apache.carbondata.hadoop.CarbonRecordReader.initialize(CarbonRecordReader.java:76)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.scala:204)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Driver stacktrace: (state=,code=0)
> HIVE:
> create table Array_com_h (CUST_ID string, YEAR int, MONTH int, AGE int,
> GENDER string, EDUCATED string, IS_MARRIED string, ARRAY_INT
> array,ARRAY_STRING array,ARRAY_DATE array,CARD_COUNT
> int,DEBIT_COUNT int, CREDIT_COUNT int, DEPOSIT double, HQ_DEPOSIT double);
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (6.541 seconds)
> load data local inpath
> '/opt/Carbon/CarbonData/TestData/Data/complex/Array.csv' INTO table
> Array_com_h;
> +-+--+
[jira] [Updated] (CARBONDATA-904) ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CARBONDATA-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-904:
-
Attachment: Test_Data1.csv
Test_Data1_h1.csv
Environment: Spark1.6
Request participants: (was: )
Fix Version/s: 1.1.0-incubating
> ArrayIndexOutOfBoundsException
> ---
>
> Key: CARBONDATA-904
> URL: https://issues.apache.org/jira/browse/CARBONDATA-904
> Project: CarbonData
> Issue Type: Bug
> Environment: Spark1.6
>Reporter: SWATI RAO
> Fix For: 1.1.0-incubating
>
> Attachments: Test_Data1_h1.csv, Test_Data1_h1.csv
>
>
> Or operator is not working properly.
> When we execute these query in hive it is working fine but when we execute
> the same in carbondata it throws an exception:
> java.lang.ArrayIndexOutOfBoundsException
> HIVE:
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary_h1 (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ',' ;
> +-+--+
> | result |
> +-+--+
> +-+--+
> No rows selected (1.177 seconds)
> 0: jdbc:hive2://hadoop-master:1> load data local inpath
> '/opt/Carbon/CarbonData/TestData/Data/Test_Data1_h1.csv' OVERWRITE INTO TABLE
> Test_Boundary_h1 ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.437 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp)
> from Test_Boundary_h1 where c6_Timestamp ='2017-07-01 12:07:28' or
> c6_Timestamp ='2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29'
> group by c6_Timestamp ;
> +++--+
> | c6_Timestamp | _c1 |
> +++--+
> | 2017-07-01 12:07:28.0 | 2017-07-01 12:07:28.0 |
> +++--+
> 1 row selected (1.637 seconds)
> CARBONDATA:
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format' ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (4.48 seconds)
> 0: jdbc:hive2://hadoop-master:1> LOAD DATA INPATH
> 'hdfs://192.168.2.145:54310/BabuStore/Data/Test_Data1.csv' INTO table
> Test_Boundary
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (4.445 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp)
> from Test_Boundary where c6_Timestamp ='2017-07-01 12:07:28' or c6_Timestamp
> =' 2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29' group by
> c6_Timestamp ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 5.0 failed 4 times, most recent failure: Lost task 0.3 in
> stage 5.0 (TID 8, hadoop-master): java.lang.RuntimeException:
> java.util.concurrent.ExecutionException:
> java.lang.ArrayIndexOutOfBoundsException: 0
> at
> org.apache.carbondata.core.scan.processor.AbstractDataBlockIterator.updateScanner(AbstractDataBlockIterator.java:136)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:50)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:32)
> at
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.getBatchResult(DetailQueryResultIterator.java:50)
> at
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:41)
> at
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:31)
> at
> org.apache.carbondata.core.scan.result.iterator.ChunkRowIterator.(ChunkRowIterator.java:41)
> at
> org.apache.carbondata.hadoop.CarbonRecordReader.initialize(CarbonRecordReader.java:79)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.scala:204)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at
>
[jira] [Updated] (CARBONDATA-904) ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CARBONDATA-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-904:
-
Attachment: (was: Test_Data1.csv)
> ArrayIndexOutOfBoundsException
> ---
>
> Key: CARBONDATA-904
> URL: https://issues.apache.org/jira/browse/CARBONDATA-904
> Project: CarbonData
> Issue Type: Bug
> Environment: Spark1.6
>Reporter: SWATI RAO
> Fix For: 1.1.0-incubating
>
> Attachments: Test_Data1_h1.csv, Test_Data1_h1.csv
>
>
> Or operator is not working properly.
> When we execute these query in hive it is working fine but when we execute
> the same in carbondata it throws an exception:
> java.lang.ArrayIndexOutOfBoundsException
> HIVE:
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary_h1 (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ',' ;
> +-+--+
> | result |
> +-+--+
> +-+--+
> No rows selected (1.177 seconds)
> 0: jdbc:hive2://hadoop-master:1> load data local inpath
> '/opt/Carbon/CarbonData/TestData/Data/Test_Data1_h1.csv' OVERWRITE INTO TABLE
> Test_Boundary_h1 ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.437 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp)
> from Test_Boundary_h1 where c6_Timestamp ='2017-07-01 12:07:28' or
> c6_Timestamp ='2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29'
> group by c6_Timestamp ;
> +++--+
> | c6_Timestamp | _c1 |
> +++--+
> | 2017-07-01 12:07:28.0 | 2017-07-01 12:07:28.0 |
> +++--+
> 1 row selected (1.637 seconds)
> CARBONDATA:
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format' ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (4.48 seconds)
> 0: jdbc:hive2://hadoop-master:1> LOAD DATA INPATH
> 'hdfs://192.168.2.145:54310/BabuStore/Data/Test_Data1.csv' INTO table
> Test_Boundary
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (4.445 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp)
> from Test_Boundary where c6_Timestamp ='2017-07-01 12:07:28' or c6_Timestamp
> =' 2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29' group by
> c6_Timestamp ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 5.0 failed 4 times, most recent failure: Lost task 0.3 in
> stage 5.0 (TID 8, hadoop-master): java.lang.RuntimeException:
> java.util.concurrent.ExecutionException:
> java.lang.ArrayIndexOutOfBoundsException: 0
> at
> org.apache.carbondata.core.scan.processor.AbstractDataBlockIterator.updateScanner(AbstractDataBlockIterator.java:136)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:50)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:32)
> at
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.getBatchResult(DetailQueryResultIterator.java:50)
> at
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:41)
> at
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:31)
> at
> org.apache.carbondata.core.scan.result.iterator.ChunkRowIterator.(ChunkRowIterator.java:41)
> at
> org.apache.carbondata.hadoop.CarbonRecordReader.initialize(CarbonRecordReader.java:79)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.scala:204)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at
>
[jira] [Updated] (CARBONDATA-904) ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CARBONDATA-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-904:
-
Attachment: (was: Test_Data1.csv)
> ArrayIndexOutOfBoundsException
> ---
>
> Key: CARBONDATA-904
> URL: https://issues.apache.org/jira/browse/CARBONDATA-904
> Project: CarbonData
> Issue Type: Bug
> Environment: Spark1.6
>Reporter: SWATI RAO
> Fix For: 1.1.0-incubating
>
> Attachments: Test_Data1_h1.csv, Test_Data1_h1.csv
>
>
> Or operator is not working properly.
> When we execute these query in hive it is working fine but when we execute
> the same in carbondata it throws an exception:
> java.lang.ArrayIndexOutOfBoundsException
> HIVE:
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary_h1 (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ',' ;
> +-+--+
> | result |
> +-+--+
> +-+--+
> No rows selected (1.177 seconds)
> 0: jdbc:hive2://hadoop-master:1> load data local inpath
> '/opt/Carbon/CarbonData/TestData/Data/Test_Data1_h1.csv' OVERWRITE INTO TABLE
> Test_Boundary_h1 ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.437 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp)
> from Test_Boundary_h1 where c6_Timestamp ='2017-07-01 12:07:28' or
> c6_Timestamp ='2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29'
> group by c6_Timestamp ;
> +++--+
> | c6_Timestamp | _c1 |
> +++--+
> | 2017-07-01 12:07:28.0 | 2017-07-01 12:07:28.0 |
> +++--+
> 1 row selected (1.637 seconds)
> CARBONDATA:
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format' ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (4.48 seconds)
> 0: jdbc:hive2://hadoop-master:1> LOAD DATA INPATH
> 'hdfs://192.168.2.145:54310/BabuStore/Data/Test_Data1.csv' INTO table
> Test_Boundary
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (4.445 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp)
> from Test_Boundary where c6_Timestamp ='2017-07-01 12:07:28' or c6_Timestamp
> =' 2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29' group by
> c6_Timestamp ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 5.0 failed 4 times, most recent failure: Lost task 0.3 in
> stage 5.0 (TID 8, hadoop-master): java.lang.RuntimeException:
> java.util.concurrent.ExecutionException:
> java.lang.ArrayIndexOutOfBoundsException: 0
> at
> org.apache.carbondata.core.scan.processor.AbstractDataBlockIterator.updateScanner(AbstractDataBlockIterator.java:136)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:50)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:32)
> at
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.getBatchResult(DetailQueryResultIterator.java:50)
> at
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:41)
> at
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:31)
> at
> org.apache.carbondata.core.scan.result.iterator.ChunkRowIterator.(ChunkRowIterator.java:41)
> at
> org.apache.carbondata.hadoop.CarbonRecordReader.initialize(CarbonRecordReader.java:79)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.scala:204)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at
>
[jira] [Created] (CARBONDATA-904) ArrayIndexOutOfBoundsException
SWATI RAO created CARBONDATA-904:
Summary: ArrayIndexOutOfBoundsException
Key: CARBONDATA-904
URL: https://issues.apache.org/jira/browse/CARBONDATA-904
Project: CarbonData
Issue Type: Bug
Reporter: SWATI RAO
Attachments: Test_Data1.csv, Test_Data1_h1.csv
Or operator is not working properly.
When we execute these query in hive it is working fine but when we execute the
same in carbondata it throws an exception:
java.lang.ArrayIndexOutOfBoundsException
HIVE:
0: jdbc:hive2://hadoop-master:1> create table Test_Boundary_h1 (c1_int
int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
string,c6_Timestamp Timestamp,c7_Datatype_Desc string) ROW FORMAT DELIMITED
FIELDS TERMINATED BY ',' ;
+-+--+
| result |
+-+--+
+-+--+
No rows selected (1.177 seconds)
0: jdbc:hive2://hadoop-master:1> load data local inpath
'/opt/Carbon/CarbonData/TestData/Data/Test_Data1_h1.csv' OVERWRITE INTO TABLE
Test_Boundary_h1 ;
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.437 seconds)
0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp) from
Test_Boundary_h1 where c6_Timestamp ='2017-07-01 12:07:28' or c6_Timestamp
='2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29' group by
c6_Timestamp ;
+++--+
| c6_Timestamp | _c1 |
+++--+
| 2017-07-01 12:07:28.0 | 2017-07-01 12:07:28.0 |
+++--+
1 row selected (1.637 seconds)
CARBONDATA:
0: jdbc:hive2://hadoop-master:1> create table Test_Boundary (c1_int
int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
'org.apache.carbondata.format' ;
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (4.48 seconds)
0: jdbc:hive2://hadoop-master:1> LOAD DATA INPATH
'hdfs://192.168.2.145:54310/BabuStore/Data/Test_Data1.csv' INTO table
Test_Boundary
OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
;
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (4.445 seconds)
0: jdbc:hive2://hadoop-master:1> select c6_Timestamp,max(c6_Timestamp) from
Test_Boundary where c6_Timestamp ='2017-07-01 12:07:28' or c6_Timestamp ='
2019-07-05 13:07:30' or c6_Timestamp = '1999-01-06 10:05:29' group by
c6_Timestamp ;
Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task
0 in stage 5.0 failed 4 times, most recent failure: Lost task 0.3 in stage 5.0
(TID 8, hadoop-master): java.lang.RuntimeException:
java.util.concurrent.ExecutionException:
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.carbondata.core.scan.processor.AbstractDataBlockIterator.updateScanner(AbstractDataBlockIterator.java:136)
at
org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:50)
at
org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:32)
at
org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.getBatchResult(DetailQueryResultIterator.java:50)
at
org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:41)
at
org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:31)
at
org.apache.carbondata.core.scan.result.iterator.ChunkRowIterator.(ChunkRowIterator.java:41)
at
org.apache.carbondata.hadoop.CarbonRecordReader.initialize(CarbonRecordReader.java:79)
at
org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.scala:204)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
at
[jira] [Updated] (CARBONDATA-883) Select query after alter is not working fine with integer datatype.
[
https://issues.apache.org/jira/browse/CARBONDATA-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-883:
-
Description:
CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string,
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB");
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
ALTER TABLE uniqdata RENAME TO uniqdata1;
0: jdbc:hive2://192.168.2.126:1> alter table default.uniqdata1 add
columns(ACTIVE_EMUI_VERSION int)
TBLPROPERTIES('DEFAULT.VALUE.ACTIVE_EMUI_VERSION'='67890');
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.265 seconds)
0: jdbc:hive2://192.168.2.126:1> select distinct(ACTIVE_EMUI_VERSION) from
uniqdata1 ;
Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task
0 in stage 22.0 failed 1 times, most recent failure: Lost task 0.0 in stage
22.0 (TID 1211, localhost, executor driver): java.lang.ClassCastException:
java.lang.Long cannot be cast to java.lang.Integer
at
org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingMeasures(RestructureBasedVectorResultCollector.java:193)
at
org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:98)
at
org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
at
org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
at
org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
at
org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
at
org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
Source)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
Source)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
Source)
at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at
org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Driver stacktrace: (state=,code=0)
0: jdbc:hive2://192.168.2.126:1>
> Select query after alter is not working fine with integer datatype.
> ---
>
> Key: CARBONDATA-883
> URL: https://issues.apache.org/jira/browse/CARBONDATA-883
> Project: CarbonData
> Issue Type: Bug
> Environment: CREATE TABLE uniqdata (CUST_ID int,CUST_NAME
> String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp,
> BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
> DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2
> double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format'
> TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
>
[jira] [Created] (CARBONDATA-883) Select query after alter is not working fine with integer datatype.
SWATI RAO created CARBONDATA-883:
Summary: Select query after alter is not working fine with integer
datatype.
Key: CARBONDATA-883
URL: https://issues.apache.org/jira/browse/CARBONDATA-883
Project: CarbonData
Issue Type: Bug
Environment: CREATE TABLE uniqdata (CUST_ID int,CUST_NAME
String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1
bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB");
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
ALTER TABLE uniqdata RENAME TO uniqdata1;
0: jdbc:hive2://192.168.2.126:1> alter table default.uniqdata1 add
columns(ACTIVE_EMUI_VERSION int)
TBLPROPERTIES('DEFAULT.VALUE.ACTIVE_EMUI_VERSION'='67890');
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.265 seconds)
0: jdbc:hive2://192.168.2.126:1> select distinct(ACTIVE_EMUI_VERSION) from
uniqdata1 ;
Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task
0 in stage 22.0 failed 1 times, most recent failure: Lost task 0.0 in stage
22.0 (TID 1211, localhost, executor driver): java.lang.ClassCastException:
java.lang.Long cannot be cast to java.lang.Integer
at
org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingMeasures(RestructureBasedVectorResultCollector.java:193)
at
org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:98)
at
org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
at
org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
at
org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
at
org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
at
org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
Source)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
Source)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
Source)
at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at
org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Driver stacktrace: (state=,code=0)
0: jdbc:hive2://192.168.2.126:1>
Reporter: SWATI RAO
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Issue Comment Deleted] (CARBONDATA-867) Shows an error on select query with date & timestamp datatype after altering the table.
[
https://issues.apache.org/jira/browse/CARBONDATA-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-867:
-
Comment: was deleted
(was: Now it displays "NULL" values
0: jdbc:hive2://192.168.2.126:1> alter table uniqdata1 drop
columns(ACTIVE_EMUI_VERSION);
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.28 seconds)
0: jdbc:hive2://192.168.2.126:1> alter table uniqdata1 add
columns(ACTIVE_EMUI_VERSION timestamp) TBLPROPERTIES
('DEFAULT.VALUE.ACTIVE_EMUI_VERSION'= '2017-01-01');
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.273 seconds)
0: jdbc:hive2://192.168.2.126:1> select distinct(ACTIVE_EMUI_VERSION) from
uniqdata1 ;
+--+--+
| ACTIVE_EMUI_VERSION |
+--+--+
| NULL |
+--+--+
1 row selected (0.816 seconds)
0: jdbc:hive2://192.168.2.126:1>
)
> Shows an error on select query with date & timestamp datatype after altering
> the table.
> ---
>
> Key: CARBONDATA-867
> URL: https://issues.apache.org/jira/browse/CARBONDATA-867
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(tmpstmp date)
> TBLPROPERTIES('DEFAULT.VALUE.tmpstmp'= '2017-01-01');
> 0: jdbc:hive2://192.168.2.126:1> select distinct(tmpstmp) from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 44.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 44.0 (TID 1038, localhost, executor driver):
> java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Long
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDirectDictionaryData(RestructureBasedVectorResultCollector.java:151)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingDimensions(RestructureBasedVectorResultCollector.java:114)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:97)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at
[jira] [Commented] (CARBONDATA-867) Shows an error on select query with date & timestamp datatype after altering the table.
[
https://issues.apache.org/jira/browse/CARBONDATA-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960436#comment-15960436
]
SWATI RAO commented on CARBONDATA-867:
--
0: jdbc:hive2://192.168.2.126:1> alter table uniqdata1 drop
columns(ACTIVE_EMUI_VERSION);
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.276 seconds)
0: jdbc:hive2://192.168.2.126:1> alter table uniqdata1 add
columns(ACTIVE_EMUI_VERSION timestamp) TBLPROPERTIES
('DEFAULT.VALUE.ACTIVE_EMUI_VERSION'= '2017-01-01 00-01-00');
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.279 seconds)
0: jdbc:hive2://192.168.2.126:1> select distinct(ACTIVE_EMUI_VERSION) from
uniqdata1 ;
+--+--+
| ACTIVE_EMUI_VERSION |
+--+--+
| NULL |
+--+--+
1 row selected (0.687 seconds)
0: jdbc:hive2://192.168.2.126:1>
> Shows an error on select query with date & timestamp datatype after altering
> the table.
> ---
>
> Key: CARBONDATA-867
> URL: https://issues.apache.org/jira/browse/CARBONDATA-867
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(tmpstmp date)
> TBLPROPERTIES('DEFAULT.VALUE.tmpstmp'= '2017-01-01');
> 0: jdbc:hive2://192.168.2.126:1> select distinct(tmpstmp) from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 44.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 44.0 (TID 1038, localhost, executor driver):
> java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Long
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDirectDictionaryData(RestructureBasedVectorResultCollector.java:151)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingDimensions(RestructureBasedVectorResultCollector.java:114)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:97)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at
[jira] [Commented] (CARBONDATA-867) Shows an error on select query with date & timestamp datatype after altering the table.
[
https://issues.apache.org/jira/browse/CARBONDATA-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960434#comment-15960434
]
SWATI RAO commented on CARBONDATA-867:
--
Now it displays "NULL" values
0: jdbc:hive2://192.168.2.126:1> alter table uniqdata1 drop
columns(ACTIVE_EMUI_VERSION);
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.28 seconds)
0: jdbc:hive2://192.168.2.126:1> alter table uniqdata1 add
columns(ACTIVE_EMUI_VERSION timestamp) TBLPROPERTIES
('DEFAULT.VALUE.ACTIVE_EMUI_VERSION'= '2017-01-01');
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.273 seconds)
0: jdbc:hive2://192.168.2.126:1> select distinct(ACTIVE_EMUI_VERSION) from
uniqdata1 ;
+--+--+
| ACTIVE_EMUI_VERSION |
+--+--+
| NULL |
+--+--+
1 row selected (0.816 seconds)
0: jdbc:hive2://192.168.2.126:1>
> Shows an error on select query with date & timestamp datatype after altering
> the table.
> ---
>
> Key: CARBONDATA-867
> URL: https://issues.apache.org/jira/browse/CARBONDATA-867
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(tmpstmp date)
> TBLPROPERTIES('DEFAULT.VALUE.tmpstmp'= '2017-01-01');
> 0: jdbc:hive2://192.168.2.126:1> select distinct(tmpstmp) from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 44.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 44.0 (TID 1038, localhost, executor driver):
> java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Long
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDirectDictionaryData(RestructureBasedVectorResultCollector.java:151)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingDimensions(RestructureBasedVectorResultCollector.java:114)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:97)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at
[jira] [Issue Comment Deleted] (CARBONDATA-864) After adding column using alter query, when we put any column in "Dictionary Exclude" then perform select query on that column then it will throws an ex
[
https://issues.apache.org/jira/browse/CARBONDATA-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-864:
-
Comment: was deleted
(was: When we put string datatype in dictionary_exclude it will display "NULL"
values when perform select query.
CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string,
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB");
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');LOAD
DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into table
uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
ALTER TABLE uniqdata RENAME TO uniqdata1;
alter table uniqdata1 drop columns(ACTIVE_EMUI_VERSION);
alter table uniqdata1 add columns(ACTIVE_EMUI_VERSION string)
TBLPROPERTIES('DICTIONARY_EXCLUDE'='ACTIVE_EMUI_VERSION',
'DEFAULT.VALUE.(ACTIVE_EMUI_VERSION'='abcd');
select distinct(ACTIVE_EMUI_VERSION) from uniqdata1 ;
Results:
0: jdbc:hive2://192.168.2.126:1> alter table uniqdata1 drop
columns(ACTIVE_EMUI_VERSION);
Error: java.lang.RuntimeException: Column active_emui_version does not exists
in the table default.uniqdata1 (state=,code=0)
0: jdbc:hive2://192.168.2.126:1>
0: jdbc:hive2://192.168.2.126:1> alter table uniqdata1 add
columns(ACTIVE_EMUI_VERSION string)
TBLPROPERTIES('DICTIONARY_EXCLUDE'='ACTIVE_EMUI_VERSION',
'DEFAULT.VALUE.(ACTIVE_EMUI_VERSION'='abcd');
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.308 seconds)
0: jdbc:hive2://192.168.2.126:1> select distinct(ACTIVE_EMUI_VERSION) from
uniqdata1 ;
+--+--+
| ACTIVE_EMUI_VERSION |
+--+--+
| NULL |
+--+--+
1 row selected (0.88 seconds)
0: jdbc:hive2://192.168.2.126:1>
)
> After adding column using alter query, when we put any column in "Dictionary
> Exclude" then perform select query on that column then it will throws an
> exception.
>
>
> Key: CARBONDATA-864
> URL: https://issues.apache.org/jira/browse/CARBONDATA-864
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv, error.png
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(dict int)
> TBLPROPERTIES('DICTIONARY_INCLUDE'='dict','DEFAULT.VALUE.dict'= '');
> select distinct(dict) from uniqdata1 ;
> when we perform select query on "DICTIONARY_INCLUDE" it is working
> but when we perform select query on "DICTIONARY_EXCLUDE" then it will throws
> an exception:
> alter table uniqdata1 add columns(nodict string)
> TBLPROPERTIES('DICTIONARY_EXCLUDE'='nodict', 'DEFAULT.VALUE.NoDict'= 'abcd');
> select distinct(nodict) from uniqdata1 ;
> 0: jdbc:hive2://192.168.2.126:1> select distinct(nodict) from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 21.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 21.0 (TID 419, localhost, executor driver):
> java.lang.ArrayIndexOutOfBoundsException: 4186
> at
> org.apache.spark.sql.execution.vectorized.OnHeapColumnVector.putByteArray(OnHeapColumnVector.java:401)
> at
>
[jira] [Commented] (CARBONDATA-864) After adding column using alter query, when we put any column in "Dictionary Exclude" then perform select query on that column then it will throws an exception.
[
https://issues.apache.org/jira/browse/CARBONDATA-864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960416#comment-15960416
]
SWATI RAO commented on CARBONDATA-864:
--
When we put string datatype in dictionary_exclude it will display "NULL" values
when perform select query.
CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string,
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB");
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');LOAD
DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into table
uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
ALTER TABLE uniqdata RENAME TO uniqdata1;
alter table uniqdata1 drop columns(ACTIVE_EMUI_VERSION);
alter table uniqdata1 add columns(ACTIVE_EMUI_VERSION string)
TBLPROPERTIES('DICTIONARY_EXCLUDE'='ACTIVE_EMUI_VERSION',
'DEFAULT.VALUE.(ACTIVE_EMUI_VERSION'='abcd');
select distinct(ACTIVE_EMUI_VERSION) from uniqdata1 ;
Results:
0: jdbc:hive2://192.168.2.126:1> alter table uniqdata1 drop
columns(ACTIVE_EMUI_VERSION);
Error: java.lang.RuntimeException: Column active_emui_version does not exists
in the table default.uniqdata1 (state=,code=0)
0: jdbc:hive2://192.168.2.126:1>
0: jdbc:hive2://192.168.2.126:1> alter table uniqdata1 add
columns(ACTIVE_EMUI_VERSION string)
TBLPROPERTIES('DICTIONARY_EXCLUDE'='ACTIVE_EMUI_VERSION',
'DEFAULT.VALUE.(ACTIVE_EMUI_VERSION'='abcd');
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.308 seconds)
0: jdbc:hive2://192.168.2.126:1> select distinct(ACTIVE_EMUI_VERSION) from
uniqdata1 ;
+--+--+
| ACTIVE_EMUI_VERSION |
+--+--+
| NULL |
+--+--+
1 row selected (0.88 seconds)
0: jdbc:hive2://192.168.2.126:1>
> After adding column using alter query, when we put any column in "Dictionary
> Exclude" then perform select query on that column then it will throws an
> exception.
>
>
> Key: CARBONDATA-864
> URL: https://issues.apache.org/jira/browse/CARBONDATA-864
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv, error.png
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(dict int)
> TBLPROPERTIES('DICTIONARY_INCLUDE'='dict','DEFAULT.VALUE.dict'= '');
> select distinct(dict) from uniqdata1 ;
> when we perform select query on "DICTIONARY_INCLUDE" it is working
> but when we perform select query on "DICTIONARY_EXCLUDE" then it will throws
> an exception:
> alter table uniqdata1 add columns(nodict string)
> TBLPROPERTIES('DICTIONARY_EXCLUDE'='nodict', 'DEFAULT.VALUE.NoDict'= 'abcd');
> select distinct(nodict) from uniqdata1 ;
> 0: jdbc:hive2://192.168.2.126:1> select distinct(nodict) from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 21.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 21.0 (TID 419, localhost, executor driver):
> java.lang.ArrayIndexOutOfBoundsException: 4186
> at
> org.apache.spark.sql.execution.vectorized.OnHeapColumnVector.putByteArray(OnHeapColumnVector.java:401)
> at
>
[jira] [Updated] (CARBONDATA-877) String datatype is throwing an error when included in DIctionary_Exclude in a alter query
[
https://issues.apache.org/jira/browse/CARBONDATA-877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-877:
-
Description:
CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string,
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB");
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
ALTER TABLE uniqdata RENAME TO uniqdata1;
alter table uniqdata1 drop columns(CUST_NAME);
alter table uniqdata1 add columns(CUST_NAME string)
TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME',
'DEFAULT.VALUE.CUST_NAME'='testuser') ;
Column added successfully. But when we execute:
select distinct(CUST_NAME) from uniqdata1 ;
&
select count(CUST_NAME) from uniqdata1 ;
it throws an error :
"Job aborted due to stage failure: Task 0 in stage 9.0 failed 1 times, most
recent failure: Lost task 0.0 in stage 9.0 (TID 206, localhost, executor
driver): java.lang.ArrayIndexOutOfBoundsException: 4186"
was:
CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string,
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB");
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
ALTER TABLE uniqdata RENAME TO uniqdata1;
alter table uniqdata1 drop columns(CUST_NAME);
alter table uniqdata1 add columns(CUST_NAME string)
TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME',
'DEFAULT.VALUE.CUST_NAME'='testuser') ;
Column added successfully. But when we execute:
select distinct(CUST_NAME) from uniqdata1 ;
&
select count(CUST_NAME) from uniqdata1 ;
it throws an error :
"Job aborted due to stage failure: Task 0 in stage 9.0 failed 1 times, most
recent failure: Lost task 0.0 in stage 9.0 (TID 206, localhost, executor
driver): java.lang.ArrayIndexOutOfBoundsException: 4186"
alter table uniqdata1 add columns(CUST_NAME string)
TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME',
'DEFAULT.VALUE.CUST_NAME'='testuser');
> String datatype is throwing an error when included in DIctionary_Exclude in a
> alter query
> -
>
> Key: CARBONDATA-877
> URL: https://issues.apache.org/jira/browse/CARBONDATA-877
> Project: CarbonData
> Issue Type: Bug
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 drop columns(CUST_NAME);
> alter table uniqdata1 add columns(CUST_NAME string)
> TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME',
> 'DEFAULT.VALUE.CUST_NAME'='testuser') ;
> Column added successfully. But when we execute:
> select distinct(CUST_NAME) from uniqdata1 ;
> &
> select count(CUST_NAME) from uniqdata1 ;
> it throws an error :
> "Job aborted due to stage failure: Task 0 in stage 9.0 failed 1 times, most
> recent failure: Lost task 0.0 in stage 9.0 (TID 206, localhost, executor
> driver): java.lang.ArrayIndexOutOfBoundsException: 4186"
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CARBONDATA-877) String datatype is throwing an error when included in DIctionary_Exclude in a alter query
SWATI RAO created CARBONDATA-877:
Summary: String datatype is throwing an error when included in
DIctionary_Exclude in a alter query
Key: CARBONDATA-877
URL: https://issues.apache.org/jira/browse/CARBONDATA-877
Project: CarbonData
Issue Type: Bug
Reporter: SWATI RAO
Attachments: 2000_UniqData.csv
CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string,
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB");
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
ALTER TABLE uniqdata RENAME TO uniqdata1;
alter table uniqdata1 drop columns(CUST_NAME);
alter table uniqdata1 add columns(CUST_NAME string)
TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME',
'DEFAULT.VALUE.CUST_NAME'='testuser') ;
Column added successfully. But when we execute:
select distinct(CUST_NAME) from uniqdata1 ;
&
select count(CUST_NAME) from uniqdata1 ;
it throws an error :
"Job aborted due to stage failure: Task 0 in stage 9.0 failed 1 times, most
recent failure: Lost task 0.0 in stage 9.0 (TID 206, localhost, executor
driver): java.lang.ArrayIndexOutOfBoundsException: 4186"
alter table uniqdata1 add columns(CUST_NAME string)
TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME',
'DEFAULT.VALUE.CUST_NAME'='testuser');
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CARBONDATA-868) Select query on decimal datatype is not working fine after adding decimal column using alter
SWATI RAO created CARBONDATA-868:
Summary: Select query on decimal datatype is not working fine
after adding decimal column using alter
Key: CARBONDATA-868
URL: https://issues.apache.org/jira/browse/CARBONDATA-868
Project: CarbonData
Issue Type: Bug
Affects Versions: 1.1.0-incubating
Environment: Spark2.1
Reporter: SWATI RAO
Attachments: 2000_UniqData.csv
CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string,
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB");
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
ALTER TABLE uniqdata RENAME TO uniqdata1;
alter table uniqdata1 add columns(msrField
decimal(5,2))TBLPROPERTIES('DEFAULT.VALUE.msrfield'= '123.45');
0: jdbc:hive2://192.168.2.126:1> select msrField from uniqdata1;
Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task
0 in stage 48.0 failed 1 times, most recent failure: Lost task 0.0 in stage
48.0 (TID 1041, localhost, executor driver):
java.lang.ArrayIndexOutOfBoundsException: 4186
at
org.apache.spark.sql.execution.vectorized.OnHeapColumnVector.putInt(OnHeapColumnVector.java:202)
at
org.apache.spark.sql.execution.vectorized.ColumnVector.putDecimal(ColumnVector.java:608)
at
org.apache.carbondata.spark.vectorreader.ColumnarVectorWrapper.putDecimal(ColumnarVectorWrapper.java:58)
at
org.apache.carbondata.spark.vectorreader.ColumnarVectorWrapper.putDecimals(ColumnarVectorWrapper.java:64)
at
org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingMeasures(RestructureBasedVectorResultCollector.java:202)
at
org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:98)
at
org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
at
org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
at
org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
at
org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
at
org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
Source)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
Source)
at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:231)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:225)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:826)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:826)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Driver stacktrace: (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CARBONDATA-864) After adding column using alter query, when we put any column in "Dictionary Exclude" then perform select query on that column then it will throws an exception.
[
https://issues.apache.org/jira/browse/CARBONDATA-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-864:
-
Request participants: (was: )
Summary: After adding column using alter query, when we put
any column in "Dictionary Exclude" then perform select query on that column
then it will throws an exception. (was: If we put any column in "Dictionary
Exclude" then perform select query on that column then it will throws an
exception.)
> After adding column using alter query, when we put any column in "Dictionary
> Exclude" then perform select query on that column then it will throws an
> exception.
>
>
> Key: CARBONDATA-864
> URL: https://issues.apache.org/jira/browse/CARBONDATA-864
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv, error.png
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(dict int)
> TBLPROPERTIES('DICTIONARY_INCLUDE'='dict','DEFAULT.VALUE.dict'= '');
> select distinct(dict) from uniqdata1 ;
> when we perform select query on "DICTIONARY_INCLUDE" it is working
> but when we perform select query on "DICTIONARY_EXCLUDE" then it will throws
> an exception:
> alter table uniqdata1 add columns(nodict string)
> TBLPROPERTIES('DICTIONARY_EXCLUDE'='nodict', 'DEFAULT.VALUE.NoDict'= 'abcd');
> select distinct(nodict) from uniqdata1 ;
> 0: jdbc:hive2://192.168.2.126:1> select distinct(nodict) from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 21.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 21.0 (TID 419, localhost, executor driver):
> java.lang.ArrayIndexOutOfBoundsException: 4186
> at
> org.apache.spark.sql.execution.vectorized.OnHeapColumnVector.putByteArray(OnHeapColumnVector.java:401)
> at
> org.apache.spark.sql.execution.vectorized.ColumnVector.putByteArray(ColumnVector.java:569)
> at
> org.apache.carbondata.spark.vectorreader.ColumnarVectorWrapper.putBytes(ColumnarVectorWrapper.java:77)
> at
> org.apache.carbondata.spark.vectorreader.ColumnarVectorWrapper.putBytes(ColumnarVectorWrapper.java:83)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillNoDictionaryData(RestructureBasedVectorResultCollector.java:167)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingDimensions(RestructureBasedVectorResultCollector.java:122)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:97)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
>
[jira] [Updated] (CARBONDATA-867) Shows an error on select query with date & timestamp datatype after altering the table.
[
https://issues.apache.org/jira/browse/CARBONDATA-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-867:
-
Request participants: (was: )
Summary: Shows an error on select query with date & timestamp
datatype after altering the table. (was: Shows an error on select query with
date & timestamp datatype)
> Shows an error on select query with date & timestamp datatype after altering
> the table.
> ---
>
> Key: CARBONDATA-867
> URL: https://issues.apache.org/jira/browse/CARBONDATA-867
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
> Attachments: 2000_UniqData.csv
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(tmpstmp date)
> TBLPROPERTIES('DEFAULT.VALUE.tmpstmp'= '2017-01-01');
> 0: jdbc:hive2://192.168.2.126:1> select distinct(tmpstmp) from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 44.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 44.0 (TID 1038, localhost, executor driver):
> java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Long
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDirectDictionaryData(RestructureBasedVectorResultCollector.java:151)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingDimensions(RestructureBasedVectorResultCollector.java:114)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:97)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Driver stacktrace: (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CARBONDATA-867) Shows an error on select query with date & timestamp datatype
SWATI RAO created CARBONDATA-867:
Summary: Shows an error on select query with date & timestamp
datatype
Key: CARBONDATA-867
URL: https://issues.apache.org/jira/browse/CARBONDATA-867
Project: CarbonData
Issue Type: Bug
Affects Versions: 1.1.0-incubating
Environment: Spark2.1
Reporter: SWATI RAO
Attachments: 2000_UniqData.csv
CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string,
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB");
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
ALTER TABLE uniqdata RENAME TO uniqdata1;
alter table uniqdata1 add columns(tmpstmp date)
TBLPROPERTIES('DEFAULT.VALUE.tmpstmp'= '2017-01-01');
0: jdbc:hive2://192.168.2.126:1> select distinct(tmpstmp) from uniqdata1 ;
Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task
0 in stage 44.0 failed 1 times, most recent failure: Lost task 0.0 in stage
44.0 (TID 1038, localhost, executor driver): java.lang.ClassCastException:
java.lang.Integer cannot be cast to java.lang.Long
at
org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDirectDictionaryData(RestructureBasedVectorResultCollector.java:151)
at
org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingDimensions(RestructureBasedVectorResultCollector.java:114)
at
org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:97)
at
org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
at
org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
at
org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
at
org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
at
org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
Source)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
Source)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
Source)
at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at
org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Driver stacktrace: (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CARBONDATA-847) Select query not working properly after alter.
SWATI RAO created CARBONDATA-847:
Summary: Select query not working properly after alter.
Key: CARBONDATA-847
URL: https://issues.apache.org/jira/browse/CARBONDATA-847
Project: CarbonData
Issue Type: Bug
Affects Versions: 1.1.0-incubating
Environment: Spark2.1
Reporter: SWATI RAO
Attachments: 2000_UniqData.csv
Execute these set of queries:
CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string,
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"=
"256 MB");
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniqdata OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
ALTER TABLE uniqdata RENAME TO uniqdata1;
alter table uniqdata1 add columns(dict int)
TBLPROPERTIES('DICTIONARY_INCLUDE'='dict','DEFAULT.VALUE.dict'= '');
select distinct(dict) from uniqdata2 ;
it will display the result but when we perform :
select * from uniqdata1 ;
it will display an error message :
Job aborted due to stage failure: Task 3 in stage 59.0 failed 1 times, most
recent failure: Lost task 3.0 in stage 59.0 (TID 714, localhost, executor
driver): java.lang.NullPointerException
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Closed] (CARBONDATA-370) Abnormal behaviour of datatype Int & bigInt
[ https://issues.apache.org/jira/browse/CARBONDATA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-370. Resolution: Fixed > Abnormal behaviour of datatype Int & bigInt > --- > > Key: CARBONDATA-370 > URL: https://issues.apache.org/jira/browse/CARBONDATA-370 > Project: CarbonData > Issue Type: Bug >Reporter: SWATI RAO >Assignee: anubhav tarar > > The specified data-type range for Int & bigInt is: > INT –––> -2147483648 to 2147483647 > BIGINT ––> -922337203685477580807 to 922337203685477580806 > but I found abnormal behaviour in these data-type . > 1.When I insert beyond range in Int it allows range even beyond bigInt. > 2.When I insert into bigInt it doesn’t accept all values following in its > range. > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-712) 'BAD_RECORDS_ACTION'='REDIRECT' is not working properly.
SWATI RAO created CARBONDATA-712:
Summary: 'BAD_RECORDS_ACTION'='REDIRECT' is not working properly.
Key: CARBONDATA-712
URL: https://issues.apache.org/jira/browse/CARBONDATA-712
Project: CarbonData
Issue Type: Bug
Components: data-load
Affects Versions: 1.0.0-incubating
Environment: Spark 1.6
Reporter: SWATI RAO
Priority: Minor
Attachments: 2000_UniqData.csv, uniq_shared_dictionary.zip
When we tried to load data using 'BAD_RECORDS_ACTION'='REDIRECT' the bad record
were not written in the file. I have set the property as
carbon.badRecords.location=/opt/Carbon/Spark/badrecords.
PFA for bad_record files which is empty & query csv.
CREATE TABLE uniq_shared_dictionary (CUST_ID int,CUST_NAME
String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1
bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES('DICTIONARY_INCLUDE'='CUST_ID,Double_COLUMN2,DECIMAL_COLUMN2','columnproperties.CUST_ID.shared_column'='shared.CUST_ID','columnproperties.decimal_column2.shared_column'='shared.decimal_column2')
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniq_shared_dictionary OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE',
'BAD_RECORDS_ACTION'='REDIRECT','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_Pass'='true')
select Cust_Id from uniq_shared_dictionary ;
The bad_records are also not loaded in the table.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CARBONDATA-711) Inconsistent data load when single_pass='true'
SWATI RAO created CARBONDATA-711:
Summary: Inconsistent data load when single_pass='true'
Key: CARBONDATA-711
URL: https://issues.apache.org/jira/browse/CARBONDATA-711
Project: CarbonData
Issue Type: Bug
Components: data-load
Affects Versions: 1.1.0-incubating
Environment: Spark 1.6
Reporter: SWATI RAO
Priority: Minor
Attachments: 2000_UniqData.csv
When we perform dataload with Single_pass='true' , it repeats some of the
values in the table whereas the csv contains empty value for that column. PFA
csv which is used for dataloading. And below is the create , load . and select
query.
CREATE TABLE uniq_shared_dictionary (CUST_ID int,CUST_NAME
String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1
bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES('DICTIONARY_INCLUDE'='CUST_ID,Double_COLUMN2,DECIMAL_COLUMN2','columnproperties.CUST_ID.shared_column'='shared.CUST_ID','columnproperties.decimal_column2.shared_column'='shared.decimal_column2')
;
LOAD DATA INPATH
'hdfs://192.168.2.145:54310/BabuStore/Data/uniqdata/2000_UniqData.csv' into
table uniq_shared_dictionary OPTIONS('DELIMITER'=',' ,
'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_Pass'='true')
;
Output:
0: jdbc:hive2://hadoop-master:1> select CUST_ID from uniq_shared_dictionary
;
+--+--+
| Cust_Id |
+--+--+
| 8999 |
| 8999 |
| 8999 |
| 8999 |
| 8999 |
| 8999 |
| 8999 |
| 8999 |
| 8999 |
| 8999 |
| 8999 |
| 8999 |
| 9000 |
| 9001 |
| 9002 |
| 9003 |
| 9004 |
| 9005 |
| 9006 |
| 9007 |
| 9008 |
| 9009 |
| 9010 |
| 9011 |
| 9012 |
| 9013 |
| 9014 |
| 9015 |
| 9016 |
| 9017 |
| 9018 |
| 9019 |
| 9020 |
| 9021 |
| 9022 |
| 9023 |
| 9024 |
| 9025 |
| 9026 |
| 9027 |
| 9028 |
| 9029 |
| 9030 |
| 9031 |
| 9032 |
| 9033 |
| 9034 |
| 9035 |
| 9036 |
| 9037 |
| 9038 |
| 9039 |
| 9040 |
| 9041 |
| 9042 |
| 9043 |
| 9044 |
| 9045 |
| 9046 |
| 9047 |
| 9048 |
| 9049 |
| 9050 |
| 9051 |
| 9052 |
| 9053 |
| 9054 |
| 9055 |
| 9056 |
| 9057 |
| 9058 |
| 9059 |
| 9060 |
| 9061 |
| 9062 |
| 9063 |
| 9064 |
| 9065 |
| 9066 |
| 9067 |
| 9068 |
| 9069 |
| 9070 |
| 9071 |
| 9072 |
| 9073 |
| 9074 |
| 9075 |
| 9076 |
| 9077 |
| 9078 |
| 9079 |
| 9080 |
| 9081 |
| 9082 |
| 9083 |
| 9084 |
| 9085 |
| 9086 |
| 9087 |
+--+--+
| Cust_Id |
+--+--+
| 9088 |
| 9089 |
| 9090 |
| 9091 |
| 9092 |
| 9093 |
| 9094 |
| 9095 |
| 9096 |
| 9097 |
| 9098 |
| 9099 |
| 9100 |
| 9101 |
| 9102 |
| 9103 |
| 9104 |
| 9105 |
| 9106 |
| 9107 |
| 9108 |
| 9109 |
| 9110 |
| 9111 |
| 9112 |
| 9113 |
| 9114 |
| 9115 |
| 9116 |
| 9117 |
| 9118 |
| 9119 |
| 9120 |
| 9121 |
| 9122 |
| 9123 |
| 9124 |
| 9125 |
| 9126 |
| 9127 |
| 9128 |
| 9129 |
| 9130 |
| 9131 |
| 9132 |
| 9133 |
| 9134 |
| 9135 |
| 9136 |
| 9137 |
| 9138 |
| 9139 |
| 9140 |
| 9141 |
| 9142 |
| 9143 |
| 9144 |
| 9145 |
| 9146 |
| 9147 |
| 9148 |
| 9149 |
| 9150 |
| 9151 |
| 9152 |
| 9153 |
| 9154 |
| 9155 |
| 9156 |
| 9157 |
| 9158 |
| 9159 |
| 9160 |
| 9161 |
| 9162 |
| 9163 |
| 9164 |
| 9165 |
| 9166 |
| 9167 |
| 9168 |
| 9169 |
| 9170 |
| 9171 |
| 9172 |
| 9173 |
| 9174 |
| 9175 |
| 9176 |
| 9177 |
| 9178 |
| 9179 |
| 9180 |
| 9181 |
| 9182 |
| 9183 |
| 9184 |
| 9185 |
| 9186 |
| 9187 |
+--+--+
| Cust_Id |
+--+--+
| 9188 |
| 9189 |
| 9190 |
| 9191 |
| 9192 |
| 9193 |
| 9194 |
| 9195 |
| 9196 |
| 9197 |
| 9198 |
| 9199 |
| 9200 |
| 9201 |
| 9202 |
| 9203 |
| 9204 |
| 9205 |
| 9206 |
| 9207 |
| 9208 |
| 9209 |
| 9210 |
| 9211 |
| 9212 |
| 9213 |
| 9214 |
| 9215 |
| 9216 |
| 9217 |
| 9218 |
| 9219 |
| 9220 |
| 9221 |
| 9222 |
[jira] [Created] (CARBONDATA-708) Between operator does not work properly in carbondata.
SWATI RAO created CARBONDATA-708:
Summary: Between operator does not work properly in carbondata.
Key: CARBONDATA-708
URL: https://issues.apache.org/jira/browse/CARBONDATA-708
Project: CarbonData
Issue Type: Bug
Components: sql
Affects Versions: 1.1.0-incubating
Environment: Spark 2.1
Reporter: SWATI RAO
Priority: Minor
Attachments: 100_hive_test.csv
Incorrect result displays.
Steps to reproduce:
1:Create table using following Command
" create table Carbon_automation (imei string,deviceInformationId int,MAC
string,deviceColor string,device_backColor string,modelId string,marketName
string,AMSize string,ROMSize string,CUPAudit string,CPIClocked string,series
string,productionDate timestamp,bomCode string,internalModels string,
deliveryTime string, channelsId string, channelsName string , deliveryAreaId
string, deliveryCountry string, deliveryProvince string, deliveryCity
string,deliveryDistrict string, deliveryStreet string, oxSingleNumber string,
ActiveCheckTime string, ActiveAreaId string, ActiveCountry string,
ActiveProvince string, Activecity string, ActiveDistrict string, ActiveStreet
string, ActiveOperatorId string, Active_releaseId string, Active_EMUIVersion
string, Active_operaSysVersion string, Active_BacVerNumber string,
Active_BacFlashVer string, Active_webUIVersion string, Active_webUITypeCarrVer
string,Active_webTypeDataVerNumber string, Active_operatorsVersion string,
Active_phonePADPartitionedVersions string, Latest_YEAR int, Latest_MONTH int,
Latest_DAY int, Latest_HOUR string, Latest_areaId string, Latest_country
string, Latest_province string, Latest_city string, Latest_district string,
Latest_street string, Latest_releaseId string, Latest_EMUIVersion string,
Latest_operaSysVersion string, Latest_BacVerNumber string, Latest_BacFlashVer
string, Latest_webUIVersion string, Latest_webUITypeCarrVer string,
Latest_webTypeDataVerNumber string, Latest_operatorsVersion string,
Latest_phonePADPartitionedVersions string, Latest_operatorId string,
gamePointDescription string,gamePointId double,contractNumber double,imei_count
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
('DICTIONARY_INCLUDE'='deviceInformationId,Latest_YEAR,Latest_MONTH,Latest_DAY')"
2:Load Data with following command
" LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/HiveData' INTO TABLE
Carbon_automation
OPTIONS('DELIMITER'=',','QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='imei,deviceInformationId,MAC,deviceColor,device_backColor,modelId,marketName,AMSize,ROMSize,CUPAudit,CPIClocked,series,productionDate,bomCode,internalModels,deliveryTime,channelsId,channelsName,deliveryAreaId,deliveryCountry,deliveryProvince,deliveryCity,deliveryDistrict,deliveryStreet,oxSingleNumber,contractNumber,ActiveCheckTime,ActiveAreaId,ActiveCountry,ActiveProvince,Activecity,ActiveDistrict,ActiveStreet,ActiveOperatorId,Active_releaseId,Active_EMUIVersion,Active_operaSysVersion,Active_BacVerNumber,Active_BacFlashVer,Active_webUIVersion,Active_webUITypeCarrVer,Active_webTypeDataVerNumber,Active_operatorsVersion,Active_phonePADPartitionedVersions,Latest_YEAR,Latest_MONTH,Latest_DAY,Latest_HOUR,Latest_areaId,Latest_country,Latest_province,Latest_city,Latest_district,Latest_street,Latest_releaseId,Latest_EMUIVersion,Latest_operaSysVersion,Latest_BacVerNumber,Latest_BacFlashVer,Latest_webUIVersion,Latest_webUITypeCarrVer,Latest_webTypeDataVerNumber,Latest_operatorsVersion,Latest_phonePADPartitionedVersions,Latest_operatorId,gamePointId,gamePointDescription,imei_count')"
3:Run the Query
select Latest_DAY,Latest_HOUR,count(distinct AMSize) as
AMSize_number,sum(gamePointId+contractNumber) as total from Carbon_automation
where Latest_HOUR between 12 and 15 group by Latest_DAY,Latest_HOUR order by
total desc
4:No Result display:
+-+--+++--+
| Latest_DAY | Latest_HOUR | AMSize_number | total |
+-+--+++--+
+-+--+++--+
No rows selected (2.133 seconds).
5:CSV Attached "100_hive_test.csv"
Expected Result:Correct Result should be display.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CARBONDATA-707) Less ( < ) than operator does not work properly in carbondata.
SWATI RAO created CARBONDATA-707:
Summary: Less ( < ) than operator does not work properly in
carbondata.
Key: CARBONDATA-707
URL: https://issues.apache.org/jira/browse/CARBONDATA-707
Project: CarbonData
Issue Type: Bug
Components: sql
Affects Versions: 1.1.0-incubating
Environment: Spark 2.1
Reporter: SWATI RAO
Priority: Minor
Attachments: 100_hive_test.csv
Incorrect result displays
Steps to Reproduces:
1:Create table using following Command
" create table Carbon_automation (imei string,deviceInformationId int,MAC
string,deviceColor string,device_backColor string,modelId string,marketName
string,AMSize string,ROMSize string,CUPAudit string,CPIClocked string,series
string,productionDate timestamp,bomCode string,internalModels string,
deliveryTime string, channelsId string, channelsName string , deliveryAreaId
string, deliveryCountry string, deliveryProvince string, deliveryCity
string,deliveryDistrict string, deliveryStreet string, oxSingleNumber string,
ActiveCheckTime string, ActiveAreaId string, ActiveCountry string,
ActiveProvince string, Activecity string, ActiveDistrict string, ActiveStreet
string, ActiveOperatorId string, Active_releaseId string, Active_EMUIVersion
string, Active_operaSysVersion string, Active_BacVerNumber string,
Active_BacFlashVer string, Active_webUIVersion string, Active_webUITypeCarrVer
string,Active_webTypeDataVerNumber string, Active_operatorsVersion string,
Active_phonePADPartitionedVersions string, Latest_YEAR int, Latest_MONTH int,
Latest_DAY int, Latest_HOUR string, Latest_areaId string, Latest_country
string, Latest_province string, Latest_city string, Latest_district string,
Latest_street string, Latest_releaseId string, Latest_EMUIVersion string,
Latest_operaSysVersion string, Latest_BacVerNumber string, Latest_BacFlashVer
string, Latest_webUIVersion string, Latest_webUITypeCarrVer string,
Latest_webTypeDataVerNumber string, Latest_operatorsVersion string,
Latest_phonePADPartitionedVersions string, Latest_operatorId string,
gamePointDescription string,gamePointId double,contractNumber double,imei_count
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
('DICTIONARY_INCLUDE'='deviceInformationId,Latest_YEAR,Latest_MONTH,Latest_DAY')"
2:Load Data with following command
" LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/HiveData' INTO TABLE
Carbon_automation
OPTIONS('DELIMITER'=',','QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='imei,deviceInformationId,MAC,deviceColor,device_backColor,modelId,marketName,AMSize,ROMSize,CUPAudit,CPIClocked,series,productionDate,bomCode,internalModels,deliveryTime,channelsId,channelsName,deliveryAreaId,deliveryCountry,deliveryProvince,deliveryCity,deliveryDistrict,deliveryStreet,oxSingleNumber,contractNumber,ActiveCheckTime,ActiveAreaId,ActiveCountry,ActiveProvince,Activecity,ActiveDistrict,ActiveStreet,ActiveOperatorId,Active_releaseId,Active_EMUIVersion,Active_operaSysVersion,Active_BacVerNumber,Active_BacFlashVer,Active_webUIVersion,Active_webUITypeCarrVer,Active_webTypeDataVerNumber,Active_operatorsVersion,Active_phonePADPartitionedVersions,Latest_YEAR,Latest_MONTH,Latest_DAY,Latest_HOUR,Latest_areaId,Latest_country,Latest_province,Latest_city,Latest_district,Latest_street,Latest_releaseId,Latest_EMUIVersion,Latest_operaSysVersion,Latest_BacVerNumber,Latest_BacFlashVer,Latest_webUIVersion,Latest_webUITypeCarrVer,Latest_webTypeDataVerNumber,Latest_operatorsVersion,Latest_phonePADPartitionedVersions,Latest_operatorId,gamePointId,gamePointDescription,imei_count')"
3:Run the Query
" Select imei,gamePointId, channelsId,series from Carbon_automation where
channelsId < 4 ORDER BY gamePointId limit 5 "
4:Incorrect Result displays as follows:
++--+-+--+--+
|imei| gamePointId | channelsId | series |
++--+-+--+--+
| 1AA100050 | 29.0 | 1 | 2Series |
| 1AA100014 | 151.0| 3 | 5Series |
| 1AA100011 | 202.0| 1 | 0Series |
| 1AA100018 | 441.0| 4 | 8Series |
| 1AA100060 | 538.0| 4 | 8Series |
++--+-+--+--+
5 rows selected (0.237 seconds)
5:CSV Attached: "100_hive_test.csv"
Expected Result: It should not display channel id 4 as per query.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CARBONDATA-706) Mulitiple OR operators does not work properly in carbondata
[
https://issues.apache.org/jira/browse/CARBONDATA-706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-706:
-
Description:
Incorrect result displays with multiple OR operators.
Steps to Reproduces:
1:Create table using following Command
" create table Carbon_automation (imei string,deviceInformationId int,MAC
string,deviceColor string,device_backColor string,modelId string,marketName
string,AMSize string,ROMSize string,CUPAudit string,CPIClocked string,series
string,productionDate timestamp,bomCode string,internalModels string,
deliveryTime string, channelsId string, channelsName string , deliveryAreaId
string, deliveryCountry string, deliveryProvince string, deliveryCity
string,deliveryDistrict string, deliveryStreet string, oxSingleNumber string,
ActiveCheckTime string, ActiveAreaId string, ActiveCountry string,
ActiveProvince string, Activecity string, ActiveDistrict string, ActiveStreet
string, ActiveOperatorId string, Active_releaseId string, Active_EMUIVersion
string, Active_operaSysVersion string, Active_BacVerNumber string,
Active_BacFlashVer string, Active_webUIVersion string, Active_webUITypeCarrVer
string,Active_webTypeDataVerNumber string, Active_operatorsVersion string,
Active_phonePADPartitionedVersions string, Latest_YEAR int, Latest_MONTH int,
Latest_DAY int, Latest_HOUR string, Latest_areaId string, Latest_country
string, Latest_province string, Latest_city string, Latest_district string,
Latest_street string, Latest_releaseId string, Latest_EMUIVersion string,
Latest_operaSysVersion string, Latest_BacVerNumber string, Latest_BacFlashVer
string, Latest_webUIVersion string, Latest_webUITypeCarrVer string,
Latest_webTypeDataVerNumber string, Latest_operatorsVersion string,
Latest_phonePADPartitionedVersions string, Latest_operatorId string,
gamePointDescription string,gamePointId double,contractNumber double,imei_count
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
('DICTIONARY_INCLUDE'='deviceInformationId,Latest_YEAR,Latest_MONTH,Latest_DAY')"
2:Load Data with following command
" LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/HiveData' INTO TABLE
Carbon_automation
OPTIONS('DELIMITER'=',','QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='imei,deviceInformationId,MAC,deviceColor,device_backColor,modelId,marketName,AMSize,ROMSize,CUPAudit,CPIClocked,series,productionDate,bomCode,internalModels,deliveryTime,channelsId,channelsName,deliveryAreaId,deliveryCountry,deliveryProvince,deliveryCity,deliveryDistrict,deliveryStreet,oxSingleNumber,contractNumber,ActiveCheckTime,ActiveAreaId,ActiveCountry,ActiveProvince,Activecity,ActiveDistrict,ActiveStreet,ActiveOperatorId,Active_releaseId,Active_EMUIVersion,Active_operaSysVersion,Active_BacVerNumber,Active_BacFlashVer,Active_webUIVersion,Active_webUITypeCarrVer,Active_webTypeDataVerNumber,Active_operatorsVersion,Active_phonePADPartitionedVersions,Latest_YEAR,Latest_MONTH,Latest_DAY,Latest_HOUR,Latest_areaId,Latest_country,Latest_province,Latest_city,Latest_district,Latest_street,Latest_releaseId,Latest_EMUIVersion,Latest_operaSysVersion,Latest_BacVerNumber,Latest_BacFlashVer,Latest_webUIVersion,Latest_webUITypeCarrVer,Latest_webTypeDataVerNumber,Latest_operatorsVersion,Latest_phonePADPartitionedVersions,Latest_operatorId,gamePointId,gamePointDescription,imei_count')"
3:Now run the Select Query:
" select imei,gamePointId, channelsId,series from Carbon_automation where
channelsId >=10 OR channelsId <=1 or series='7Series' "
4:Result Displays
" 0: jdbc:hive2://localhost:1> select imei,gamePointId, channelsId,series
from Carbon_automation where channelsId >=10 OR channelsId <=1 or
series='7Series';
+-+--+-+--+--+
|imei | gamePointId | channelsId | series |
+-+--+-+--+--+
| 1AA1| 2738.562 | 4 | 7Series |
| 1AA10 | 1714.635 | 4 | 7Series |
| 1AA100 | 1271.0 | 6 | 5Series |
| 1AA1000 | 692.0| 3 | 5Series |
| 1AA1| 2175.0 | 1 | 7Series |
| 1AA10 | 136.0| 6 | 9Series |
| 1AA100 | 1600.0 | 6 | 7Series |
| 1AA11 | 505.0| 7 | 0Series |
| 1AA12 | 1341.0 | 3 | 0Series |
| 1AA13 | 2239.0 | 3 | 5Series |
| 1AA14 | 2970.0 | 2 | 4Series |
| 1AA15 | 2593.0 | 1 | 1Series |
| 1AA16 | 2572.0 | 2 | 6Series |
| 1AA17 | 1991.0 | 3 | 9Series |
| 1AA18 | 1442.0 | 1 | 8Series |
| 1AA19 | 1841.0 | 3 | 0Series |
| 1AA10001| 298.0| 5 | 2Series |
| 1AA100010 | 79.0 | 6 | 3Series |
| 1AA100011 | 202.0| 1 |
[jira] [Created] (CARBONDATA-706) Mulitiple OR operators does not work properly in carbondata
SWATI RAO created CARBONDATA-706:
Summary: Mulitiple OR operators does not work properly in
carbondata
Key: CARBONDATA-706
URL: https://issues.apache.org/jira/browse/CARBONDATA-706
Project: CarbonData
Issue Type: Bug
Components: sql
Affects Versions: 1.1.0-incubating
Environment: Spark 2.1
Reporter: SWATI RAO
Priority: Minor
Attachments: 100_hive_test.csv
Multiple OR operators result does not match with hive.
Steps to Reproduces:
1:Create table using following Command
" create table Carbon_automation (imei string,deviceInformationId int,MAC
string,deviceColor string,device_backColor string,modelId string,marketName
string,AMSize string,ROMSize string,CUPAudit string,CPIClocked string,series
string,productionDate timestamp,bomCode string,internalModels string,
deliveryTime string, channelsId string, channelsName string , deliveryAreaId
string, deliveryCountry string, deliveryProvince string, deliveryCity
string,deliveryDistrict string, deliveryStreet string, oxSingleNumber string,
ActiveCheckTime string, ActiveAreaId string, ActiveCountry string,
ActiveProvince string, Activecity string, ActiveDistrict string, ActiveStreet
string, ActiveOperatorId string, Active_releaseId string, Active_EMUIVersion
string, Active_operaSysVersion string, Active_BacVerNumber string,
Active_BacFlashVer string, Active_webUIVersion string, Active_webUITypeCarrVer
string,Active_webTypeDataVerNumber string, Active_operatorsVersion string,
Active_phonePADPartitionedVersions string, Latest_YEAR int, Latest_MONTH int,
Latest_DAY int, Latest_HOUR string, Latest_areaId string, Latest_country
string, Latest_province string, Latest_city string, Latest_district string,
Latest_street string, Latest_releaseId string, Latest_EMUIVersion string,
Latest_operaSysVersion string, Latest_BacVerNumber string, Latest_BacFlashVer
string, Latest_webUIVersion string, Latest_webUITypeCarrVer string,
Latest_webTypeDataVerNumber string, Latest_operatorsVersion string,
Latest_phonePADPartitionedVersions string, Latest_operatorId string,
gamePointDescription string,gamePointId double,contractNumber double,imei_count
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
('DICTIONARY_INCLUDE'='deviceInformationId,Latest_YEAR,Latest_MONTH,Latest_DAY')"
2:Load Data with following command
" LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/HiveData' INTO TABLE
Carbon_automation
OPTIONS('DELIMITER'=',','QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='imei,deviceInformationId,MAC,deviceColor,device_backColor,modelId,marketName,AMSize,ROMSize,CUPAudit,CPIClocked,series,productionDate,bomCode,internalModels,deliveryTime,channelsId,channelsName,deliveryAreaId,deliveryCountry,deliveryProvince,deliveryCity,deliveryDistrict,deliveryStreet,oxSingleNumber,contractNumber,ActiveCheckTime,ActiveAreaId,ActiveCountry,ActiveProvince,Activecity,ActiveDistrict,ActiveStreet,ActiveOperatorId,Active_releaseId,Active_EMUIVersion,Active_operaSysVersion,Active_BacVerNumber,Active_BacFlashVer,Active_webUIVersion,Active_webUITypeCarrVer,Active_webTypeDataVerNumber,Active_operatorsVersion,Active_phonePADPartitionedVersions,Latest_YEAR,Latest_MONTH,Latest_DAY,Latest_HOUR,Latest_areaId,Latest_country,Latest_province,Latest_city,Latest_district,Latest_street,Latest_releaseId,Latest_EMUIVersion,Latest_operaSysVersion,Latest_BacVerNumber,Latest_BacFlashVer,Latest_webUIVersion,Latest_webUITypeCarrVer,Latest_webTypeDataVerNumber,Latest_operatorsVersion,Latest_phonePADPartitionedVersions,Latest_operatorId,gamePointId,gamePointDescription,imei_count')"
3:Now run the Select Query:
" select imei,gamePointId, channelsId,series from Carbon_automation where
channelsId >=10 OR channelsId <=1 or series='7Series' "
4:Result Displays
" 0: jdbc:hive2://localhost:1> select imei,gamePointId, channelsId,series
from Carbon_automation where channelsId >=10 OR channelsId <=1 or
series='7Series';
+-+--+-+--+--+
|imei | gamePointId | channelsId | series |
+-+--+-+--+--+
| 1AA1| 2738.562 | 4 | 7Series |
| 1AA10 | 1714.635 | 4 | 7Series |
| 1AA100 | 1271.0 | 6 | 5Series |
| 1AA1000 | 692.0| 3 | 5Series |
| 1AA1| 2175.0 | 1 | 7Series |
| 1AA10 | 136.0| 6 | 9Series |
| 1AA100 | 1600.0 | 6 | 7Series |
| 1AA11 | 505.0| 7 | 0Series |
| 1AA12 | 1341.0 | 3 | 0Series |
| 1AA13 | 2239.0 | 3 | 5Series |
| 1AA14 | 2970.0 | 2 | 4Series |
| 1AA15 | 2593.0 | 1 | 1Series |
| 1AA16 | 2572.0 | 2 | 6Series |
| 1AA17 |
[jira] [Created] (CARBONDATA-704) data mismatch between hive and carbondata after loading for bigint values
SWATI RAO created CARBONDATA-704:
Summary: data mismatch between hive and carbondata after loading
for bigint values
Key: CARBONDATA-704
URL: https://issues.apache.org/jira/browse/CARBONDATA-704
Project: CarbonData
Issue Type: Bug
Components: data-load
Affects Versions: 1.0.0-incubating
Reporter: SWATI RAO
Attachments: Test_Data1 (4).csv
carbondata
0: jdbc:hive2://localhost:1> create table Test_Boundary (c1_int
int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
'org.apache.carbondata.format' ;
0: jdbc:hive2://localhost:1> LOAD DATA INPATH
'hdfs://localhost:54310/Test_Data1.csv' INTO table Test_Boundary OPTIONS
('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='');
0: jdbc:hive2://localhost:1> select c2_Bigint from Test_Boundary;
+--+--+
| c2_Bigint |
+--+--+
| NULL |
| NULL |
| NULL |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
+--+--+
but in hive
create table Test_Boundary_hive (c1_int int,c2_Bigint Bigint,c3_Decimal
Decimal(38,30),c4_double double,c5_string string,c6_Timestamp
Timestamp,c7_Datatype_Desc string) ROW FORMAT DELIMITED FIELDS TERMINATED BY
",";
LOAD DATA LOCAL INPATH 'Test_Data1.csv' into table Test_Boundary_hive;
select c2_Bigint from Test_Boundary_hive;
+---+--+
| c2_Bigint |
+---+--+
| 1234 |
| 2345 |
| 3456 |
| 4567 |
| 9223372036854775807 |
| -9223372036854775808 |
| -9223372036854775807 |
| -9223372036854775806 |
| -9223372036854775805 |
| 0 |
| 9223372036854775807 |
| 9223372036854775807 |
| 9223372036854775807 |
| NULL |
| NULL |
| NULL |
+---+--+
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Closed] (CARBONDATA-419) Show tables in query for non-existing database do not show any exception
[ https://issues.apache.org/jira/browse/CARBONDATA-419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-419. Resolution: Fixed invalid issue > Show tables in query for non-existing database do not show any exception > - > > Key: CARBONDATA-419 > URL: https://issues.apache.org/jira/browse/CARBONDATA-419 > Project: CarbonData > Issue Type: Bug > Components: data-query >Reporter: SWATI RAO >Assignee: Manohar Vanam > > When executing command 'show tables in' for an non-existing database, it do > not show any error, > Example: show tables in test_tableMg; > (where test_tableMg database do not exist) > whereas when executing the same query on hive, it says : > FAILED: SemanticException [Error 10072]: Database does not exist: test_tableMg -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (CARBONDATA-678) Corr function is not working for double datatype.
[ https://issues.apache.org/jira/browse/CARBONDATA-678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-678. Resolution: Fixed this issue is invalid > Corr function is not working for double datatype. > - > > Key: CARBONDATA-678 > URL: https://issues.apache.org/jira/browse/CARBONDATA-678 > Project: CarbonData > Issue Type: Bug > Components: sql >Affects Versions: 1.0.0-incubating > Environment: Spark 1.6 >Reporter: SWATI RAO >Priority: Trivial > Fix For: 1.1.0-incubating > > Attachments: 2000_UniqData.csv, 3000_1_UniqData.csv, > 4000_UniqData.csv, 5000_UniqData.csv, 6000_UniqData.csv, 7000_UniqData.csv > > > We have created a table uniqdata_h and then we upload data in this table then > we perform select query having data-type double then it displays null value > but when we perform select query with integer or bigint data-type it displays > a value. > "Double" data-type is not working. > CREATE QUERY : > 0: jdbc:hive2://localhost:1> CREATE TABLE uniqdata_h (CUST_ID > int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ > timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 > decimal(30,10), DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, > Double_COLUMN2 double,INTEGER_COLUMN1 int) ROW FORMAT DELIMITED FIELDS > TERMINATED BY ','; > +-+--+ > | result | > +-+--+ > +-+--+ > No rows selected (4.249 seconds) > LOAD 1 : > 0: jdbc:hive2://localhost:1> load data local inpath > '/opt/Carbon/CarbonData/TestData/Data/uniqdata/2000_UniqData.csv' into table > uniqdata_h; > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (1.176 seconds) > LOAD 2 : > 0: jdbc:hive2://localhost:1> load data local inpath > '/opt/Carbon/CarbonData/TestData/Data/uniqdata/4000_UniqData.csv' into table > uniqdata_h; > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (0.485 seconds) > LOAD 3 : > 0: jdbc:hive2://localhost:1> load data local inpath > '/opt/Carbon/CarbonData/TestData/Data/uniqdata/5000_UniqData.csv' into table > uniqdata_h; > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (0.365 seconds) > LOAD 4 : > 0: jdbc:hive2://localhost:1> load data local inpath > '/opt/Carbon/CarbonData/TestData/Data/uniqdata/6000_UniqData.csv' into table > uniqdata_h; > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (0.594 seconds) > LOAD 5 : > 0: jdbc:hive2://localhost:1> load data local inpath > '/opt/Carbon/CarbonData/TestData/Data/uniqdata/7000_UniqData.csv' into table > uniqdata_h; > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (0.753 seconds) > LOAD 6 : > 0: jdbc:hive2://localhost:1> load data local inpath > '/opt/Carbon/CarbonData/TestData/Data/uniqdata/3000_1_UniqData.csv' into > table uniqdata_h; > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (0.417 seconds) > SELECT QUERY : double data-type displays NULL value > 0: jdbc:hive2://localhost:1> select corr(Double_COLUMN1,Double_COLUMN1) > as a from uniqdata_h ; > +---+--+ > | a | > +---+--+ > | NULL | > +---+--+ > whereas select query with integer & bigdata type displays some value. > 0: jdbc:hive2://localhost:1> select corr(integer_COLUMN1,bigint_COLUMN1) > as a from uniqdata_h ; > +-+--+ > | a | > +-+--+ > | 1.0002 | > +-+--+ > 1 row selected (0.54 seconds) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (CARBONDATA-693) Issue in select query for function Ceiling & Floor
[
https://issues.apache.org/jira/browse/CARBONDATA-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO closed CARBONDATA-693.
Resolution: Fixed
this issue is invalid
> Issue in select query for function Ceiling & Floor
> --
>
> Key: CARBONDATA-693
> URL: https://issues.apache.org/jira/browse/CARBONDATA-693
> Project: CarbonData
> Issue Type: Bug
> Components: sql
>Affects Versions: 1.0.0-incubating
> Environment: Spark1.6
>Reporter: SWATI RAO
>Priority: Trivial
> Attachments: Test_Data1.csv
>
>
> When executed in Hive it is working Fine, But when we execute same query in
> carbondata it gives an error.
> PushUP_FILTER_Test_Boundary_TC059
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary2 (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) ;
> +-+--+
> | result |
> +-+--+
> +-+--+
> No rows selected (0.101 seconds)
> 0: jdbc:hive2://hadoop-master:1> show tables ;
> +-+--+--+
> |tableName| isTemporary |
> +-+--+--+
> | test_boundary | false|
> | test_boundary2 | false|
> +-+--+--+
> 2 rows selected (0.02 seconds)
> 0: jdbc:hive2://hadoop-master:1> load data local inpath
> '/home/server/Desktop/Test_Data1.csv' into table test_boundary2 ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.201 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c3_Decimal from Test_Boundary
> where floor(c3_Decimal)=0.00 or floor(c3_Decimal) IS NULL ;
> +-+--+
> | c3_Decimal |
> +-+--+
> +-+--+
> No rows selected (0.315 seconds)
> 0: jdbc:hive2://hadoop-master:1> select count(*) from Test_Boundary ;
> +--+--+
> | _c0 |
> +--+--+
> | 0|
> +--+--+
> CarbonData
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format' ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.535 seconds)
> 0: jdbc:hive2://hadoop-master:1> select count(*) from Test_Boundary ;
> +--+--+
> | _c0 |
> +--+--+
> | 0|
> +--+--+
> 1 row selected (1.883 seconds)
>
> hdfs://192.168.2.145:54310hdfs://192.168.2.145:54310/HDFS_URL/BabuStore/Data/Test_Data1.csv
> (state=,code=0)
> 0: jdbc:hive2://hadoop-master:1> LOAD DATA INPATH
> 'hdfs://192.168.2.145:54310/BabuStore/Data/Test_Data1.csv' INTO table
> Test_Boundary
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (2.436 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c3_Decimal from Test_Boundary
> where floor(c3_Decimal)=0.00 or floor(c3_Decimal) IS NULL ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 7.0 failed 4 times, most recent failure: Lost task 0.3 in
> stage 7.0 (TID 16, hadoop-master): org.apache.spark.sql.AnalysisException:
> Decimal scale (0) cannot be greater than precision (-28).;
> at org.apache.spark.sql.types.PrecisionInfo.(DecimalType.scala:32)
> at org.apache.spark.sql.types.DecimalType.(DecimalType.scala:68)
> at
> org.apache.spark.sql.types.DecimalType$.bounded(DecimalType.scala:155)
> at org.apache.spark.sql.types.Decimal.floor(Decimal.scala:326)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificPredicate.eval(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.codegen.GeneratePredicate$$anonfun$create$2.apply(GeneratePredicate.scala:68)
> at
> org.apache.spark.sql.catalyst.expressions.codegen.GeneratePredicate$$anonfun$create$2.apply(GeneratePredicate.scala:68)
> at
> org.apache.spark.sql.execution.Filter$$anonfun$2$$anonfun$apply$2.apply(basicOperators.scala:74)
> at
> org.apache.spark.sql.execution.Filter$$anonfun$2$$anonfun$apply$2.apply(basicOperators.scala:72)
> at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:390)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> at
> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
> at
> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
> at
>
[jira] [Updated] (CARBONDATA-693) Issue in select query for function Ceiling & Floor
[
https://issues.apache.org/jira/browse/CARBONDATA-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-693:
-
Request participants: (was: )
Priority: Minor (was: Major)
> Issue in select query for function Ceiling & Floor
> --
>
> Key: CARBONDATA-693
> URL: https://issues.apache.org/jira/browse/CARBONDATA-693
> Project: CarbonData
> Issue Type: Bug
> Components: sql
>Affects Versions: 1.0.0-incubating
> Environment: Spark1.6
>Reporter: SWATI RAO
>Priority: Minor
> Attachments: Test_Data1.csv
>
>
> When executed in Hive it is working Fine, But when we execute same query in
> carbondata it gives an error.
> PushUP_FILTER_Test_Boundary_TC059
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary2 (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) ;
> +-+--+
> | result |
> +-+--+
> +-+--+
> No rows selected (0.101 seconds)
> 0: jdbc:hive2://hadoop-master:1> show tables ;
> +-+--+--+
> |tableName| isTemporary |
> +-+--+--+
> | test_boundary | false|
> | test_boundary2 | false|
> +-+--+--+
> 2 rows selected (0.02 seconds)
> 0: jdbc:hive2://hadoop-master:1> load data local inpath
> '/home/server/Desktop/Test_Data1.csv' into table test_boundary2 ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.201 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c3_Decimal from Test_Boundary
> where floor(c3_Decimal)=0.00 or floor(c3_Decimal) IS NULL ;
> +-+--+
> | c3_Decimal |
> +-+--+
> +-+--+
> No rows selected (0.315 seconds)
> 0: jdbc:hive2://hadoop-master:1> select count(*) from Test_Boundary ;
> +--+--+
> | _c0 |
> +--+--+
> | 0|
> +--+--+
> CarbonData
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format' ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.535 seconds)
> 0: jdbc:hive2://hadoop-master:1> select count(*) from Test_Boundary ;
> +--+--+
> | _c0 |
> +--+--+
> | 0|
> +--+--+
> 1 row selected (1.883 seconds)
>
> hdfs://192.168.2.145:54310hdfs://192.168.2.145:54310/HDFS_URL/BabuStore/Data/Test_Data1.csv
> (state=,code=0)
> 0: jdbc:hive2://hadoop-master:1> LOAD DATA INPATH
> 'hdfs://192.168.2.145:54310/BabuStore/Data/Test_Data1.csv' INTO table
> Test_Boundary
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (2.436 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c3_Decimal from Test_Boundary
> where floor(c3_Decimal)=0.00 or floor(c3_Decimal) IS NULL ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 7.0 failed 4 times, most recent failure: Lost task 0.3 in
> stage 7.0 (TID 16, hadoop-master): org.apache.spark.sql.AnalysisException:
> Decimal scale (0) cannot be greater than precision (-28).;
> at org.apache.spark.sql.types.PrecisionInfo.(DecimalType.scala:32)
> at org.apache.spark.sql.types.DecimalType.(DecimalType.scala:68)
> at
> org.apache.spark.sql.types.DecimalType$.bounded(DecimalType.scala:155)
> at org.apache.spark.sql.types.Decimal.floor(Decimal.scala:326)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificPredicate.eval(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.codegen.GeneratePredicate$$anonfun$create$2.apply(GeneratePredicate.scala:68)
> at
> org.apache.spark.sql.catalyst.expressions.codegen.GeneratePredicate$$anonfun$create$2.apply(GeneratePredicate.scala:68)
> at
> org.apache.spark.sql.execution.Filter$$anonfun$2$$anonfun$apply$2.apply(basicOperators.scala:74)
> at
> org.apache.spark.sql.execution.Filter$$anonfun$2$$anonfun$apply$2.apply(basicOperators.scala:72)
> at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:390)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> at
> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
> at
> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
> at
>
[jira] [Updated] (CARBONDATA-693) Issue in select query for function Ceiling & Floor
[
https://issues.apache.org/jira/browse/CARBONDATA-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-693:
-
Request participants: (was: )
Priority: Trivial (was: Minor)
> Issue in select query for function Ceiling & Floor
> --
>
> Key: CARBONDATA-693
> URL: https://issues.apache.org/jira/browse/CARBONDATA-693
> Project: CarbonData
> Issue Type: Bug
> Components: sql
>Affects Versions: 1.0.0-incubating
> Environment: Spark1.6
>Reporter: SWATI RAO
>Priority: Trivial
> Attachments: Test_Data1.csv
>
>
> When executed in Hive it is working Fine, But when we execute same query in
> carbondata it gives an error.
> PushUP_FILTER_Test_Boundary_TC059
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary2 (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) ;
> +-+--+
> | result |
> +-+--+
> +-+--+
> No rows selected (0.101 seconds)
> 0: jdbc:hive2://hadoop-master:1> show tables ;
> +-+--+--+
> |tableName| isTemporary |
> +-+--+--+
> | test_boundary | false|
> | test_boundary2 | false|
> +-+--+--+
> 2 rows selected (0.02 seconds)
> 0: jdbc:hive2://hadoop-master:1> load data local inpath
> '/home/server/Desktop/Test_Data1.csv' into table test_boundary2 ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.201 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c3_Decimal from Test_Boundary
> where floor(c3_Decimal)=0.00 or floor(c3_Decimal) IS NULL ;
> +-+--+
> | c3_Decimal |
> +-+--+
> +-+--+
> No rows selected (0.315 seconds)
> 0: jdbc:hive2://hadoop-master:1> select count(*) from Test_Boundary ;
> +--+--+
> | _c0 |
> +--+--+
> | 0|
> +--+--+
> CarbonData
> 0: jdbc:hive2://hadoop-master:1> create table Test_Boundary (c1_int
> int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format' ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.535 seconds)
> 0: jdbc:hive2://hadoop-master:1> select count(*) from Test_Boundary ;
> +--+--+
> | _c0 |
> +--+--+
> | 0|
> +--+--+
> 1 row selected (1.883 seconds)
>
> hdfs://192.168.2.145:54310hdfs://192.168.2.145:54310/HDFS_URL/BabuStore/Data/Test_Data1.csv
> (state=,code=0)
> 0: jdbc:hive2://hadoop-master:1> LOAD DATA INPATH
> 'hdfs://192.168.2.145:54310/BabuStore/Data/Test_Data1.csv' INTO table
> Test_Boundary
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> ;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (2.436 seconds)
> 0: jdbc:hive2://hadoop-master:1> select c3_Decimal from Test_Boundary
> where floor(c3_Decimal)=0.00 or floor(c3_Decimal) IS NULL ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 7.0 failed 4 times, most recent failure: Lost task 0.3 in
> stage 7.0 (TID 16, hadoop-master): org.apache.spark.sql.AnalysisException:
> Decimal scale (0) cannot be greater than precision (-28).;
> at org.apache.spark.sql.types.PrecisionInfo.(DecimalType.scala:32)
> at org.apache.spark.sql.types.DecimalType.(DecimalType.scala:68)
> at
> org.apache.spark.sql.types.DecimalType$.bounded(DecimalType.scala:155)
> at org.apache.spark.sql.types.Decimal.floor(Decimal.scala:326)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificPredicate.eval(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.codegen.GeneratePredicate$$anonfun$create$2.apply(GeneratePredicate.scala:68)
> at
> org.apache.spark.sql.catalyst.expressions.codegen.GeneratePredicate$$anonfun$create$2.apply(GeneratePredicate.scala:68)
> at
> org.apache.spark.sql.execution.Filter$$anonfun$2$$anonfun$apply$2.apply(basicOperators.scala:74)
> at
> org.apache.spark.sql.execution.Filter$$anonfun$2$$anonfun$apply$2.apply(basicOperators.scala:72)
> at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:390)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> at
> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
> at
> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
> at
>
[jira] [Updated] (CARBONDATA-665) Comparision Failure occurs when we execute the same query in hive and Carbondata
[
https://issues.apache.org/jira/browse/CARBONDATA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-665:
-
Request participants: (was: )
Priority: Minor (was: Major)
> Comparision Failure occurs when we execute the same query in hive and
> Carbondata
>
>
> Key: CARBONDATA-665
> URL: https://issues.apache.org/jira/browse/CARBONDATA-665
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.0.0-incubating
> Environment: Spark 1.6
>Reporter: SWATI RAO
>Priority: Minor
> Attachments: Test_Data1.csv
>
>
> Orderby is not working , so records are not coming in sequence as well there
> is data difference and some values being stored as null
> Data itself is stored incorrectly and is different from Hive
> Spark version :1.6.2
> Create 1 query : create table Test_Boundary (c1_int int,c2_Bigint
> Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format'
> Load 1 Query : LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO
> table Test_Boundary
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> Create 2 query : create table Test_Boundary1 (c1_int int,c2_Bigint
> Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format'
> Load 2 query: LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO
> table Test_Boundary1
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> Select Query :
> select
> c1_int,c2_Bigint,c3_Decimal,c4_double,c5_string,c6_Timestamp,c7_Datatype_Desc
> from Test_Boundary where c2_bigint=c2_bigint
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CARBONDATA-665) Comparision Failure occurs when we execute the same query in hive and Carbondata
[
https://issues.apache.org/jira/browse/CARBONDATA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-665:
-
Request participants: (was: )
> Comparision Failure occurs when we execute the same query in hive and
> Carbondata
>
>
> Key: CARBONDATA-665
> URL: https://issues.apache.org/jira/browse/CARBONDATA-665
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.0.0-incubating
> Environment: Spark 1.6
>Reporter: SWATI RAO
>Priority: Minor
> Attachments: Test_Data1.csv
>
>
> Orderby is not working , so records are not coming in sequence as well there
> is data difference and some values being stored as null
> Data itself is stored incorrectly and is different from Hive
> Spark version :1.6.2
> Create 1 query : create table Test_Boundary (c1_int int,c2_Bigint
> Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format'
> Load 1 Query : LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO
> table Test_Boundary
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> Create 2 query : create table Test_Boundary1 (c1_int int,c2_Bigint
> Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format'
> Load 2 query: LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO
> table Test_Boundary1
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> Select Query :
> select
> c1_int,c2_Bigint,c3_Decimal,c4_double,c5_string,c6_Timestamp,c7_Datatype_Desc
> from Test_Boundary where c2_bigint=c2_bigint
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CARBONDATA-668) Dataloads fails for this particular query
[ https://issues.apache.org/jira/browse/CARBONDATA-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO updated CARBONDATA-668: - Description: Dataloads fail for this query Create Query: create table smart_500_DE (MSISDN string,IMSI string,IMEI string,INTERFACEID int,GROUPID int,GGSN_ID double,SGSN_ID double,SESSION_INDICATOR int,BEGIN_TIME double,BEGIN_TIME_MSEL int,END_TIME double,END_TIME_MSEL int,PROT_CATEGORY int,PROT_TYPE int,L7_CARRIER_PROT int,SUB_PROT_TYPE int,SID double,PROBEID double,ENCRYPT_VERSION int,ROAMING_TYPE int,ROAM_DIRECTION int,MS_IP string,SERVER_IP string,MS_PORT int,SERVER_DECIMAL Decimal,APN string,SGSN_SIG_IP string,GGSN_SIG_IP_BigInt_NEGATIVE bigint,SGSN_USER_IP string,GGSN_USER_IP string,MCC string,MNC string,RAT int,LAC string,RAC string,SAC string,CI string,BROWSER_TIMESTAMP timestamp,TCP_CONN_STATES int,TCP_STATES_BIGINTPOSITIVE int,TCP_WIN_SIZE int,TCP_MSS int,TCP_CONN_TIMES int,TCP_CONN_2_FAILED_TIMES int,TCP_CONN_3_FAILED_TIMES int,HOST string,STREAMING_URL string,GET_STREAMING_FAILED_CODE int,GET_STREAMING_FLAG int,GET_NUM int,GET_SUCCEED_NUM int,GET_RETRANS_NUM int,GET_TIMEOUT_NUM int,INTBUFFER_FST_FLAG int,INTBUFFER_FULL_FLAG int,STALL_NUM int,VIDEO_FRAME_RATE int,VIDEO_CODEC_ID string,VIDEO_WIDTH int,VIDEO_HEIGHT int,AUDIO_CODEC_ID string,MEDIA_FILE_TYPE int,PLAY_STATE int,STREAMING_FLAG int,TCP_STATUS_INDICATOR int,DISCONNECTION_FLAG int,FAILURE_CODE int,FLAG int,TAC string,ECI string,TCP_SYN_TIME_MSEL int,TCP_FST_SYN_DIRECTION int,RAN_NE_USER_IP string,HOMEMCC string,HOMEMNC string,CHARGE_FLAG int,PREPAID_FLAG int,USER_AGENT string,MS_WIN_STAT_TOTAL_NUM int,MS_WIN_STAT_SMALL_NUM int,MS_ACK_TO_1STGET_DELAY int,SERVER_ACK_TO_1STDATA_DELAY int,STREAMING_TYPE int,SOURCE_VIDEO_QUALITY int,TETHERING_FLAG int,CARRIER_ID double,LAYER1ID int,LAYER2ID int,LAYER3ID int,LAYER4ID int,LAYER5ID int,LAYER6ID int,CHARGING_RULE_BASE_NAME string,SP string,EXTENDED_URL string,SV string,FIRST_SAI_CGI_ECGI string,EXTENDED_URL_OTHER string,SIGNALING_USE_FLAG int,DNS_RETRANS_NUM int,DNS_FAIL_CODE int,FIRST_RAT int,MS_INDICATOR string,LAST_SAI_CGI_ECGI string,LAST_RAT int,FIRST_LONGITUDE double,FIRST_LATITUDE double,FIRST_ALTITUDE int,FIRST_RASTERLONGITUDE double,FIRST_RASTERLATITUDE double,FIRST_RASTERALTITUDE int,FIRST_FREQUENCYSPOT int,FIRST_CLUTTER int,FIRST_USERBEHAVIOR int,FIRST_SPEED int,FIRST_CREDIBILITY int,LAST_LONGITUDE double,LAST_LATITUDE double,LAST_ALTITUDE int,LAST_RASTERLONGITUDE double,LAST_RASTERLATITUDE double,LAST_RASTERALTITUDE int,LAST_FREQUENCYSPOT int,LAST_CLUTTER int,LAST_USERBEHAVIOR int,LAST_SPEED int,LAST_CREDIBILITY int,IMEI_CIPHERTEXT string,APP_ID int,DOMAIN_NAME string,STREAMING_CACHE_IP string,STOP_LONGER_THAN_MIN_THRESHOLD int,STOP_LONGER_THAN_MAX_THRESHOLD int,PLAY_END_STAT int,STOP_START_TIME1 double,STOP_END_TIME1 double,STOP_START_TIME2 double,STOP_END_TIME2 double,STOP_START_TIME3 double,STOP_END_TIME3 double,STOP_START_TIME4 double,STOP_END_TIME4 double,STOP_START_TIME5 double,STOP_END_TIME5 double,STOP_START_TIME6 double,STOP_END_TIME6 double,STOP_START_TIME7 double,STOP_END_TIME7 double,STOP_START_TIME8 double,STOP_END_TIME8 double,STOP_START_TIME9 double,STOP_END_TIME9 double,STOP_START_TIME10 double,STOP_END_TIME10 double,FAIL_CLASS double,RECORD_TYPE double,NODATA_COUNT double,VIDEO_NODATA_DURATION double,VIDEO_SMOOTH_DURATION double,VIDEO_SD_DURATION double,VIDEO_HD_DURATION double,VIDEO_UHD_DURATION double,VIDEO_FHD_DURATION double,FLUCTUATION double,START_DOWNLOAD_THROUGHPUT double,L7_UL_GOODPUT_FULL_MSS double,SESSIONKEY string,FIRST_UCELLID double,LAST_UCELLID double,UCELLID1 double,LONGITUDE1 double,LATITUDE1 double,UCELLID2 double,LONGITUDE2 double,LATITUDE2 double,UCELLID3 double,LONGITUDE3 double,LATITUDE3 double,UCELLID4 double,LONGITUDE4 double,LATITUDE4 double,UCELLID5 double,LONGITUDE5 double,LATITUDE5 double,UCELLID6 double,LONGITUDE6 double,LATITUDE6 double,UCELLID7 double,LONGITUDE7 double,LATITUDE7 double,UCELLID8 double,LONGITUDE8 double,LATITUDE8 double,UCELLID9 double,LONGITUDE9 double,LATITUDE9 double,UCELLID10 double,LONGITUDE10 double,LATITUDE10 double,INTBUFFER_FULL_DELAY double,STALL_DURATION double,STREAMING_DW_PACKETS double,STREAMING_DOWNLOAD_DELAY double,PLAY_DURATION double,STREAMING_QUALITY int,VIDEO_DATA_RATE double,AUDIO_DATA_RATE double,STREAMING_FILESIZE double,STREAMING_DURATIOIN double,TCP_SYN_TIME double,TCP_RTT_STEP1 double,CHARGE_ID double,UL_REVERSE_TO_DL_DELAY double,DL_REVERSE_TO_UL_DELAY double,DATATRANS_DW_GOODPUT double,DATATRANS_DW_TOTAL_DURATION double,SUM_FRAGMENT_INTERVAL double,TCP_FIN_TIMES double,TCP_RESET_TIMES double,URL_CLASSIFICATION double,STREAMING_LQ_DURATIOIN double,MAX_DNS_DELAY double,MAX_DNS2SYN double,MAX_LATANCY_OF_LINK_SETUP double,MAX_SYNACK2FIRSTACK double,MAX_SYNACK2LASTACK double,MAX_ACK2GET_DELAY double,MAX_FRAG_INTERVAL_PREDELAY
[jira] [Created] (CARBONDATA-693) Issue in select query for function Ceiling & Floor
SWATI RAO created CARBONDATA-693:
Summary: Issue in select query for function Ceiling & Floor
Key: CARBONDATA-693
URL: https://issues.apache.org/jira/browse/CARBONDATA-693
Project: CarbonData
Issue Type: Bug
Components: sql
Affects Versions: 1.0.0-incubating
Environment: Spark1.6
Reporter: SWATI RAO
Attachments: Test_Data1.csv
When executed in Hive it is working Fine, But when we execute same query in
carbondata it gives an error.
PushUP_FILTER_Test_Boundary_TC059
0: jdbc:hive2://hadoop-master:1> create table Test_Boundary2 (c1_int
int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
string,c6_Timestamp Timestamp,c7_Datatype_Desc string) ;
+-+--+
| result |
+-+--+
+-+--+
No rows selected (0.101 seconds)
0: jdbc:hive2://hadoop-master:1> show tables ;
+-+--+--+
|tableName| isTemporary |
+-+--+--+
| test_boundary | false|
| test_boundary2 | false|
+-+--+--+
2 rows selected (0.02 seconds)
0: jdbc:hive2://hadoop-master:1> load data local inpath
'/home/server/Desktop/Test_Data1.csv' into table test_boundary2 ;
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.201 seconds)
0: jdbc:hive2://hadoop-master:1> select c3_Decimal from Test_Boundary where
floor(c3_Decimal)=0.00 or floor(c3_Decimal) IS NULL ;
+-+--+
| c3_Decimal |
+-+--+
+-+--+
No rows selected (0.315 seconds)
0: jdbc:hive2://hadoop-master:1> select count(*) from Test_Boundary ;
+--+--+
| _c0 |
+--+--+
| 0|
+--+--+
CarbonData
0: jdbc:hive2://hadoop-master:1> create table Test_Boundary (c1_int
int,c2_Bigint Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
'org.apache.carbondata.format' ;
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (0.535 seconds)
0: jdbc:hive2://hadoop-master:1> select count(*) from Test_Boundary ;
+--+--+
| _c0 |
+--+--+
| 0|
+--+--+
1 row selected (1.883 seconds)
hdfs://192.168.2.145:54310hdfs://192.168.2.145:54310/HDFS_URL/BabuStore/Data/Test_Data1.csv
(state=,code=0)
0: jdbc:hive2://hadoop-master:1> LOAD DATA INPATH
'hdfs://192.168.2.145:54310/BabuStore/Data/Test_Data1.csv' INTO table
Test_Boundary
OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
;
+-+--+
| Result |
+-+--+
+-+--+
No rows selected (2.436 seconds)
0: jdbc:hive2://hadoop-master:1> select c3_Decimal from Test_Boundary where
floor(c3_Decimal)=0.00 or floor(c3_Decimal) IS NULL ;
Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task
0 in stage 7.0 failed 4 times, most recent failure: Lost task 0.3 in stage 7.0
(TID 16, hadoop-master): org.apache.spark.sql.AnalysisException: Decimal scale
(0) cannot be greater than precision (-28).;
at org.apache.spark.sql.types.PrecisionInfo.(DecimalType.scala:32)
at org.apache.spark.sql.types.DecimalType.(DecimalType.scala:68)
at
org.apache.spark.sql.types.DecimalType$.bounded(DecimalType.scala:155)
at org.apache.spark.sql.types.Decimal.floor(Decimal.scala:326)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificPredicate.eval(Unknown
Source)
at
org.apache.spark.sql.catalyst.expressions.codegen.GeneratePredicate$$anonfun$create$2.apply(GeneratePredicate.scala:68)
at
org.apache.spark.sql.catalyst.expressions.codegen.GeneratePredicate$$anonfun$create$2.apply(GeneratePredicate.scala:68)
at
org.apache.spark.sql.execution.Filter$$anonfun$2$$anonfun$apply$2.apply(basicOperators.scala:74)
at
org.apache.spark.sql.execution.Filter$$anonfun$2$$anonfun$apply$2.apply(basicOperators.scala:72)
at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:390)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at
scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
at scala.collection.AbstractIterator.to(Iterator.scala:1157)
at
scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
at
[jira] [Updated] (CARBONDATA-668) Dataloads fail when no. of column in load query is greater than the no. of column in create table
[ https://issues.apache.org/jira/browse/CARBONDATA-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO updated CARBONDATA-668: - Priority: Minor (was: Major) Description: Dataloads fail when no. of column in load query is greater than the no. of column in create table Create Query: create table smart_500_DE (MSISDN string,IMSI string,IMEI string,INTERFACEID int,GROUPID int,GGSN_ID double,SGSN_ID double,SESSION_INDICATOR int,BEGIN_TIME double,BEGIN_TIME_MSEL int,END_TIME double,END_TIME_MSEL int,PROT_CATEGORY int,PROT_TYPE int,L7_CARRIER_PROT int,SUB_PROT_TYPE int,SID double,PROBEID double,ENCRYPT_VERSION int,ROAMING_TYPE int,ROAM_DIRECTION int,MS_IP string,SERVER_IP string,MS_PORT int,SERVER_DECIMAL Decimal,APN string,SGSN_SIG_IP string,GGSN_SIG_IP_BigInt_NEGATIVE bigint,SGSN_USER_IP string,GGSN_USER_IP string,MCC string,MNC string,RAT int,LAC string,RAC string,SAC string,CI string,BROWSER_TIMESTAMP timestamp,TCP_CONN_STATES int,TCP_STATES_BIGINTPOSITIVE int,TCP_WIN_SIZE int,TCP_MSS int,TCP_CONN_TIMES int,TCP_CONN_2_FAILED_TIMES int,TCP_CONN_3_FAILED_TIMES int,HOST string,STREAMING_URL string,GET_STREAMING_FAILED_CODE int,GET_STREAMING_FLAG int,GET_NUM int,GET_SUCCEED_NUM int,GET_RETRANS_NUM int,GET_TIMEOUT_NUM int,INTBUFFER_FST_FLAG int,INTBUFFER_FULL_FLAG int,STALL_NUM int,VIDEO_FRAME_RATE int,VIDEO_CODEC_ID string,VIDEO_WIDTH int,VIDEO_HEIGHT int,AUDIO_CODEC_ID string,MEDIA_FILE_TYPE int,PLAY_STATE int,STREAMING_FLAG int,TCP_STATUS_INDICATOR int,DISCONNECTION_FLAG int,FAILURE_CODE int,FLAG int,TAC string,ECI string,TCP_SYN_TIME_MSEL int,TCP_FST_SYN_DIRECTION int,RAN_NE_USER_IP string,HOMEMCC string,HOMEMNC string,CHARGE_FLAG int,PREPAID_FLAG int,USER_AGENT string,MS_WIN_STAT_TOTAL_NUM int,MS_WIN_STAT_SMALL_NUM int,MS_ACK_TO_1STGET_DELAY int,SERVER_ACK_TO_1STDATA_DELAY int,STREAMING_TYPE int,SOURCE_VIDEO_QUALITY int,TETHERING_FLAG int,CARRIER_ID double,LAYER1ID int,LAYER2ID int,LAYER3ID int,LAYER4ID int,LAYER5ID int,LAYER6ID int,CHARGING_RULE_BASE_NAME string,SP string,EXTENDED_URL string,SV string,FIRST_SAI_CGI_ECGI string,EXTENDED_URL_OTHER string,SIGNALING_USE_FLAG int,DNS_RETRANS_NUM int,DNS_FAIL_CODE int,FIRST_RAT int,MS_INDICATOR string,LAST_SAI_CGI_ECGI string,LAST_RAT int,FIRST_LONGITUDE double,FIRST_LATITUDE double,FIRST_ALTITUDE int,FIRST_RASTERLONGITUDE double,FIRST_RASTERLATITUDE double,FIRST_RASTERALTITUDE int,FIRST_FREQUENCYSPOT int,FIRST_CLUTTER int,FIRST_USERBEHAVIOR int,FIRST_SPEED int,FIRST_CREDIBILITY int,LAST_LONGITUDE double,LAST_LATITUDE double,LAST_ALTITUDE int,LAST_RASTERLONGITUDE double,LAST_RASTERLATITUDE double,LAST_RASTERALTITUDE int,LAST_FREQUENCYSPOT int,LAST_CLUTTER int,LAST_USERBEHAVIOR int,LAST_SPEED int,LAST_CREDIBILITY int,IMEI_CIPHERTEXT string,APP_ID int,DOMAIN_NAME string,STREAMING_CACHE_IP string,STOP_LONGER_THAN_MIN_THRESHOLD int,STOP_LONGER_THAN_MAX_THRESHOLD int,PLAY_END_STAT int,STOP_START_TIME1 double,STOP_END_TIME1 double,STOP_START_TIME2 double,STOP_END_TIME2 double,STOP_START_TIME3 double,STOP_END_TIME3 double,STOP_START_TIME4 double,STOP_END_TIME4 double,STOP_START_TIME5 double,STOP_END_TIME5 double,STOP_START_TIME6 double,STOP_END_TIME6 double,STOP_START_TIME7 double,STOP_END_TIME7 double,STOP_START_TIME8 double,STOP_END_TIME8 double,STOP_START_TIME9 double,STOP_END_TIME9 double,STOP_START_TIME10 double,STOP_END_TIME10 double,FAIL_CLASS double,RECORD_TYPE double,NODATA_COUNT double,VIDEO_NODATA_DURATION double,VIDEO_SMOOTH_DURATION double,VIDEO_SD_DURATION double,VIDEO_HD_DURATION double,VIDEO_UHD_DURATION double,VIDEO_FHD_DURATION double,FLUCTUATION double,START_DOWNLOAD_THROUGHPUT double,L7_UL_GOODPUT_FULL_MSS double,SESSIONKEY string,FIRST_UCELLID double,LAST_UCELLID double,UCELLID1 double,LONGITUDE1 double,LATITUDE1 double,UCELLID2 double,LONGITUDE2 double,LATITUDE2 double,UCELLID3 double,LONGITUDE3 double,LATITUDE3 double,UCELLID4 double,LONGITUDE4 double,LATITUDE4 double,UCELLID5 double,LONGITUDE5 double,LATITUDE5 double,UCELLID6 double,LONGITUDE6 double,LATITUDE6 double,UCELLID7 double,LONGITUDE7 double,LATITUDE7 double,UCELLID8 double,LONGITUDE8 double,LATITUDE8 double,UCELLID9 double,LONGITUDE9 double,LATITUDE9 double,UCELLID10 double,LONGITUDE10 double,LATITUDE10 double,INTBUFFER_FULL_DELAY double,STALL_DURATION double,STREAMING_DW_PACKETS double,STREAMING_DOWNLOAD_DELAY double,PLAY_DURATION double,STREAMING_QUALITY int,VIDEO_DATA_RATE double,AUDIO_DATA_RATE double,STREAMING_FILESIZE double,STREAMING_DURATIOIN double,TCP_SYN_TIME double,TCP_RTT_STEP1 double,CHARGE_ID double,UL_REVERSE_TO_DL_DELAY double,DL_REVERSE_TO_UL_DELAY double,DATATRANS_DW_GOODPUT double,DATATRANS_DW_TOTAL_DURATION double,SUM_FRAGMENT_INTERVAL double,TCP_FIN_TIMES double,TCP_RESET_TIMES double,URL_CLASSIFICATION double,STREAMING_LQ_DURATIOIN double,MAX_DNS_DELAY double,MAX_DNS2SYN double,MAX_LATANCY_OF_LINK_SETUP
[jira] [Updated] (CARBONDATA-668) Data loading
[ https://issues.apache.org/jira/browse/CARBONDATA-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO updated CARBONDATA-668: - Summary: Data loading (was: Data loading failed on the latest build) > Data loading > - > > Key: CARBONDATA-668 > URL: https://issues.apache.org/jira/browse/CARBONDATA-668 > Project: CarbonData > Issue Type: Bug > Components: data-load > Environment: Spark-1.6 >Reporter: SWATI RAO > Attachments: seq_500Records.csv > > > Create Query: create table smart_500_DE (MSISDN string,IMSI string,IMEI > string,INTERFACEID int,GROUPID int,GGSN_ID double,SGSN_ID > double,SESSION_INDICATOR int,BEGIN_TIME double,BEGIN_TIME_MSEL int,END_TIME > double,END_TIME_MSEL int,PROT_CATEGORY int,PROT_TYPE int,L7_CARRIER_PROT > int,SUB_PROT_TYPE int,SID double,PROBEID double,ENCRYPT_VERSION > int,ROAMING_TYPE int,ROAM_DIRECTION int,MS_IP string,SERVER_IP string,MS_PORT > int,SERVER_DECIMAL Decimal,APN string,SGSN_SIG_IP > string,GGSN_SIG_IP_BigInt_NEGATIVE bigint,SGSN_USER_IP string,GGSN_USER_IP > string,MCC string,MNC string,RAT int,LAC string,RAC string,SAC string,CI > string,BROWSER_TIMESTAMP timestamp,TCP_CONN_STATES > int,TCP_STATES_BIGINTPOSITIVE int,TCP_WIN_SIZE int,TCP_MSS int,TCP_CONN_TIMES > int,TCP_CONN_2_FAILED_TIMES int,TCP_CONN_3_FAILED_TIMES int,HOST > string,STREAMING_URL string,GET_STREAMING_FAILED_CODE int,GET_STREAMING_FLAG > int,GET_NUM int,GET_SUCCEED_NUM int,GET_RETRANS_NUM int,GET_TIMEOUT_NUM > int,INTBUFFER_FST_FLAG int,INTBUFFER_FULL_FLAG int,STALL_NUM > int,VIDEO_FRAME_RATE int,VIDEO_CODEC_ID string,VIDEO_WIDTH int,VIDEO_HEIGHT > int,AUDIO_CODEC_ID string,MEDIA_FILE_TYPE int,PLAY_STATE int,STREAMING_FLAG > int,TCP_STATUS_INDICATOR int,DISCONNECTION_FLAG int,FAILURE_CODE int,FLAG > int,TAC string,ECI string,TCP_SYN_TIME_MSEL int,TCP_FST_SYN_DIRECTION > int,RAN_NE_USER_IP string,HOMEMCC string,HOMEMNC string,CHARGE_FLAG > int,PREPAID_FLAG int,USER_AGENT string,MS_WIN_STAT_TOTAL_NUM > int,MS_WIN_STAT_SMALL_NUM int,MS_ACK_TO_1STGET_DELAY > int,SERVER_ACK_TO_1STDATA_DELAY int,STREAMING_TYPE int,SOURCE_VIDEO_QUALITY > int,TETHERING_FLAG int,CARRIER_ID double,LAYER1ID int,LAYER2ID int,LAYER3ID > int,LAYER4ID int,LAYER5ID int,LAYER6ID int,CHARGING_RULE_BASE_NAME string,SP > string,EXTENDED_URL string,SV string,FIRST_SAI_CGI_ECGI > string,EXTENDED_URL_OTHER string,SIGNALING_USE_FLAG int,DNS_RETRANS_NUM > int,DNS_FAIL_CODE int,FIRST_RAT int,MS_INDICATOR string,LAST_SAI_CGI_ECGI > string,LAST_RAT int,FIRST_LONGITUDE double,FIRST_LATITUDE > double,FIRST_ALTITUDE int,FIRST_RASTERLONGITUDE double,FIRST_RASTERLATITUDE > double,FIRST_RASTERALTITUDE int,FIRST_FREQUENCYSPOT int,FIRST_CLUTTER > int,FIRST_USERBEHAVIOR int,FIRST_SPEED int,FIRST_CREDIBILITY > int,LAST_LONGITUDE double,LAST_LATITUDE double,LAST_ALTITUDE > int,LAST_RASTERLONGITUDE double,LAST_RASTERLATITUDE > double,LAST_RASTERALTITUDE int,LAST_FREQUENCYSPOT int,LAST_CLUTTER > int,LAST_USERBEHAVIOR int,LAST_SPEED int,LAST_CREDIBILITY int,IMEI_CIPHERTEXT > string,APP_ID int,DOMAIN_NAME string,STREAMING_CACHE_IP > string,STOP_LONGER_THAN_MIN_THRESHOLD int,STOP_LONGER_THAN_MAX_THRESHOLD > int,PLAY_END_STAT int,STOP_START_TIME1 double,STOP_END_TIME1 > double,STOP_START_TIME2 double,STOP_END_TIME2 double,STOP_START_TIME3 > double,STOP_END_TIME3 double,STOP_START_TIME4 double,STOP_END_TIME4 > double,STOP_START_TIME5 double,STOP_END_TIME5 double,STOP_START_TIME6 > double,STOP_END_TIME6 double,STOP_START_TIME7 double,STOP_END_TIME7 > double,STOP_START_TIME8 double,STOP_END_TIME8 double,STOP_START_TIME9 > double,STOP_END_TIME9 double,STOP_START_TIME10 double,STOP_END_TIME10 > double,FAIL_CLASS double,RECORD_TYPE double,NODATA_COUNT > double,VIDEO_NODATA_DURATION double,VIDEO_SMOOTH_DURATION > double,VIDEO_SD_DURATION double,VIDEO_HD_DURATION double,VIDEO_UHD_DURATION > double,VIDEO_FHD_DURATION double,FLUCTUATION double,START_DOWNLOAD_THROUGHPUT > double,L7_UL_GOODPUT_FULL_MSS double,SESSIONKEY string,FIRST_UCELLID > double,LAST_UCELLID double,UCELLID1 double,LONGITUDE1 double,LATITUDE1 > double,UCELLID2 double,LONGITUDE2 double,LATITUDE2 double,UCELLID3 > double,LONGITUDE3 double,LATITUDE3 double,UCELLID4 double,LONGITUDE4 > double,LATITUDE4 double,UCELLID5 double,LONGITUDE5 double,LATITUDE5 > double,UCELLID6 double,LONGITUDE6 double,LATITUDE6 double,UCELLID7 > double,LONGITUDE7 double,LATITUDE7 double,UCELLID8 double,LONGITUDE8 > double,LATITUDE8 double,UCELLID9 double,LONGITUDE9 double,LATITUDE9 > double,UCELLID10 double,LONGITUDE10 double,LATITUDE10 > double,INTBUFFER_FULL_DELAY double,STALL_DURATION double,STREAMING_DW_PACKETS > double,STREAMING_DOWNLOAD_DELAY double,PLAY_DURATION double,STREAMING_QUALITY > int,VIDEO_DATA_RATE double,AUDIO_DATA_RATE double,STREAMING_FILESIZE > double,STREAMING_DURATIOIN
[jira] [Updated] (CARBONDATA-668) Data loading failed on the latest build
[ https://issues.apache.org/jira/browse/CARBONDATA-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO updated CARBONDATA-668: - Attachment: seq_500Records.csv Environment: Spark-1.6 Description: Create Query: create table smart_500_DE (MSISDN string,IMSI string,IMEI string,INTERFACEID int,GROUPID int,GGSN_ID double,SGSN_ID double,SESSION_INDICATOR int,BEGIN_TIME double,BEGIN_TIME_MSEL int,END_TIME double,END_TIME_MSEL int,PROT_CATEGORY int,PROT_TYPE int,L7_CARRIER_PROT int,SUB_PROT_TYPE int,SID double,PROBEID double,ENCRYPT_VERSION int,ROAMING_TYPE int,ROAM_DIRECTION int,MS_IP string,SERVER_IP string,MS_PORT int,SERVER_DECIMAL Decimal,APN string,SGSN_SIG_IP string,GGSN_SIG_IP_BigInt_NEGATIVE bigint,SGSN_USER_IP string,GGSN_USER_IP string,MCC string,MNC string,RAT int,LAC string,RAC string,SAC string,CI string,BROWSER_TIMESTAMP timestamp,TCP_CONN_STATES int,TCP_STATES_BIGINTPOSITIVE int,TCP_WIN_SIZE int,TCP_MSS int,TCP_CONN_TIMES int,TCP_CONN_2_FAILED_TIMES int,TCP_CONN_3_FAILED_TIMES int,HOST string,STREAMING_URL string,GET_STREAMING_FAILED_CODE int,GET_STREAMING_FLAG int,GET_NUM int,GET_SUCCEED_NUM int,GET_RETRANS_NUM int,GET_TIMEOUT_NUM int,INTBUFFER_FST_FLAG int,INTBUFFER_FULL_FLAG int,STALL_NUM int,VIDEO_FRAME_RATE int,VIDEO_CODEC_ID string,VIDEO_WIDTH int,VIDEO_HEIGHT int,AUDIO_CODEC_ID string,MEDIA_FILE_TYPE int,PLAY_STATE int,STREAMING_FLAG int,TCP_STATUS_INDICATOR int,DISCONNECTION_FLAG int,FAILURE_CODE int,FLAG int,TAC string,ECI string,TCP_SYN_TIME_MSEL int,TCP_FST_SYN_DIRECTION int,RAN_NE_USER_IP string,HOMEMCC string,HOMEMNC string,CHARGE_FLAG int,PREPAID_FLAG int,USER_AGENT string,MS_WIN_STAT_TOTAL_NUM int,MS_WIN_STAT_SMALL_NUM int,MS_ACK_TO_1STGET_DELAY int,SERVER_ACK_TO_1STDATA_DELAY int,STREAMING_TYPE int,SOURCE_VIDEO_QUALITY int,TETHERING_FLAG int,CARRIER_ID double,LAYER1ID int,LAYER2ID int,LAYER3ID int,LAYER4ID int,LAYER5ID int,LAYER6ID int,CHARGING_RULE_BASE_NAME string,SP string,EXTENDED_URL string,SV string,FIRST_SAI_CGI_ECGI string,EXTENDED_URL_OTHER string,SIGNALING_USE_FLAG int,DNS_RETRANS_NUM int,DNS_FAIL_CODE int,FIRST_RAT int,MS_INDICATOR string,LAST_SAI_CGI_ECGI string,LAST_RAT int,FIRST_LONGITUDE double,FIRST_LATITUDE double,FIRST_ALTITUDE int,FIRST_RASTERLONGITUDE double,FIRST_RASTERLATITUDE double,FIRST_RASTERALTITUDE int,FIRST_FREQUENCYSPOT int,FIRST_CLUTTER int,FIRST_USERBEHAVIOR int,FIRST_SPEED int,FIRST_CREDIBILITY int,LAST_LONGITUDE double,LAST_LATITUDE double,LAST_ALTITUDE int,LAST_RASTERLONGITUDE double,LAST_RASTERLATITUDE double,LAST_RASTERALTITUDE int,LAST_FREQUENCYSPOT int,LAST_CLUTTER int,LAST_USERBEHAVIOR int,LAST_SPEED int,LAST_CREDIBILITY int,IMEI_CIPHERTEXT string,APP_ID int,DOMAIN_NAME string,STREAMING_CACHE_IP string,STOP_LONGER_THAN_MIN_THRESHOLD int,STOP_LONGER_THAN_MAX_THRESHOLD int,PLAY_END_STAT int,STOP_START_TIME1 double,STOP_END_TIME1 double,STOP_START_TIME2 double,STOP_END_TIME2 double,STOP_START_TIME3 double,STOP_END_TIME3 double,STOP_START_TIME4 double,STOP_END_TIME4 double,STOP_START_TIME5 double,STOP_END_TIME5 double,STOP_START_TIME6 double,STOP_END_TIME6 double,STOP_START_TIME7 double,STOP_END_TIME7 double,STOP_START_TIME8 double,STOP_END_TIME8 double,STOP_START_TIME9 double,STOP_END_TIME9 double,STOP_START_TIME10 double,STOP_END_TIME10 double,FAIL_CLASS double,RECORD_TYPE double,NODATA_COUNT double,VIDEO_NODATA_DURATION double,VIDEO_SMOOTH_DURATION double,VIDEO_SD_DURATION double,VIDEO_HD_DURATION double,VIDEO_UHD_DURATION double,VIDEO_FHD_DURATION double,FLUCTUATION double,START_DOWNLOAD_THROUGHPUT double,L7_UL_GOODPUT_FULL_MSS double,SESSIONKEY string,FIRST_UCELLID double,LAST_UCELLID double,UCELLID1 double,LONGITUDE1 double,LATITUDE1 double,UCELLID2 double,LONGITUDE2 double,LATITUDE2 double,UCELLID3 double,LONGITUDE3 double,LATITUDE3 double,UCELLID4 double,LONGITUDE4 double,LATITUDE4 double,UCELLID5 double,LONGITUDE5 double,LATITUDE5 double,UCELLID6 double,LONGITUDE6 double,LATITUDE6 double,UCELLID7 double,LONGITUDE7 double,LATITUDE7 double,UCELLID8 double,LONGITUDE8 double,LATITUDE8 double,UCELLID9 double,LONGITUDE9 double,LATITUDE9 double,UCELLID10 double,LONGITUDE10 double,LATITUDE10 double,INTBUFFER_FULL_DELAY double,STALL_DURATION double,STREAMING_DW_PACKETS double,STREAMING_DOWNLOAD_DELAY double,PLAY_DURATION double,STREAMING_QUALITY int,VIDEO_DATA_RATE double,AUDIO_DATA_RATE double,STREAMING_FILESIZE double,STREAMING_DURATIOIN double,TCP_SYN_TIME double,TCP_RTT_STEP1 double,CHARGE_ID double,UL_REVERSE_TO_DL_DELAY double,DL_REVERSE_TO_UL_DELAY double,DATATRANS_DW_GOODPUT double,DATATRANS_DW_TOTAL_DURATION double,SUM_FRAGMENT_INTERVAL double,TCP_FIN_TIMES double,TCP_RESET_TIMES double,URL_CLASSIFICATION double,STREAMING_LQ_DURATIOIN double,MAX_DNS_DELAY double,MAX_DNS2SYN double,MAX_LATANCY_OF_LINK_SETUP double,MAX_SYNACK2FIRSTACK double,MAX_SYNACK2LASTACK double,MAX_ACK2GET_DELAY
[jira] [Created] (CARBONDATA-668) Data loading failed on the latest build
SWATI RAO created CARBONDATA-668: Summary: Data loading failed on the latest build Key: CARBONDATA-668 URL: https://issues.apache.org/jira/browse/CARBONDATA-668 Project: CarbonData Issue Type: Bug Reporter: SWATI RAO Create Query: create table smart_500_DE (MSISDN string,IMSI string,IMEI string,INTERFACEID int,GROUPID int,GGSN_ID double,SGSN_ID double,SESSION_INDICATOR int,BEGIN_TIME double,BEGIN_TIME_MSEL int,END_TIME double,END_TIME_MSEL int,PROT_CATEGORY int,PROT_TYPE int,L7_CARRIER_PROT int,SUB_PROT_TYPE int,SID double,PROBEID double,ENCRYPT_VERSION int,ROAMING_TYPE int,ROAM_DIRECTION int,MS_IP string,SERVER_IP string,MS_PORT int,SERVER_DECIMAL Decimal,APN string,SGSN_SIG_IP string,GGSN_SIG_IP_BigInt_NEGATIVE bigint,SGSN_USER_IP string,GGSN_USER_IP string,MCC string,MNC string,RAT int,LAC string,RAC string,SAC string,CI string,BROWSER_TIMESTAMP timestamp,TCP_CONN_STATES int,TCP_STATES_BIGINTPOSITIVE int,TCP_WIN_SIZE int,TCP_MSS int,TCP_CONN_TIMES int,TCP_CONN_2_FAILED_TIMES int,TCP_CONN_3_FAILED_TIMES int,HOST string,STREAMING_URL string,GET_STREAMING_FAILED_CODE int,GET_STREAMING_FLAG int,GET_NUM int,GET_SUCCEED_NUM int,GET_RETRANS_NUM int,GET_TIMEOUT_NUM int,INTBUFFER_FST_FLAG int,INTBUFFER_FULL_FLAG int,STALL_NUM int,VIDEO_FRAME_RATE int,VIDEO_CODEC_ID string,VIDEO_WIDTH int,VIDEO_HEIGHT int,AUDIO_CODEC_ID string,MEDIA_FILE_TYPE int,PLAY_STATE int,STREAMING_FLAG int,TCP_STATUS_INDICATOR int,DISCONNECTION_FLAG int,FAILURE_CODE int,FLAG int,TAC string,ECI string,TCP_SYN_TIME_MSEL int,TCP_FST_SYN_DIRECTION int,RAN_NE_USER_IP string,HOMEMCC string,HOMEMNC string,CHARGE_FLAG int,PREPAID_FLAG int,USER_AGENT string,MS_WIN_STAT_TOTAL_NUM int,MS_WIN_STAT_SMALL_NUM int,MS_ACK_TO_1STGET_DELAY int,SERVER_ACK_TO_1STDATA_DELAY int,STREAMING_TYPE int,SOURCE_VIDEO_QUALITY int,TETHERING_FLAG int,CARRIER_ID double,LAYER1ID int,LAYER2ID int,LAYER3ID int,LAYER4ID int,LAYER5ID int,LAYER6ID int,CHARGING_RULE_BASE_NAME string,SP string,EXTENDED_URL string,SV string,FIRST_SAI_CGI_ECGI string,EXTENDED_URL_OTHER string,SIGNALING_USE_FLAG int,DNS_RETRANS_NUM int,DNS_FAIL_CODE int,FIRST_RAT int,MS_INDICATOR string,LAST_SAI_CGI_ECGI string,LAST_RAT int,FIRST_LONGITUDE double,FIRST_LATITUDE double,FIRST_ALTITUDE int,FIRST_RASTERLONGITUDE double,FIRST_RASTERLATITUDE double,FIRST_RASTERALTITUDE int,FIRST_FREQUENCYSPOT int,FIRST_CLUTTER int,FIRST_USERBEHAVIOR int,FIRST_SPEED int,FIRST_CREDIBILITY int,LAST_LONGITUDE double,LAST_LATITUDE double,LAST_ALTITUDE int,LAST_RASTERLONGITUDE double,LAST_RASTERLATITUDE double,LAST_RASTERALTITUDE int,LAST_FREQUENCYSPOT int,LAST_CLUTTER int,LAST_USERBEHAVIOR int,LAST_SPEED int,LAST_CREDIBILITY int,IMEI_CIPHERTEXT string,APP_ID int,DOMAIN_NAME string,STREAMING_CACHE_IP string,STOP_LONGER_THAN_MIN_THRESHOLD int,STOP_LONGER_THAN_MAX_THRESHOLD int,PLAY_END_STAT int,STOP_START_TIME1 double,STOP_END_TIME1 double,STOP_START_TIME2 double,STOP_END_TIME2 double,STOP_START_TIME3 double,STOP_END_TIME3 double,STOP_START_TIME4 double,STOP_END_TIME4 double,STOP_START_TIME5 double,STOP_END_TIME5 double,STOP_START_TIME6 double,STOP_END_TIME6 double,STOP_START_TIME7 double,STOP_END_TIME7 double,STOP_START_TIME8 double,STOP_END_TIME8 double,STOP_START_TIME9 double,STOP_END_TIME9 double,STOP_START_TIME10 double,STOP_END_TIME10 double,FAIL_CLASS double,RECORD_TYPE double,NODATA_COUNT double,VIDEO_NODATA_DURATION double,VIDEO_SMOOTH_DURATION double,VIDEO_SD_DURATION double,VIDEO_HD_DURATION double,VIDEO_UHD_DURATION double,VIDEO_FHD_DURATION double,FLUCTUATION double,START_DOWNLOAD_THROUGHPUT double,L7_UL_GOODPUT_FULL_MSS double,SESSIONKEY string,FIRST_UCELLID double,LAST_UCELLID double,UCELLID1 double,LONGITUDE1 double,LATITUDE1 double,UCELLID2 double,LONGITUDE2 double,LATITUDE2 double,UCELLID3 double,LONGITUDE3 double,LATITUDE3 double,UCELLID4 double,LONGITUDE4 double,LATITUDE4 double,UCELLID5 double,LONGITUDE5 double,LATITUDE5 double,UCELLID6 double,LONGITUDE6 double,LATITUDE6 double,UCELLID7 double,LONGITUDE7 double,LATITUDE7 double,UCELLID8 double,LONGITUDE8 double,LATITUDE8 double,UCELLID9 double,LONGITUDE9 double,LATITUDE9 double,UCELLID10 double,LONGITUDE10 double,LATITUDE10 double,INTBUFFER_FULL_DELAY double,STALL_DURATION double,STREAMING_DW_PACKETS double,STREAMING_DOWNLOAD_DELAY double,PLAY_DURATION double,STREAMING_QUALITY int,VIDEO_DATA_RATE double,AUDIO_DATA_RATE double,STREAMING_FILESIZE double,STREAMING_DURATIOIN double,TCP_SYN_TIME double,TCP_RTT_STEP1 double,CHARGE_ID double,UL_REVERSE_TO_DL_DELAY double,DL_REVERSE_TO_UL_DELAY double,DATATRANS_DW_GOODPUT double,DATATRANS_DW_TOTAL_DURATION double,SUM_FRAGMENT_INTERVAL double,TCP_FIN_TIMES double,TCP_RESET_TIMES double,URL_CLASSIFICATION double,STREAMING_LQ_DURATIOIN double,MAX_DNS_DELAY double,MAX_DNS2SYN double,MAX_LATANCY_OF_LINK_SETUP double,MAX_SYNACK2FIRSTACK
[jira] [Updated] (CARBONDATA-665) Comparision Failure occurs when we execute the same query in hive and Carbondata
[
https://issues.apache.org/jira/browse/CARBONDATA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO updated CARBONDATA-665:
-
Attachment: Test_Data1.csv
Description:
Orderby is not working , so records are not coming in sequence as well there is
data difference and some values being stored as null
Data itself is stored incorrectly and is different from Hive
Spark version :1.6.2
Create 1 query : create table Test_Boundary (c1_int int,c2_Bigint
Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string string,c6_Timestamp
Timestamp,c7_Datatype_Desc string) STORED BY 'org.apache.carbondata.format'
Load 1 Query : LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO
table Test_Boundary
OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
Create 2 query : create table Test_Boundary1 (c1_int int,c2_Bigint
Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string string,c6_Timestamp
Timestamp,c7_Datatype_Desc string) STORED BY 'org.apache.carbondata.format'
Load 2 query: LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO
table Test_Boundary1
OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
Select Query :
select
c1_int,c2_Bigint,c3_Decimal,c4_double,c5_string,c6_Timestamp,c7_Datatype_Desc
from Test_Boundary where c2_bigint=c2_bigint
was:
Create 1 query : create table Test_Boundary (c1_int int,c2_Bigint
Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string string,c6_Timestamp
Timestamp,c7_Datatype_Desc string) STORED BY 'org.apache.carbondata.format'
Load 1 Query : LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO
table Test_Boundary
OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
Create 2 query : create table Test_Boundary1 (c1_int int,c2_Bigint
Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string string,c6_Timestamp
Timestamp,c7_Datatype_Desc string) STORED BY 'org.apache.carbondata.format'
Load 2 query: LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO
table Test_Boundary1
OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
Select Query :
select
c1_int,c2_Bigint,c3_Decimal,c4_double,c5_string,c6_Timestamp,c7_Datatype_Desc
from Test_Boundary where c2_bigint=c2_bigint
> Comparision Failure occurs when we execute the same query in hive and
> Carbondata
>
>
> Key: CARBONDATA-665
> URL: https://issues.apache.org/jira/browse/CARBONDATA-665
> Project: CarbonData
> Issue Type: Bug
>Reporter: SWATI RAO
> Attachments: Test_Data1.csv
>
>
> Orderby is not working , so records are not coming in sequence as well there
> is data difference and some values being stored as null
> Data itself is stored incorrectly and is different from Hive
> Spark version :1.6.2
> Create 1 query : create table Test_Boundary (c1_int int,c2_Bigint
> Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format'
> Load 1 Query : LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO
> table Test_Boundary
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> Create 2 query : create table Test_Boundary1 (c1_int int,c2_Bigint
> Bigint,c3_Decimal Decimal(38,30),c4_double double,c5_string
> string,c6_Timestamp Timestamp,c7_Datatype_Desc string) STORED BY
> 'org.apache.carbondata.format'
> Load 2 query: LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/Test_Data1.csv' INTO
> table Test_Boundary1
> OPTIONS('DELIMITER'=',','QUOTECHAR'='','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> Select Query :
> select
> c1_int,c2_Bigint,c3_Decimal,c4_double,c5_string,c6_Timestamp,c7_Datatype_Desc
> from Test_Boundary where c2_bigint=c2_bigint
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (CARBONDATA-395) Unit Test cases for package org.apache.carbondata.scan.expression.ExpressionResult
[ https://issues.apache.org/jira/browse/CARBONDATA-395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-395. > Unit Test cases for package > org.apache.carbondata.scan.expression.ExpressionResult > -- > > Key: CARBONDATA-395 > URL: https://issues.apache.org/jira/browse/CARBONDATA-395 > Project: CarbonData > Issue Type: Test >Reporter: SWATI RAO >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-410) Implement test cases for core.datastore.file system
[ https://issues.apache.org/jira/browse/CARBONDATA-410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-410. > Implement test cases for core.datastore.file system > --- > > Key: CARBONDATA-410 > URL: https://issues.apache.org/jira/browse/CARBONDATA-410 > Project: CarbonData > Issue Type: Test >Reporter: SWATI RAO > Fix For: 1.0.0-incubating > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-416) Add unit test case for result.impl package
[ https://issues.apache.org/jira/browse/CARBONDATA-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-416. > Add unit test case for result.impl package > -- > > Key: CARBONDATA-416 > URL: https://issues.apache.org/jira/browse/CARBONDATA-416 > Project: CarbonData > Issue Type: Test >Reporter: SWATI RAO >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (CARBONDATA-427) In documentation ,applicable is not define properly in Parameters of Compaction
[ https://issues.apache.org/jira/browse/CARBONDATA-427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO reopened CARBONDATA-427: -- > In documentation ,applicable is not define properly in Parameters of > Compaction > > > Key: CARBONDATA-427 > URL: https://issues.apache.org/jira/browse/CARBONDATA-427 > Project: CarbonData > Issue Type: Bug > Components: docs >Reporter: SWATI RAO > > Applicable is not define properly in Parameters of Compaction like > In carbon.numberof.preserve.segments applicable is Minor/Major, it should be > minor > # carbon.allowed.compaction.days applicable is Minor/Major, it should be > minor > #carbon.number.of.cores.while.compacting applicable is Minor/Major, it should > be minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-427) In documentation ,applicable is not define properly in Parameters of Compaction
[ https://issues.apache.org/jira/browse/CARBONDATA-427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-427. Resolution: Fixed > In documentation ,applicable is not define properly in Parameters of > Compaction > > > Key: CARBONDATA-427 > URL: https://issues.apache.org/jira/browse/CARBONDATA-427 > Project: CarbonData > Issue Type: Bug > Components: docs >Reporter: SWATI RAO > > Applicable is not define properly in Parameters of Compaction like > In carbon.numberof.preserve.segments applicable is Minor/Major, it should be > minor > # carbon.allowed.compaction.days applicable is Minor/Major, it should be > minor > #carbon.number.of.cores.while.compacting applicable is Minor/Major, it should > be minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (CARBONDATA-419) Show tables in query for non-existing database do not show any exception
[ https://issues.apache.org/jira/browse/CARBONDATA-419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO reopened CARBONDATA-419: -- > Show tables in query for non-existing database do not show any exception > - > > Key: CARBONDATA-419 > URL: https://issues.apache.org/jira/browse/CARBONDATA-419 > Project: CarbonData > Issue Type: Bug > Components: data-query >Reporter: SWATI RAO >Assignee: Manohar Vanam > > When executing command 'show tables in' for an non-existing database, it do > not show any error, > Example: show tables in test_tableMg; > (where test_tableMg database do not exist) > whereas when executing the same query on hive, it says : > FAILED: SemanticException [Error 10072]: Database does not exist: test_tableMg -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-442) Query result mismatching with Hive
[
https://issues.apache.org/jira/browse/CARBONDATA-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO closed CARBONDATA-442.
> Query result mismatching with Hive
> --
>
> Key: CARBONDATA-442
> URL: https://issues.apache.org/jira/browse/CARBONDATA-442
> Project: CarbonData
> Issue Type: Bug
>Reporter: SWATI RAO
> Fix For: 1.0.0-incubating
>
>
> => I created table using following command :
> create table Carbon_automation_test5 (imei string,deviceInformationId int,MAC
> string,deviceColor string,device_backColor string,modelId string,marketName
> string,AMSize string,ROMSize string,CUPAudit string,CPIClocked string,series
> string,productionDate string,bomCode string,internalModels string,
> deliveryTime string, channelsId string,channelsName string , deliveryAreaId
> string, deliveryCountry string, deliveryProvince string, deliveryCity
> string,deliveryDistrict string, deliveryStreet string,oxSingleNumber string,
> ActiveCheckTime string, ActiveAreaId string, ActiveCountry string,
> ActiveProvince string, Activecity string, ActiveDistrict string, ActiveStreet
> string, ActiveOperatorId string, Active_releaseId string, Active_EMUIVersion
> string,Active_operaSysVersion string, Active_BacVerNumber string,
> Active_BacFlashVer string,Active_webUIVersion string, Active_webUITypeCarrVer
> string,Active_webTypeDataVerNumber string, Active_operatorsVersion string,
> Active_phonePADPartitionedVersions string,Latest_YEAR int, Latest_MONTH int,
> Latest_DAY int, Latest_HOUR string, Latest_areaId string, Latest_country
> string, Latest_province string, Latest_city string,Latest_district string,
> Latest_street string, Latest_releaseId string,Latest_EMUIVersion string,
> Latest_operaSysVersion string, Latest_BacVerNumber string,Latest_BacFlashVer
> string, Latest_webUIVersion string, Latest_webUITypeCarrVer
> string,Latest_webTypeDataVerNumber string, Latest_operatorsVersion
> string,Latest_phonePADPartitionedVersions string, Latest_operatorId
> string,gamePointDescription string, gamePointId int,contractNumber int)
> stored by 'org.apache.carbondata.format'
> => Load csv to table :
> LOAD DATA INPATH 'hdfs://localhost:54310/user/hduser/100_olap.csv' INTO table
> Carbon_automation_test5 OPTIONS('DELIMITER'= ',' ,'QUOTECHAR'= '"',
> 'FILEHEADER'=
> 'imei,deviceInformationId,MAC,deviceColor,device_backColor,modelId,marketName,AMSize,ROMSize,CUPAudit,CPIClocked,series,productionDate,bomCode,internalModels,deliveryTime,channelsId,channelsName,deliveryAreaId,deliveryCountry,deliveryProvince,deliveryCity,deliveryDistrict,deliveryStreet,oxSingleNumber,contractNumber,ActiveCheckTime,ActiveAreaId,ActiveCountry,ActiveProvince,Activecity,ActiveDistrict,ActiveStreet,ActiveOperatorId,Active_releaseId,Active_EMUIVersion,Active_operaSysVersion,Active_BacVerNumber,Active_BacFlashVer,Active_webUIVersion,Active_webUITypeCarrVer,Active_webTypeDataVerNumber,Active_operatorsVersion,Active_phonePADPartitionedVersions,Latest_YEAR,Latest_MONTH,Latest_DAY,Latest_HOUR,Latest_areaId,Latest_country,Latest_province,Latest_city,Latest_district,Latest_street,Latest_releaseId,Latest_EMUIVersion,Latest_operaSysVersion,Latest_BacVerNumber,Latest_BacFlashVer,Latest_webUIVersion,Latest_webUITypeCarrVer,Latest_webTypeDataVerNumber,Latest_operatorsVersion,Latest_phonePADPartitionedVersions,Latest_operatorId,gamePointId,gamePointDescription')
> =>now executed SELECT querry :
> SELECT Carbon_automation_test5.AMSize AS AMSize,
> Carbon_automation_test5.ActiveCountry AS ActiveCountry,
> Carbon_automation_test5.Activecity AS Activecity ,
> SUM(Carbon_automation_test5.gamePointId) AS Sum_gamePointId FROM ( SELECT
> AMSize,ActiveCountry,gamePointId, Activecity FROM (select * from
> Carbon_automation_test5) SUB_QRY ) Carbon_automation_test5 INNER JOIN (
> SELECT ActiveCountry, Activecity, AMSize FROM (select * from
> Carbon_automation_test5) SUB_QRY ) Carbon_automation_vmall_test1 ON
> Carbon_automation_test5.AMSize = Carbon_automation_vmall_test1.AMSize WHERE
> NOT(Carbon_automation_test5.AMSize <= '3RAM size') GROUP BY
> Carbon_automation_test5.AMSize, Carbon_automation_test5.ActiveCountry,
> Carbon_automation_test5.Activecity ORDER BY Carbon_automation_test5.AMSize
> ASC, Carbon_automation_test5.ActiveCountry ASC,
> Carbon_automation_test5.Activecity ASC;
> +++-+--+--+
> | AMSize | ActiveCountry | Activecity | Sum_gamePointId |
> +++-+--+--+
> | 4RAM size | Chinese| changsha| 200860 |
> | 4RAM size | Chinese| guangzhou | 38016|
> | 4RAM size | Chinese| shenzhen| 49610|
> | 4RAM size | Chinese| wuhan | 117568 |
> | 4RAM size |
[jira] [Closed] (CARBONDATA-444) Improved integration test-case for AllDataTypesTestCase1
[ https://issues.apache.org/jira/browse/CARBONDATA-444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-444. > Improved integration test-case for AllDataTypesTestCase1 > > > Key: CARBONDATA-444 > URL: https://issues.apache.org/jira/browse/CARBONDATA-444 > Project: CarbonData > Issue Type: Task >Reporter: SWATI RAO > Fix For: 1.0.0-incubating > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-438) Add unit test for scan.scanner.impl package
[ https://issues.apache.org/jira/browse/CARBONDATA-438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-438. > Add unit test for scan.scanner.impl package > --- > > Key: CARBONDATA-438 > URL: https://issues.apache.org/jira/browse/CARBONDATA-438 > Project: CarbonData > Issue Type: Test >Reporter: SWATI RAO >Assignee: Kunal Kapoor >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-460) Add Unit Tests For core.writer.sortindex package
[ https://issues.apache.org/jira/browse/CARBONDATA-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-460. > Add Unit Tests For core.writer.sortindex package > - > > Key: CARBONDATA-460 > URL: https://issues.apache.org/jira/browse/CARBONDATA-460 > Project: CarbonData > Issue Type: Test >Reporter: SWATI RAO >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-472) Improve code coverage for core.cache package.
[ https://issues.apache.org/jira/browse/CARBONDATA-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-472. > Improve code coverage for core.cache package. > - > > Key: CARBONDATA-472 > URL: https://issues.apache.org/jira/browse/CARBONDATA-472 > Project: CarbonData > Issue Type: Test >Reporter: SWATI RAO >Assignee: Kunal Kapoor >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-445) Improved integration test-case for AllDataTypesTestCase3
[ https://issues.apache.org/jira/browse/CARBONDATA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-445. > Improved integration test-case for AllDataTypesTestCase3 > - > > Key: CARBONDATA-445 > URL: https://issues.apache.org/jira/browse/CARBONDATA-445 > Project: CarbonData > Issue Type: Task >Reporter: SWATI RAO > Fix For: 1.0.0-incubating > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-450) Increase Test Coverage for Core.reader module
[ https://issues.apache.org/jira/browse/CARBONDATA-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-450. > Increase Test Coverage for Core.reader module > - > > Key: CARBONDATA-450 > URL: https://issues.apache.org/jira/browse/CARBONDATA-450 > Project: CarbonData > Issue Type: Test >Reporter: SWATI RAO > Fix For: 1.0.0-incubating > > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-483) Add Unit Tests For core.carbon.metadata package
[ https://issues.apache.org/jira/browse/CARBONDATA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO closed CARBONDATA-483. > Add Unit Tests For core.carbon.metadata package > --- > > Key: CARBONDATA-483 > URL: https://issues.apache.org/jira/browse/CARBONDATA-483 > Project: CarbonData > Issue Type: Test >Reporter: SWATI RAO >Priority: Trivial > Fix For: 1.0.0-incubating > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (CARBONDATA-394) Carbon Loading data from files having invalid extensions or no extension
[
https://issues.apache.org/jira/browse/CARBONDATA-394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SWATI RAO closed CARBONDATA-394.
Resolution: Not A Problem
> Carbon Loading data from files having invalid extensions or no extension
>
>
> Key: CARBONDATA-394
> URL: https://issues.apache.org/jira/browse/CARBONDATA-394
> Project: CarbonData
> Issue Type: Bug
>Reporter: SWATI RAO
>Assignee: xbkaishui
>Priority: Trivial
>
> When I try to run the following queries :
> LOAD DATA inpath 'hdfs://localhost:54310/user/hive/warehouse/file1.csv.csv'
> INTO table empdata options('DELIMITER'=',', 'FILEHEADER'='id,
> name','QUOTECHAR'='"');
> LOAD DATA inpath
> 'hdfs://localhost:54310/user/hive/warehouse/file2.csv.csv.csv.csv' INTO table
> empdata options('DELIMITER'=',', 'FILEHEADER'='id, name','QUOTECHAR'='"');
> LOAD DATA inpath 'hdfs://localhost:54310/user/hive/warehouse/file3.txttt'
> INTO table empdata options('DELIMITER'=',', 'FILEHEADER'='id,
> name','QUOTECHAR'='"');
> LOAD DATA inpath 'hdfs://localhost:54310/user/hive/warehouse/file4' INTO
> table empdata options('DELIMITER'=',', 'FILEHEADER'='id,
> name','QUOTECHAR'='"');
> LOAD DATA inpath
> 'hdfs://localhost:54310/user/hive/warehouse/file5.txt.bat.csv' INTO table
> empdata options('DELIMITER'=',', 'FILEHEADER'='id, name','QUOTECHAR'='"');
> We should get Input File Errors, but the data is loaded successfully into the
> Carbon table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CARBONDATA-394) Carbon Loading data from files having invalid extensions or no extension
[
https://issues.apache.org/jira/browse/CARBONDATA-394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750987#comment-15750987
]
SWATI RAO commented on CARBONDATA-394:
--
So we are closing this issue.
> Carbon Loading data from files having invalid extensions or no extension
>
>
> Key: CARBONDATA-394
> URL: https://issues.apache.org/jira/browse/CARBONDATA-394
> Project: CarbonData
> Issue Type: Bug
>Reporter: SWATI RAO
>Assignee: xbkaishui
>Priority: Trivial
>
> When I try to run the following queries :
> LOAD DATA inpath 'hdfs://localhost:54310/user/hive/warehouse/file1.csv.csv'
> INTO table empdata options('DELIMITER'=',', 'FILEHEADER'='id,
> name','QUOTECHAR'='"');
> LOAD DATA inpath
> 'hdfs://localhost:54310/user/hive/warehouse/file2.csv.csv.csv.csv' INTO table
> empdata options('DELIMITER'=',', 'FILEHEADER'='id, name','QUOTECHAR'='"');
> LOAD DATA inpath 'hdfs://localhost:54310/user/hive/warehouse/file3.txttt'
> INTO table empdata options('DELIMITER'=',', 'FILEHEADER'='id,
> name','QUOTECHAR'='"');
> LOAD DATA inpath 'hdfs://localhost:54310/user/hive/warehouse/file4' INTO
> table empdata options('DELIMITER'=',', 'FILEHEADER'='id,
> name','QUOTECHAR'='"');
> LOAD DATA inpath
> 'hdfs://localhost:54310/user/hive/warehouse/file5.txt.bat.csv' INTO table
> empdata options('DELIMITER'=',', 'FILEHEADER'='id, name','QUOTECHAR'='"');
> We should get Input File Errors, but the data is loaded successfully into the
> Carbon table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CARBONDATA-483) Add Unit Tests For core.carbon.metadata package
SWATI RAO created CARBONDATA-483: Summary: Add Unit Tests For core.carbon.metadata package Key: CARBONDATA-483 URL: https://issues.apache.org/jira/browse/CARBONDATA-483 Project: CarbonData Issue Type: Test Reporter: SWATI RAO Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CARBONDATA-361) SMALL INT data showing unsupported datatype
[ https://issues.apache.org/jira/browse/CARBONDATA-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15707814#comment-15707814 ] SWATI RAO commented on CARBONDATA-361: -- [~ravi.pesala] : As mentioned on your wiki page which is lastly updated on 10th Nov'16: SMALL INT data type should work but currently it isnot working. Either update the wiki page or we need to resolve this bug. https://cwiki.apache.org/confluence/display/CARBONDATA/Carbon+Data+Types > SMALL INT data showing unsupported datatype > --- > > Key: CARBONDATA-361 > URL: https://issues.apache.org/jira/browse/CARBONDATA-361 > Project: CarbonData > Issue Type: Bug >Reporter: Prabhat Kashyap > > When I tried executing the following query with SMALLINT datatype > create table testTable2(id SMALLINT, name String) stored by 'carbondata' ; > It is giving me following exception. > Error: org.apache.carbondata.spark.exception.MalformedCarbonCommandException: > Unsupported data type : FieldSchema(name:id, type:smallint, > comment:null).getType (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-472) Improve code coverage for core.cache package.
SWATI RAO created CARBONDATA-472: Summary: Improve code coverage for core.cache package. Key: CARBONDATA-472 URL: https://issues.apache.org/jira/browse/CARBONDATA-472 Project: CarbonData Issue Type: Test Reporter: SWATI RAO Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-457) Add Unit Tests For core.writer package
SWATI RAO created CARBONDATA-457: Summary: Add Unit Tests For core.writer package Key: CARBONDATA-457 URL: https://issues.apache.org/jira/browse/CARBONDATA-457 Project: CarbonData Issue Type: Test Reporter: SWATI RAO Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-450) Increase Test Coverage for Core.reader module
SWATI RAO created CARBONDATA-450: Summary: Increase Test Coverage for Core.reader module Key: CARBONDATA-450 URL: https://issues.apache.org/jira/browse/CARBONDATA-450 Project: CarbonData Issue Type: Test Reporter: SWATI RAO -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CARBONDATA-444) Improved integration test-case for AllDataTypesTestCase1
[ https://issues.apache.org/jira/browse/CARBONDATA-444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SWATI RAO updated CARBONDATA-444: - Issue Type: Task (was: Bug) > Improved integration test-case for AllDataTypesTestCase1 > > > Key: CARBONDATA-444 > URL: https://issues.apache.org/jira/browse/CARBONDATA-444 > Project: CarbonData > Issue Type: Task >Reporter: SWATI RAO > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-444) Improved integration test-case for AllDataTypesTestCase1
SWATI RAO created CARBONDATA-444: Summary: Improved integration test-case for AllDataTypesTestCase1 Key: CARBONDATA-444 URL: https://issues.apache.org/jira/browse/CARBONDATA-444 Project: CarbonData Issue Type: Bug Reporter: SWATI RAO -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-438) Add unit test for scan.scanner.impl package
SWATI RAO created CARBONDATA-438: Summary: Add unit test for scan.scanner.impl package Key: CARBONDATA-438 URL: https://issues.apache.org/jira/browse/CARBONDATA-438 Project: CarbonData Issue Type: Test Reporter: SWATI RAO Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-434) Update test cases for AllDataTypesTestCase2
SWATI RAO created CARBONDATA-434: Summary: Update test cases for AllDataTypesTestCase2 Key: CARBONDATA-434 URL: https://issues.apache.org/jira/browse/CARBONDATA-434 Project: CarbonData Issue Type: Improvement Reporter: SWATI RAO -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-435) improve integration test case for AllDataTypesTestCase4
SWATI RAO created CARBONDATA-435: Summary: improve integration test case for AllDataTypesTestCase4 Key: CARBONDATA-435 URL: https://issues.apache.org/jira/browse/CARBONDATA-435 Project: CarbonData Issue Type: Improvement Reporter: SWATI RAO -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-427) In documentation ,applicable is not define properly in Parameters of Compaction
SWATI RAO created CARBONDATA-427: Summary: In documentation ,applicable is not define properly in Parameters of Compaction Key: CARBONDATA-427 URL: https://issues.apache.org/jira/browse/CARBONDATA-427 Project: CarbonData Issue Type: Bug Components: docs Reporter: SWATI RAO Applicable is not define properly in Parameters of Compaction like In carbon.numberof.preserve.segmentsapplicable is Minor/Major, it should be minor # carbon.allowed.compaction.days applicable is Minor/Major, it should be minor #carbon.number.of.cores.while.compacting applicable is Minor/Major, it should be minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-410) Implement test cases for core.datastore.file system
SWATI RAO created CARBONDATA-410: Summary: Implement test cases for core.datastore.file system Key: CARBONDATA-410 URL: https://issues.apache.org/jira/browse/CARBONDATA-410 Project: CarbonData Issue Type: Task Reporter: SWATI RAO -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-408) Unable to create view from a table
SWATI RAO created CARBONDATA-408: Summary: Unable to create view from a table Key: CARBONDATA-408 URL: https://issues.apache.org/jira/browse/CARBONDATA-408 Project: CarbonData Issue Type: Bug Reporter: SWATI RAO Priority: Trivial When we tried to execute the following query to create view in carbon : create view emp_view AS Select name,sal from demo2; NOTE :demo2 table contains following columns: id Int, name String, sal decimal we got the following exception: Error: org.apache.spark.sql.execution.QueryExecutionException: FAILED: SemanticException [Error 10004]: Line 1:31 Invalid table alias or column reference 'name': (possible column names are: col) (state=,code=0) where as we are able to create view in hive using the same query. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-407) Missing documentation for data load using data frames
SWATI RAO created CARBONDATA-407: Summary: Missing documentation for data load using data frames Key: CARBONDATA-407 URL: https://issues.apache.org/jira/browse/CARBONDATA-407 Project: CarbonData Issue Type: Bug Components: data-load, docs Reporter: SWATI RAO Problem: Could not load data from a json file using data frames. Suggestion: Data can be loaded using data frames for the following cases : 1) Using case class (we have created a sample example for this) 2) From Json Data 3) Load data from existing parquet table to carbon data 4) Load data from one carbon table to another. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-395) Unit Test cases for package org.apache.carbondata.scan.expression.ExpressionResult
SWATI RAO created CARBONDATA-395: Summary: Unit Test cases for package org.apache.carbondata.scan.expression.ExpressionResult Key: CARBONDATA-395 URL: https://issues.apache.org/jira/browse/CARBONDATA-395 Project: CarbonData Issue Type: Test Reporter: SWATI RAO Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-394) Carbon Loading data from files having invalid extensions or no extension
SWATI RAO created CARBONDATA-394:
Summary: Carbon Loading data from files having invalid extensions
or no extension
Key: CARBONDATA-394
URL: https://issues.apache.org/jira/browse/CARBONDATA-394
Project: CarbonData
Issue Type: Bug
Reporter: SWATI RAO
Priority: Trivial
When I try to run the following queries :
LOAD DATA inpath 'hdfs://localhost:54310/user/hive/warehouse/file1.csv.csv'
INTO table empdata options('DELIMITER'=',', 'FILEHEADER'='id,
name','QUOTECHAR'='"');
LOAD DATA inpath
'hdfs://localhost:54310/user/hive/warehouse/file2.csv.csv.csv.csv' INTO table
empdata options('DELIMITER'=',', 'FILEHEADER'='id, name','QUOTECHAR'='"');
LOAD DATA inpath 'hdfs://localhost:54310/user/hive/warehouse/file3.txttt' INTO
table empdata options('DELIMITER'=',', 'FILEHEADER'='id, name','QUOTECHAR'='"');
LOAD DATA inpath 'hdfs://localhost:54310/user/hive/warehouse/file4' INTO table
empdata options('DELIMITER'=',', 'FILEHEADER'='id, name','QUOTECHAR'='"');
LOAD DATA inpath 'hdfs://localhost:54310/user/hive/warehouse/file5.txt.bat.csv'
INTO table empdata options('DELIMITER'=',', 'FILEHEADER'='id,
name','QUOTECHAR'='"');
We should get Input File Errors, but the data is loaded successfully into the
Carbon table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CARBONDATA-378) Long datatype is not working
[ https://issues.apache.org/jira/browse/CARBONDATA-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643479#comment-15643479 ] SWATI RAO commented on CARBONDATA-378: -- Okay. Thanks. > Long datatype is not working > > > Key: CARBONDATA-378 > URL: https://issues.apache.org/jira/browse/CARBONDATA-378 > Project: CarbonData > Issue Type: Bug >Reporter: SWATI RAO > > As it is mentioned in DataType.java file that long is supported datatype but > when I run query using long datatype its displays an error. > e.g: > create table testTable(id long, name String) stored by 'carbondata' ; > Error: org.apache.spark.sql.AnalysisException: cannot recognize input near > 'long' ',' 'name' in column type; line 1 pos 29 (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-378) Long datatype is not working
SWATI RAO created CARBONDATA-378: Summary: Long datatype is not working Key: CARBONDATA-378 URL: https://issues.apache.org/jira/browse/CARBONDATA-378 Project: CarbonData Issue Type: Bug Reporter: SWATI RAO As it is mentioned in DataType.java file that long is supported datatype but when I run query using long datatype its displays an error. e.g: create table testTable(id long, name String) stored by 'carbondata' ; Error: org.apache.spark.sql.AnalysisException: cannot recognize input near 'long' ',' 'name' in column type; line 1 pos 29 (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CARBONDATA-370) Abnormal behaviour of datatype Int & bigInt
SWATI RAO created CARBONDATA-370: Summary: Abnormal behaviour of datatype Int & bigInt Key: CARBONDATA-370 URL: https://issues.apache.org/jira/browse/CARBONDATA-370 Project: CarbonData Issue Type: Bug Reporter: SWATI RAO The specified data-type range for Int & bigInt is: INT –––> -2147483648 to 2147483647 BIGINT ––> -922337203685477580807 to 922337203685477580806 but I found abnormal behaviour in these data-type . 1.When I insert beyond range in Int it allows range even beyond bigInt. 2.When I insert into bigInt it doesn’t accept all values following in its range. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
