[jira] [Created] (CARBONDATA-346) Update unit test for core module
Kunal Kapoor created CARBONDATA-346: --- Summary: Update unit test for core module Key: CARBONDATA-346 URL: https://issues.apache.org/jira/browse/CARBONDATA-346 Project: CarbonData Issue Type: Test Reporter: Kunal Kapoor Priority: Trivial Improve code coverage for util package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CARBONDATA-689) User unable to create table using another database with carbon source.
[
https://issues.apache.org/jira/browse/CARBONDATA-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851291#comment-15851291
]
Kunal Kapoor commented on CARBONDATA-689:
-
This is Invalid because database name is mandatory in carbon source options.
> User unable to create table using another database with carbon source.
> --
>
> Key: CARBONDATA-689
> URL: https://issues.apache.org/jira/browse/CARBONDATA-689
> Project: CarbonData
> Issue Type: Bug
> Components: spark-integration
>Affects Versions: 1.1.0-incubating
> Environment: Spark 2.1
>Reporter: Vinod Rohilla
>Priority: Trivial
> Attachments: Screenshots.png
>
>
> I try to create the table in default database it was successfully created
> successfully and showing same in HDFS but when I create a new database(vinod)
> and inside I try to create the same table then as per the log it's created
> successfully but the table does not exist in HDFS(under vinod database).
> Steps to Reproduces:
> 1: User should be default database.
> 2: Create table by using the following command:
> " CREATE TABLE table11 (ID Int, date Timestamp, country String, name String,
> phonetype String, serialname String, salary Int) USING
> org.apache.spark.sql.CarbonSource OPTIONS("tableName"="table11"); "
> 3: Table created successfully in the default database.
> 4: Create database vinod;
> 5: Use vinod;
> 6: Create the same table:
> " CREATE TABLE table11 (ID Int, date Timestamp, country String, name
> String, phonetype String, serialname String, salary Int) USING
> org.apache.spark.sql.CarbonSource OPTIONS("tableName"="table11"); "
> 7: Tables shows created as per log.
> " 0: jdbc:hive2://localhost:1> CREATE TABLE table11 (ID Int, date
> Timestamp, country String, name String, phonetype String, serialname String,
> salary Int) USING org.apache.spark.sql.CarbonSource
> OPTIONS("tableName"="table11");
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.095 seconds)
> 0: jdbc:hive2://localhost:1> use vinod;
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.02 seconds)
> 0: jdbc:hive2://localhost:1> CREATE TABLE table11 (ID Int, date
> Timestamp, country String, name String, phonetype String, serialname String,
> salary Int) USING org.apache.spark.sql.CarbonSource
> OPTIONS("tableName"="table11");
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.093 seconds)
> "
> 8: Table does not create on HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CARBONDATA-773) During parallel load multiple instances of DictionaryServer are being created.
Kunal Kapoor created CARBONDATA-773: --- Summary: During parallel load multiple instances of DictionaryServer are being created. Key: CARBONDATA-773 URL: https://issues.apache.org/jira/browse/CARBONDATA-773 Project: CarbonData Issue Type: Improvement Reporter: Kunal Kapoor Priority: Minor During parallel load multiple instances of DictionaryServer are being created. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-773) During parallel load multiple instances of DictionaryServer are being created.
[ https://issues.apache.org/jira/browse/CARBONDATA-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor reassigned CARBONDATA-773: --- Assignee: Kunal Kapoor > During parallel load multiple instances of DictionaryServer are being created. > -- > > Key: CARBONDATA-773 > URL: https://issues.apache.org/jira/browse/CARBONDATA-773 > Project: CarbonData > Issue Type: Improvement >Reporter: Kunal Kapoor >Assignee: Kunal Kapoor >Priority: Minor > > During parallel load multiple instances of DictionaryServer are being created. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CARBONDATA-785) Compilation error in core module util package.
[ https://issues.apache.org/jira/browse/CARBONDATA-785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927868#comment-15927868 ] Kunal Kapoor commented on CARBONDATA-785: - Try running the following command mvn -Pspark-2.1 -Pbuild-with-format -DskipTests -Dscalastyle.skip clean install > Compilation error in core module util package. > -- > > Key: CARBONDATA-785 > URL: https://issues.apache.org/jira/browse/CARBONDATA-785 > Project: CarbonData > Issue Type: Bug > Components: build >Affects Versions: 1.1.0-incubating > Environment: Spark 2.1 >Reporter: Vinod Rohilla >Priority: Blocker > > Getting error while build creation. > Error: > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /opt/jenkins/workspace/CarbonData_Functional_Suite_New/incubator-carbondata/core/src/main/java/org/apache/carbondata/core/util/DataFileFooterConverterV3.java:[60,56] > cannot find symbol > symbol: method getTime_stamp() > location: variable fileHeader of type > org.apache.carbondata.format.FileHeader > [ERROR] > /opt/jenkins/workspace/CarbonData_Functional_Suite_New/incubator-carbondata/core/src/main/java/org/apache/carbondata/core/util/CarbonMetadataUtil.java:[903,15] > cannot find symbol > symbol: method setTime_stamp(long) > location: variable fileHeader of type > org.apache.carbondata.format.FileHeader > [INFO] 2 errors > Expected Result: Compilation error should not exist while build creation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (CARBONDATA-785) Compilation error in core module util package.
[ https://issues.apache.org/jira/browse/CARBONDATA-785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927868#comment-15927868 ] Kunal Kapoor edited comment on CARBONDATA-785 at 3/16/17 11:35 AM: --- Try running the following command mvn -Pspark-2.1 -Pbuild-with-format -DskipTests clean install was (Author: kunal): Try running the following command mvn -Pspark-2.1 -Pbuild-with-format -DskipTests -Dscalastyle.skip clean install > Compilation error in core module util package. > -- > > Key: CARBONDATA-785 > URL: https://issues.apache.org/jira/browse/CARBONDATA-785 > Project: CarbonData > Issue Type: Bug > Components: build >Affects Versions: 1.1.0-incubating > Environment: Spark 2.1 >Reporter: Vinod Rohilla >Priority: Blocker > > Getting error while build creation. > Error: > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /opt/jenkins/workspace/CarbonData_Functional_Suite_New/incubator-carbondata/core/src/main/java/org/apache/carbondata/core/util/DataFileFooterConverterV3.java:[60,56] > cannot find symbol > symbol: method getTime_stamp() > location: variable fileHeader of type > org.apache.carbondata.format.FileHeader > [ERROR] > /opt/jenkins/workspace/CarbonData_Functional_Suite_New/incubator-carbondata/core/src/main/java/org/apache/carbondata/core/util/CarbonMetadataUtil.java:[903,15] > cannot find symbol > symbol: method setTime_stamp(long) > location: variable fileHeader of type > org.apache.carbondata.format.FileHeader > [INFO] 2 errors > Expected Result: Compilation error should not exist while build creation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-696) NPE when select query run on measure having double data type without fraction.
[
https://issues.apache.org/jira/browse/CARBONDATA-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Kapoor reassigned CARBONDATA-696:
---
Assignee: Kunal Kapoor (was: kumar vishal)
> NPE when select query run on measure having double data type without
> fraction.
> ---
>
> Key: CARBONDATA-696
> URL: https://issues.apache.org/jira/browse/CARBONDATA-696
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.0.0-incubating
>Reporter: Babulal
>Assignee: Kunal Kapoor
> Attachments: logs, oscon_10.csv
>
>
> Create table as below
> cc.sql("create table oscon_carbon_old (CUST_PRFRD_FLG String,PROD_BRAND_NAME
> String,PROD_COLOR String,CUST_LAST_RVW_DATE String,CUST_COUNTRY
> String,CUST_CITY String,PRODUCT_NAME String,CUST_JOB_TITLE String,CUST_STATE
> String,CUST_BUY_POTENTIAL String,PRODUCT_MODEL String,ITM_ID String,ITM_NAME
> String,PRMTION_ID String,PRMTION_NAME String,SHP_MODE_ID String,SHP_MODE
> String,DELIVERY_COUNTRY String,DELIVERY_STATE String,DELIVERY_CITY
> String,DELIVERY_DISTRICT String,ACTIVE_EMUI_VERSION String,WH_NAME
> String,STR_ORDER_DATE String,OL_ORDER_NO String,OL_ORDER_DATE String,OL_SITE
> String,CUST_FIRST_NAME String,CUST_LAST_NAME String,CUST_BIRTH_DY
> String,CUST_BIRTH_MM String,CUST_BIRTH_YR String,CUST_BIRTH_COUNTRY
> String,CUST_SEX String,CUST_ADDRESS_ID String,CUST_STREET_NO
> String,CUST_STREET_NAME String,CUST_AGE String,CUST_SUITE_NO String,CUST_ZIP
> String,CUST_COUNTY String,PRODUCT_ID String,PROD_SHELL_COLOR
> String,DEVICE_NAME String,PROD_SHORT_DESC String,PROD_LONG_DESC
> String,PROD_THUMB String,PROD_IMAGE String,PROD_UPDATE_DATE String,PROD_LIVE
> String,PROD_LOC String,PROD_RAM String,PROD_ROM String,PROD_CPU_CLOCK
> String,PROD_SERIES String,ITM_REC_START_DATE String,ITM_REC_END_DATE
> String,ITM_BRAND_ID String,ITM_BRAND String,ITM_CLASS_ID String,ITM_CLASS
> String,ITM_CATEGORY_ID String,ITM_CATEGORY String,ITM_MANUFACT_ID
> String,ITM_MANUFACT String,ITM_FORMULATION String,ITM_COLOR
> String,ITM_CONTAINER String,ITM_MANAGER_ID String,PRM_START_DATE
> String,PRM_END_DATE String,PRM_CHANNEL_DMAIL String,PRM_CHANNEL_EMAIL
> String,PRM_CHANNEL_CAT String,PRM_CHANNEL_TV String,PRM_CHANNEL_RADIO
> String,PRM_CHANNEL_PRESS String,PRM_CHANNEL_EVENT String,PRM_CHANNEL_DEMO
> String,PRM_CHANNEL_DETAILS String,PRM_PURPOSE String,PRM_DSCNT_ACTIVE
> String,SHP_CODE String,SHP_CARRIER String,SHP_CONTRACT String,CHECK_DATE
> String,CHECK_YR String,CHECK_MM String,CHECK_DY String,CHECK_HOUR String,BOM
> String,INSIDE_NAME String,PACKING_DATE String,PACKING_YR String,PACKING_MM
> String,PACKING_DY String,PACKING_HOUR String,DELIVERY_PROVINCE
> String,PACKING_LIST_NO String,ACTIVE_CHECK_TIME String,ACTIVE_CHECK_YR
> String,ACTIVE_CHECK_MM String,ACTIVE_CHECK_DY String,ACTIVE_CHECK_HOUR
> String,ACTIVE_AREA_ID String,ACTIVE_COUNTRY String,ACTIVE_PROVINCE
> String,ACTIVE_CITY String,ACTIVE_DISTRICT String,ACTIVE_NETWORK
> String,ACTIVE_FIRMWARE_VER String,ACTIVE_OS_VERSION String,LATEST_CHECK_TIME
> String,LATEST_CHECK_YR String,LATEST_CHECK_MM String,LATEST_CHECK_DY
> String,LATEST_CHECK_HOUR String,LATEST_AREAID String,LATEST_COUNTRY
> String,LATEST_PROVINCE String,LATEST_CITY String,LATEST_DISTRICT
> String,LATEST_FIRMWARE_VER String,LATEST_EMUI_VERSION
> String,LATEST_OS_VERSION String,LATEST_NETWORK String,WH_ID
> String,WH_STREET_NO String,WH_STREET_NAME String,WH_STREET_TYPE
> String,WH_SUITE_NO String,WH_CITY String,WH_COUNTY String,WH_STATE
> String,WH_ZIP String,WH_COUNTRY String,OL_SITE_DESC String,OL_RET_ORDER_NO
> String,OL_RET_DATE String,PROD_MODEL_ID String,CUST_ID String,PROD_UNQ_MDL_ID
> String,CUST_NICK_NAME String,CUST_LOGIN String,CUST_EMAIL_ADDR
> String,PROD_UNQ_DEVICE_ADDR String,PROD_UQ_UUID String,PROD_BAR_CODE
> String,TRACKING_NO String,STR_ORDER_NO String,CUST_DEP_COUNT
> double,CUST_VEHICLE_COUNT double,CUST_ADDRESS_CNT double,CUST_CRNT_CDEMO_CNT
> double,CUST_CRNT_HDEMO_CNT double,CUST_CRNT_ADDR_DM
> double,CUST_FIRST_SHIPTO_CNT double,CUST_FIRST_SALES_CNT
> double,CUST_GMT_OFFSET double,CUST_DEMO_CNT double,CUST_INCOME
> double,PROD_UNLIMITED double,PROD_OFF_PRICE double,PROD_UNITS
> double,TOTAL_PRD_COST double,TOTAL_PRD_DISC double,PROD_WEIGHT
> double,REG_UNIT_PRICE double,EXTENDED_AMT double,UNIT_PRICE_DSCNT_PCT
> double,DSCNT_AMT double,PROD_STD_CST double,TOTAL_TX_AMT double,FREIGHT_CHRG
> double,WAITING_PERIOD double,DELIVERY_PERIOD double,ITM_CRNT_PRICE
> double,ITM_UNITS double,ITM_WSLE_CST double,ITM_SIZE double,PRM_CST
> double,PRM_RESPONSE_TARGET double,PRM_ITM_DM double,SHP_MODE_CNT
> double,WH_GMT_OFFSET double,WH_SQ_FT double,STR_ORD_QTY double,STR_WSLE_CST
> double,STR_LIST_PRICE double,STR_SAL
[jira] [Created] (CARBONDATA-839) Table lock file is not getting deleted after table rename is successful
Kunal Kapoor created CARBONDATA-839: --- Summary: Table lock file is not getting deleted after table rename is successful Key: CARBONDATA-839 URL: https://issues.apache.org/jira/browse/CARBONDATA-839 Project: CarbonData Issue Type: Bug Reporter: Kunal Kapoor Assignee: Kunal Kapoor Priority: Minor Table lock file is not getting deleted after table rename is successful -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-846) Add support to revert changes to alter table commands if there is a failure while executing the changes on hive.
Kunal Kapoor created CARBONDATA-846: --- Summary: Add support to revert changes to alter table commands if there is a failure while executing the changes on hive. Key: CARBONDATA-846 URL: https://issues.apache.org/jira/browse/CARBONDATA-846 Project: CarbonData Issue Type: Improvement Reporter: Kunal Kapoor Assignee: Kunal Kapoor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-847) Select query not working properly after alter.
[
https://issues.apache.org/jira/browse/CARBONDATA-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Kapoor reassigned CARBONDATA-847:
---
Assignee: Kunal Kapoor
> Select query not working properly after alter.
> --
>
> Key: CARBONDATA-847
> URL: https://issues.apache.org/jira/browse/CARBONDATA-847
> Project: CarbonData
> Issue Type: Bug
>Affects Versions: 1.1.0-incubating
> Environment: Spark2.1
>Reporter: SWATI RAO
>Assignee: Kunal Kapoor
> Attachments: 2000_UniqData.csv
>
>
> Execute these set of queries:
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 add columns(dict int)
> TBLPROPERTIES('DICTIONARY_INCLUDE'='dict','DEFAULT.VALUE.dict'= '');
> select distinct(dict) from uniqdata2 ;
> it will display the result but when we perform :
> select * from uniqdata1 ;
> it will display an error message :
> Job aborted due to stage failure: Task 3 in stage 59.0 failed 1 times, most
> recent failure: Lost task 3.0 in stage 59.0 (TID 714, localhost, executor
> driver): java.lang.NullPointerException
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CARBONDATA-810) Index builder exception display to user using select query.
[
https://issues.apache.org/jira/browse/CARBONDATA-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15956387#comment-15956387
]
Kunal Kapoor commented on CARBONDATA-810:
-
I am not able to reproduce this issue. Please re-verify and close
> Index builder exception display to user using select query.
> ---
>
> Key: CARBONDATA-810
> URL: https://issues.apache.org/jira/browse/CARBONDATA-810
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.1.0-incubating
> Environment: Spark 2.1
>Reporter: Vinod Rohilla
>Priority: Minor
> Attachments: FACT_UNITED_DATA_INFO_sample_cube.csv
>
>
> Error message display as Job aborted due to stage failure.
> Steps to reproduce:
> 1: Create the table in Carbon:
> create table IF NOT EXISTS traffic_2g_3g_4g (SOURCE_INFO String
> ,APP_CATEGORY_ID String ,APP_CATEGORY_NAME String ,APP_SUB_CATEGORY_ID String
> ,APP_SUB_CATEGORY_NAME String ,RAT_NAME String ,IMSI String ,OFFER_MSISDN
> String ,OFFER_ID String ,OFFER_OPTION_1 String ,OFFER_OPTION_2 String
> ,OFFER_OPTION_3 String ,MSISDN String ,PACKAGE_TYPE String ,PACKAGE_PRICE
> String ,TAG_IMSI String ,TAG_MSISDN String ,PROVINCE String ,CITY String
> ,AREA_CODE String ,TAC String ,IMEI String ,TERMINAL_TYPE String
> ,TERMINAL_BRAND String ,TERMINAL_MODEL String ,PRICE_LEVEL String ,NETWORK
> String ,SHIPPED_OS String ,WIFI String ,WIFI_HOTSPOT String ,GSM String
> ,WCDMA String ,TD_SCDMA String ,LTE_FDD String ,LTE_TDD String ,CDMA String
> ,SCREEN_SIZE String ,SCREEN_RESOLUTION String ,HOST_NAME String ,WEBSITE_NAME
> String ,OPERATOR String ,SRV_TYPE_NAME String ,TAG_HOST String ,CGI String
> ,CELL_NAME String ,COVERITY_TYPE1 String ,COVERITY_TYPE2 String
> ,COVERITY_TYPE3 String ,COVERITY_TYPE4 String ,COVERITY_TYPE5 String
> ,LATITUDE String ,LONGITUDE String ,AZIMUTH String ,TAG_CGI String ,APN
> String ,USER_AGENT String ,DAY String ,HOUR String ,MIN String
> ,IS_DEFAULT_BEAR int ,EPS_BEARER_ID String ,QCI int ,USER_FILTER String
> ,ANALYSIS_PERIOD String, UP_THROUGHPUT int,DOWN_THROUGHPUT int,UP_PKT_NUM
> int,DOWN_PKT_NUM int,APP_REQUEST_NUM int,PKT_NUM_LEN_1_64
> int,PKT_NUM_LEN_64_128 int,PKT_NUM_LEN_128_256 int,PKT_NUM_LEN_256_512
> int,PKT_NUM_LEN_512_768 int,PKT_NUM_LEN_768_1024 int,PKT_NUM_LEN_1024_ALL
> int,IP_FLOW_MARK int) STORED BY 'org.apache.carbondata.format'
> 2: Load the Data
> LOAD DATA INPATH 'HDFS_URL/FACT_UNITED_DATA_INFO_sample_cube.csv' INTO table
> traffic_2g_3g_4g OPTIONS ('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='')
> 3: Perform the Select Query
> select MSISDN,sum(UP_THROUGHPUT)+sum(DOWN_THROUGHPUT) as total from
> traffic_2g_3g_4g where TERMINAL_BRAND='HTC' and
> APP_CATEGORY_NAME='Web_Browsing' group by MSISDN
> Error on ThriftServer
> Job aborted due to stage failure: Task 1 in stage 12259.0 failed 4 times,
> most recent failure: Lost task 1.3 in stage 12259.0 (TID 642677,
> hadoop-slave-3, executor 9):
> org.apache.carbondata.core.datastore.exception.IndexBuilderException: Block
> B-tree loading failed
> at
> org.apache.carbondata.core.datastore.BlockIndexStore.fillLoadedBlocks(BlockIndexStore.java:264)
> at
> org.apache.carbondata.core.datastore.BlockIndexStore.getAll(BlockIndexStore.java:189)
> at
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:131)
> at
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfos(AbstractQueryExecutor.java:186)
> at
> org.apache.carbondata.core.scan.executor.impl.DetailQueryExecutor.execute(DetailQueryExecutor.java:39)
> at
> org.apache.carbondata.hadoop.CarbonRecordReader.initialize(CarbonRecordReader.java:79)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.scala:204)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scal
[jira] [Commented] (CARBONDATA-869) Greater than operator(>) does not display correct result for some attribute in range filter
[
https://issues.apache.org/jira/browse/CARBONDATA-869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15956817#comment-15956817
]
Kunal Kapoor commented on CARBONDATA-869:
-
hi
An issue has been already raised for this CARBONDATA-852.
Please mark as duplicate and close this.
> Greater than operator(>) does not display correct result for some attribute
> in range filter
> ---
>
> Key: CARBONDATA-869
> URL: https://issues.apache.org/jira/browse/CARBONDATA-869
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
> Environment: spark 2.1
>Reporter: Vandana Yadav
>Priority: Minor
> Attachments: 2000_UniqData.csv
>
>
> Greater than operator(>) does not display accurate result for some attributes
> in range filter
> Steps to reproduces:
> 1)Create table:
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> 2)Load Data in a table:
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> 3: Run the Query.
> select dob from uniqdata where dob > '1974-12-25' and dob < '1975';
> 4:Result on beeline:
> ++--+
> | dob |
> ++--+
> | 1974-12-25 01:00:03.0 |
> | 1974-12-26 01:00:03.0 |
> | 1974-12-27 01:00:03.0 |
> | 1974-12-28 01:00:03.0 |
> | 1974-12-29 01:00:03.0 |
> | 1974-12-30 01:00:03.0 |
> | 1974-12-31 01:00:03.0 |
> ++--+
> 7 rows selected (0.132 seconds)
> Expected Result: resultset should include dates from "1974-12-26 to
> 1974-12-31" and exclude "1974-12-25"
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-877) String datatype is throwing an error when included in DIctionary_Exclude in a alter query
[
https://issues.apache.org/jira/browse/CARBONDATA-877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Kapoor reassigned CARBONDATA-877:
---
Assignee: Kunal Kapoor
> String datatype is throwing an error when included in DIctionary_Exclude in a
> alter query
> -
>
> Key: CARBONDATA-877
> URL: https://issues.apache.org/jira/browse/CARBONDATA-877
> Project: CarbonData
> Issue Type: Bug
>Reporter: SWATI RAO
>Assignee: Kunal Kapoor
> Attachments: 2000_UniqData.csv
>
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> alter table uniqdata1 drop columns(CUST_NAME);
> alter table uniqdata1 add columns(CUST_NAME string)
> TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME',
> 'DEFAULT.VALUE.CUST_NAME'='testuser') ;
> Column added successfully. But when we execute:
> select distinct(CUST_NAME) from uniqdata1 ;
> &
> select count(CUST_NAME) from uniqdata1 ;
> it throws an error :
> "Job aborted due to stage failure: Task 0 in stage 9.0 failed 1 times, most
> recent failure: Lost task 0.0 in stage 9.0 (TID 206, localhost, executor
> driver): java.lang.ArrayIndexOutOfBoundsException: 4186"
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-883) Select query after alter is not working fine with integer datatype.
[
https://issues.apache.org/jira/browse/CARBONDATA-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Kapoor reassigned CARBONDATA-883:
---
Assignee: Kunal Kapoor
> Select query after alter is not working fine with integer datatype.
> ---
>
> Key: CARBONDATA-883
> URL: https://issues.apache.org/jira/browse/CARBONDATA-883
> Project: CarbonData
> Issue Type: Bug
> Environment: CREATE TABLE uniqdata (CUST_ID int,CUST_NAME
> String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp,
> BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
> DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2
> double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format'
> TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> ALTER TABLE uniqdata RENAME TO uniqdata1;
> 0: jdbc:hive2://192.168.2.126:1> alter table default.uniqdata1 add
> columns(ACTIVE_EMUI_VERSION int)
> TBLPROPERTIES('DEFAULT.VALUE.ACTIVE_EMUI_VERSION'='67890');
> +-+--+
> | Result |
> +-+--+
> +-+--+
> No rows selected (0.265 seconds)
> 0: jdbc:hive2://192.168.2.126:1> select distinct(ACTIVE_EMUI_VERSION)
> from uniqdata1 ;
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 22.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 22.0 (TID 1211, localhost, executor driver):
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.Integer
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingMeasures(RestructureBasedVectorResultCollector.java:193)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:98)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Driver stacktrace: (state=,code=0)
> 0: jdbc:hive2://192.168.2.126:1>
>Reporter: SWATI RAO
>Assignee: Kunal Kapoor
>
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES
> ("TABLE_BLOCKSIZE"= "256 MB");
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into
> table uniqdata OPTION
[jira] [Assigned] (CARBONDATA-892) IndexOutOf Bound exception while running query with 2nd level sub-query
[ https://issues.apache.org/jira/browse/CARBONDATA-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor reassigned CARBONDATA-892: --- Assignee: Kunal Kapoor > IndexOutOf Bound exception while running query with 2nd level sub-query > --- > > Key: CARBONDATA-892 > URL: https://issues.apache.org/jira/browse/CARBONDATA-892 > Project: CarbonData > Issue Type: Bug > Components: sql >Affects Versions: 0.2.0-incubating >Reporter: Ramakrishna >Assignee: Kunal Kapoor >Priority: Minor > Fix For: 1.2.0-incubating > > > select s_name, s_address from tpchcarbon_1.supplier where s_suppkey in > (select ps_suppkey from tpchcarbon_1.partsupp where ps_partkey in (select > p_partkey from tpchcarbon_1.part where p_name like 'forest%')) limit 10; > Error: java.lang.IndexOutOfBoundsException: 0 (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CARBONDATA-911) Exception raised while creating table using bucketing
[
https://issues.apache.org/jira/browse/CARBONDATA-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15965472#comment-15965472
]
Kunal Kapoor commented on CARBONDATA-911:
-
Column Groups are not supported for columns defined with DICTIONARY_EXCLUDE
> Exception raised while creating table using bucketing
> --
>
> Key: CARBONDATA-911
> URL: https://issues.apache.org/jira/browse/CARBONDATA-911
> Project: CarbonData
> Issue Type: Bug
> Components: docs
> Environment: Spark 2.1
>Reporter: Shivangi Gupta
>Priority: Minor
>
> Exception raised while creating table using bucketing: Column group is not
> supported for no dictionary columns
> Steps to reproduce:
> 1. Create table
> CREATE TABLE IF NOT EXISTS productSchema.productSalesTable (
> productNumber Int,
> productName String,
> storeCity String,
> storeProvince String,
> productCategory String,
> productBatch String,
> saleQuantity Int,
> revenue Int)
>STORED BY 'carbondata'
>TBLPROPERTIES ('COLUMN_GROUPS'='(productName,productNumber)',
> 'DICTIONARY_EXCLUDE'='productName',
> 'DICTIONARY_INCLUDE'='productNumber',
> 'NO_INVERTED_INDEX'='productBatch',
> 'BUCKETNUMBER'='4',
> 'BUCKETCOLUMNS'='productName');
> 2. Expected result :
> It should create table successfully
> 3. Result on beeline :
> Error: org.apache.carbondata.spark.exception.MalformedCarbonCommandException:
> Column group is not supported for no dictionary columns:productname
> (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CARBONDATA-926) Higher MAXCOLUMNS value in load DML options is leading to out of memory error
Kunal Kapoor created CARBONDATA-926: --- Summary: Higher MAXCOLUMNS value in load DML options is leading to out of memory error Key: CARBONDATA-926 URL: https://issues.apache.org/jira/browse/CARBONDATA-926 Project: CarbonData Issue Type: Improvement Reporter: Kunal Kapoor Assignee: Kunal Kapoor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (CARBONDATA-926) Higher MAXCOLUMNS value in load DML options is leading to out of memory error
[ https://issues.apache.org/jira/browse/CARBONDATA-926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-926: Priority: Minor (was: Major) > Higher MAXCOLUMNS value in load DML options is leading to out of memory error > - > > Key: CARBONDATA-926 > URL: https://issues.apache.org/jira/browse/CARBONDATA-926 > Project: CarbonData > Issue Type: Improvement >Reporter: Kunal Kapoor >Assignee: Kunal Kapoor >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (CARBONDATA-926) Set the max column from options in csv parser settings
[ https://issues.apache.org/jira/browse/CARBONDATA-926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-926: Summary: Set the max column from options in csv parser settings (was: Higher MAXCOLUMNS value in load DML options is leading to out of memory error) > Set the max column from options in csv parser settings > -- > > Key: CARBONDATA-926 > URL: https://issues.apache.org/jira/browse/CARBONDATA-926 > Project: CarbonData > Issue Type: Improvement >Reporter: Kunal Kapoor >Assignee: Kunal Kapoor >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-957) Table not found exception in rename table after lock acquire failure
Kunal Kapoor created CARBONDATA-957: --- Summary: Table not found exception in rename table after lock acquire failure Key: CARBONDATA-957 URL: https://issues.apache.org/jira/browse/CARBONDATA-957 Project: CarbonData Issue Type: Bug Reporter: Kunal Kapoor Assignee: Kunal Kapoor Priority: Minor During rename table if an exception is thrown during acquiring locks then table not exists exception was thrown while reverting the changes. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
