[jira] [Created] (CARBONDATA-346) Update unit test for core module
Kunal Kapoor created CARBONDATA-346: --- Summary: Update unit test for core module Key: CARBONDATA-346 URL: https://issues.apache.org/jira/browse/CARBONDATA-346 Project: CarbonData Issue Type: Test Reporter: Kunal Kapoor Priority: Trivial Improve code coverage for util package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CARBONDATA-689) User unable to create table using another database with carbon source.
[ https://issues.apache.org/jira/browse/CARBONDATA-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851291#comment-15851291 ] Kunal Kapoor commented on CARBONDATA-689: - This is Invalid because database name is mandatory in carbon source options. > User unable to create table using another database with carbon source. > -- > > Key: CARBONDATA-689 > URL: https://issues.apache.org/jira/browse/CARBONDATA-689 > Project: CarbonData > Issue Type: Bug > Components: spark-integration >Affects Versions: 1.1.0-incubating > Environment: Spark 2.1 >Reporter: Vinod Rohilla >Priority: Trivial > Attachments: Screenshots.png > > > I try to create the table in default database it was successfully created > successfully and showing same in HDFS but when I create a new database(vinod) > and inside I try to create the same table then as per the log it's created > successfully but the table does not exist in HDFS(under vinod database). > Steps to Reproduces: > 1: User should be default database. > 2: Create table by using the following command: > " CREATE TABLE table11 (ID Int, date Timestamp, country String, name String, > phonetype String, serialname String, salary Int) USING > org.apache.spark.sql.CarbonSource OPTIONS("tableName"="table11"); " > 3: Table created successfully in the default database. > 4: Create database vinod; > 5: Use vinod; > 6: Create the same table: > " CREATE TABLE table11 (ID Int, date Timestamp, country String, name > String, phonetype String, serialname String, salary Int) USING > org.apache.spark.sql.CarbonSource OPTIONS("tableName"="table11"); " > 7: Tables shows created as per log. > " 0: jdbc:hive2://localhost:1> CREATE TABLE table11 (ID Int, date > Timestamp, country String, name String, phonetype String, serialname String, > salary Int) USING org.apache.spark.sql.CarbonSource > OPTIONS("tableName"="table11"); > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (0.095 seconds) > 0: jdbc:hive2://localhost:1> use vinod; > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (0.02 seconds) > 0: jdbc:hive2://localhost:1> CREATE TABLE table11 (ID Int, date > Timestamp, country String, name String, phonetype String, serialname String, > salary Int) USING org.apache.spark.sql.CarbonSource > OPTIONS("tableName"="table11"); > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (0.093 seconds) > " > 8: Table does not create on HDFS. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-773) During parallel load multiple instances of DictionaryServer are being created.
Kunal Kapoor created CARBONDATA-773: --- Summary: During parallel load multiple instances of DictionaryServer are being created. Key: CARBONDATA-773 URL: https://issues.apache.org/jira/browse/CARBONDATA-773 Project: CarbonData Issue Type: Improvement Reporter: Kunal Kapoor Priority: Minor During parallel load multiple instances of DictionaryServer are being created. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-773) During parallel load multiple instances of DictionaryServer are being created.
[ https://issues.apache.org/jira/browse/CARBONDATA-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor reassigned CARBONDATA-773: --- Assignee: Kunal Kapoor > During parallel load multiple instances of DictionaryServer are being created. > -- > > Key: CARBONDATA-773 > URL: https://issues.apache.org/jira/browse/CARBONDATA-773 > Project: CarbonData > Issue Type: Improvement >Reporter: Kunal Kapoor >Assignee: Kunal Kapoor >Priority: Minor > > During parallel load multiple instances of DictionaryServer are being created. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CARBONDATA-785) Compilation error in core module util package.
[ https://issues.apache.org/jira/browse/CARBONDATA-785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927868#comment-15927868 ] Kunal Kapoor commented on CARBONDATA-785: - Try running the following command mvn -Pspark-2.1 -Pbuild-with-format -DskipTests -Dscalastyle.skip clean install > Compilation error in core module util package. > -- > > Key: CARBONDATA-785 > URL: https://issues.apache.org/jira/browse/CARBONDATA-785 > Project: CarbonData > Issue Type: Bug > Components: build >Affects Versions: 1.1.0-incubating > Environment: Spark 2.1 >Reporter: Vinod Rohilla >Priority: Blocker > > Getting error while build creation. > Error: > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /opt/jenkins/workspace/CarbonData_Functional_Suite_New/incubator-carbondata/core/src/main/java/org/apache/carbondata/core/util/DataFileFooterConverterV3.java:[60,56] > cannot find symbol > symbol: method getTime_stamp() > location: variable fileHeader of type > org.apache.carbondata.format.FileHeader > [ERROR] > /opt/jenkins/workspace/CarbonData_Functional_Suite_New/incubator-carbondata/core/src/main/java/org/apache/carbondata/core/util/CarbonMetadataUtil.java:[903,15] > cannot find symbol > symbol: method setTime_stamp(long) > location: variable fileHeader of type > org.apache.carbondata.format.FileHeader > [INFO] 2 errors > Expected Result: Compilation error should not exist while build creation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (CARBONDATA-785) Compilation error in core module util package.
[ https://issues.apache.org/jira/browse/CARBONDATA-785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927868#comment-15927868 ] Kunal Kapoor edited comment on CARBONDATA-785 at 3/16/17 11:35 AM: --- Try running the following command mvn -Pspark-2.1 -Pbuild-with-format -DskipTests clean install was (Author: kunal): Try running the following command mvn -Pspark-2.1 -Pbuild-with-format -DskipTests -Dscalastyle.skip clean install > Compilation error in core module util package. > -- > > Key: CARBONDATA-785 > URL: https://issues.apache.org/jira/browse/CARBONDATA-785 > Project: CarbonData > Issue Type: Bug > Components: build >Affects Versions: 1.1.0-incubating > Environment: Spark 2.1 >Reporter: Vinod Rohilla >Priority: Blocker > > Getting error while build creation. > Error: > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /opt/jenkins/workspace/CarbonData_Functional_Suite_New/incubator-carbondata/core/src/main/java/org/apache/carbondata/core/util/DataFileFooterConverterV3.java:[60,56] > cannot find symbol > symbol: method getTime_stamp() > location: variable fileHeader of type > org.apache.carbondata.format.FileHeader > [ERROR] > /opt/jenkins/workspace/CarbonData_Functional_Suite_New/incubator-carbondata/core/src/main/java/org/apache/carbondata/core/util/CarbonMetadataUtil.java:[903,15] > cannot find symbol > symbol: method setTime_stamp(long) > location: variable fileHeader of type > org.apache.carbondata.format.FileHeader > [INFO] 2 errors > Expected Result: Compilation error should not exist while build creation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-696) NPE when select query run on measure having double data type without fraction.
[ https://issues.apache.org/jira/browse/CARBONDATA-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor reassigned CARBONDATA-696: --- Assignee: Kunal Kapoor (was: kumar vishal) > NPE when select query run on measure having double data type without > fraction. > --- > > Key: CARBONDATA-696 > URL: https://issues.apache.org/jira/browse/CARBONDATA-696 > Project: CarbonData > Issue Type: Bug > Components: data-query >Affects Versions: 1.0.0-incubating >Reporter: Babulal >Assignee: Kunal Kapoor > Attachments: logs, oscon_10.csv > > > Create table as below > cc.sql("create table oscon_carbon_old (CUST_PRFRD_FLG String,PROD_BRAND_NAME > String,PROD_COLOR String,CUST_LAST_RVW_DATE String,CUST_COUNTRY > String,CUST_CITY String,PRODUCT_NAME String,CUST_JOB_TITLE String,CUST_STATE > String,CUST_BUY_POTENTIAL String,PRODUCT_MODEL String,ITM_ID String,ITM_NAME > String,PRMTION_ID String,PRMTION_NAME String,SHP_MODE_ID String,SHP_MODE > String,DELIVERY_COUNTRY String,DELIVERY_STATE String,DELIVERY_CITY > String,DELIVERY_DISTRICT String,ACTIVE_EMUI_VERSION String,WH_NAME > String,STR_ORDER_DATE String,OL_ORDER_NO String,OL_ORDER_DATE String,OL_SITE > String,CUST_FIRST_NAME String,CUST_LAST_NAME String,CUST_BIRTH_DY > String,CUST_BIRTH_MM String,CUST_BIRTH_YR String,CUST_BIRTH_COUNTRY > String,CUST_SEX String,CUST_ADDRESS_ID String,CUST_STREET_NO > String,CUST_STREET_NAME String,CUST_AGE String,CUST_SUITE_NO String,CUST_ZIP > String,CUST_COUNTY String,PRODUCT_ID String,PROD_SHELL_COLOR > String,DEVICE_NAME String,PROD_SHORT_DESC String,PROD_LONG_DESC > String,PROD_THUMB String,PROD_IMAGE String,PROD_UPDATE_DATE String,PROD_LIVE > String,PROD_LOC String,PROD_RAM String,PROD_ROM String,PROD_CPU_CLOCK > String,PROD_SERIES String,ITM_REC_START_DATE String,ITM_REC_END_DATE > String,ITM_BRAND_ID String,ITM_BRAND String,ITM_CLASS_ID String,ITM_CLASS > String,ITM_CATEGORY_ID String,ITM_CATEGORY String,ITM_MANUFACT_ID > String,ITM_MANUFACT String,ITM_FORMULATION String,ITM_COLOR > String,ITM_CONTAINER String,ITM_MANAGER_ID String,PRM_START_DATE > String,PRM_END_DATE String,PRM_CHANNEL_DMAIL String,PRM_CHANNEL_EMAIL > String,PRM_CHANNEL_CAT String,PRM_CHANNEL_TV String,PRM_CHANNEL_RADIO > String,PRM_CHANNEL_PRESS String,PRM_CHANNEL_EVENT String,PRM_CHANNEL_DEMO > String,PRM_CHANNEL_DETAILS String,PRM_PURPOSE String,PRM_DSCNT_ACTIVE > String,SHP_CODE String,SHP_CARRIER String,SHP_CONTRACT String,CHECK_DATE > String,CHECK_YR String,CHECK_MM String,CHECK_DY String,CHECK_HOUR String,BOM > String,INSIDE_NAME String,PACKING_DATE String,PACKING_YR String,PACKING_MM > String,PACKING_DY String,PACKING_HOUR String,DELIVERY_PROVINCE > String,PACKING_LIST_NO String,ACTIVE_CHECK_TIME String,ACTIVE_CHECK_YR > String,ACTIVE_CHECK_MM String,ACTIVE_CHECK_DY String,ACTIVE_CHECK_HOUR > String,ACTIVE_AREA_ID String,ACTIVE_COUNTRY String,ACTIVE_PROVINCE > String,ACTIVE_CITY String,ACTIVE_DISTRICT String,ACTIVE_NETWORK > String,ACTIVE_FIRMWARE_VER String,ACTIVE_OS_VERSION String,LATEST_CHECK_TIME > String,LATEST_CHECK_YR String,LATEST_CHECK_MM String,LATEST_CHECK_DY > String,LATEST_CHECK_HOUR String,LATEST_AREAID String,LATEST_COUNTRY > String,LATEST_PROVINCE String,LATEST_CITY String,LATEST_DISTRICT > String,LATEST_FIRMWARE_VER String,LATEST_EMUI_VERSION > String,LATEST_OS_VERSION String,LATEST_NETWORK String,WH_ID > String,WH_STREET_NO String,WH_STREET_NAME String,WH_STREET_TYPE > String,WH_SUITE_NO String,WH_CITY String,WH_COUNTY String,WH_STATE > String,WH_ZIP String,WH_COUNTRY String,OL_SITE_DESC String,OL_RET_ORDER_NO > String,OL_RET_DATE String,PROD_MODEL_ID String,CUST_ID String,PROD_UNQ_MDL_ID > String,CUST_NICK_NAME String,CUST_LOGIN String,CUST_EMAIL_ADDR > String,PROD_UNQ_DEVICE_ADDR String,PROD_UQ_UUID String,PROD_BAR_CODE > String,TRACKING_NO String,STR_ORDER_NO String,CUST_DEP_COUNT > double,CUST_VEHICLE_COUNT double,CUST_ADDRESS_CNT double,CUST_CRNT_CDEMO_CNT > double,CUST_CRNT_HDEMO_CNT double,CUST_CRNT_ADDR_DM > double,CUST_FIRST_SHIPTO_CNT double,CUST_FIRST_SALES_CNT > double,CUST_GMT_OFFSET double,CUST_DEMO_CNT double,CUST_INCOME > double,PROD_UNLIMITED double,PROD_OFF_PRICE double,PROD_UNITS > double,TOTAL_PRD_COST double,TOTAL_PRD_DISC double,PROD_WEIGHT > double,REG_UNIT_PRICE double,EXTENDED_AMT double,UNIT_PRICE_DSCNT_PCT > double,DSCNT_AMT double,PROD_STD_CST double,TOTAL_TX_AMT double,FREIGHT_CHRG > double,WAITING_PERIOD double,DELIVERY_PERIOD double,ITM_CRNT_PRICE > double,ITM_UNITS double,ITM_WSLE_CST double,ITM_SIZE double,PRM_CST > double,PRM_RESPONSE_TARGET double,PRM_ITM_DM double,SHP_MODE_CNT > double,WH_GMT_OFFSET double,WH_SQ_FT double,STR_ORD_QTY double,STR_WSLE_CST > double,STR_LIST_PRICE double,STR_SAL
[jira] [Created] (CARBONDATA-839) Table lock file is not getting deleted after table rename is successful
Kunal Kapoor created CARBONDATA-839: --- Summary: Table lock file is not getting deleted after table rename is successful Key: CARBONDATA-839 URL: https://issues.apache.org/jira/browse/CARBONDATA-839 Project: CarbonData Issue Type: Bug Reporter: Kunal Kapoor Assignee: Kunal Kapoor Priority: Minor Table lock file is not getting deleted after table rename is successful -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-846) Add support to revert changes to alter table commands if there is a failure while executing the changes on hive.
Kunal Kapoor created CARBONDATA-846: --- Summary: Add support to revert changes to alter table commands if there is a failure while executing the changes on hive. Key: CARBONDATA-846 URL: https://issues.apache.org/jira/browse/CARBONDATA-846 Project: CarbonData Issue Type: Improvement Reporter: Kunal Kapoor Assignee: Kunal Kapoor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-847) Select query not working properly after alter.
[ https://issues.apache.org/jira/browse/CARBONDATA-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor reassigned CARBONDATA-847: --- Assignee: Kunal Kapoor > Select query not working properly after alter. > -- > > Key: CARBONDATA-847 > URL: https://issues.apache.org/jira/browse/CARBONDATA-847 > Project: CarbonData > Issue Type: Bug >Affects Versions: 1.1.0-incubating > Environment: Spark2.1 >Reporter: SWATI RAO >Assignee: Kunal Kapoor > Attachments: 2000_UniqData.csv > > > Execute these set of queries: > CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION > string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 > bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 > decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 > int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES > ("TABLE_BLOCKSIZE"= "256 MB"); > LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into > table uniqdata OPTIONS('DELIMITER'=',' , > 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1'); > ALTER TABLE uniqdata RENAME TO uniqdata1; > alter table uniqdata1 add columns(dict int) > TBLPROPERTIES('DICTIONARY_INCLUDE'='dict','DEFAULT.VALUE.dict'= ''); > select distinct(dict) from uniqdata2 ; > it will display the result but when we perform : > select * from uniqdata1 ; > it will display an error message : > Job aborted due to stage failure: Task 3 in stage 59.0 failed 1 times, most > recent failure: Lost task 3.0 in stage 59.0 (TID 714, localhost, executor > driver): java.lang.NullPointerException -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CARBONDATA-810) Index builder exception display to user using select query.
[ https://issues.apache.org/jira/browse/CARBONDATA-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15956387#comment-15956387 ] Kunal Kapoor commented on CARBONDATA-810: - I am not able to reproduce this issue. Please re-verify and close > Index builder exception display to user using select query. > --- > > Key: CARBONDATA-810 > URL: https://issues.apache.org/jira/browse/CARBONDATA-810 > Project: CarbonData > Issue Type: Bug > Components: data-query >Affects Versions: 1.1.0-incubating > Environment: Spark 2.1 >Reporter: Vinod Rohilla >Priority: Minor > Attachments: FACT_UNITED_DATA_INFO_sample_cube.csv > > > Error message display as Job aborted due to stage failure. > Steps to reproduce: > 1: Create the table in Carbon: > create table IF NOT EXISTS traffic_2g_3g_4g (SOURCE_INFO String > ,APP_CATEGORY_ID String ,APP_CATEGORY_NAME String ,APP_SUB_CATEGORY_ID String > ,APP_SUB_CATEGORY_NAME String ,RAT_NAME String ,IMSI String ,OFFER_MSISDN > String ,OFFER_ID String ,OFFER_OPTION_1 String ,OFFER_OPTION_2 String > ,OFFER_OPTION_3 String ,MSISDN String ,PACKAGE_TYPE String ,PACKAGE_PRICE > String ,TAG_IMSI String ,TAG_MSISDN String ,PROVINCE String ,CITY String > ,AREA_CODE String ,TAC String ,IMEI String ,TERMINAL_TYPE String > ,TERMINAL_BRAND String ,TERMINAL_MODEL String ,PRICE_LEVEL String ,NETWORK > String ,SHIPPED_OS String ,WIFI String ,WIFI_HOTSPOT String ,GSM String > ,WCDMA String ,TD_SCDMA String ,LTE_FDD String ,LTE_TDD String ,CDMA String > ,SCREEN_SIZE String ,SCREEN_RESOLUTION String ,HOST_NAME String ,WEBSITE_NAME > String ,OPERATOR String ,SRV_TYPE_NAME String ,TAG_HOST String ,CGI String > ,CELL_NAME String ,COVERITY_TYPE1 String ,COVERITY_TYPE2 String > ,COVERITY_TYPE3 String ,COVERITY_TYPE4 String ,COVERITY_TYPE5 String > ,LATITUDE String ,LONGITUDE String ,AZIMUTH String ,TAG_CGI String ,APN > String ,USER_AGENT String ,DAY String ,HOUR String ,MIN String > ,IS_DEFAULT_BEAR int ,EPS_BEARER_ID String ,QCI int ,USER_FILTER String > ,ANALYSIS_PERIOD String, UP_THROUGHPUT int,DOWN_THROUGHPUT int,UP_PKT_NUM > int,DOWN_PKT_NUM int,APP_REQUEST_NUM int,PKT_NUM_LEN_1_64 > int,PKT_NUM_LEN_64_128 int,PKT_NUM_LEN_128_256 int,PKT_NUM_LEN_256_512 > int,PKT_NUM_LEN_512_768 int,PKT_NUM_LEN_768_1024 int,PKT_NUM_LEN_1024_ALL > int,IP_FLOW_MARK int) STORED BY 'org.apache.carbondata.format' > 2: Load the Data > LOAD DATA INPATH 'HDFS_URL/FACT_UNITED_DATA_INFO_sample_cube.csv' INTO table > traffic_2g_3g_4g OPTIONS ('DELIMITER'=',' , > 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='') > 3: Perform the Select Query > select MSISDN,sum(UP_THROUGHPUT)+sum(DOWN_THROUGHPUT) as total from > traffic_2g_3g_4g where TERMINAL_BRAND='HTC' and > APP_CATEGORY_NAME='Web_Browsing' group by MSISDN > Error on ThriftServer > Job aborted due to stage failure: Task 1 in stage 12259.0 failed 4 times, > most recent failure: Lost task 1.3 in stage 12259.0 (TID 642677, > hadoop-slave-3, executor 9): > org.apache.carbondata.core.datastore.exception.IndexBuilderException: Block > B-tree loading failed > at > org.apache.carbondata.core.datastore.BlockIndexStore.fillLoadedBlocks(BlockIndexStore.java:264) > at > org.apache.carbondata.core.datastore.BlockIndexStore.getAll(BlockIndexStore.java:189) > at > org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:131) > at > org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfos(AbstractQueryExecutor.java:186) > at > org.apache.carbondata.core.scan.executor.impl.DetailQueryExecutor.execute(DetailQueryExecutor.java:39) > at > org.apache.carbondata.hadoop.CarbonRecordReader.initialize(CarbonRecordReader.java:79) > at > org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.scala:204) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) > at > org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) > at > org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) > at > org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96) > at > org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53) > at org.apache.spark.scheduler.Task.run(Task.scala:99) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scal
[jira] [Commented] (CARBONDATA-869) Greater than operator(>) does not display correct result for some attribute in range filter
[ https://issues.apache.org/jira/browse/CARBONDATA-869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15956817#comment-15956817 ] Kunal Kapoor commented on CARBONDATA-869: - hi An issue has been already raised for this CARBONDATA-852. Please mark as duplicate and close this. > Greater than operator(>) does not display correct result for some attribute > in range filter > --- > > Key: CARBONDATA-869 > URL: https://issues.apache.org/jira/browse/CARBONDATA-869 > Project: CarbonData > Issue Type: Bug > Components: data-query > Environment: spark 2.1 >Reporter: Vandana Yadav >Priority: Minor > Attachments: 2000_UniqData.csv > > > Greater than operator(>) does not display accurate result for some attributes > in range filter > Steps to reproduces: > 1)Create table: > CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION > string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 > bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 > decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 > int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES > ("TABLE_BLOCKSIZE"= "256 MB"); > 2)Load Data in a table: > LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into > table uniqdata OPTIONS('DELIMITER'=',' , > 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1'); > 3: Run the Query. > select dob from uniqdata where dob > '1974-12-25' and dob < '1975'; > 4:Result on beeline: > ++--+ > | dob | > ++--+ > | 1974-12-25 01:00:03.0 | > | 1974-12-26 01:00:03.0 | > | 1974-12-27 01:00:03.0 | > | 1974-12-28 01:00:03.0 | > | 1974-12-29 01:00:03.0 | > | 1974-12-30 01:00:03.0 | > | 1974-12-31 01:00:03.0 | > ++--+ > 7 rows selected (0.132 seconds) > Expected Result: resultset should include dates from "1974-12-26 to > 1974-12-31" and exclude "1974-12-25" -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-877) String datatype is throwing an error when included in DIctionary_Exclude in a alter query
[ https://issues.apache.org/jira/browse/CARBONDATA-877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor reassigned CARBONDATA-877: --- Assignee: Kunal Kapoor > String datatype is throwing an error when included in DIctionary_Exclude in a > alter query > - > > Key: CARBONDATA-877 > URL: https://issues.apache.org/jira/browse/CARBONDATA-877 > Project: CarbonData > Issue Type: Bug >Reporter: SWATI RAO >Assignee: Kunal Kapoor > Attachments: 2000_UniqData.csv > > > CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION > string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 > bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 > decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 > int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES > ("TABLE_BLOCKSIZE"= "256 MB"); > LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into > table uniqdata OPTIONS('DELIMITER'=',' , > 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1'); > ALTER TABLE uniqdata RENAME TO uniqdata1; > alter table uniqdata1 drop columns(CUST_NAME); > alter table uniqdata1 add columns(CUST_NAME string) > TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME', > 'DEFAULT.VALUE.CUST_NAME'='testuser') ; > Column added successfully. But when we execute: > select distinct(CUST_NAME) from uniqdata1 ; > & > select count(CUST_NAME) from uniqdata1 ; > it throws an error : > "Job aborted due to stage failure: Task 0 in stage 9.0 failed 1 times, most > recent failure: Lost task 0.0 in stage 9.0 (TID 206, localhost, executor > driver): java.lang.ArrayIndexOutOfBoundsException: 4186" -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (CARBONDATA-883) Select query after alter is not working fine with integer datatype.
[ https://issues.apache.org/jira/browse/CARBONDATA-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor reassigned CARBONDATA-883: --- Assignee: Kunal Kapoor > Select query after alter is not working fine with integer datatype. > --- > > Key: CARBONDATA-883 > URL: https://issues.apache.org/jira/browse/CARBONDATA-883 > Project: CarbonData > Issue Type: Bug > Environment: CREATE TABLE uniqdata (CUST_ID int,CUST_NAME > String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, > BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), > DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 > double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' > TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB"); > LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into > table uniqdata OPTIONS('DELIMITER'=',' , > 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1'); > ALTER TABLE uniqdata RENAME TO uniqdata1; > 0: jdbc:hive2://192.168.2.126:1> alter table default.uniqdata1 add > columns(ACTIVE_EMUI_VERSION int) > TBLPROPERTIES('DEFAULT.VALUE.ACTIVE_EMUI_VERSION'='67890'); > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (0.265 seconds) > 0: jdbc:hive2://192.168.2.126:1> select distinct(ACTIVE_EMUI_VERSION) > from uniqdata1 ; > Error: org.apache.spark.SparkException: Job aborted due to stage failure: > Task 0 in stage 22.0 failed 1 times, most recent failure: Lost task 0.0 in > stage 22.0 (TID 1211, localhost, executor driver): > java.lang.ClassCastException: java.lang.Long cannot be cast to > java.lang.Integer > at > org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.fillDataForNonExistingMeasures(RestructureBasedVectorResultCollector.java:193) > at > org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:98) > at > org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65) > at > org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46) > at > org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:246) > at > org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:140) > at > org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:222) > at > org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown > Source) > at > org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithKeys$(Unknown > Source) > at > org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown > Source) > at > org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) > at > org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377) > at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408) > at > org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126) > at > org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96) > at > org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53) > at org.apache.spark.scheduler.Task.run(Task.scala:99) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Driver stacktrace: (state=,code=0) > 0: jdbc:hive2://192.168.2.126:1> >Reporter: SWATI RAO >Assignee: Kunal Kapoor > > CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION > string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 > bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 > decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 > int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES > ("TABLE_BLOCKSIZE"= "256 MB"); > LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into > table uniqdata OPTION
[jira] [Assigned] (CARBONDATA-892) IndexOutOf Bound exception while running query with 2nd level sub-query
[ https://issues.apache.org/jira/browse/CARBONDATA-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor reassigned CARBONDATA-892: --- Assignee: Kunal Kapoor > IndexOutOf Bound exception while running query with 2nd level sub-query > --- > > Key: CARBONDATA-892 > URL: https://issues.apache.org/jira/browse/CARBONDATA-892 > Project: CarbonData > Issue Type: Bug > Components: sql >Affects Versions: 0.2.0-incubating >Reporter: Ramakrishna >Assignee: Kunal Kapoor >Priority: Minor > Fix For: 1.2.0-incubating > > > select s_name, s_address from tpchcarbon_1.supplier where s_suppkey in > (select ps_suppkey from tpchcarbon_1.partsupp where ps_partkey in (select > p_partkey from tpchcarbon_1.part where p_name like 'forest%')) limit 10; > Error: java.lang.IndexOutOfBoundsException: 0 (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CARBONDATA-911) Exception raised while creating table using bucketing
[ https://issues.apache.org/jira/browse/CARBONDATA-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15965472#comment-15965472 ] Kunal Kapoor commented on CARBONDATA-911: - Column Groups are not supported for columns defined with DICTIONARY_EXCLUDE > Exception raised while creating table using bucketing > -- > > Key: CARBONDATA-911 > URL: https://issues.apache.org/jira/browse/CARBONDATA-911 > Project: CarbonData > Issue Type: Bug > Components: docs > Environment: Spark 2.1 >Reporter: Shivangi Gupta >Priority: Minor > > Exception raised while creating table using bucketing: Column group is not > supported for no dictionary columns > Steps to reproduce: > 1. Create table > CREATE TABLE IF NOT EXISTS productSchema.productSalesTable ( > productNumber Int, > productName String, > storeCity String, > storeProvince String, > productCategory String, > productBatch String, > saleQuantity Int, > revenue Int) >STORED BY 'carbondata' >TBLPROPERTIES ('COLUMN_GROUPS'='(productName,productNumber)', > 'DICTIONARY_EXCLUDE'='productName', > 'DICTIONARY_INCLUDE'='productNumber', > 'NO_INVERTED_INDEX'='productBatch', > 'BUCKETNUMBER'='4', > 'BUCKETCOLUMNS'='productName'); > 2. Expected result : > It should create table successfully > 3. Result on beeline : > Error: org.apache.carbondata.spark.exception.MalformedCarbonCommandException: > Column group is not supported for no dictionary columns:productname > (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-926) Higher MAXCOLUMNS value in load DML options is leading to out of memory error
Kunal Kapoor created CARBONDATA-926: --- Summary: Higher MAXCOLUMNS value in load DML options is leading to out of memory error Key: CARBONDATA-926 URL: https://issues.apache.org/jira/browse/CARBONDATA-926 Project: CarbonData Issue Type: Improvement Reporter: Kunal Kapoor Assignee: Kunal Kapoor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (CARBONDATA-926) Higher MAXCOLUMNS value in load DML options is leading to out of memory error
[ https://issues.apache.org/jira/browse/CARBONDATA-926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-926: Priority: Minor (was: Major) > Higher MAXCOLUMNS value in load DML options is leading to out of memory error > - > > Key: CARBONDATA-926 > URL: https://issues.apache.org/jira/browse/CARBONDATA-926 > Project: CarbonData > Issue Type: Improvement >Reporter: Kunal Kapoor >Assignee: Kunal Kapoor >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (CARBONDATA-926) Set the max column from options in csv parser settings
[ https://issues.apache.org/jira/browse/CARBONDATA-926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor updated CARBONDATA-926: Summary: Set the max column from options in csv parser settings (was: Higher MAXCOLUMNS value in load DML options is leading to out of memory error) > Set the max column from options in csv parser settings > -- > > Key: CARBONDATA-926 > URL: https://issues.apache.org/jira/browse/CARBONDATA-926 > Project: CarbonData > Issue Type: Improvement >Reporter: Kunal Kapoor >Assignee: Kunal Kapoor >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-957) Table not found exception in rename table after lock acquire failure
Kunal Kapoor created CARBONDATA-957: --- Summary: Table not found exception in rename table after lock acquire failure Key: CARBONDATA-957 URL: https://issues.apache.org/jira/browse/CARBONDATA-957 Project: CarbonData Issue Type: Bug Reporter: Kunal Kapoor Assignee: Kunal Kapoor Priority: Minor During rename table if an exception is thrown during acquiring locks then table not exists exception was thrown while reverting the changes. -- This message was sent by Atlassian JIRA (v6.3.15#6346)