[GitHub] carbondata issue #1355: [CARBONDATA-1452] Issue with loading timestamp data ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1355 LGTM ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1322 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/747/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user dhatchayani commented on the issue: https://github.com/apache/carbondata/pull/1322 retest this please ---
[GitHub] carbondata pull request #1281: [CARBONDATA-1326] Fixed findbug issue and uni...
GitHub user pawanmalwal reopened a pull request: https://github.com/apache/carbondata/pull/1281 [CARBONDATA-1326] Fixed findbug issue and univocity-parsers jar version to 2.2.1 Updated univocity-parsers jar version from 1.5.6 to 2.2.1 in processing/pom.xml and fixed one findbug issue. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pawanmalwal/carbondata findbug_univocity_pom_changes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1281.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1281 commit 2a36440af46fc9ed361109990dc9a7c8f367e19e Author: Pawan Malwal Date: 2017-08-22T10:19:00Z [CARBONDATA-1326] Fixed findbug issue and univocity-parsers jar version to 2.2.1 ---
[GitHub] carbondata pull request #1281: [CARBONDATA-1326] Fixed findbug issue and uni...
Github user pawanmalwal closed the pull request at: https://github.com/apache/carbondata/pull/1281 ---
[GitHub] carbondata issue #1358: [SDV]Disable tests in other modules except cluster
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1358 Build Success with Spark 1.6, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6/ ---
[GitHub] carbondata issue #1358: [SDV]Disable tests in other modules except cluster
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1358 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/132/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1322 Build Failed with Spark 1.6, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1322 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/131/ ---
[GitHub] carbondata issue #1358: [SDV]Disable tests in other modules except cluster
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1358 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/744/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1322 Build Success with Spark 1.6, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4/ ---
[GitHub] carbondata issue #1358: [SDV]Disable tests in other modules except cluster
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1358 Build Success with Spark 1.6, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3/ ---
[GitHub] carbondata issue #1358: [SDV]Disable tests in other modules except cluster
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1358 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/130/ ---
[GitHub] carbondata issue #249: [CARBONDATA-329] constant final class changed to inte...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/249 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #892: [CARBONDATA - 1036] - Added Implementation for Flink ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/892 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1101: [CARBONDATA-1143] fix for null struct type
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1101 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1316: [CARBONDATA-1412] - Fixed bug related to incorrect b...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1316 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1352: [CARBONDATA-1174] Streaming Ingestion - schema valid...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1352 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1330: [CARBONDATA-1408]:Data loading with globalSort is fa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1330 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1281: [CARBONDATA-1326] Fixed findbug issue and univocity-...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1281 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1280: [CARBONDATA-1326]- Findbugs Fixes
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1280 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1311: [CARBONDATA-1439] Wrong Error message shown for Bad ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1311 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1324: [CARBONDATA-1291]:carbonData query performance impro...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1324 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1321 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/129/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1321 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/128/ ---
[GitHub] carbondata issue #1337: [CARBONDATA-1445] Fix update fail when carbon.update...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1337 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/23/ ---
[GitHub] carbondata issue #1353: [CARBONDATA-1476] Added Unit Test Case For Presto
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1353 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/742/ ---
[GitHub] carbondata issue #1297: [CARBONDATA-1429] Add a value based compression for ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1297 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/743/ ---
[GitHub] carbondata pull request #1358: [SDV]Disable tests in other modules except cl...
GitHub user ravipesala opened a pull request: https://github.com/apache/carbondata/pull/1358 [SDV]Disable tests in other modules except cluster You can merge this pull request into a Git repository by running: $ git pull https://github.com/ravipesala/incubator-carbondata sdv-test-fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1358.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1358 commit c43526f51f7b83c72d3a61cb2f8f31eeb2fe1074 Author: Ravindra Pesala Date: 2017-09-14T03:44:21Z Disable tests in other modules except cluster ---
[GitHub] carbondata issue #1286: [CARBONDATA-1404] Added Unit test cases for Hive Int...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1286 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/127/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/1321 retest this please ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/1321 retest this please ---
[GitHub] carbondata issue #1337: [CARBONDATA-1445] Fix update fail when carbon.update...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1337 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/126/ ---
[GitHub] carbondata issue #1281: [CARBONDATA-1326] Fixed findbug issue and univocity-...
Github user sraghunandan commented on the issue: https://github.com/apache/carbondata/pull/1281 retest this please ---
[GitHub] carbondata issue #1337: [CARBONDATA-1445] Fix update fail when carbon.update...
Github user zzcclp commented on the issue: https://github.com/apache/carbondata/pull/1337 retest this please. ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1321 Build Failed with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/22/ ---
[GitHub] carbondata issue #1297: [CARBONDATA-1429] Add a value based compression for ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1297 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/741/ ---
[GitHub] carbondata issue #1354: [CARBONDATA-1348] Sort_Column should not supported f...
Github user xuchuanyin commented on the issue: https://github.com/apache/carbondata/pull/1354 I used numeric-no-dictionary column in SORT_COLUMN and did not find any problems for now. ---
[GitHub] carbondata issue #1337: [CARBONDATA-1445] Fix update fail when carbon.update...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1337 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/125/ ---
[GitHub] carbondata issue #1337: [CARBONDATA-1445] Fix update fail when carbon.update...
Github user zzcclp commented on the issue: https://github.com/apache/carbondata/pull/1337 retest this please. ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1321 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/21/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1321 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/740/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1321 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/124/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user chenerlu commented on the issue: https://github.com/apache/carbondata/pull/1321 retest this please ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user chenerlu commented on the issue: https://github.com/apache/carbondata/pull/1321 please retest this ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1322 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/739/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1321 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/123/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1322 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/738/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1321 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/122/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1321 Build Failed with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/20/ ---
[GitHub] carbondata issue #1336: [CARBONDATA-1425] Inappropriate Exception displays w...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1336 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/737/ ---
[GitHub] carbondata issue #1355: [CARBONDATA-1452] Issue with loading timestamp data ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1355 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/736/ ---
[GitHub] carbondata issue #1356: [CARBONDATA-1477] corrected the bug for wrong values...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1356 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/735/ ---
[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1317 Build Failed with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/19/ ---
[GitHub] carbondata issue #1353: [CARBONDATA-1476] Added Unit Test Case For Presto
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1353 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/18/ ---
[GitHub] carbondata issue #1357: [CARBONDATA-1478]update compaction documentation
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1357 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/734/ ---
[GitHub] carbondata issue #1297: [CARBONDATA-1429] Add a value based compression for ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1297 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/17/ ---
[GitHub] carbondata issue #1353: [CARBONDATA-1476] Added Unit Test Case For Presto
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1353 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/16/ ---
[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1317 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/121/ ---
[GitHub] carbondata pull request #1317: [CARBONDATA-1316] drop partition
Github user lionelcao commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1317#discussion_r138692030 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchema.scala --- @@ -320,7 +322,143 @@ case class AlterTableSplitPartitionCommand(splitPartitionModel: AlterTableSplitP } } -case class CreateTable(cm: TableModel, createDSTable: Boolean = true) extends RunnableCommand +case class AlterTableDropPartition(alterTableDropPartitionModel: AlterTableDropPartitionModel) + extends RunnableCommand with DataProcessCommand with SchemaProcessCommand { + val LOGGER = LogServiceFactory.getLogService(this.getClass.getName) --- End diff -- The variables could only get from sparkSession and both processSchema and processData will use those variables. ---
[GitHub] carbondata issue #1354: [CARBONDATA-1348] Sort_Column should not supported f...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1354 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/733/ ---
[GitHub] carbondata issue #1353: [CARBONDATA-1476] Added Unit Test Case For Presto
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1353 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/120/ ---
[GitHub] carbondata issue #1297: [CARBONDATA-1429] Add a value based compression for ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1297 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/15/ ---
[GitHub] carbondata issue #1353: [CARBONDATA-1476] Added Unit Test Case For Presto
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1353 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/119/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1321 Build Failed with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/14/ ---
[GitHub] carbondata issue #1273: [CARBONDATA-1400] Fix bug of array column out of bou...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1273 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/730/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1322 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/13/ ---
[GitHub] carbondata issue #1337: [CARBONDATA-1445] Fix update fail when carbon.update...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1337 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/729/ ---
[GitHub] carbondata issue #1355: [CARBONDATA-1452] Issue with loading timestamp data ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1355 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/732/ ---
[jira] [Created] (CARBONDATA-1479) Tree exception displays to user in spark 1.6
manoj mathpal created CARBONDATA-1479: - Summary: Tree exception displays to user in spark 1.6 Key: CARBONDATA-1479 URL: https://issues.apache.org/jira/browse/CARBONDATA-1479 Project: CarbonData Issue Type: Bug Components: sql Affects Versions: 1.1.1 Environment: spark 1.6 Reporter: manoj mathpal Attachments: 2000_UniqData.csv, 3000_1_UniqData.csv, 3000_UniqData.csv, 4000_UniqData.csv, 5000_UniqData.csv, 6000_UniqData.csv, 7000_UniqData.csv Step to reproduce: 1:Create a table in carbon data : CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB") 2: Load Data into the table: LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData_tabdelm.csv' into table uniqdata OPTIONS('DELIMITER'='/' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1') LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/3000_UniqDatatdelm.csv' into table uniqdata OPTIONS('DELIMITER'='\t' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1') LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/5000_UniqData!delm.csv' into table uniqdata OPTIONS('DELIMITER'='!' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1') LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/6000_UniqData|delm.csv' into table uniqdata OPTIONS('DELIMITER'='|' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1') LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/7000_UniqData;delm.csv' into table uniqdata OPTIONS('DELIMITER'=';' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1') LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/3000_1_UniqData.csv' into table uniqdata OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1') 3: Select Query: select cust_id,cust_name,to_date(doj),quarter(dob),month(doj),day(dob),hour(doj),minute(dob),second(doj),weekofyear(dob),datediff(doj,current_date),date_add(dob,4),date_sub(doj,1),to_utc_timestamp(doj,current_date),add_months(dob,5),last_day(doj),months_between(doj,current_date),date_format(dob,current_date) from uniqdata where substr(cust_name,0,4)='CUST' and length(cust_name)in(15,14,13,16) and ACTIVE_EMUI_VERSION rlike 'ACTIVE' and month(dob)=01 and minute(dob)=0 group by doj,cust_id,cust_name,ACTIVE_EMUI_VERSION,dob having max(cust_id)=10830 and count(distinct(cust_id))<=2001 and max(cust_name) not like '%def%' order by cust_id,cust_name Result: execute, tree: CarbonDictionaryDecoder [CarbonDecoderRelation(Map(decimal_column1#17584 -> decimal_column1#17584, dob#17580 -> dob#17580, double_column1#17586 -> double_column1#17586, cust_name#17578 -> cust_name#17578, integer_column1#17588 -> integer_column1#17588, bigint_column2#17583L -> bigint_column2#17583L, bigint_column1#17582L -> bigint_column1#17582L, decimal_column2#17585 -> decimal_column2#17585, active_emui_version#17579 -> active_emui_version#17579, double_column2#17587 -> double_column2#17587, cust_id#17577 -> cust_id#17577, doj#17581 -> doj#17581),CarbonDatasourceRelation(`default`.`uniqdata`,None))], ExcludeProfile(ArrayBuffer(ACTIVE_EMUI_VERSION#17579, cust_name#17578, #19968, dob#17580)), CarbonAliasDecoderRelation() +- Sort [cust_id#17577 ASC,cust_name#17578 ASC], true, 0+- ConvertToUnsafe +- Exchange rangepartitioning(cust_id#17577 ASC,cust_name#17578 ASC,200), None +- ConvertToSafe +- Project [cust_id#17577,cust_name#17578,_c2#19946,_c3#19947,_c4#19948,_c5#19949,_c6#19943,_c7#19944,_c8#19945,_c9#19950,_c10#19951,_c11#19952,_c12#19953,_c13#19954,_c14#19955,_c15#19956,_c16#19957,_c17#19958] +- Filter havingCondit
[GitHub] carbondata issue #1273: [CARBONDATA-1400] Fix bug of array column out of bou...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1273 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/731/ ---
[GitHub] carbondata issue #1297: [CARBONDATA-1429] Add a value based compression for ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1297 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/118/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1322 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/12/ ---
[GitHub] carbondata pull request #1356: [CARBONDATA-1477] corrected the bug for wrong...
Github user anubhav100 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1356#discussion_r138659199 --- Diff: integration/hive/src/main/java/org/apache/carbondata/hive/CarbonDictionaryDecodeReadSupport.java --- @@ -228,9 +229,13 @@ private Writable createWritablePrimitive(Object obj, CarbonColumn carbonColumn) case LONG: return new LongWritable((long) obj); case SHORT: -return new ShortWritable((Short) obj); +return new ShortWritable((short) obj); case DATE: -return new DateWritable(new Date(((Integer) obj).longValue())); +Calendar c = Calendar.getInstance(); +c.setTime(new Date(0)); --- End diff -- There are changes in master code for date so these changes are required for getting the right values at hive side ---
[GitHub] carbondata issue #1336: [CARBONDATA-1425] Inappropriate Exception displays w...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1336 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/11/ ---
[GitHub] carbondata pull request #1356: [CARBONDATA-1477] corrected the bug for wrong...
Github user anubhav100 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1356#discussion_r138656706 --- Diff: integration/hive/src/main/java/org/apache/carbondata/hive/CarbonDictionaryDecodeReadSupport.java --- @@ -228,9 +229,13 @@ private Writable createWritablePrimitive(Object obj, CarbonColumn carbonColumn) case LONG: return new LongWritable((long) obj); case SHORT: -return new ShortWritable((Short) obj); +return new ShortWritable((short) obj); case DATE: -return new DateWritable(new Date(((Integer) obj).longValue())); +Calendar c = Calendar.getInstance(); +c.setTime(new Date(0)); --- End diff -- Yup ---
[GitHub] carbondata issue #1353: [CARBONDATA-1476] Added Unit Test Case For Presto
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1353 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/728/ ---
[GitHub] carbondata pull request #1356: [CARBONDATA-1477] corrected the bug for wrong...
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1356#discussion_r138654019 --- Diff: integration/hive/src/main/java/org/apache/carbondata/hive/CarbonDictionaryDecodeReadSupport.java --- @@ -228,9 +229,13 @@ private Writable createWritablePrimitive(Object obj, CarbonColumn carbonColumn) case LONG: return new LongWritable((long) obj); case SHORT: -return new ShortWritable((Short) obj); +return new ShortWritable((short) obj); case DATE: -return new DateWritable(new Date(((Integer) obj).longValue())); +Calendar c = Calendar.getInstance(); +c.setTime(new Date(0)); --- End diff -- Does it mean current time? and then reset the year to current year? ---
[jira] [Updated] (CARBONDATA-1477) wrong values shown when fetching date type values in hive
[ https://issues.apache.org/jira/browse/CARBONDATA-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated CARBONDATA-1477: -- Description: {code} import org.apache.spark.sql.CarbonSession._ val carbonSession = SparkSession .builder() .master("local") .appName("HiveExample") .config("carbonSession.sql.warehouse.dir", warehouse).enableHiveSupport() .getOrCreateCarbonSession( store) carbonSession.sql("""DROP TABLE IF EXISTS HIVE_CARBON_EXAMPLE""".stripMargin) carbonSession .sql( """CREATE TABLE HIVE_CARBON_EXAMPLE (ID int,NAME string,SALARY double,JOININGDATE date) STORED BY |'CARBONDATA' """ .stripMargin) carbonSession.sql( s""" LOAD DATA LOCAL INPATH '$rootPath/integration/hive/src/main/resources/data.csv' INTO TABLE HIVE_CARBON_EXAMPLE """) carbonSession.sql("SELECT * FROM HIVE_CARBON_EXAMPLE").show() carbonSession.stop() try { Class.forName(driverName) } catch { case classNotFoundException: ClassNotFoundException => classNotFoundException.printStackTrace() } HiveEmbeddedServer.start() val port = HiveEmbeddedServer.getFreePort val connection = DriverManager.getConnection(s"jdbc:hive2://localhost:8000/default", "", "") val statement: Statement = connection.createStatement logger.info(s"HIVE CLI IS STARTED ON PORT $port ==") statement.execute("CREATE TABLE IF NOT EXISTS " + "HIVE_CARBON_EXAMPLE " + " (ID int, NAME string,SALARY double,JOININGDATE date)") statement .execute( "ALTER TABLE HIVE_CARBON_EXAMPLE SET FILEFORMAT INPUTFORMAT \"org.apache.carbondata." + "hive.MapredCarbonInputFormat\"OUTPUTFORMAT \"org.apache.carbondata.hive." + "MapredCarbonOutputFormat\"SERDE \"org.apache.carbondata.hive." + "CarbonHiveSerDe\" ") statement .execute( "ALTER TABLE HIVE_CARBON_EXAMPLE SET LOCATION " + s"'file:///$store/default/hive_carbon_example' ") val sql = "SELECT * FROM HIVE_CARBON_EXAMPLE" val resultSet: ResultSet = statement.executeQuery(sql) var rowsFetched = 0 while (resultSet.next) { println("*"+resultSet.getString("JOININGDATE")) } println(s"**Total Number Of Rows Fetched ** $rowsFetched") logger.info("Fetching the Individual Columns ") HiveEmbeddedServer.stop() {code} actual result *null *1970-01-01 values in my csv are ID,NAME,SALARY,JOININGDATE 1,'liang',20,2016-03-14 2,'anubhav',2,2019/03/17 was: import org.apache.spark.sql.CarbonSession._ val carbonSession = SparkSession .builder() .master("local") .appName("HiveExample") .config("carbonSession.sql.warehouse.dir", warehouse).enableHiveSupport() .getOrCreateCarbonSession( store) carbonSession.sql("""DROP TABLE IF EXISTS HIVE_CARBON_EXAMPLE""".stripMargin) carbonSession .sql( """CREATE TABLE HIVE_CARBON_EXAMPLE (ID int,NAME string,SALARY double,JOININGDATE date) STORED BY |'CARBONDATA' """ .stripMargin) carbonSession.sql( s""" LOAD DATA LOCAL INPATH '$rootPath/integration/hive/src/main/resources/data.csv' INTO TABLE HIVE_CARBON_EXAMPLE """) carbonSession.sql("SELECT * FROM HIVE_CARBON_EXAMPLE").show() carbonSession.stop() try { Class.forName(driverName) } catch { case classNotFoundException: ClassNotFoundException => classNotFoundException.printStackTrace() } HiveEmbeddedServer.start() val port = HiveEmbeddedServer.getFreePort val connection = DriverManager.getConnection(s"jdbc:hive2://localhost:8000/default", "", "") val statement: Statement = connection.createStatement logger.info(s"HIVE CLI IS STARTED ON PORT $port ==") statement.execute("CREATE TABLE IF NOT EXISTS " + "HIVE_CARBON_EXAMPLE " + " (ID int, NAME string,SALARY double,JOININGDATE date)") statement .execute( "ALTER TABLE HIVE_CARBON_EXAMPLE SET FILEFORMAT INPUTFORMAT \"org.apache.carbondata." + "hive.MapredCarbonInputFormat\"OUTPUTFORMAT \"org.apache.carbondata.hive." + "MapredCarbonOutputFormat\"SERDE \"org.apache.carbondata.hive." + "CarbonHiveSerDe\" ") statement .execute( "ALTER TABLE HIVE_CARBON_EXAMPLE SET LOCATION " + s"'file:///$store/default/hive_carbon_example' ") val sql = "SELECT * FROM HIVE_CARBON_EXAMPLE" val resultSet: ResultSet = statement.executeQuery(sql) var rowsFetched = 0 while (resultSet.next) { println("*"+resultSet.getString("JOININGDATE")) } println(s"**Total Number Of Rows Fetched
[GitHub] carbondata issue #1355: [CARBONDATA-1452] Issue with loading timestamp data ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1355 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/10/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1321 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/117/ ---
[GitHub] carbondata issue #1356: [CARBONDATA-1477] corrected the bug for wrong values...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1356 Build Failed with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/9/ ---
[GitHub] carbondata issue #1286: [CARBONDATA-1404] Added Unit test cases for Hive Int...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1286 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/727/ ---
[GitHub] carbondata issue #1356: [CARBONDATA-1477] corrected the bug for wrong values...
Github user anubhav100 commented on the issue: https://github.com/apache/carbondata/pull/1356 @chenliang613 @cenyuhai ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1322 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/116/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user chenerlu commented on the issue: https://github.com/apache/carbondata/pull/1321 please retest this ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user chenerlu commented on the issue: https://github.com/apache/carbondata/pull/1321 please retest it ---
[jira] [Resolved] (CARBONDATA-1465) Hive unable to query carbondata when column names is in small letters
[ https://issues.apache.org/jira/browse/CARBONDATA-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Chen resolved CARBONDATA-1465. Resolution: Fixed > Hive unable to query carbondata when column names is in small letters > - > > Key: CARBONDATA-1465 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1465 > Project: CarbonData > Issue Type: Bug > Components: hive-integration >Affects Versions: 1.2.0 > Environment: hive 1.2 ,spark 2.1 >Reporter: anubhav tarar >Assignee: anubhav tarar >Priority: Trivial > Fix For: 1.2.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > inside hive example i tried this > hiveEmbeddedServer2 = new HiveEmbeddedServer2 > hiveEmbeddedServer2.start() > hiveEmbeddedServer2 > .execute(s"CREATE TABLE IF NOT EXISTS TESTTABLE(ID INT,JOINNINGDATE > date,COUNTRY string,NAME STRING," + >s" PHONETYPE STRING, SERIALNAME STRING, SALARY DOUBLE,BONUS > DECIMAL(10,4)) ROW FORMAT " + >s"SERDE 'org.apache.carbondata.hive.CarbonHiveSerDe' STORED AS > INPUTFORMAT 'org" + >s".apache.carbondata.hive.MapredCarbonInputFormat' > OUTPUTFORMAT 'org.apache" + >s".carbondata.hive.MapredCarbonOutputFormat' TBLPROPERTIES > ('spark.sql.sources" + >s".provider'='org.apache.spark.sql.CarbonSource')") > hiveEmbeddedServer2 > .execute(s"ALTER TABLE TESTTABLE SET LOCATION " + > > s"'file:///$rootPath/integration/hive/target/store/testdb/testtable' ") > hiveEmbeddedServer2.execute("SELECT name FROM TESTTABLE") > Caused by: java.lang.StringIndexOutOfBoundsException: String index out of > range: -1 > at > java.lang.AbstractStringBuilder.substring(AbstractStringBuilder.java:935) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] carbondata pull request #1347: [CARBONDATA-1465] resolved bug for hive cant ...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/1347 ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1322 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/115/ ---
[GitHub] carbondata issue #1347: [CARBONDATA-1465] resolved bug for hive cant query c...
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/1347 LGTM ---
[GitHub] carbondata issue #1297: [CARBONDATA-1429] Add a value based compression for ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1297 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/114/ ---
[jira] [Resolved] (CARBONDATA-1472) Optimize memory and fix nosort queries
[ https://issues.apache.org/jira/browse/CARBONDATA-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacky Li resolved CARBONDATA-1472. -- Resolution: Fixed Fix Version/s: 1.2.0 > Optimize memory and fix nosort queries > -- > > Key: CARBONDATA-1472 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1472 > Project: CarbonData > Issue Type: Bug >Reporter: Ravindra Pesala > Fix For: 1.2.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Use UnsafeManager for dimension chunks as well to avoid leaks > Fix filters on nosort columns. > Optimize scanRDD -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] carbondata pull request #1346: [CARBONDATA-1472] Optimize memory and fix nos...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/1346 ---
[GitHub] carbondata issue #1346: [CARBONDATA-1472] Optimize memory and fix nosort que...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/1346 LGTM ---
[GitHub] carbondata issue #1297: [CARBONDATA-1429] Add a value based compression for ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1297 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/113/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1322 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/112/ ---
[GitHub] carbondata issue #1336: [CARBONDATA-1425] Inappropriate Exception displays w...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1336 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/111/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1322 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/110/ ---
[GitHub] carbondata issue #1357: [CARBONDATA-1478]update compaction documentation
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1357 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder1/8/ ---