[GitHub] carbondata issue #1303: [CARBONDATA-1423] added integration test cases for p...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1303 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/696/ ---
[GitHub] carbondata issue #1346: Optimize memory and fix nosort queries
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1346 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/697/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1322 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/698/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user dhatchayani commented on the issue: https://github.com/apache/carbondata/pull/1322 retest this please ---
[GitHub] carbondata issue #1320: [CARBONDATA-1446] Fixed Bug for error message on inv...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1320 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/699/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1322 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/700/ ---
[GitHub] carbondata issue #1346: Optimize memory and fix nosort queries
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1346 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/701/ ---
[GitHub] carbondata issue #1349: [CARBONDATA-1470] csv data should not show in error ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1349 LGTM ---
[GitHub] carbondata pull request #1349: [CARBONDATA-1470] csv data should not show in...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/1349 ---
[jira] [Resolved] (CARBONDATA-1470) csv data should not show in error log when data column length is greater than 100000 characters
[ https://issues.apache.org/jira/browse/CARBONDATA-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-1470. - Resolution: Fixed Fix Version/s: 1.2.0 > csv data should not show in error log when data column length is greater > than 10 characters > > > Key: CARBONDATA-1470 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1470 > Project: CarbonData > Issue Type: Bug >Reporter: kapil reja >Priority: Minor > Fix For: 1.2.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] carbondata issue #1293: [CARBONDATA-1417]Added cluster tests for IUD, batch ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1293 LGTM ---
[GitHub] carbondata issue #1337: [CARBONDATA-1445] Fix update fail when carbon.update...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1337 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/702/ ---
[GitHub] carbondata pull request #1293: [CARBONDATA-1417]Added cluster tests for IUD,...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/1293 ---
[jira] [Resolved] (CARBONDATA-1417) Add Cluster tests for IUD, batch sort and Global sort features
[ https://issues.apache.org/jira/browse/CARBONDATA-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-1417. - Resolution: Fixed Fix Version/s: 1.2.0 > Add Cluster tests for IUD, batch sort and Global sort features > -- > > Key: CARBONDATA-1417 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1417 > Project: CarbonData > Issue Type: Bug >Reporter: Raghunandan > Fix For: 1.2.0 > > Time Spent: 2h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1471) Replace BigDecimal to double to improve performance
Ravindra Pesala created CARBONDATA-1471: --- Summary: Replace BigDecimal to double to improve performance Key: CARBONDATA-1471 URL: https://issues.apache.org/jira/browse/CARBONDATA-1471 Project: CarbonData Issue Type: Bug Reporter: Ravindra Pesala Replace BigDecimal to double in AdaptiveFloatingCodec to improve performance -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (CARBONDATA-1417) Add Cluster tests for IUD, batch sort and Global sort features
[ https://issues.apache.org/jira/browse/CARBONDATA-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala reassigned CARBONDATA-1417: --- Assignee: Raghunandan S > Add Cluster tests for IUD, batch sort and Global sort features > -- > > Key: CARBONDATA-1417 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1417 > Project: CarbonData > Issue Type: Bug >Reporter: Raghunandan >Assignee: Raghunandan S > Fix For: 1.2.0 > > Time Spent: 2h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1472) Optimize memory and fix nosort queries
Ravindra Pesala created CARBONDATA-1472: --- Summary: Optimize memory and fix nosort queries Key: CARBONDATA-1472 URL: https://issues.apache.org/jira/browse/CARBONDATA-1472 Project: CarbonData Issue Type: Bug Reporter: Ravindra Pesala Use UnsafeManager for dimension chunks as well to avoid leaks Fix filters on nosort columns. Optimize scanRDD -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] carbondata issue #1337: [CARBONDATA-1445] Fix update fail when carbon.update...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1337 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/703/ ---
[GitHub] carbondata issue #1282: [CARBONDATA-1221] Documentation - removed the unused...
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/1282 retest this please ---
[jira] [Updated] (CARBONDATA-1452) Issue with loading timestamp data beyond cutoff
[ https://issues.apache.org/jira/browse/CARBONDATA-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhatchayani updated CARBONDATA-1452: Summary: Issue with loading timestamp data beyond cutoff (was: Issue with loading timestamp data within cutoff when timestamp column is dictionary_include) > Issue with loading timestamp data beyond cutoff > --- > > Key: CARBONDATA-1452 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1452 > Project: CarbonData > Issue Type: Bug >Reporter: dhatchayani >Assignee: dhatchayani >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (CARBONDATA-1452) Issue with loading timestamp data beyond cutoff
[ https://issues.apache.org/jira/browse/CARBONDATA-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhatchayani updated CARBONDATA-1452: Description: While generating surrogate for timestamp dictionary column, we are casting the value to int. Then we are considering only the +ve values for generating dictionary, when the value is out of range,overflow occurs and cyclic rotation happens while casting, in the cyclic rotation there is possibility of getting +ve values in overflow cases too. > Issue with loading timestamp data beyond cutoff > --- > > Key: CARBONDATA-1452 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1452 > Project: CarbonData > Issue Type: Bug >Reporter: dhatchayani >Assignee: dhatchayani >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > While generating surrogate for timestamp dictionary column, we are casting > the value to int. Then we are considering only the +ve values for generating > dictionary, when the value is out of range,overflow occurs and cyclic > rotation happens while casting, in the cyclic rotation there is possibility > of getting +ve values in overflow cases too. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1473) Unable To use Greater than Operator on Date Type In Hive
anubhav tarar created CARBONDATA-1473:
-
Summary: Unable To use Greater than Operator on Date Type In Hive
Key: CARBONDATA-1473
URL: https://issues.apache.org/jira/browse/CARBONDATA-1473
Project: CarbonData
Issue Type: Bug
Components: hive-integration
Affects Versions: 1.2.0
Environment: hive1.2.1.,spark 2.1
Reporter: anubhav tarar
Assignee: anubhav tarar
Priority: Trivial
Fix For: 1.2.0
HiveEmbeddedServer.execute("SELECT ID FROM ALLDATATYPETEST WHERE JOININGDATE >
2016-04-14")
org.apache.hive.service.cli.HiveSQLException: java.io.IOException:
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:352)
at
org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:220)
at
org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:685)
at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:454)
at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:447)
at server.HiveEmbeddedServer$.execute(HiveEmbeddedServer.scala:150)
at
org.apache.carbondata.hive.integrationtest.AllDataTypeTest$$anonfun$2.apply$mcV$sp(AllDataTypeTest.scala:50)
at
org.apache.carbondata.hive.integrationtest.AllDataTypeTest$$anonfun$2.apply(AllDataTypeTest.scala:50)
at
org.apache.carbondata.hive.integrationtest.AllDataTypeTest$$anonfun$2.apply(AllDataTypeTest.scala:50)
at
org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
...
Cause: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:154)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1670)
at
org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:347)
at
org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:220)
at
org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:685)
at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:454)
at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:447)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] carbondata issue #1346: [CARBONDATA-1472] Optimize memory and fix nosort que...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1346 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/704/ ---
[GitHub] carbondata issue #1345: [CARBONDATA-1471] Replace BigDecimal to double to im...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1345 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/705/ ---
[GitHub] carbondata issue #1303: [CARBONDATA-1423] added integration test cases for p...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1303 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/706/ ---
[GitHub] carbondata pull request #1273: [CARBONDATA-1400] Fix bug of array column out...
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1273#discussion_r138330052
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/page/SafeVarLengthColumnPage.java
---
@@ -67,6 +70,17 @@ public void setByteArrayPage(byte[][] byteArray) {
}
@Override
+ public byte[] getLVFlattenedBytePage() throws IOException {
+ByteArrayOutputStream stream = new ByteArrayOutputStream();
+DataOutputStream out = new DataOutputStream(stream);
+for (byte[] byteArrayDatum : byteArrayData) {
+ out.writeInt(byteArrayDatum.length);
--- End diff --
This is for backward compatible, old code use 4 bytes to store length
---
[GitHub] carbondata pull request #1327: [CARBONDATA-1333]:Codex_Issue
Github user kushalsaha closed the pull request at: https://github.com/apache/carbondata/pull/1327 ---
[GitHub] carbondata pull request #1331: [CARBONDATA-1326][WIP]: Normal/Low priority f...
Github user kushalsaha closed the pull request at: https://github.com/apache/carbondata/pull/1331 ---
[GitHub] carbondata pull request #1346: [CARBONDATA-1472] Optimize memory and fix nos...
Github user kumarvishal09 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1346#discussion_r138253487
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/impl/unsafe/UnsafeAbstractDimensionDataChunkStore.java
---
@@ -68,10 +72,9 @@
* @param numberOfRows total number of rows
*/
public UnsafeAbstractDimensionDataChunkStore(long totalSize, boolean
isInvertedIdex,
- int numberOfRows) {
+ int numberOfRows) throws MemoryException {
// allocating the data page
-this.dataPageMemoryBlock =
-
MemoryAllocatorFactory.INSATANCE.getMemoryAllocator().allocate(totalSize);
+this.dataPageMemoryBlock =
UnsafeMemoryManager.allocateMemoryWithRetry(taskId, totalSize);
--- End diff --
From this class it self we can throw runtime exception if Unsafesafe memory
manager not able to allocate requested memory
---
[GitHub] carbondata pull request #1346: [CARBONDATA-1472] Optimize memory and fix nos...
Github user kumarvishal09 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1346#discussion_r138254137 --- Diff: core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeLessThanEqualFilterExecuterImpl.java --- @@ -265,11 +265,15 @@ private BitSet getFilteredIndexes(DimensionColumnDataChunk dimensionColumnDataCh DirectDictionaryGenerator directDictionaryGenerator = DirectDictionaryKeyGeneratorFactory --- End diff -- Now as we are not adding one in default value, while searching for last index of null value in binary search, we need to add +1 in search result as search will be last index of null value inclusive ---
[GitHub] carbondata pull request #1346: [CARBONDATA-1472] Optimize memory and fix nos...
Github user kumarvishal09 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1346#discussion_r138254259 --- Diff: core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeLessThanFiterExecuterImpl.java --- @@ -267,11 +267,15 @@ private BitSet getFilteredIndexes(DimensionColumnDataChunk dimensionColumnDataCh DirectDictionaryGenerator directDictionaryGenerator = DirectDictionaryKeyGeneratorFactory --- End diff -- Now as we are not adding one in default value, while searching for last index of null value in binary search, we need to add +1 in search result as search will be last index of null value inclusive ---
[GitHub] carbondata issue #1282: [CARBONDATA-1221] Documentation - removed the unused...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1282 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/707/ ---
[GitHub] carbondata pull request #1346: [CARBONDATA-1472] Optimize memory and fix nos...
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1346#discussion_r138338482
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/impl/unsafe/UnsafeAbstractDimensionDataChunkStore.java
---
@@ -68,10 +72,9 @@
* @param numberOfRows total number of rows
*/
public UnsafeAbstractDimensionDataChunkStore(long totalSize, boolean
isInvertedIdex,
- int numberOfRows) {
+ int numberOfRows) throws MemoryException {
// allocating the data page
-this.dataPageMemoryBlock =
-
MemoryAllocatorFactory.INSATANCE.getMemoryAllocator().allocate(totalSize);
+this.dataPageMemoryBlock =
UnsafeMemoryManager.allocateMemoryWithRetry(taskId, totalSize);
--- End diff --
ok
---
[GitHub] carbondata pull request #1346: [CARBONDATA-1472] Optimize memory and fix nos...
Github user ravipesala commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1346#discussion_r138339157 --- Diff: core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeLessThanFiterExecuterImpl.java --- @@ -267,11 +267,15 @@ private BitSet getFilteredIndexes(DimensionColumnDataChunk dimensionColumnDataCh DirectDictionaryGenerator directDictionaryGenerator = DirectDictionaryKeyGeneratorFactory --- End diff -- ok ---
[GitHub] carbondata pull request #1346: [CARBONDATA-1472] Optimize memory and fix nos...
Github user ravipesala commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1346#discussion_r138339130 --- Diff: core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeLessThanEqualFilterExecuterImpl.java --- @@ -265,11 +265,15 @@ private BitSet getFilteredIndexes(DimensionColumnDataChunk dimensionColumnDataCh DirectDictionaryGenerator directDictionaryGenerator = DirectDictionaryKeyGeneratorFactory --- End diff -- ok ---
[GitHub] carbondata issue #1282: [CARBONDATA-1221] Documentation - removed the unused...
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/1282 retest this please ---
[GitHub] carbondata issue #1303: [CARBONDATA-1423] added integration test cases for p...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1303 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/708/ ---
[GitHub] carbondata issue #1320: [CARBONDATA-1446] Fixed Bug for error message on inv...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/1320 LGTM ---
[GitHub] carbondata pull request #1320: [CARBONDATA-1446] Fixed Bug for error message...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/1320 ---
[jira] [Resolved] (CARBONDATA-1446) Alter query throws invalid exception while executing on range partitioned table
[
https://issues.apache.org/jira/browse/CARBONDATA-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacky Li resolved CARBONDATA-1446.
--
Resolution: Fixed
Fix Version/s: 1.2.0
> Alter query throws invalid exception while executing on range partitioned
> table
> ---
>
> Key: CARBONDATA-1446
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1446
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 1.2.0
> Environment: spark 2.1
>Reporter: Vandana Yadav
>Priority: Minor
> Fix For: 1.2.0
>
> Attachments: 2000_UniqData.csv
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Alter query throws invalid exception while executing on range partitioned
> table with invalid partition id
> Steps to reproduce:
> 1) Create table
> CREATE TABLE uniqdata_part1 (CUST_NAME String,ACTIVE_EMUI_VERSION string,DOB
> Timestamp,DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) PARTITIONED BY (CUST_ID int) STORED BY 'org.apache.carbondata.format'
> TBLPROPERTIES
> ('PARTITION_TYPE'='RANGE','RANGE_INFO'='9090,9500,9800',"TABLE_BLOCKSIZE"=
> "256 MB")
> 2)Load data into the table:
> LOAD DATA INPATH 'hdfs://localhost:54310/uniqdata/2000_UniqData.csv' into
> table uniqdata_part1 OPTIONS('DELIMITER'=',' ,
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')
> 3)Execute Query:
> ALTER TABLE uniqdata_part1 SPLIT PARTITION(6) INTO ('9800','9900')
> 4) Result On beeline:
> Error: java.lang.RuntimeException: Range info must be in ascending order,
> please check again! (state=,code=0)
> 5) Expected Result:
> As we are giving invalid partition id then error/exception should be like
> invalid partition id (related to partition id rather than range info).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CARBONDATA-1474) Memory leak issue in case of vector reader
kumar vishal created CARBONDATA-1474: Summary: Memory leak issue in case of vector reader Key: CARBONDATA-1474 URL: https://issues.apache.org/jira/browse/CARBONDATA-1474 Project: CarbonData Issue Type: Bug Reporter: kumar vishal Assignee: kumar vishal Memory leak issue in case of vector reader -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CARBONDATA-1475) fixed dependencies for Intellij Idea
QiangCai created CARBONDATA-1475: Summary: fixed dependencies for Intellij Idea Key: CARBONDATA-1475 URL: https://issues.apache.org/jira/browse/CARBONDATA-1475 Project: CarbonData Issue Type: Improvement Reporter: QiangCai Attachments: Selection_007.png When we choose profile spark-1.6, libraries contain dependencies of scala-2.11. !Selection_007.png|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] carbondata pull request #1350: [CARBONDATA-1475] fix default maven dependenc...
GitHub user QiangCai opened a pull request: https://github.com/apache/carbondata/pull/1350 [CARBONDATA-1475] fix default maven dependencies for Intellij Idea When we choose spark-1.6, Intellij Idea dependency scala-2.11. This PR will fix this issue.  You can merge this pull request into a Git repository by running: $ git pull https://github.com/QiangCai/carbondata default_dependencies Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1350.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1350 commit 29978231931c5af6c6eb7adf1df9e2594dad3c7b Author: QiangCai Date: 2017-09-12T14:36:13Z fixed maven dependencies for intellij idea ---
[GitHub] carbondata issue #1174: Update installation-guide.md
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/1174 LGTM ---
[GitHub] carbondata pull request #1174: [DOC] Update installation-guide.md
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/1174 ---
[GitHub] carbondata pull request #1117: [CARBONDATA-757] Big decimal optimization
Github user ravipesala closed the pull request at: https://github.com/apache/carbondata/pull/1117 ---
[GitHub] carbondata pull request #1350: [CARBONDATA-1475] fix default maven dependenc...
Github user chenliang613 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1350#discussion_r138381266
--- Diff: examples/spark2/pom.xml ---
@@ -38,32 +38,6 @@
org.apache.carbondata
carbondata-spark2
${project.version}
-
-
- org.apache.spark
--- End diff --
why need remove these exclusions ? in spark2 module, need to exclude 2.10
dependency jars
---
[GitHub] carbondata issue #1335: [CARBONDATA-1452] Issue with loading timestamp data ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1335 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/709/ ---
[GitHub] carbondata issue #1350: [CARBONDATA-1475] fix default maven dependencies for...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1350 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/710/ ---
[GitHub] carbondata issue #1324: [CARBONDATA-1291]:carbonData query performance impro...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1324 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1316: [CARBONDATA-1412] - Fixed bug related to incorrect b...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1316 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1282: [CARBONDATA-1221] Documentation - removed the unused...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1282 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1330: [CARBONDATA-1408]:Data loading with globalSort is fa...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1330 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1311: [CARBONDATA-1439] Wrong Error message shown for Bad ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1311 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1281: [CARBONDATA-1326] Fixed findbug issue and univocity-...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1281 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1280: [CARBONDATA-1326]- Findbugs Fixes
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1280 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1330: [CARBONDATA-1408]:Data loading with globalSort is fa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1330 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1316: [CARBONDATA-1412] - Fixed bug related to incorrect b...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1316 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1324: [CARBONDATA-1291]:carbonData query performance impro...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1324 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1311: [CARBONDATA-1439] Wrong Error message shown for Bad ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1311 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1280: [CARBONDATA-1326]- Findbugs Fixes
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1280 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1282: [CARBONDATA-1221] Documentation - removed the unused...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1282 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1281: [CARBONDATA-1326] Fixed findbug issue and univocity-...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1281 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1101: [CARBONDATA-1143] fix for null struct type
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1101 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #892: [CARBONDATA - 1036] - Added Implementation for Flink ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/892 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1303: [CARBONDATA-1423] added integration test cases for p...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1303 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/827/ ---
[GitHub] carbondata issue #249: [CARBONDATA-329] constant final class changed to inte...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/249 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1350: [CARBONDATA-1475] fix default maven dependencies for...
Github user QiangCai commented on the issue: https://github.com/apache/carbondata/pull/1350 please retest this ---
[GitHub] carbondata issue #1297: [CARBONDATA-1429] Add a value based compression for ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1297 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/828/ ---
[GitHub] carbondata issue #1303: [CARBONDATA-1423] added integration test cases for p...
Github user anubhav100 commented on the issue: https://github.com/apache/carbondata/pull/1303 retest this please ---
[GitHub] carbondata issue #1350: [CARBONDATA-1475] fix default maven dependencies for...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1350 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2/ ---
[GitHub] carbondata issue #1273: [CARBONDATA-1400] Fix bug of array column out of bou...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1273 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/711/ ---
[GitHub] carbondata issue #1134: [CARBONDATA-1262] Remove unnecessary LoadConfigurati...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1134 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/712/ ---
[GitHub] carbondata issue #1230: [WIP]corrected sdv test cases
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1230 Build Failed with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/829/ ---
[GitHub] carbondata issue #1347: [CARBONDATA-1465] resolved bug for hive cant query c...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1347 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3/ ---
[GitHub] carbondata issue #1346: [CARBONDATA-1472] Optimize memory and fix nosort que...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1346 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4/ ---
[GitHub] carbondata pull request #1351: [WIP]Add maven-antrun-plugin into build profi...
GitHub user sraghunandan opened a pull request: https://github.com/apache/carbondata/pull/1351 [WIP]Add maven-antrun-plugin into build profiles so that class files of different profiles do not clash when genrating coverage report You can merge this pull request into a Git repository by running: $ git pull https://github.com/sraghunandan/carbondata-1 profile_based_coverage Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1351.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1351 commit a4590e2e30b722e65d0a4d2aa77bea40b89e9138 Author: Raghunandan S Date: 2017-09-12T17:51:21Z Add maven-antrun-plugin into build profiles so that class files of different profiles do not clash when genrating coverage report ---
[GitHub] carbondata issue #1345: [CARBONDATA-1471] Replace BigDecimal to double to im...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1345 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5/ ---
[GitHub] carbondata issue #1273: [CARBONDATA-1400] Fix bug of array column out of bou...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1273 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/713/ ---
[GitHub] carbondata pull request #1352: [CARBONDATA-1174] Streaming Ingestion - schem...
GitHub user aniketadnaik opened a pull request: https://github.com/apache/carbondata/pull/1352 [CARBONDATA-1174] Streaming Ingestion - schema validation and streaming examples - Description: This change is mainly targeted for "streaming_ingest" development branch. Following changes are added on top of previous framework changes (pr-1064): 1. schema validation of input data if its from a file source when schema is specified. We validate source schema against existing table schema. For socket source , there is no schema validation required since there is no schema attached to it. 2. added streaming examples - for file stream and socket stream sources, CarbonStreamingIngestFileSourceExample.scala , CarbonStreamingIngestSocketSourceExample.scala these examples are added to facilitate development activity to understand and analyze code flow. The examples would run in its totality when carbondata is able write into carbondata file format. - Whether new unit test cases have been added or why no new tests are required? Yes , new unit test for schema validation has been added - What manual testing you have done? $> mvn clean -Pspark-2.1 -Dspark.version=2.1.0 verify [INFO] [INFO] Reactor Summary: [INFO] [INFO] Apache CarbonData :: Parent SUCCESS [ 1.320 s] [INFO] Apache CarbonData :: Common SUCCESS [ 1.509 s] [INFO] Apache CarbonData :: Core .. SUCCESS [ 26.109 s] [INFO] Apache CarbonData :: Processing SUCCESS [ 4.892 s] [INFO] Apache CarbonData :: Hadoop SUCCESS [ 8.910 s] [INFO] Apache CarbonData :: Spark Common .. SUCCESS [ 13.876 s] [INFO] Apache CarbonData :: Spark2 SUCCESS [02:29 min] [INFO] Apache CarbonData :: Spark Common Test . SUCCESS [07:06 min] [INFO] Apache CarbonData :: Assembly .. SUCCESS [ 1.724 s] [INFO] Apache CarbonData :: Flink Examples SUCCESS [ 2.480 s] [INFO] Apache CarbonData :: Hive .. SUCCESS [ 4.776 s] [INFO] Apache CarbonData :: presto SUCCESS [ 5.786 s] [INFO] Apache CarbonData :: Spark2 Examples ... SUCCESS [ 4.957 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 10:52 min [INFO] Finished at: 2017-09-12T10:50:40-07:00 [INFO] Final Memory: 119M/1223M [INFO] $> mvn clean verify [INFO] [INFO] Reactor Summary: [INFO] [INFO] Apache CarbonData :: Parent SUCCESS [ 6.925 s] [INFO] Apache CarbonData :: Common SUCCESS [ 10.383 s] [INFO] Apache CarbonData :: Core .. SUCCESS [02:07 min] [INFO] Apache CarbonData :: Processing SUCCESS [ 21.376 s] [INFO] Apache CarbonData :: Hadoop SUCCESS [ 18.568 s] [INFO] Apache CarbonData :: Spark Common .. SUCCESS [01:03 min] [INFO] Apache CarbonData :: Spark . SUCCESS [04:34 min] [INFO] Apache CarbonData :: Spark Common Test . SUCCESS [24:33 min] [INFO] Apache CarbonData :: Assembly .. SUCCESS [ 8.661 s] [INFO] Apache CarbonData :: Spark Examples SUCCESS [ 22.520 s] [INFO] Apache CarbonData :: Flink Examples SUCCESS [ 6.592 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 33:55 min [INFO] Finished at: 2017-09-12T08:12:30-07:00 [INFO] Final Memory: 62M/298M [INFO] * Made sure write path class invocation and schema validation happens correctly with with spark structured streaming (2.1) and parquet file source * Made sure write path execution work flow with structured streaming(2.1) for both socket and file sources - Any additional information to help reviewers in testing this change. For invalid schema carbondata throws exception and no record writer will be be instantiated. This is kind of first level of validation of input streaming data at CarbonSource entry point, another level of input data validation
[GitHub] carbondata issue #1352: [CARBONDATA-1174] Streaming Ingestion - schema valid...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1352 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1337: [CARBONDATA-1445] Fix update fail when carbon.update...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1337 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6/ ---
[GitHub] carbondata issue #1336: [CARBONDATA-1425] Inappropriate Exception displays w...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1336 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7/ ---
[GitHub] carbondata issue #1335: [CARBONDATA-1452] Issue with loading timestamp data ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1335 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8/ ---
[GitHub] carbondata issue #1329: [WIP] Min Max DataMap Example
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1329 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/9/ ---
[GitHub] carbondata issue #1323: [CARBONDATA-1413]Validate for invalid range info in ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1323 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/714/ ---
[GitHub] carbondata issue #1323: [CARBONDATA-1413]Validate for invalid range info in ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1323 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/10/ ---
[GitHub] carbondata issue #1303: [CARBONDATA-1423] added integration test cases for p...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1303 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/715/ ---
[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1322 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/11/ ---
[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1321 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/12/ ---
[GitHub] carbondata issue #1352: [CARBONDATA-1174] Streaming Ingestion - schema valid...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1352 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1352: [CARBONDATA-1174] Streaming Ingestion - schema valid...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1352 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1317 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/13/ ---
[jira] [Assigned] (CARBONDATA-1174) Streaming Ingest: Write path schema validation/inference
[ https://issues.apache.org/jira/browse/CARBONDATA-1174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Adnaik reassigned CARBONDATA-1174: - Assignee: Aniket Adnaik > Streaming Ingest: Write path schema validation/inference > > > Key: CARBONDATA-1174 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1174 > Project: CarbonData > Issue Type: Sub-task > Components: core, spark-integration >Affects Versions: 1.2.0 >Reporter: Aniket Adnaik >Assignee: Aniket Adnaik > Labels: features > Fix For: 1.2.0 > > > Streaming Ingest: Write path > - schema validation / schema inference from existing carbondata table > - streaming ingest allowed to existing tables only -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] carbondata issue #1351: [WIP]Add maven-antrun-plugin into build profiles so ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/1351 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/716/ ---
[GitHub] carbondata issue #1308: [CARBONDATA-1440]Fix coverity issues
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1308 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/14/ ---
[GitHub] carbondata issue #1303: [CARBONDATA-1423] added integration test cases for p...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1303 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/15/ ---
[GitHub] carbondata issue #1297: [CARBONDATA-1429] Add a value based compression for ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1297 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/16/ ---
[GitHub] carbondata issue #1291: [CARBONDATA-1343] Hive can't query data when the car...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1291 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/17/ ---
[GitHub] carbondata issue #1286: [CARBONDATA-1404] Added Unit test cases for Hive Int...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1286 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/18/ ---
