[GitHub] carbondata pull request #1432: [CARBONDATA-1608]Support Column Comment for C...
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1432#discussion_r150883724
--- Diff:
integration/spark2/src/test/scala/org/apache/carbondata/spark/testsuite/booleantype/BooleanDataTypesInsertTest.scala
---
@@ -945,4 +948,41 @@ class BooleanDataTypesInsertTest extends QueryTest
with BeforeAndAfterEach with
)
}
+ test("Inserting table with bad records, and SORT_COLUMNS is boolean
column") {
+
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.ENABLE_AUTO_LOAD_MERGE,
"true")
--- End diff --
this test is not related to col comment, this test is just added in this PR
which is for boolean datatype
---
[GitHub] carbondata pull request #1500: [WIP]Remove spark broadcast for gettting hado...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1500 [WIP]Remove spark broadcast for gettting hadoop configurations why? with the new conf set in hadoop conf, after broadcast conf will not get reflected, hence these changes Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [*] Any interfaces changed? NONE - [*] Any backward compatibility impacted? NONE - [ ] Document update required? NONE - [*] Testing done test cases are not required, SDV will take care - [*] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NONE You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata conf Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1500.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1500 commit e86bf0809dd1270126b99f5c61f92c84ba5a2ed8 Author: akashrn5 Date: 2017-10-27T12:41:03Z remove spark broadcast for gettting hadoop configurations ---
[GitHub] carbondata issue #1500: [CARBONDATA-1717]Remove spark broadcast for gettting...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1500 @QiangCai please review ---
[GitHub] carbondata issue #1500: [CARBONDATA-1717]Remove spark broadcast for gettting...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1500 @ravipesala @jackylk please review ---
[GitHub] carbondata pull request #1438: [CARBONDATA-1649]insert overwrite fix during ...
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1438#discussion_r151332421
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/LoadTableCommand.scala
---
@@ -186,6 +186,12 @@ case class LoadTableCommand(
LOGGER.error(ex, s"Dataload failure for $dbName.$tableName")
throw new RuntimeException(s"Dataload failure for
$dbName.$tableName, ${ex.getMessage}")
case ex: Exception =>
+ if (ex.isInstanceOf[InterruptedException] &&
+ ex.getMessage.contains("update fail status")) {
--- End diff --
ok, i will add a new Exception for this, and throw that exception
---
[GitHub] carbondata issue #1438: [CARBONDATA-1649]insert overwrite fix during job int...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1438 @jackylk handled your comment, please review ---
[GitHub] carbondata pull request #1432: [CARBONDATA-1608]Support Column Comment for C...
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1432#discussion_r151345725
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/parser/CarbonSpark2SqlParser.scala
---
@@ -424,11 +424,20 @@ class CarbonSpark2SqlParser extends
CarbonDDLSqlParser {
def getFields(schema: Seq[StructField]): Seq[Field] = {
schema.map { col =>
- val x = if (col.dataType.catalogString == "float") {
-'`' + col.name + '`' + " double"
- }
- else {
-'`' + col.name + '`' + ' ' + col.dataType.catalogString
+ val colComment = col.getComment()
+ val x = if (colComment.isDefined) {
+val columnComment = " comment \"" + colComment.get + "\""
--- End diff --
handled, please review
---
[GitHub] carbondata pull request #1432: [CARBONDATA-1608]Support Column Comment for C...
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1432#discussion_r151345870
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/CarbonDescribeFormattedCommand.scala
---
@@ -65,6 +65,7 @@ private[sql] case class CarbonDescribeFormattedCommand(
val dims = relation.metaData.dims.map(x => x.toLowerCase)
var results: Seq[(String, String, String)] = child.schema.fields.map {
field =>
val fieldName = field.name.toLowerCase
+ val colComment = field.getComment().getOrElse("null")
--- End diff --
verified, it is in sync with hive, if comment is not there, by default it
will be null
---
[GitHub] carbondata issue #1500: [CARBONDATA-1717]Remove spark broadcast for gettting...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1500 retest sdv pleaase ---
[GitHub] carbondata issue #1500: [CARBONDATA-1717]Remove spark broadcast for gettting...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1500 retest this please ---
[GitHub] carbondata pull request #1506: [CARBONDATA-1734] Ignore empty line while rea...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1506 [CARBONDATA-1734] Ignore empty line while reading CSV - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata csv_reading Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1506.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1506 commit cd29f161efbb30d2a402cf383624b2085e37d9fb Author: akashrn5 Date: 2017-11-16T10:51:05Z [CARBONDATA-1734] Ignore empty line while reading CSV ---
[GitHub] carbondata pull request #1507: [CARBONDATA-1326] Fixed high priority findbug...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1507 [CARBONDATA-1326] Fixed high priority findbug issue - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata findbugs_issue Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1507.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1507 commit c76b57f5dab88e2a7ffe7b11c570cca95370a91a Author: dhatchayani Date: 2017-11-16T11:22:18Z [CARBONDATA-1326] Fixed high priority findbug issue ---
[GitHub] carbondata issue #1512: [CARBONDATA-1742] Fix NullPointerException in Segmen...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1512 @xubo245 this change is already being handled in #1507 ---
[GitHub] carbondata pull request #1435: [CARBONDATA-1626]add data size and index size...
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1435#discussion_r151421484
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/impl/FileFactory.java
---
@@ -606,4 +609,53 @@ public static FileSystem getFileSystem(Path path)
throws IOException {
return path.getFileSystem(configuration);
}
+ // Get the total size of carbon data and the total size of carbon index
+ public static HashMap
getDataSizeAndIndexSize(CarbonTablePath carbonTablePath,
--- End diff --
handled comments
---
[GitHub] carbondata pull request #1506: [CARBONDATA-1734] Ignore empty line while rea...
Github user akashrn5 closed the pull request at: https://github.com/apache/carbondata/pull/1506 ---
[GitHub] carbondata pull request #1524: [CARBONDATA-1762] Remove existing column leve...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1524 [CARBONDATA-1762] Remove existing column level dateformat and support dateformat, timestampformat in the load option (1) Remove column level dateformat option (2) Support dateformat and timestampformat in load options(table level) Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [X] Document update required? 2 new load level properties are added. Document to be updated accordingly. - [X] Testing done UT Added - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata timeformat Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1524.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1524 commit f7b253c672c4eed21466806cd4bc9990264a8a37 Author: akashrn5 Date: 2017-11-17T11:25:33Z [CARBONDATA-1762] Remove existing column level dateformat and support dateformat, timestampformat in the load option ---
[GitHub] carbondata pull request #1567: [CARBONDATA-1809]Add create table event
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1567 [CARBONDATA-1809]Add create table event Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [*] Any interfaces changed? NA - [*] Any backward compatibility impacted? NA - [*] Document update required? NA - [*] Testing done UT not required, firing events will test this - [*] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata table Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1567.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1567 commit 259f8c5c304623d73b2d1c0e6722aed52bec6a77 Author: akashrn5 Date: 2017-11-25T09:06:28Z add create table event ---
[GitHub] carbondata issue #1567: [CARBONDATA-1809]Add create table event
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1567 retest this please ---
[GitHub] carbondata pull request #1608: [CARBONDATA-1761]do not change status of segm...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1608 [CARBONDATA-1761]do not change status of segment to marked for delete if it is in progress do not change status of segment to marked for delete if it is in progress Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [*] Any interfaces changed? NA - [*] Any backward compatibility impacted? NA - [*] Document update required? NA - [*] Testing done tested in local - [*] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata delete Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1608.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1608 commit de63d8c843a7860f49d3e5b5627aceee1332f37d Author: akashrn5 Date: 2017-12-04T12:31:18Z do not change status of segment to marked for delete if it is in progress ---
[GitHub] carbondata pull request #1610: [CARBONDATA-1879]do not drop the table if the...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1610 [CARBONDATA-1879]do not drop the table if the load, insert or insert overwrite is in progress do not drop the table if the load, insert or insert overwrite is in progress Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [*] Any interfaces changed? NA - [*] Any backward compatibility impacted? NA - [*] Document update required? NA - [*] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. local testing is done - [*] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata drop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1610.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1610 commit a0e516709f6b2053845fc882c02a0ee7aee91fac Author: akashrn5 Date: 2017-12-04T15:06:15Z do not drop the table if the load, insert or insert overwrite is in progress ---
[GitHub] carbondata issue #1608: [CARBONDATA-1761]do not change status of segment dur...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1608 @ravipesala handled comments please review ---
[GitHub] carbondata issue #1610: [CARBONDATA-1879][CARBONDATA-1791]do not drop the ta...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1610 retest this please ---
[GitHub] carbondata issue #1610: [CARBONDATA-1789][CARBONDATA-1791]do not drop the ta...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1610 @ravipesala i have added test case for both scenario ---
[GitHub] carbondata issue #1610: [CARBONDATA-1789][CARBONDATA-1791]do not drop the ta...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1610 retest this please ---
[GitHub] carbondata pull request #1615: [CARBONDATA-1592]added new parameters for cre...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1615 [CARBONDATA-1592]added new parameters for create and load events added new parameters for create and load events, which will help for some specific validations on table during create and load Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [*] Any interfaces changed? NA - [*] Any backward compatibility impacted? NA - [*] Document update required? NA - [*] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. only parameters are added for events, testing is not needed, unless listeners are added - [*] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata param Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1615.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1615 commit 7818e1b57054463060f863ebc29da36d374560b4 Author: akashrn5 Date: 2017-12-05T13:22:35Z added new parameters for create and load events ---
[GitHub] carbondata issue #1610: [CARBONDATA-1789][CARBONDATA-1791]do not drop the ta...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1610 retest sdv please ---
[GitHub] carbondata issue #1608: [CARBONDATA-1761]do not change status of segment dur...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1608 retest this please ---
[GitHub] carbondata issue #1615: [CARBONDATA-1592]added new parameters for create and...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1615 handled comment, plesae review @jackylk ---
[GitHub] carbondata pull request #1567: [CARBONDATA-1809]Add create table event
Github user akashrn5 closed the pull request at: https://github.com/apache/carbondata/pull/1567 ---
[GitHub] carbondata issue #1608: [CARBONDATA-1761]do not change status of segment dur...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1608 retest this please ---
[GitHub] carbondata pull request #1615: [CARBONDATA-1592]added new parameters for cre...
Github user akashrn5 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1615#discussion_r155253089 --- Diff: integration/spark-common/src/main/scala/org/apache/carbondata/events/CreateTableEvents.scala --- @@ -20,14 +20,16 @@ package org.apache.carbondata.events import org.apache.spark.sql._ import org.apache.carbondata.core.metadata.AbsoluteTableIdentifier +import org.apache.carbondata.core.metadata.schema.table.TableInfo /** * Class for handling operations before start of a load process. * Example usage: For validation purpose */ case class CreateTablePreExecutionEvent( sparkSession: SparkSession, -identifier: AbsoluteTableIdentifier) extends Event with TableEventInfo +identifier: AbsoluteTableIdentifier, +tableModel: Option[TableInfo]) extends Event with TableEventInfo --- End diff -- done, please check ---
[GitHub] carbondata pull request #1615: [CARBONDATA-1592]added new parameters for cre...
Github user akashrn5 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1615#discussion_r155267087 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala --- @@ -148,7 +148,9 @@ case class CarbonLoadDataCommand( carbonLoadModel, factPath, dataFrame.isDefined, -optionsFinal) +optionsFinal, +options, --- End diff -- from finalOptions we will get all the default values of that particular option, but if we need only the options given from user,we need options also, so this one is added ---
[GitHub] carbondata pull request #1627: [CARBONDATA-1759]make visibility of segments ...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1627 [CARBONDATA-1759]make visibility of segments as false eventhough file is not present To take care show segments after clean files operation make visibility of segments as false eventhough file is not present To take care show segments after clean files operation Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? - [X] Testing done test cases needs to be added - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata cleanfiles Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1627.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1627 ---
[GitHub] carbondata issue #1627: [CARBONDATA-1759]make visibility of segments as fals...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1627 retest sdv please ---
[GitHub] carbondata pull request #1610: [CARBONDATA-1789][CARBONDATA-1791]do not drop...
Github user akashrn5 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1610#discussion_r155435306 --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/concurrent/TestLoadTableConcurrentScenario.scala --- @@ -0,0 +1,61 @@ +package org.apache.carbondata.spark.testsuite.concurrent --- End diff -- header added ---
[GitHub] carbondata pull request #1637: [CARBONDATA-1876]clean all the InProgress seg...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1637 [CARBONDATA-1876]clean all the InProgress segments for all databases during session initialization clean all the InProgress segments for all databases during session initialization clean all the InProgress segments for all databases during session initialization. when carbon session initialize , clean all the in progress segments for all the databases created by user. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done local testing is done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata clean_progress Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1637.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1637 commit db891b8638fe56e8bfec4162d19a8d5b045d265f Author: akashrn5 Date: 2017-12-07T23:06:55Z clean all the InProgress segments for all databases during session initialization ---
[GitHub] carbondata issue #1627: [CARBONDATA-1759]make visibility of segments as fals...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1627 retest this please ---
[GitHub] carbondata issue #1637: [CARBONDATA-1876]clean all the InProgress segments f...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1637 retest this please ---
[GitHub] carbondata pull request #1438: [CARBONDATA-1649]insert overwrite fix during ...
Github user akashrn5 closed the pull request at: https://github.com/apache/carbondata/pull/1438 ---
[GitHub] carbondata pull request #1682: [CARBONDATA-1910]do not allow tupleid, refere...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1682 [CARBONDATA-1910]do not allow tupleid, referenceid and positionReference as columns names Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. UTs are added to test the scenarios - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata tuple Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1682.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1682 commit 59ec63f614ce69ad2b6d8fe1fae6db2b6af93424 Author: akashrn5 Date: 2017-12-19T13:04:10Z do not allow tupleid, referenceid and positionReference as columns names ---
[GitHub] carbondata issue #1682: [CARBONDATA-1910]do not allow tupleid, referenceid a...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1682 retest this please ---
[GitHub] carbondata issue #1682: [CARBONDATA-1910]do not allow tupleid, referenceid a...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1682 retest sdv please ---
[GitHub] carbondata pull request #1687: in the insert into and update flow when stati...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1687 in the insert into and update flow when static values are inserted then preferred locations are coming empty in the insert into and update flow when static values are inserted then preferred locations are coming empty Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: update the table,then query table was not working (if the dynamic executors are enabled and in insert and update flow when static content is used then nodedata is coming as 0) so, if the number of nodes are coming less than 1, assign 1 executor - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. Yes - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata node Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1687.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1687 commit 0b6619e198c0ad3073d312698d607b1f22587f72 Author: akashrn5 Date: 2017-12-20T04:50:51Z in the insert into and update flow when static values are inserted then preferred locations are coming empty ---
[GitHub] carbondata pull request #1688: [CARBONDATA-1916]Correct the database locatio...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1688 [CARBONDATA-1916]Correct the database location path during carbon drop database Correct the database location path during carbon drop database when drop database is called, to delete the databsae directory, the path formed is wrong, so when drop datasbe is executed, operation is successful , but the database directory is still present in hdfs Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. YES, DONE IN ANT CLUSTER - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata dropdb Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1688.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1688 commit 9d4278fad8a030096c99dcf547533d6bb78d250a Author: akashrn5 Date: 2017-12-20T05:28:23Z Correct the database location path during carbon drop database ---
[GitHub] carbondata pull request #1687: [CARBONDATA-1915]In the insert into and updat...
Github user akashrn5 closed the pull request at: https://github.com/apache/carbondata/pull/1687 ---
[GitHub] carbondata issue #1688: [CARBONDATA-1916]Correct the database location path ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1688 retest sdv please ---
[GitHub] carbondata pull request #1682: [CARBONDATA-1910]do not allow tupleid, refere...
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1682#discussion_r158022056
--- Diff:
integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/iud/UpdateCarbonTableTestCase.scala
---
@@ -569,6 +569,39 @@ class UpdateCarbonTableTestCase extends QueryTest with
BeforeAndAfterAll {
CarbonCommonConstants.defaultValueIsPersistEnabled)
}
+ test("test create table with tupleid as column name") {
--- End diff --
added
---
[GitHub] carbondata pull request #1682: [CARBONDATA-1910]do not allow tupleid, refere...
Github user akashrn5 closed the pull request at: https://github.com/apache/carbondata/pull/1682 ---
[GitHub] carbondata pull request #1682: [CARBONDATA-1910]do not allow tupleid, refere...
GitHub user akashrn5 reopened a pull request: https://github.com/apache/carbondata/pull/1682 [CARBONDATA-1910]do not allow tupleid, referenceid and positionReference as columns names Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. UTs are added to test the scenarios - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata tuple Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1682.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1682 commit 1dc5dd8b4490848cdab187790c0db28e40dbd354 Author: akashrn5 Date: 2017-12-19T13:04:10Z do not allow tupleid, referenceid and positionReference as columns names ---
[GitHub] carbondata issue #1688: [CARBONDATA-1916]Correct the database location path ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1688 retest this please ---
[GitHub] carbondata pull request #1720: [WIP]fix the backword compatibility issue for...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1720 [WIP]fix the backword compatibility issue for tableInfo deserialization Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata compatibility Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1720.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1720 commit 3a3d67ce245541efb2608368c23b80eb57e63e94 Author: akashrn5 Date: 2017-12-22T13:07:20Z fix the backword compatibility issue for tableInfo deserialization ---
[GitHub] carbondata pull request #1688: [CARBONDATA-1916]Correct the database locatio...
Github user akashrn5 closed the pull request at: https://github.com/apache/carbondata/pull/1688 ---
[GitHub] carbondata pull request #1688: [CARBONDATA-1916]Correct the database locatio...
GitHub user akashrn5 reopened a pull request: https://github.com/apache/carbondata/pull/1688 [CARBONDATA-1916]Correct the database location path during carbon drop database Correct the database location path during carbon drop database when drop database is called, to delete the databsae directory, the path formed is wrong, so when drop datasbe is executed, operation is successful , but the database directory is still present in hdfs Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. YES, DONE IN ANT CLUSTER - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata dropdb Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1688.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1688 commit 8b813816b23a911c4ea4b42c2b55b98ee61c3bbe Author: akashrn5 Date: 2017-12-20T05:28:23Z Correct the database location path during carbon drop database ---
[GitHub] carbondata pull request #1720: [CARBONDATA-1935]fix the backword compatibili...
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1720#discussion_r158512806
--- Diff:
core/src/main/java/org/apache/carbondata/core/metadata/datatype/DataTypes.java
---
@@ -112,6 +118,40 @@ public static DataType valueOf(int id) {
}
}
+ public static DataType valueOf(String name) {
--- End diff --
ok
---
[GitHub] carbondata pull request #1720: [CARBONDATA-1935]fix the backword compatibili...
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1720#discussion_r158515688
--- Diff:
core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java ---
@@ -1938,6 +1939,12 @@ public static TableInfo
convertGsonToTableInfo(Map properties) {
}
builder.append(part);
}
+
+// Datatype GSON adapter is added to support backward compatibility
for tableInfo desrialization
+GsonBuilder gsonBuilder = new GsonBuilder();
+gsonBuilder.registerTypeAdapter(DataType.class,
DataTypes.DataTypeAdapter.getInstance());
--- End diff --
ok
---
[GitHub] carbondata issue #1720: [CARBONDATA-1935]fix the backword compatibility issu...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1720 retest this please ---
[GitHub] carbondata issue #1720: [CARBONDATA-1935]fix the backword compatibility issu...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1720 retest this please ---
[GitHub] carbondata issue #1682: [CARBONDATA-1910]do not allow tupleid, referenceid a...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1682 retest this please ---
[GitHub] carbondata issue #1732: [WIP] Fixed select query failure after alter change ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1732 retest this please ---
[GitHub] carbondata issue #1720: [CARBONDATA-1935]fix the backword compatibility issu...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1720 retest this please ---
[GitHub] carbondata pull request #1735: [CARBONDATA-1947]fix select * issue after com...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1735 [CARBONDATA-1947]fix select * issue after compaction, delete and clean files operation **Problem**: fix select * issue after compaction, delete and clean files operation **Analysis**: during compaction the timestamp of fact file and the load starttime present in metadata details are different for the same segments, so when the delete operation is executed and after that clean files is called, since there is a timestamp difference, it will delete the carbondata file which inturn gives empty result for select query. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done Please provide details on - Units test cases are added to test the scenario - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata clean_after_delete Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1735.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1735 commit 47e9a7753473936607f54321d7b0602fcf50e90a Author: akashrn5 Date: 2017-12-28T08:13:15Z fix select * issue after compaction, delete and clean files operation ---
[GitHub] carbondata issue #1735: [CARBONDATA-1947]fix select * issue after compaction...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1735 retest this please ---
[GitHub] carbondata issue #1735: [CARBONDATA-1947]fix select * issue after compaction...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1735 retest this please ---
[GitHub] carbondata issue #1720: [CARBONDATA-1935]fix the backword compatibility issu...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1720 retest this please ---
[GitHub] carbondata issue #1741: [Carbondata 1951]: Sort Scope displayed multiple tim...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1741 this issue will be fixed with #1740 , so you can close this PR ---
[GitHub] carbondata issue #1720: [CARBONDATA-1935]fix the backword compatibility issu...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1720 retest this please ---
[GitHub] carbondata issue #1688: [CARBONDATA-1916]Correct the database location path ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1688 retest this please ---
[GitHub] carbondata issue #1688: [CARBONDATA-1916]Correct the database location path ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1688 retest this please ---
[GitHub] carbondata pull request #1766: [WIP]enable hive metastore and test
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1766 [WIP]enable hive metastore and test Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata hive_metastore Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1766.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1766 commit 16cf74053f018db879a2b78beb2818ba32aa8dfb Author: akashrn5 Date: 2018-01-05T14:42:17Z enable hive metastore and test ---
[GitHub] carbondata pull request #1769: [WIP]update table status for failure only aft...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1769 [WIP]update table status for failure only after first entry and IndexOutOfBound Fix Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata updateStatus Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1769.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1769 commit 17a29461e47e6cf7ef121f3d7769403ce0c80390 Author: akashrn5 Date: 2018-01-06T10:59:30Z update table status for failure only after first entry and IndexOutOfBound Fix ---
[GitHub] carbondata pull request #1793: [CARBONDATA-2021]fix clean up issue when upda...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1793 [CARBONDATA-2021]fix clean up issue when update operation is abprutly stopped when delete is success and update is failed while writing status file then a stale carbon data file is created. so removing that file on clean up . and also not considering that one during query. =>when the update operation is running and the user stops it abruptly, then the carbon data file will be remained in the store . so extra data is coming. =>so during the next update the clean up of the files need to be handled. and in query also new data file should be excluded. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [x] Any interfaces changed? NA - [x] Any backward compatibility impacted? NA - [x] Document update required? NA - [x] Testing done manual testing Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata update_abrupt Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1793.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1793 commit 8983ce1cdeaf89c1f2e79f259a2f796a996fffd9 Author: akashrn5 Date: 2018-01-10T14:59:43Z fix clean up issue when update operation is abprutly stopped ---
[GitHub] carbondata pull request #1809: [CARBONDATA-2031] Fix ArrayIndexOutOfBoundExc...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1809 [CARBONDATA-2031] Fix ArrayIndexOutOfBoundException when filter query is applied on column where all values are null and column is noinverted index column PROBLEM: Fix ArrayIndexOutOfBoundException when filter query is applied on column where all values are null and column is noinverted index column when all the values are null in no inverted index column the number of exclude filter keys are null, hence just return the bitset if the exclude filters to be applied are none. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done UT is added Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata AIOB Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1809.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1809 commit 6eec40e7c9cc82174e3beaf451a74fb86c4d84da Author: akashrn5 Date: 2018-01-16T06:47:53Z Fix ArrayIndexOutOfBoundException when filter query is applied on column where all values are null and column is noinverted index column ---
[GitHub] carbondata issue #1793: [CARBONDATA-2021]fix clean up issue when update oper...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1793 retest this please ---
[GitHub] carbondata issue #1793: [CARBONDATA-2021]fix clean up issue when update oper...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1793 retest sdv please ---
[GitHub] carbondata issue #1809: [CARBONDATA-2031] Fix ArrayIndexOutOfBoundException ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1809 retest this please ---
[GitHub] carbondata pull request #1793: [CARBONDATA-2021]fix clean up issue when upda...
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1793#discussion_r161996456
--- Diff:
core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentUpdateStatusManager.java
---
@@ -956,9 +964,16 @@ public UpdateVO getInvalidTimestampRange(String
segmentId) {
long timestamp = CarbonUpdateUtil.getTimeStampAsLong(
CarbonTablePath.DataFileUtil.getTimeStampFromDeleteDeltaFile(fileName));
-if (block.getBlockName().equalsIgnoreCase(blkName) && (
-Long.compare(timestamp, deltaStartTimestamp) < 0)) {
- files.add(eachFile);
+if (block.getBlockName().equalsIgnoreCase(blkName)) {
+
+ if (getOnlyAbortedFiles) {
+if (Long.compare(timestamp, deltaEndTimestamp) > 0) {
--- End diff --
yes, but for all the timestamp comparision in code, we were using
Long.compare, so i follwed the same , just to make same for all
---
[GitHub] carbondata issue #1793: [CARBONDATA-2021]fix clean up issue when update oper...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1793 retest this please ---
[GitHub] carbondata issue #1793: [CARBONDATA-2021]fix clean up issue when update oper...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1793 retest sdv please ---
[GitHub] carbondata issue #1793: [CARBONDATA-2021]fix clean up issue when update oper...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1793 retest sdv please ---
[GitHub] carbondata pull request #1766: [WIP]enable hive metastore and test
Github user akashrn5 closed the pull request at: https://github.com/apache/carbondata/pull/1766 ---
[GitHub] carbondata pull request #1838: [CARBONDATA-2060]fix insert overwrite on part...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1838 [CARBONDATA-2060]fix insert overwrite on partition table Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: Problem: when insert overwrite is done on partition table with the table which has empty data, it was not overwriting. Solution: when insert OverWrite is fired on partition table from empty table, it should create new empty segment and it should delete old segments. - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done Unit tests cases are added to test the scenario Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata partition_overwrite Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1838.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1838 commit 56e3054ade07bfd39e7ec1050fe2962166d4ad23 Author: akashrn5 Date: 2018-01-19T14:57:05Z fix insert overwrite on partition table ---
[GitHub] carbondata issue #1793: [CARBONDATA-2021]fix clean up issue when update oper...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1793 retest sdv please ---
[GitHub] carbondata issue #1793: [CARBONDATA-2021]fix clean up issue when update oper...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1793 retest this please ---
[GitHub] carbondata pull request #1852: [CARBONDATA-2070]fix create preaggregate on d...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1852 [CARBONDATA-2070]fix create preaggregate on decimal column in hive metastore Problem: when hive metastore is enabled and aggregate table is tried to create on the decimal column of main table, cast exception is thrown for Decimal datatype solution:During creation of TableInfo from hivemetastore the DataMapSchemas and the columns dataTypes are not converted to the appropriate child classes. convert to actual datatype Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done Added UT Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata hive_pre Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1852.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1852 commit d8c38d5d11fbb1f4fcfb590168b8785497ab868a Author: akashrn5 Date: 2018-01-23T13:29:51Z fix create preaggregate on decimal column in hive metastore ---
[GitHub] carbondata issue #1838: [CARBONDATA-2060]fix insert overwrite on partition t...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1838 retest this please ---
[GitHub] carbondata issue #1852: [CARBONDATA-2070]fix create preaggregate on decimal ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1852 retest this please ---
[GitHub] carbondata issue #1852: [CARBONDATA-2070]fix create preaggregate on decimal ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1852 retest sdv please ---
[GitHub] carbondata issue #1852: [CARBONDATA-2070]fix create preaggregate on decimal ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1852 retest this please ---
[GitHub] carbondata pull request #1855: [CARBONDATA-1796]while submiting new job, pas...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1855 [CARBONDATA-1796]while submiting new job, pass credentials in jobConf object In hadoop secure mode cluster, while submitting job to hadoopRdd token should be generated for the path in JobConf, else Delegation Token exception will be thrown during load. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? NA - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done done in cluster mode Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata passConf Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1855.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1855 commit f2b793cc341c5499f46e1d8aea547da9cbc7250a Author: akashrn5 Date: 2018-01-24T08:56:28Z while submiting new job, pass credentials in jobConf object ---
[GitHub] carbondata pull request #1901: [wip]compatibility fix for v2
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1901 [wip]compatibility fix for v2 Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata compatibility_v2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1901.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1901 commit b4e6885dab8f01d5e2980d30e98375db551be6f2 Author: akashrn5 Date: 2018-02-01T07:12:49Z compatibility fix for v2 ---
[GitHub] carbondata issue #1901: [Compatibility]compatibility fix for v2
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1901 retest this please ---
[GitHub] carbondata pull request #1947: [CARBONDATA-2119]deserialization issue for ca...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1947 [CARBONDATA-2119]deserialization issue for carbonloadmodel Problem: Load model was not getting de-serialized in the executor due to which 2 different carbon table objects were being created. Solution: Reconstruct carbonTable from tableInfo if not already created. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [x] Any interfaces changed? NA - [x] Any backward compatibility impacted? NA - [x] Document update required? NA - [x] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata dat Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1947.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1947 commit 62a33b4245ccf3999a8d94121b0f15b37a711a76 Author: akashrn5 Date: 2018-02-07T13:14:33Z deserialization issue for carbonloadmodel ---
[GitHub] carbondata issue #1947: [CARBONDATA-2119]deserialization issue for carbonloa...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1947 retest sdv please ---
[GitHub] carbondata pull request #1957: [CARBONDATA-2150] Unwanted updatetable status...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1957 [CARBONDATA-2150] Unwanted updatetable status files are being generated for the delete operation where no records are deleted **Problem:** Unwanted updatetable status files are being generated for the delete operation where no records are deleted **Analysis:**when the filter value for delete operation is less than the maximum value in that column, then getsplits() will return the block and hence in delete logic, it was creating update table status file even though delete operation was not done. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [x] Any interfaces changed? NA - [x] Any backward compatibility impacted? NA - [x] Document update required? NA - [x] Testing done test cases are added Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata del Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1957.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1957 commit af1ae4cdbcec7f590a5eff5dca3e88899cda6916 Author: akashrn5 Date: 2018-02-08T17:40:43Z unwanted updatetablestatus files are being created for delete operation where no records are selected ---
[GitHub] carbondata issue #1957: [CARBONDATA-2150] Unwanted updatetable status files ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1957 retest sdv please ---
[GitHub] carbondata issue #1957: [CARBONDATA-2150] Unwanted updatetable status files ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1957 retest this please ---
[GitHub] carbondata issue #1957: [CARBONDATA-2150] Unwanted updatetable status files ...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1957 retest sdv please ---
[GitHub] carbondata pull request #1978: [CARBONDATA-2182]added one more params called...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1978 [CARBONDATA-2182]added one more params called extraParams in SessionParams and add carbonSessionInfo to CarbonEnvInitPreEvent add one more param called ExtraParmas in SessionParams for session Level operations and pass the carbonSessionInfo to event, so that user can save information in that at session level, in carbonSessionInfo Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [x] Any interfaces changed? NA - [x] Any backward compatibility impacted? NA - [x] Document update required? NA - [x] Testing done manual testing Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata extra Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1978.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1978 commit adfdad79debe8990a5de6b2d5c63b970a360cd0a Author: akashrn5 Date: 2018-02-13T12:29:47Z added one more params called extraParams in SessionParams ---
[GitHub] carbondata pull request #1979: [CARBONDATA-2183]fix compaction when segment ...
GitHub user akashrn5 opened a pull request: https://github.com/apache/carbondata/pull/1979 [CARBONDATA-2183]fix compaction when segment is delete during compaction and remove unnecessary parameters in functions **Problem:** when compaction is started and job is running, and parallelly the segment involved in the compaction is deleted using DeleteSegmentByID, then compaction is success. **Solution:** when compaction is started and job is running, and parallelly the segment involved in the compaction is deleted using DeleteSegmentByID, then compaction should be aborted and failed. and proper error message should thrown to user. THis PR also removes the unnecessary parameters in functions. - [x] Any interfaces changed? NA - [x] Any backward compatibility impacted? NA - [x] Document update required? NA - [x] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/akashrn5/incubator-carbondata compact_del Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1979.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1979 commit 500b4f01bd390a57c846de6d3309f143957248dd Author: akashrn5 Date: 2018-02-14T10:15:04Z fix compaction when segment is delete during compaction and remove unnecessary parameters in functions ---
[GitHub] carbondata issue #1978: [CARBONDATA-2182]added one more params called extraP...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/1978 retest sdv please ---
