[GitHub] incubator-carbondata pull request #207: [CARBONDATA-283] VT enhancement for ...
Github user ManoharVanam commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/207#discussion_r86523287
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/util/CarbonTableStatusUtil.java
---
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.carbondata.processing.util;
+
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Date;
+import java.util.List;
+
+import org.apache.carbondata.common.logging.LogService;
+import org.apache.carbondata.common.logging.LogServiceFactory;
+import org.apache.carbondata.core.constants.CarbonCommonConstants;
+import org.apache.carbondata.core.load.LoadMetadataDetails;
+
+/**
+ * This class contains all table status file utilities
+ */
+public final class CarbonTableStatusUtil {
+ private static final LogService LOGGER =
+
LogServiceFactory.getLogService(CarbonTableStatusUtil.class.getName());
+
+ private CarbonTableStatusUtil() {
+
+ }
+
+ /**
+ * updates table status details using latest metadata
+ *
+ * @param oldMetadata
+ * @param newMetadata
+ * @return
+ */
+
+ public static List updateLatestTableStatusDetails(
--- End diff --
ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #207: [CARBONDATA-283] VT enhancement for ...
Github user ManoharVanam commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/207#discussion_r86523222
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/util/CarbonTableStatusUtil.java
---
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.carbondata.processing.util;
+
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Date;
+import java.util.List;
+
+import org.apache.carbondata.common.logging.LogService;
+import org.apache.carbondata.common.logging.LogServiceFactory;
+import org.apache.carbondata.core.constants.CarbonCommonConstants;
+import org.apache.carbondata.core.load.LoadMetadataDetails;
+
+/**
+ * This class contains all table status file utilities
+ */
+public final class CarbonTableStatusUtil {
+ private static final LogService LOGGER =
+
LogServiceFactory.getLogService(CarbonTableStatusUtil.class.getName());
+
+ private CarbonTableStatusUtil() {
+
+ }
+
+ /**
+ * updates table status details using latest metadata
+ *
+ * @param oldMetadata
+ * @param newMetadata
+ * @return
+ */
+
+ public static List updateLatestTableStatusDetails(
+ LoadMetadataDetails[] oldMetadata, LoadMetadataDetails[]
newMetadata) {
+
+List newListMetadata =
+new ArrayList(Arrays.asList(newMetadata));
+for (LoadMetadataDetails oldSegment : oldMetadata) {
+ if
(CarbonCommonConstants.MARKED_FOR_DELETE.equalsIgnoreCase(oldSegment.getLoadStatus()))
{
+
updateSegmentMetadataDetails(newListMetadata.get(newListMetadata.indexOf(oldSegment)));
+ }
+}
+return newListMetadata;
+ }
+
+ /**
+ * returns current time
+ *
+ * @return
+ */
+ private static String readCurrentTime() {
+SimpleDateFormat sdf = new
SimpleDateFormat(CarbonCommonConstants.CARBON_TIMESTAMP);
+String date = null;
+
+date = sdf.format(new Date());
+
+return date;
+ }
+
+ /**
+ * updates segment status and modificaton time details
+ *
+ * @param loadMetadata
+ */
+ public static void updateSegmentMetadataDetails(LoadMetadataDetails
loadMetadata) {
--- End diff --
ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #207: [CARBONDATA-283] VT enhancement for ...
GitHub user ManoharVanam opened a pull request: https://github.com/apache/incubator-carbondata/pull/207 [CARBONDATA-283] VT enhancement for data retention concurrent scenarios why raise this PR ? 1 . Handling concurrency scenarios for retention feature 2. Added test cases for concurrent scenarios like load, delete segment by date/ id & clean files operations. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ManoharVanam/incubator-carbondata Concurrency Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-carbondata/pull/207.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #207 commit 1af96b9bfc258fba4110f39dd0a5ca5e119a8189 Author: Manohar Date: 2016-09-30T10:01:34Z Handle delete segment concurrent scenarios --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #194: [CARBONDATA-270] Double data type va...
Github user ManoharVanam commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/194#discussion_r80212063
--- Diff:
core/src/main/java/org/apache/carbondata/scan/filter/FilterUtil.java ---
@@ -1401,8 +1401,7 @@ public static void logError(Throwable e, boolean
invalidRowsPresent) {
public static boolean nanSafeEqualsDoubles(Double d1, Double d2) {
Boolean xIsNan = Double.isNaN(d1);
Boolean yIsNan = Double.isNaN(d2);
-if ((xIsNan && yIsNan) || (d1.doubleValue() == d2.doubleValue())) {
-
+if ((d1.doubleValue() == d2.doubleValue()) || (xIsNan && yIsNan)) {
--- End diff --
I think we can use Double.compare instead of these two checks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #192: [CARBONDATA-269]change delete segmen...
Github user ManoharVanam commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/192#discussion_r80060992
--- Diff:
integration/spark/src/main/scala/org/apache/spark/sql/CarbonSqlParser.scala ---
@@ -1181,23 +1182,21 @@ class CarbonSqlParser()
p.getClass.getSimpleName.equals("DecimalLit") } ) ^^ (_.chars)
protected lazy val deleteLoadsByID: Parser[LogicalPlan] =
-DELETE ~> SEGMENT ~> repsep(segmentId, ",") ~ (FROM ~> TABLE ~>
- (ident <~ ".").? ~ ident) <~
+DELETE ~> FROM ~ TABLE ~> (ident <~ ".").? ~ ident ~
+ (WHERE ~> SEGMENT ~> EQUAL ~> repsep(segmentId, ",")) <~
--- End diff --
I think its better to use segments in both places deleteLoadsByID ,
deleteLoadsByLoadDate to avoid confusion
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #167: [CARBONDATA-250] Throw exception and...
GitHub user ManoharVanam opened a pull request: https://github.com/apache/incubator-carbondata/pull/167 [CARBONDATA-250] Throw exception and fail the data load if provided MAXCOLUMNS value is not proper [Problem] : If user executes data load query with improper maxcolumns value, data load is showing success instead of error in the console [Solution] : Throw exception and fail the data load if provided MAXCOLUMNS value is not proper. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ManoharVanam/incubator-carbondata MAXCOLUMN Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-carbondata/pull/167.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #167 commit 650f194050835a28b8b450d12df5f0ba98a7790b Author: Manohar Date: 2016-09-17T13:47:58Z Throwing exception in case of improper MAXCOULUMN value --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #165: [WIP] Changed long to Bigint
Github user ManoharVanam closed the pull request at: https://github.com/apache/incubator-carbondata/pull/165 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #165: [WIP] Changed long to Bigint
GitHub user ManoharVanam opened a pull request: https://github.com/apache/incubator-carbondata/pull/165 [WIP] Changed long to Bigint Changed long to Bigint Testing scenarios You can merge this pull request into a Git repository by running: $ git pull https://github.com/ManoharVanam/incubator-carbondata BIGINT_Change Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-carbondata/pull/165.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #165 commit 4ec155ba0c785a628c283328d077dd8390c5e51d Author: Manohar Date: 2016-09-17T10:29:57Z Changed long to Bigint --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #164: [WIP] Changed Long to BigInt
Github user ManoharVanam closed the pull request at: https://github.com/apache/incubator-carbondata/pull/164 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #164: [WIP] Changed Long to BigInt
GitHub user ManoharVanam opened a pull request: https://github.com/apache/incubator-carbondata/pull/164 [WIP] Changed Long to BigInt Changed Long to BigInt You can merge this pull request into a Git repository by running: $ git pull https://github.com/ManoharVanam/incubator-carbondata BIGINT Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-carbondata/pull/164.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #164 commit c6b28cf321e394adfa5d8f390fe681d479863628 Author: Manohar Date: 2016-09-14T17:18:23Z Added table Status lock while deleting the segments commit 70ebe25303a0cdcf7e2728940997b9399f4f5b8e Author: Manohar Date: 2016-09-15T14:29:18Z Added locking while delete date, clean files commit 10c0cf830e7962db0a0a00eb8380c5f7c1de767a Author: Manohar Date: 2016-09-15T14:56:14Z Corrected logs commit 5c3fe5dc367c13de285594a0a8ff7a3f50b18697 Author: Manohar Date: 2016-09-16T06:18:29Z Added test cases commit 346f04aaf71a34fd326cdd736a312131241c10c5 Author: Manohar Date: 2016-09-16T07:09:49Z Added test cases commit 0e9833ca256d9b809c4e859b2cabcc63bc3df055 Author: Manohar Date: 2016-09-16T07:35:53Z added test cases commit 37c1c01c005e2b90596140c8e5994eb9f746b601 Author: Manohar Date: 2016-09-16T08:44:08Z corrected test cases commit eed727d7e0d4a8e4e3b1ac33ce1a7015bd3164ba Author: Manohar Date: 2016-09-16T19:28:51Z Fixed Review comments commit c18cf141d696aabb7c24e8a6dc390ee08d65ca06 Author: Manohar Date: 2016-09-17T10:06:30Z Renamed long to bigint --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #156: [CARBONDATA-244] Load and delete seg...
Github user ManoharVanam commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/156#discussion_r79222066
--- Diff:
processing/src/main/java/org/apache/carbondata/lcm/status/SegmentStatusManager.java
---
@@ -306,13 +330,18 @@ private Integer compareDateValues(Long loadValue,
Long userValue) {
* @return
*/
public List updateDeletionStatus(String loadDate, String
tableFolderPath,
- Long loadStartTime) {
-ICarbonLock carbonLock = CarbonLockFactory
-
.getCarbonLockObj(absoluteTableIdentifier.getCarbonTableIdentifier(),
-LockUsage.METADATA_LOCK);
+ Long loadStartTime) throws Exception {
+CarbonTableIdentifier carbonTableIdentifier =
+absoluteTableIdentifier.getCarbonTableIdentifier();
+ICarbonLock carbonMetadataLock =
+CarbonLockFactory.getCarbonLockObj(carbonTableIdentifier,
LockUsage.METADATA_LOCK);
+ICarbonLock carbonTableStatusLock =
+CarbonLockFactory.getCarbonLockObj(carbonTableIdentifier,
LockUsage.TABLE_STATUS_LOCK);
+String tableDetails =
+carbonTableIdentifier.getDatabaseName() + "." +
carbonTableIdentifier.getTableName();
List invalidLoadTimestamps = new ArrayList(0);
try {
- if (carbonLock.lockWithRetries()) {
+ if (carbonMetadataLock.lockWithRetries()) {
--- End diff --
ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #156: [CARBONDATA-244] Load and delete seg...
Github user ManoharVanam commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/156#discussion_r79217913
--- Diff:
integration/spark/src/main/scala/org/apache/carbondata/spark/rdd/CarbonDataRDDFactory.scala
---
@@ -1077,19 +1077,38 @@ object CarbonDataRDDFactory extends Logging {
val segmentStatusManager = new
SegmentStatusManager(table.getAbsoluteTableIdentifier)
val details = segmentStatusManager
.readLoadMetadata(loadMetadataFilePath)
+ val carbonTableStatusLock = CarbonLockFactory
+
.getCarbonLockObj(table.getAbsoluteTableIdentifier.getCarbonTableIdentifier,
+ LockUsage.TABLE_STATUS_LOCK)
// Delete marked loads
val isUpdationRequired = DeleteLoadFolders
.deleteLoadFoldersFromFileSystem(carbonLoadModel,
hdfsStoreLocation,
partitioner.partitionCount, isForceDeletion, details)
if (isUpdationRequired) {
+try {
// Update load metadate file after cleaning deleted nodes
-CarbonLoaderUtil.writeLoadMetadata(
- carbonLoadModel.getCarbonDataLoadSchema,
- carbonLoadModel.getDatabaseName,
- carbonLoadModel.getTableName, details.toList.asJava
-)
+if (carbonTableStatusLock.lockWithRetries()) {
+ logger.info("Table status lock has been successfully acquired.")
+ CarbonLoaderUtil.writeLoadMetadata(
+carbonLoadModel.getCarbonDataLoadSchema,
+carbonLoadModel.getDatabaseName,
+carbonLoadModel.getTableName, details.toList.asJava
+ )
+}
+else {
+ val errorMsg = "Clean files request is failed for " +
carbonLoadModel.getDatabaseName +
+ "." + carbonLoadModel.getTableName +
+ ". Not able to acquire the table status lock."
--- End diff --
ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #156: [WIP] Added table Status lock while ...
Github user ManoharVanam commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/156#discussion_r79119024
--- Diff:
processing/src/main/java/org/apache/carbondata/lcm/status/SegmentStatusManager.java
---
@@ -246,15 +247,22 @@ private Integer compareDateValues(Long loadValue,
Long userValue) {
* updates deletion status
* @param loadIds
* @param tableFolderPath
+ * @param dbName
+ * @param tableName
* @return
*/
- public List updateDeletionStatus(List loadIds, String
tableFolderPath) {
-ICarbonLock carbonLock = CarbonLockFactory
+ public List updateDeletionStatus(List loadIds, String
tableFolderPath,
+ String dbName, String tableName) throws Exception {
--- End diff --
ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #156: [WIP] Added table Status lock while ...
Github user ManoharVanam commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/156#discussion_r79119039
--- Diff:
processing/src/main/java/org/apache/carbondata/lcm/status/SegmentStatusManager.java
---
@@ -306,13 +330,17 @@ private Integer compareDateValues(Long loadValue,
Long userValue) {
* @return
*/
public List updateDeletionStatus(String loadDate, String
tableFolderPath,
- Long loadStartTime) {
-ICarbonLock carbonLock = CarbonLockFactory
+ Long loadStartTime, String dbName, String tableName) throws
Exception {
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #156: [WIP] Added table Status lock while ...
GitHub user ManoharVanam opened a pull request: https://github.com/apache/incubator-carbondata/pull/156 [WIP] Added table Status lock while deleting the segments Added table Status lock while deleting the segments throwing exception if lock is not available You can merge this pull request into a Git repository by running: $ git pull https://github.com/ManoharVanam/incubator-carbondata DeleteSEg Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-carbondata/pull/156.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #156 commit d6d5304b6e4743f1c081232d449b20691a6ca0e1 Author: Manohar Date: 2016-09-14T17:18:23Z Added table Status lock while deleting the segments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #141: Removed Unsupported features from de...
GitHub user ManoharVanam opened a pull request: https://github.com/apache/incubator-carbondata/pull/141 Removed Unsupported features from describe formatted command description Removed Unsupported feature (Aggregate Tables) from describe formatted command description You can merge this pull request into a Git repository by running: $ git pull https://github.com/ManoharVanam/incubator-carbondata DESCNEW Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-carbondata/pull/141.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #141 commit c88432d8f98d818cfef10e6bb43b3c60bd0f943f Author: Manohar Date: 2016-09-08T06:55:54Z Removed Unsupported features from describe formatted command description --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #109: [CARBONDATA-192] Invalidating the ta...
GitHub user ManoharVanam opened a pull request: https://github.com/apache/incubator-carbondata/pull/109 [CARBONDATA-192] Invalidating the table from hive context while dropping the table Select Query is throwing exception in the below scenario : Create table table1 load table1 select * from table1 drop table1 create same table table1 load table1 select * from table1 throws exception You can merge this pull request into a Git repository by running: $ git pull https://github.com/ManoharVanam/incubator-carbondata InvalidateTable Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-carbondata/pull/109.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #109 commit d9bc40c5beff74136db579601a6455b1f59fc630 Author: Manohar V Date: 2016-08-30T06:30:39Z Invalidating the table from hive context while dropping the table --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-carbondata pull request #92: [CARBONDATA-176] Deletion of compacte...
Github user ManoharVanam commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/92#discussion_r76608195
--- Diff:
processing/src/main/java/org/apache/carbondata/lcm/status/SegmentStatusManager.java
---
@@ -449,6 +457,12 @@ public void writeLoadDetailsIntoFile(String
dataLoadLocation,
for (LoadMetadataDetails loadMetadata : listOfLoadFolderDetailsArray) {
Integer result =
compareDateValues(loadMetadata.getLoadStartTimeAsLong(), loadStartTime);
if (result < 0) {
+if (CarbonCommonConstants.SEGMENT_COMPACTED
--- End diff --
Add test case for this scenario
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #92: [CARBONDATA-176] Deletion of compacte...
Github user ManoharVanam commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/92#discussion_r76606495
--- Diff:
processing/src/main/java/org/apache/carbondata/lcm/status/SegmentStatusManager.java
---
@@ -410,18 +410,26 @@ public void writeLoadDetailsIntoFile(String
dataLoadLocation,
for (LoadMetadataDetails loadMetadata :
listOfLoadFolderDetailsArray) {
if (loadId.equalsIgnoreCase(loadMetadata.getLoadName())) {
+ // if the segment is compacted then no need to delete that.
+ if (CarbonCommonConstants.SEGMENT_COMPACTED
+ .equalsIgnoreCase(loadMetadata.getLoadStatus())) {
+LOG.error("Cannot delete the Segment which is compacted.
Segment is " + loadId);
+loadFound = true;
+invalidLoadIds.add(loadId);
--- End diff --
Above two lines are not required as we are deleting all or none
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
