Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
ChenSammi merged PR #9614: URL: https://github.com/apache/ozone/pull/9614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
ChenSammi commented on PR #9614: URL: https://github.com/apache/ozone/pull/9614#issuecomment-4167253331 Thanks @chungen0126 @xichen01 for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
ChenSammi commented on PR #9614: URL: https://github.com/apache/ozone/pull/9614#issuecomment-4167251502 > +1 LGTM @xichen01 Would you like to take a look? I have offline discussed this patch with @xichen01. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
YutaLin commented on PR #9614: URL: https://github.com/apache/ozone/pull/9614#issuecomment-4148591063 Hi @ChenSammi, thanks for review, i've made some change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
github-actions[bot] commented on PR #9614: URL: https://github.com/apache/ozone/pull/9614#issuecomment-4146047153 This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
ChenSammi commented on code in PR #9614: URL: https://github.com/apache/ozone/pull/9614#discussion_r2894621600 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/lifecycle/OMLifecycleConfigurationSetRequest.java: ## @@ -190,4 +196,21 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager, Execut return omClientResponse; } } + + @RequestFeatureValidator( + conditions = ValidationCondition.CLUSTER_NEEDS_FINALIZATION, + processingPhase = RequestProcessingPhase.PRE_PROCESS, + requestType = Type.SetLifecycleConfiguration + ) + public static OMRequest disallowSetLifecycleConfigurationBeforeFinalization( Review Comment: @YutaLin , thanks working on this. There is one more OMLifecycleSetServiceStatusRequest request to add this validator. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
ChenSammi commented on code in PR #9614:
URL: https://github.com/apache/ozone/pull/9614#discussion_r2894625598
##
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMLayoutFeature.java:
##
@@ -46,7 +46,8 @@ public enum OMLayoutFeature implements LayoutFeature {
QUOTA(6, "Ozone quota re-calculate"),
HBASE_SUPPORT(7, "Full support of hsync, lease recovery and listOpenFiles
APIs for HBase"),
DELEGATION_TOKEN_SYMMETRIC_SIGN(8, "Delegation token signed by symmetric
key"),
- SNAPSHOT_DEFRAG(9, "Supporting defragmentation of snapshot");
+ SNAPSHOT_DEFRAG(9, "Supporting defragmentation of snapshot"),
+ LIFECYCLE_SUPPORT(10, "Life Cycle configuration");
Review Comment:
Can we call it S3_LIFECYCLE_SUPPORT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
ChenSammi commented on code in PR #9614: URL: https://github.com/apache/ozone/pull/9614#discussion_r2894621600 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/lifecycle/OMLifecycleConfigurationSetRequest.java: ## @@ -190,4 +196,21 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager, Execut return omClientResponse; } } + + @RequestFeatureValidator( + conditions = ValidationCondition.CLUSTER_NEEDS_FINALIZATION, + processingPhase = RequestProcessingPhase.PRE_PROCESS, + requestType = Type.SetLifecycleConfiguration + ) + public static OMRequest disallowSetLifecycleConfigurationBeforeFinalization( Review Comment: There is one more OMLifecycleSetServiceStatusRequest request to add this validator. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
YutaLin commented on PR #9614: URL: https://github.com/apache/ozone/pull/9614#issuecomment-3956604065 Hi @xichen01 @chungen0126, could you help me review this, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
github-actions[bot] commented on PR #9614: URL: https://github.com/apache/ozone/pull/9614#issuecomment-3948082661 This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
YutaLin commented on PR #9614: URL: https://github.com/apache/ozone/pull/9614#issuecomment-3797945225 Hi @chungen0126 @xichen01, thanks for the review, i've add the tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
chungen0126 commented on PR #9614: URL: https://github.com/apache/ozone/pull/9614#issuecomment-3754627537 > @YutaLin Thanks for your patch. > > I may be missing something, but do we need to upgrade the framework for the lifecycle feature? > > My understanding is the lifecycle feature only adds a few DB definition and interfaces without altering the existing DB field. If we roll back after upgrading, it might leave some useless data in the DB, but it shouldn't cause data to become unreadable. Thanks @xichen01 for the review. You are right that we don't change existing fields, but I'm concerned about skipping the upgrade process for a couple of reasons: 1. Based on my understanding of the design doc. See https://github.com/apache/ozone/blob/master/hadoop-hdds/docs/content/design/upgrade-dev-primer.md?plain=1#L76-L81, adding a new Column Family (CF) must happen during the ON_FINALIZE stage. 2. In my opinion, It is risky If we write lifecycle data and then rollback, some "ghost data" will stay in RocksDB. When we upgrade again later, this data might start the lifecycle check and delete things without notice. I think it's safer to follow the upgrade framework here. Let me know your thoughts! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] HDDS-14347. Handle upgrades to support S3 lifecycle [ozone]
xichen01 commented on PR #9614: URL: https://github.com/apache/ozone/pull/9614#issuecomment-3754107232 @YutaLin Thanks for your patch. I may be missing something, but do we need to upgrade the framework for the lifecycle feature? My understanding is the lifecycle feature only adds a few DB definition and interfaces without altering the existing DB field. If we roll back after upgrading, it might leave some useless data in the DB, but it shouldn't cause data to become unreadable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
