István Fajth created HDDS-4676:
----------------------------------
Summary: Revisit LayoutFeature, and UpgradeAction related code
Key: HDDS-4676
URL: https://issues.apache.org/jira/browse/HDDS-4676
Project: Hadoop Distributed Data Store
Issue Type: Sub-task
Reporter: István Fajth
The following came up as part of the review and related discussions with
[~avijayan] about HDDS-4219 but as these things came in earlier, we decided to
open up a new JIRA for this, to discuss what can be done. CC [~ppogde] as he
worked the most on implementing the HDDS part.
UpgradeAction is part of the LayoutFeature interface, but its descendants
HDDSUpgradeAction and OMUpgradeAction are separate interfaces, without a
specific LayoutFeature. This seems to be a flaw, especially because concrete
implementations of the UpgradeAction interface are not doing any component
specific thing that is not covered by the base interface, and probably they
won’t do.
The way I look at it the UpgradeAction at the moment is, and may remain in the
future also, a way to support a callback mechanism during the upgrade for the
component to add necessary steps in order to activate a new feature. If this is
true, probably we can even use the Consumer interface instead (more on this
idea later), or at least mark the interface to be a functional interface, and
use it that way.
The relationship between LayoutFeatures and UpgradeActions are having two kind
of binding in OM and in HDDS, I think the OM one is the better approach.
In OM, we have OMLayoutFeature enum, which implements the LayoutFeature
interface, and which defines the LayoutFeatures with layoutVersion,
description, and UpgradeAction triplets, while in HDDS, we have
DataNodeLayoutAction, and SCMLayoutAction enums that binds the UpgradeActions
to LayoutFeatures independently from HDDSLayoutFeature, and therefore we have
HDDSLayoutFeature as an enum, as it is by nature, but with mutable fields,
which we then carefully mutate at initialization time.
The biggest problem with this is that, HDDSLayoutFeature basically redefines
the LayoutFeature API, adds onFinalizeSCMAction, and onFinalizeDataNodeAction
methods to the interface, and let the rest of the code neglect the
onFinalizeAction part of the HDDSLayoutFeatures.
The rationale behind this is probably the fact that HDDS has two components,
and a LayoutFeature may belong to only one of them, or both of them, and in
case of both, the LayoutFeature API is not able to provide a type consistent
view of the UpgradeActions, as there can be UpgradeAction<DatanodeStateMachine>
and UpgradeAction<StorageContainerManager> actions associated with a
LayoutFeature.
The control flow that is using this part of the LayoutFeature is finalization.
We have three very similar way of finalizing the 3 component which we support
at the moment. But all of them has subtle differences, some may be accidental,
and some comes from different APIs, but I firmly believe this can be a generic
flow of the following steps:
- general prepare of finalization if any
- loop over the actions
- call the finalization callback of actions if any
- report the results back
- finish finalization do some after the fact things if any
- report ready to the caller
I am unsure yet what has to be done for this generalization, but I believe
there is an approach where these problems are not present and the control flow
remains the same with more generic and with that shorter and standardized
solutions.
I am happy to put work into this but I don't have time until the next week for
sure.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
