[
https://issues.apache.org/jira/browse/IGNITE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-18093:
-------------------------------------
Description:
h3. Motivation
Distribution zones manager is an ignite component that is responsible for both
zones configuration(create/update/drop) and dataNodes calculation that is
triggered by corresponding topology and zone modification events.
Within the scope of current ticket it's expected that DistributionZoneManager
will be introduced with an internal API to create, drop and alter distribution
zones.
h3. Definition of Done
* New ignite component DistributionZoneManager is introduced.
* It is inlined into ignite node start/stop flow.
* It's possible to create/alter/drop distribution zone with the help of ignite
internal api that technically leads to cluster configuration changes.
h3. Implementation Notes
1. First of all we should introduce new root zones configuration schema:
{code:java}
@ConfigurationRoot(rootName = "zone", type = ConfigurationType.DISTRIBUTED)
public class DistributionZonesConfigurationSchema {
/** Global integer id counter. Used as an auto-increment counter to generate
integer identifiers for distribution zone. */
@Value(hasDefault = true)
public int globalIdCounter = 0;
/** List of configured distribution zones. */
@NamedConfigValue
public DistributionZoneConfigurationSchema distributionZones;
}{code}
and zone configuration itself:
{code:java}
/**
* Distribution zone configuration schema class.
*/
@Config
public class DistributionZoneConfigurationSchema {
/** Zone name. */
@InjectedName
public String name;
/** Integer zone id. */
@Immutable
@Range(min = 1)
@Value(hasDefault = true)
public int zoneId = 1;
public int dataNodesAutoAdjustScaleUp;
public int dataNodesAutoAdjustScaleDown;
public int dataNodesAutoAdjust;
} {code}
Filter and affinity function type will be added to configuration later.
2. It's also required to introduce DistributionZoneManager as an
IgniteComponent that should be started and stopped during node start and stop
flow. The proper place to insert distributionZoneManager.start() is somewhere
after clusterCfgMgr.start().
3. DistributionZoneManager should expose (as in internal API) methods to create
alter and drop distribution zones.
4. Aforementioned create/alter/dropZone API should be available as part of
Ignite @TestOnly internal api.
5. Internally all that methods should call update configuration closures.
6. The only interesting part here is input data and drop validations.
* In case of drop we should check whether there are tables binded to given
zone. Let's say that CASCADE is not supported for now.
* Zone name should be checked for uniqueness. Let's ask SE teem about proper
way of doing such checks, maybe there's an annotation that will help.
* Neither scaleUp or scaleDown are compatible with common auto adjust, so we
should complete create result future (yep it's async) exceptionally if user
specifies both (scaleUp || scaleDown) and common auto adjust. In case of alter
proposed property should override non-compatible ones.
* All-in-all some explicit validation, some configuration validators (see
@TableValidator) and tests are expected.
7. Lot's of unit tests are expected to be written, that will check that
distibutionZoneManager.create()/update()/drop() is either returns future
completed exceptionally with proper exception or actually change the
configuration.
8. Please pay attention, that within given ticket its' not expected for
partitions and replicas to be moved inside Zones. Let it still be tables for
now.
was:
h3. Motivation
Distribution zones manager is an ignite component that is responsible for both
zones configuration(create/update/drop) and dataNodes calculation that is
triggered by corresponding topology and zone modification events.
Within the scope of current ticket it's expected that DistributionZoneManager
will be introduced with an internal API to create, drop and alter distribution
zones.
h3. Definition of Done
* New ignite component DistributionZoneManager is introduced.
* It is inlined into ignite node start/stop flow.
* It's possible to create/alter/drop distribution zone that internally leads
to cluster configuration changes.
h3. Implementation Notes
1. First of all we should introduce new root zones configuration schema:
{code:java}
@ConfigurationRoot(rootName = "zone", type = ConfigurationType.DISTRIBUTED)
public class DistributionZonesConfigurationSchema {
/** Global integer id counter. Used as an auto-increment counter to generate
integer identifiers for distribution zone. */
@Value(hasDefault = true)
public int globalIdCounter = 0;
/** List of configured distribution zones. */
@NamedConfigValue
public DistributionZoneConfigurationSchema distributionZones;
}{code}
and zone configuration itself:
{code:java}
/**
* Distribution zone configuration schema class.
*/
@Config
public class DistributionZoneConfigurationSchema {
/** Zone name. */
@InjectedName
public String name;
/** Integer zone id. */
@Immutable
@Range(min = 1)
@Value(hasDefault = true)
public int zoneId = 1;
/** Partitions count. */
@Range(min = 0, max = 65_000)
@Value(hasDefault = true)
public int partitions = 25;
/** Count of partition replicas. */
@Range(min = 1)
@Value(hasDefault = true)
public int replicas = 1;
public int dataNodesAutoAdjustScaleUp;
public int dataNodesAutoAdjustScaleDown;
public int dataNodesAutoAdjust;
} {code}
Filter and affinity function type will be added to configuration later.
2. It's also required to introduce DistributionZoneManager as an
IgniteComponent that should be started and stopped during node start and stop
flow. The proper place to insert distributionZoneManager.start() is somewhere
after clusterCfgMgr.start().
3. DistributionZoneManager should expose (as in internal API) methods to create
alter and drop distribution zones.
4. Internally all that methods should call update configuration closures.
5. The only interesting part here is input data and drop validations.
* In case of drop we should check whether there are tables binded to given
zone. Let's say that CASCADE is not supported for now.
* Zone name should be checked for uniqueness. Let's ask SE teem about proper
way of doing such checks, maybe there's an annotation that will help.
* Neither scaleUp or scaleDown are compatible with common auto adjust, so we
should complete create result future (yep it's async) exceptionally if user
specifies both (scaleUp || scaleDown) and common auto adjust. In case of alter
proposed property should override non-compatible ones.
* All-in-all some explicit validation, some configuration validators (see
@TableValidator) and lot's of tests are expected.
6. Lot's of unit tests are expected to be written, that will check that
distibutionZoneManager.create()/update()/drop() is either returns future
completed exceptionally with proper exception or actually change the
configuration.
> Introduce distribution zone manager for the purposes of zones configuration
> ---------------------------------------------------------------------------
>
> Key: IGNITE-18093
> URL: https://issues.apache.org/jira/browse/IGNITE-18093
> Project: Ignite
> Issue Type: Improvement
> Reporter: Alexander Lapin
> Priority: Major
>
> h3. Motivation
> Distribution zones manager is an ignite component that is responsible for
> both zones configuration(create/update/drop) and dataNodes calculation that
> is triggered by corresponding topology and zone modification events.
> Within the scope of current ticket it's expected that DistributionZoneManager
> will be introduced with an internal API to create, drop and alter
> distribution zones.
> h3. Definition of Done
> * New ignite component DistributionZoneManager is introduced.
> * It is inlined into ignite node start/stop flow.
> * It's possible to create/alter/drop distribution zone with the help of
> ignite internal api that technically leads to cluster configuration changes.
> h3. Implementation Notes
> 1. First of all we should introduce new root zones configuration schema:
> {code:java}
> @ConfigurationRoot(rootName = "zone", type = ConfigurationType.DISTRIBUTED)
> public class DistributionZonesConfigurationSchema {
> /** Global integer id counter. Used as an auto-increment counter to generate
> integer identifiers for distribution zone. */
> @Value(hasDefault = true)
> public int globalIdCounter = 0;
>
> /** List of configured distribution zones. */
> @NamedConfigValue
> public DistributionZoneConfigurationSchema distributionZones;
> }{code}
> and zone configuration itself:
> {code:java}
> /**
> * Distribution zone configuration schema class.
> */
> @Config
> public class DistributionZoneConfigurationSchema {
> /** Zone name. */
> @InjectedName
> public String name;
> /** Integer zone id. */
> @Immutable
> @Range(min = 1)
> @Value(hasDefault = true)
> public int zoneId = 1;
> public int dataNodesAutoAdjustScaleUp;
> public int dataNodesAutoAdjustScaleDown;
> public int dataNodesAutoAdjust;
> } {code}
> Filter and affinity function type will be added to configuration later.
> 2. It's also required to introduce DistributionZoneManager as an
> IgniteComponent that should be started and stopped during node start and stop
> flow. The proper place to insert distributionZoneManager.start() is somewhere
> after clusterCfgMgr.start().
> 3. DistributionZoneManager should expose (as in internal API) methods to
> create alter and drop distribution zones.
> 4. Aforementioned create/alter/dropZone API should be available as part of
> Ignite @TestOnly internal api.
> 5. Internally all that methods should call update configuration closures.
> 6. The only interesting part here is input data and drop validations.
> * In case of drop we should check whether there are tables binded to given
> zone. Let's say that CASCADE is not supported for now.
> * Zone name should be checked for uniqueness. Let's ask SE teem about proper
> way of doing such checks, maybe there's an annotation that will help.
> * Neither scaleUp or scaleDown are compatible with common auto adjust, so we
> should complete create result future (yep it's async) exceptionally if user
> specifies both (scaleUp || scaleDown) and common auto adjust. In case of
> alter proposed property should override non-compatible ones.
> * All-in-all some explicit validation, some configuration validators (see
> @TableValidator) and tests are expected.
> 7. Lot's of unit tests are expected to be written, that will check that
> distibutionZoneManager.create()/update()/drop() is either returns future
> completed exceptionally with proper exception or actually change the
> configuration.
> 8. Please pay attention, that within given ticket its' not expected for
> partitions and replicas to be moved inside Zones. Let it still be tables for
> now.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
