Re: [PR] IGNITE-25144 Make throttling dirty pages bounds configurable [ignite-3]
rpuch commented on code in PR #5638:
URL: https://github.com/apache/ignite-3/pull/5638#discussion_r2044272557
##
modules/storage-page-memory/src/main/java/org/apache/ignite/internal/storage/pagememory/PersistentPageMemoryDataRegion.java:
##
@@ -199,52 +202,81 @@ private void initThrottling(PersistentPageMemory
pageMemory) {
}
private ThrottlingType getThrottlingType() {
-SystemPropertyView throttlingTypeCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_TYPE_SYSTEM_PROPERTY);
-
-ThrottlingType throttlingType = ThrottlingType.SPEED_BASED;
-
-if (throttlingTypeCfg != null) {
-try {
-throttlingType =
ThrottlingType.valueOf(throttlingTypeCfg.propertyValue().toUpperCase());
-} catch (IllegalArgumentException e) {
-LOG.warn(
-"Invalid throttling configuration {}={}, using default
value {}",
-THROTTLING_TYPE_SYSTEM_PROPERTY,
-throttlingTypeCfg.propertyValue(),
-throttlingType
-);
-}
-}
-return throttlingType;
+return getSystemConfig(THROTTLING_TYPE_SYSTEM_PROPERTY,
+ThrottlingType.SPEED_BASED,
+value -> ThrottlingType.valueOf(value.toUpperCase())
+);
}
private long getLoggingThreshold() {
-SystemPropertyView logThresholdCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY);
+return TimeUnit.MILLISECONDS.toNanos(getSystemConfig(
+THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY,
+
TimeUnit.NANOSECONDS.toMillis(PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD),
+value -> {
+long logThresholdMillis = Long.parseLong(value);
-long logThresholdNanos =
PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD;
-try {
-if (logThresholdCfg != null) {
-long logThresholdMillis =
Long.parseLong(logThresholdCfg.propertyValue());
+if (logThresholdMillis <= 0) {
+throw new IllegalArgumentException();
Review Comment:
Should we use the error message in the catch then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] IGNITE-25144 Make throttling dirty pages bounds configurable [ignite-3]
ibessonov commented on code in PR #5638:
URL: https://github.com/apache/ignite-3/pull/5638#discussion_r2044247908
##
modules/storage-page-memory/src/main/java/org/apache/ignite/internal/storage/pagememory/PersistentPageMemoryDataRegion.java:
##
@@ -199,52 +202,81 @@ private void initThrottling(PersistentPageMemory
pageMemory) {
}
private ThrottlingType getThrottlingType() {
-SystemPropertyView throttlingTypeCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_TYPE_SYSTEM_PROPERTY);
-
-ThrottlingType throttlingType = ThrottlingType.SPEED_BASED;
-
-if (throttlingTypeCfg != null) {
-try {
-throttlingType =
ThrottlingType.valueOf(throttlingTypeCfg.propertyValue().toUpperCase());
-} catch (IllegalArgumentException e) {
-LOG.warn(
-"Invalid throttling configuration {}={}, using default
value {}",
-THROTTLING_TYPE_SYSTEM_PROPERTY,
-throttlingTypeCfg.propertyValue(),
-throttlingType
-);
-}
-}
-return throttlingType;
+return getSystemConfig(THROTTLING_TYPE_SYSTEM_PROPERTY,
+ThrottlingType.SPEED_BASED,
+value -> ThrottlingType.valueOf(value.toUpperCase())
+);
}
private long getLoggingThreshold() {
-SystemPropertyView logThresholdCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY);
+return TimeUnit.MILLISECONDS.toNanos(getSystemConfig(
+THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY,
+
TimeUnit.NANOSECONDS.toMillis(PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD),
+value -> {
+long logThresholdMillis = Long.parseLong(value);
-long logThresholdNanos =
PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD;
-try {
-if (logThresholdCfg != null) {
-long logThresholdMillis =
Long.parseLong(logThresholdCfg.propertyValue());
+if (logThresholdMillis <= 0) {
+throw new IllegalArgumentException();
Review Comment:
The message is ignored in the "catch", but I agree that some clarification
would be helpful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] IGNITE-25144 Make throttling dirty pages bounds configurable [ignite-3]
ibessonov merged PR #5638: URL: https://github.com/apache/ignite-3/pull/5638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] IGNITE-25144 Make throttling dirty pages bounds configurable [ignite-3]
ibessonov commented on code in PR #5638:
URL: https://github.com/apache/ignite-3/pull/5638#discussion_r2044297388
##
modules/storage-page-memory/src/main/java/org/apache/ignite/internal/storage/pagememory/PersistentPageMemoryDataRegion.java:
##
@@ -199,52 +202,81 @@ private void initThrottling(PersistentPageMemory
pageMemory) {
}
private ThrottlingType getThrottlingType() {
-SystemPropertyView throttlingTypeCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_TYPE_SYSTEM_PROPERTY);
-
-ThrottlingType throttlingType = ThrottlingType.SPEED_BASED;
-
-if (throttlingTypeCfg != null) {
-try {
-throttlingType =
ThrottlingType.valueOf(throttlingTypeCfg.propertyValue().toUpperCase());
-} catch (IllegalArgumentException e) {
-LOG.warn(
-"Invalid throttling configuration {}={}, using default
value {}",
-THROTTLING_TYPE_SYSTEM_PROPERTY,
-throttlingTypeCfg.propertyValue(),
-throttlingType
-);
-}
-}
-return throttlingType;
+return getSystemConfig(THROTTLING_TYPE_SYSTEM_PROPERTY,
+ThrottlingType.SPEED_BASED,
+value -> ThrottlingType.valueOf(value.toUpperCase())
+);
}
private long getLoggingThreshold() {
-SystemPropertyView logThresholdCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY);
+return TimeUnit.MILLISECONDS.toNanos(getSystemConfig(
+THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY,
+
TimeUnit.NANOSECONDS.toMillis(PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD),
+value -> {
+long logThresholdMillis = Long.parseLong(value);
-long logThresholdNanos =
PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD;
-try {
-if (logThresholdCfg != null) {
-long logThresholdMillis =
Long.parseLong(logThresholdCfg.propertyValue());
+if (logThresholdMillis <= 0) {
+throw new IllegalArgumentException();
Review Comment:
Nah, it would make the message too big
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] IGNITE-25144 Make throttling dirty pages bounds configurable [ignite-3]
rpuch commented on code in PR #5638:
URL: https://github.com/apache/ignite-3/pull/5638#discussion_r2044231717
##
modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/persistence/throttling/PagesWriteSpeedBasedThrottle.java:
##
@@ -42,6 +42,12 @@
* Otherwise, uses average checkpoint write speed and instant speed of marking
pages as dirty.
*/
public class PagesWriteSpeedBasedThrottle implements PagesWriteThrottlePolicy {
+/** Maximum fraction of dirty pages in a region. */
+public static final double MAX_DIRTY_PAGES = 0.75;
Review Comment:
```suggestion
public static final double DEFAULT_MAX_DIRTY_PAGES = 0.75;
```
##
modules/storage-page-memory/src/main/java/org/apache/ignite/internal/storage/pagememory/PersistentPageMemoryDataRegion.java:
##
@@ -199,52 +202,81 @@ private void initThrottling(PersistentPageMemory
pageMemory) {
}
private ThrottlingType getThrottlingType() {
-SystemPropertyView throttlingTypeCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_TYPE_SYSTEM_PROPERTY);
-
-ThrottlingType throttlingType = ThrottlingType.SPEED_BASED;
-
-if (throttlingTypeCfg != null) {
-try {
-throttlingType =
ThrottlingType.valueOf(throttlingTypeCfg.propertyValue().toUpperCase());
-} catch (IllegalArgumentException e) {
-LOG.warn(
-"Invalid throttling configuration {}={}, using default
value {}",
-THROTTLING_TYPE_SYSTEM_PROPERTY,
-throttlingTypeCfg.propertyValue(),
-throttlingType
-);
-}
-}
-return throttlingType;
+return getSystemConfig(THROTTLING_TYPE_SYSTEM_PROPERTY,
+ThrottlingType.SPEED_BASED,
+value -> ThrottlingType.valueOf(value.toUpperCase())
+);
}
private long getLoggingThreshold() {
-SystemPropertyView logThresholdCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY);
+return TimeUnit.MILLISECONDS.toNanos(getSystemConfig(
+THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY,
+
TimeUnit.NANOSECONDS.toMillis(PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD),
+value -> {
+long logThresholdMillis = Long.parseLong(value);
-long logThresholdNanos =
PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD;
-try {
-if (logThresholdCfg != null) {
-long logThresholdMillis =
Long.parseLong(logThresholdCfg.propertyValue());
+if (logThresholdMillis <= 0) {
+throw new IllegalArgumentException();
+}
-if (logThresholdMillis <= 0) {
-throw new IllegalArgumentException();
+return logThresholdMillis;
}
+));
+}
+
+private double getMinDirtyPages() {
+return getSystemConfig(
+THROTTLING_MIN_DIRTY_PAGES_SYSTEM_PROPERTY,
+PagesWriteSpeedBasedThrottle.MIN_RATIO_NO_THROTTLE,
+value -> {
+double maxDirtyPages = Double.parseDouble(value);
+
+if (maxDirtyPages <= 0.01 || maxDirtyPages > 0.75) {
+throw new IllegalArgumentException();
+}
-logThresholdNanos =
TimeUnit.MILLISECONDS.toNanos(logThresholdMillis);
+return maxDirtyPages;
+}
+);
+}
+
+private double getMaxDirtyPages() {
+return getSystemConfig(
+THROTTLING_MAX_DIRTY_PAGES_SYSTEM_PROPERTY,
+PagesWriteSpeedBasedThrottle.MAX_DIRTY_PAGES,
+value -> {
+double maxDirtyPages = Double.parseDouble(value);
+
+if (maxDirtyPages <= 0.5 || maxDirtyPages > 0.9) {
+throw new IllegalArgumentException();
Review Comment:
Same thing about validation message. Also, I think there is a machinery that
allows to add validators on the config side. It would greatly improve the UX if
we had such validators for the properties you are adding.
##
modules/storage-page-memory/src/main/java/org/apache/ignite/internal/storage/pagememory/PersistentPageMemoryDataRegion.java:
##
@@ -199,52 +202,81 @@ private void initThrottling(PersistentPageMemory
pageMemory) {
}
private ThrottlingType getThrottlingType() {
-SystemPropertyView throttlingTypeCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_TYPE_SYSTEM_PROPERTY);
-
-ThrottlingType
[PR] IGNITE-25144 Make throttling dirty pages bounds configurable [ignite-3]
ibessonov opened a new pull request, #5638: URL: https://github.com/apache/ignite-3/pull/5638 https://issues.apache.org/jira/browse/IGNITE-25144 Thank you for submitting the pull request. To streamline the review process of the patch and ensure better code quality we ask both an author and a reviewer to verify the following: ### The Review Checklist - [ ] **Formal criteria:** TC status, codestyle, mandatory documentation. Also make sure to complete the following: \- There is a single JIRA ticket related to the pull request. \- The web-link to the pull request is attached to the JIRA ticket. \- The JIRA ticket has the Patch Available state. \- The description of the JIRA ticket explains WHAT was made, WHY and HOW. \- The pull request title is treated as the final commit message. The following pattern must be used: IGNITE- Change summary where - number of JIRA issue. - [ ] **Design:** new code conforms with the design principles of the components it is added to. - [ ] **Patch quality:** patch cannot be split into smaller pieces, its size must be reasonable. - [ ] **Code quality:** code is clean and readable, necessary developer documentation is added if needed. - [ ] **Tests code quality:** test set covers positive/negative scenarios, happy/edge cases. Tests are effective in terms of execution time and resources. ### Notes - [Apache Ignite Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
