Re: [PR] IGNITE-25144 Make throttling dirty pages bounds configurable [ignite-3]
rpuch commented on code in PR #5638:
URL: https://github.com/apache/ignite-3/pull/5638#discussion_r2044272557
##
modules/storage-page-memory/src/main/java/org/apache/ignite/internal/storage/pagememory/PersistentPageMemoryDataRegion.java:
##
@@ -199,52 +202,81 @@ private void initThrottling(PersistentPageMemory
pageMemory) {
}
private ThrottlingType getThrottlingType() {
-SystemPropertyView throttlingTypeCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_TYPE_SYSTEM_PROPERTY);
-
-ThrottlingType throttlingType = ThrottlingType.SPEED_BASED;
-
-if (throttlingTypeCfg != null) {
-try {
-throttlingType =
ThrottlingType.valueOf(throttlingTypeCfg.propertyValue().toUpperCase());
-} catch (IllegalArgumentException e) {
-LOG.warn(
-"Invalid throttling configuration {}={}, using default
value {}",
-THROTTLING_TYPE_SYSTEM_PROPERTY,
-throttlingTypeCfg.propertyValue(),
-throttlingType
-);
-}
-}
-return throttlingType;
+return getSystemConfig(THROTTLING_TYPE_SYSTEM_PROPERTY,
+ThrottlingType.SPEED_BASED,
+value -> ThrottlingType.valueOf(value.toUpperCase())
+);
}
private long getLoggingThreshold() {
-SystemPropertyView logThresholdCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY);
+return TimeUnit.MILLISECONDS.toNanos(getSystemConfig(
+THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY,
+
TimeUnit.NANOSECONDS.toMillis(PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD),
+value -> {
+long logThresholdMillis = Long.parseLong(value);
-long logThresholdNanos =
PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD;
-try {
-if (logThresholdCfg != null) {
-long logThresholdMillis =
Long.parseLong(logThresholdCfg.propertyValue());
+if (logThresholdMillis <= 0) {
+throw new IllegalArgumentException();
Review Comment:
Should we use the error message in the catch then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] IGNITE-25144 Make throttling dirty pages bounds configurable [ignite-3]
ibessonov commented on code in PR #5638:
URL: https://github.com/apache/ignite-3/pull/5638#discussion_r2044247908
##
modules/storage-page-memory/src/main/java/org/apache/ignite/internal/storage/pagememory/PersistentPageMemoryDataRegion.java:
##
@@ -199,52 +202,81 @@ private void initThrottling(PersistentPageMemory
pageMemory) {
}
private ThrottlingType getThrottlingType() {
-SystemPropertyView throttlingTypeCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_TYPE_SYSTEM_PROPERTY);
-
-ThrottlingType throttlingType = ThrottlingType.SPEED_BASED;
-
-if (throttlingTypeCfg != null) {
-try {
-throttlingType =
ThrottlingType.valueOf(throttlingTypeCfg.propertyValue().toUpperCase());
-} catch (IllegalArgumentException e) {
-LOG.warn(
-"Invalid throttling configuration {}={}, using default
value {}",
-THROTTLING_TYPE_SYSTEM_PROPERTY,
-throttlingTypeCfg.propertyValue(),
-throttlingType
-);
-}
-}
-return throttlingType;
+return getSystemConfig(THROTTLING_TYPE_SYSTEM_PROPERTY,
+ThrottlingType.SPEED_BASED,
+value -> ThrottlingType.valueOf(value.toUpperCase())
+);
}
private long getLoggingThreshold() {
-SystemPropertyView logThresholdCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY);
+return TimeUnit.MILLISECONDS.toNanos(getSystemConfig(
+THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY,
+
TimeUnit.NANOSECONDS.toMillis(PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD),
+value -> {
+long logThresholdMillis = Long.parseLong(value);
-long logThresholdNanos =
PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD;
-try {
-if (logThresholdCfg != null) {
-long logThresholdMillis =
Long.parseLong(logThresholdCfg.propertyValue());
+if (logThresholdMillis <= 0) {
+throw new IllegalArgumentException();
Review Comment:
The message is ignored in the "catch", but I agree that some clarification
would be helpful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] IGNITE-25144 Make throttling dirty pages bounds configurable [ignite-3]
ibessonov merged PR #5638: URL: https://github.com/apache/ignite-3/pull/5638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] IGNITE-25144 Make throttling dirty pages bounds configurable [ignite-3]
ibessonov commented on code in PR #5638:
URL: https://github.com/apache/ignite-3/pull/5638#discussion_r2044297388
##
modules/storage-page-memory/src/main/java/org/apache/ignite/internal/storage/pagememory/PersistentPageMemoryDataRegion.java:
##
@@ -199,52 +202,81 @@ private void initThrottling(PersistentPageMemory
pageMemory) {
}
private ThrottlingType getThrottlingType() {
-SystemPropertyView throttlingTypeCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_TYPE_SYSTEM_PROPERTY);
-
-ThrottlingType throttlingType = ThrottlingType.SPEED_BASED;
-
-if (throttlingTypeCfg != null) {
-try {
-throttlingType =
ThrottlingType.valueOf(throttlingTypeCfg.propertyValue().toUpperCase());
-} catch (IllegalArgumentException e) {
-LOG.warn(
-"Invalid throttling configuration {}={}, using default
value {}",
-THROTTLING_TYPE_SYSTEM_PROPERTY,
-throttlingTypeCfg.propertyValue(),
-throttlingType
-);
-}
-}
-return throttlingType;
+return getSystemConfig(THROTTLING_TYPE_SYSTEM_PROPERTY,
+ThrottlingType.SPEED_BASED,
+value -> ThrottlingType.valueOf(value.toUpperCase())
+);
}
private long getLoggingThreshold() {
-SystemPropertyView logThresholdCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY);
+return TimeUnit.MILLISECONDS.toNanos(getSystemConfig(
+THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY,
+
TimeUnit.NANOSECONDS.toMillis(PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD),
+value -> {
+long logThresholdMillis = Long.parseLong(value);
-long logThresholdNanos =
PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD;
-try {
-if (logThresholdCfg != null) {
-long logThresholdMillis =
Long.parseLong(logThresholdCfg.propertyValue());
+if (logThresholdMillis <= 0) {
+throw new IllegalArgumentException();
Review Comment:
Nah, it would make the message too big
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] IGNITE-25144 Make throttling dirty pages bounds configurable [ignite-3]
rpuch commented on code in PR #5638:
URL: https://github.com/apache/ignite-3/pull/5638#discussion_r2044231717
##
modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/persistence/throttling/PagesWriteSpeedBasedThrottle.java:
##
@@ -42,6 +42,12 @@
* Otherwise, uses average checkpoint write speed and instant speed of marking
pages as dirty.
*/
public class PagesWriteSpeedBasedThrottle implements PagesWriteThrottlePolicy {
+/** Maximum fraction of dirty pages in a region. */
+public static final double MAX_DIRTY_PAGES = 0.75;
Review Comment:
```suggestion
public static final double DEFAULT_MAX_DIRTY_PAGES = 0.75;
```
##
modules/storage-page-memory/src/main/java/org/apache/ignite/internal/storage/pagememory/PersistentPageMemoryDataRegion.java:
##
@@ -199,52 +202,81 @@ private void initThrottling(PersistentPageMemory
pageMemory) {
}
private ThrottlingType getThrottlingType() {
-SystemPropertyView throttlingTypeCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_TYPE_SYSTEM_PROPERTY);
-
-ThrottlingType throttlingType = ThrottlingType.SPEED_BASED;
-
-if (throttlingTypeCfg != null) {
-try {
-throttlingType =
ThrottlingType.valueOf(throttlingTypeCfg.propertyValue().toUpperCase());
-} catch (IllegalArgumentException e) {
-LOG.warn(
-"Invalid throttling configuration {}={}, using default
value {}",
-THROTTLING_TYPE_SYSTEM_PROPERTY,
-throttlingTypeCfg.propertyValue(),
-throttlingType
-);
-}
-}
-return throttlingType;
+return getSystemConfig(THROTTLING_TYPE_SYSTEM_PROPERTY,
+ThrottlingType.SPEED_BASED,
+value -> ThrottlingType.valueOf(value.toUpperCase())
+);
}
private long getLoggingThreshold() {
-SystemPropertyView logThresholdCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY);
+return TimeUnit.MILLISECONDS.toNanos(getSystemConfig(
+THROTTLING_LOG_THRESHOLD_SYSTEM_PROPERTY,
+
TimeUnit.NANOSECONDS.toMillis(PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD),
+value -> {
+long logThresholdMillis = Long.parseLong(value);
-long logThresholdNanos =
PagesWriteThrottlePolicy.DEFAULT_LOGGING_THRESHOLD;
-try {
-if (logThresholdCfg != null) {
-long logThresholdMillis =
Long.parseLong(logThresholdCfg.propertyValue());
+if (logThresholdMillis <= 0) {
+throw new IllegalArgumentException();
+}
-if (logThresholdMillis <= 0) {
-throw new IllegalArgumentException();
+return logThresholdMillis;
}
+));
+}
+
+private double getMinDirtyPages() {
+return getSystemConfig(
+THROTTLING_MIN_DIRTY_PAGES_SYSTEM_PROPERTY,
+PagesWriteSpeedBasedThrottle.MIN_RATIO_NO_THROTTLE,
+value -> {
+double maxDirtyPages = Double.parseDouble(value);
+
+if (maxDirtyPages <= 0.01 || maxDirtyPages > 0.75) {
+throw new IllegalArgumentException();
+}
-logThresholdNanos =
TimeUnit.MILLISECONDS.toNanos(logThresholdMillis);
+return maxDirtyPages;
+}
+);
+}
+
+private double getMaxDirtyPages() {
+return getSystemConfig(
+THROTTLING_MAX_DIRTY_PAGES_SYSTEM_PROPERTY,
+PagesWriteSpeedBasedThrottle.MAX_DIRTY_PAGES,
+value -> {
+double maxDirtyPages = Double.parseDouble(value);
+
+if (maxDirtyPages <= 0.5 || maxDirtyPages > 0.9) {
+throw new IllegalArgumentException();
Review Comment:
Same thing about validation message. Also, I think there is a machinery that
allows to add validators on the config side. It would greatly improve the UX if
we had such validators for the properties you are adding.
##
modules/storage-page-memory/src/main/java/org/apache/ignite/internal/storage/pagememory/PersistentPageMemoryDataRegion.java:
##
@@ -199,52 +202,81 @@ private void initThrottling(PersistentPageMemory
pageMemory) {
}
private ThrottlingType getThrottlingType() {
-SystemPropertyView throttlingTypeCfg = systemLocalConfig == null
-? null
-:
systemLocalConfig.value().properties().get(THROTTLING_TYPE_SYSTEM_PROPERTY);
-
-ThrottlingType
