[GitHub] [druid] jihoonson commented on a change in pull request #10732: Add a config for monitorScheduler type
jihoonson commented on a change in pull request #10732:
URL: https://github.com/apache/druid/pull/10732#discussion_r555262086
##
File path:
server/src/main/java/org/apache/druid/server/metrics/DruidMonitorSchedulerConfig.java
##
@@ -28,9 +29,17 @@
*/
public class DruidMonitorSchedulerConfig extends MonitorSchedulerConfig
{
+ @JsonProperty
+ private String schedulerClassName =
ClockDriftSafeMonitorScheduler.class.getName();
Review comment:
It was changed in https://github.com/apache/druid/pull/10448, not in
this PR. This PR is just to make it configurable because I'm not sure how
stable it is. As noted in
https://github.com/apache/druid/pull/10448#issuecomment-756367474,
CronScheduler seems to have a not-bad test coverage and worked well in my
testing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] jihoonson commented on a change in pull request #10732: Add a config for monitorScheduler type
jihoonson commented on a change in pull request #10732:
URL: https://github.com/apache/druid/pull/10732#discussion_r555262535
##
File path:
core/src/main/java/org/apache/druid/java/util/metrics/ClockDriftSafeMonitorScheduler.java
##
@@ -0,0 +1,134 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.java.util.metrics;
+
+import io.timeandspace.cronscheduler.CronScheduler;
+import io.timeandspace.cronscheduler.CronTask;
+import org.apache.druid.java.util.common.logger.Logger;
+import org.apache.druid.java.util.emitter.service.ServiceEmitter;
+
+import java.util.List;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicReference;
+
+/**
+ * A {@link MonitorScheduler} implementation based on {@link CronScheduler}.
+ */
+public class ClockDriftSafeMonitorScheduler extends MonitorScheduler
+{
+ private static final Logger LOG = new
Logger(ClockDriftSafeMonitorScheduler.class);
+
+ private final CronScheduler monitorScheduler;
+ private final ExecutorService monitorRunner;
+
+ public ClockDriftSafeMonitorScheduler(
+ MonitorSchedulerConfig config,
+ ServiceEmitter emitter,
+ List monitors,
+ CronScheduler monitorScheduler,
+ ExecutorService monitorRunner
+ )
+ {
+super(config, emitter, monitors);
+this.monitorScheduler = monitorScheduler;
+this.monitorRunner = monitorRunner;
+ }
+
+ @Override
+ void startMonitor(final Monitor monitor)
+ {
+monitor.start();
+long rate = getConfig().getEmitterPeriod().getMillis();
+final AtomicReference> scheduleFutureReference = new
AtomicReference<>();
+Future scheduledFuture = monitorScheduler.scheduleAtFixedRate(
+rate,
+rate,
+TimeUnit.MILLISECONDS,
+new CronTask()
+{
+ private Future scheduleFuture = null;
+ private Future monitorFuture = null;
+
+ @Override
+ public void run(long scheduledRunTimeMillis)
+ {
+waitForScheduleFutureToBeSet();
Review comment:
We can use it, but it seems not matter much to me since this loop is not
supposed to run at all in production.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] jihoonson commented on a change in pull request #10732: Add a config for monitorScheduler type
jihoonson commented on a change in pull request #10732:
URL: https://github.com/apache/druid/pull/10732#discussion_r1
##
File path:
server/src/main/java/org/apache/druid/server/metrics/DruidMonitorSchedulerConfig.java
##
@@ -28,9 +29,17 @@
*/
public class DruidMonitorSchedulerConfig extends MonitorSchedulerConfig
{
+ @JsonProperty
+ private String schedulerClassName =
ClockDriftSafeMonitorScheduler.class.getName();
Review comment:
Yes, I have done some testing before and will do more.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] jihoonson commented on a change in pull request #10732: Add a config for monitorScheduler type
jihoonson commented on a change in pull request #10732:
URL: https://github.com/apache/druid/pull/10732#discussion_r555600385
##
File path:
core/src/main/java/org/apache/druid/java/util/metrics/ClockDriftSafeMonitorScheduler.java
##
@@ -0,0 +1,134 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.java.util.metrics;
+
+import io.timeandspace.cronscheduler.CronScheduler;
+import io.timeandspace.cronscheduler.CronTask;
+import org.apache.druid.java.util.common.logger.Logger;
+import org.apache.druid.java.util.emitter.service.ServiceEmitter;
+
+import java.util.List;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicReference;
+
+/**
+ * A {@link MonitorScheduler} implementation based on {@link CronScheduler}.
+ */
+public class ClockDriftSafeMonitorScheduler extends MonitorScheduler
+{
+ private static final Logger LOG = new
Logger(ClockDriftSafeMonitorScheduler.class);
+
+ private final CronScheduler monitorScheduler;
+ private final ExecutorService monitorRunner;
+
+ public ClockDriftSafeMonitorScheduler(
+ MonitorSchedulerConfig config,
+ ServiceEmitter emitter,
+ List monitors,
+ CronScheduler monitorScheduler,
+ ExecutorService monitorRunner
+ )
+ {
+super(config, emitter, monitors);
+this.monitorScheduler = monitorScheduler;
+this.monitorRunner = monitorRunner;
+ }
+
+ @Override
+ void startMonitor(final Monitor monitor)
+ {
+monitor.start();
+long rate = getConfig().getEmitterPeriod().getMillis();
+final AtomicReference> scheduleFutureReference = new
AtomicReference<>();
+Future scheduledFuture = monitorScheduler.scheduleAtFixedRate(
+rate,
+rate,
+TimeUnit.MILLISECONDS,
+new CronTask()
+{
+ private Future scheduleFuture = null;
Review comment:
Ah, forgot to clean them up before commit. Thanks :+1:
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] jihoonson commented on a change in pull request #10732: Add a config for monitorScheduler type
jihoonson commented on a change in pull request #10732:
URL: https://github.com/apache/druid/pull/10732#discussion_r556211566
##
File path:
server/src/main/java/org/apache/druid/server/metrics/DruidMonitorSchedulerConfig.java
##
@@ -28,9 +29,17 @@
*/
public class DruidMonitorSchedulerConfig extends MonitorSchedulerConfig
{
+ @JsonProperty
+ private String schedulerClassName =
ClockDriftSafeMonitorScheduler.class.getName();
Review comment:
> So, because the potential reward has a small impact, and the potential
risk has a large impact, I think it's best to default to the old scheduler for
another release or so. Just until such time as people have been able to do
long-running tests in production and have found that there are no issues.
This makes sense to me. I think we can do more extensive testing by
ourselves instead of rushing to change the default.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] jihoonson commented on a change in pull request #10732: Add a config for monitorScheduler type
jihoonson commented on a change in pull request #10732:
URL: https://github.com/apache/druid/pull/10732#discussion_r556211781
##
File path:
server/src/main/java/org/apache/druid/server/metrics/DruidMonitorSchedulerConfig.java
##
@@ -28,9 +29,17 @@
*/
public class DruidMonitorSchedulerConfig extends MonitorSchedulerConfig
{
+ @JsonProperty
+ private String schedulerClassName =
ClockDriftSafeMonitorScheduler.class.getName();
Review comment:
Changed the default back to `BasicMonitorScheduler`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
