ORuteMa commented on PR #14324: URL: https://github.com/apache/dolphinscheduler/pull/14324#issuecomment-1646495477
> hi @zhuangchong @ORuteMa 我也遇到了这种情况，但是我觉得这个轮询时间需要根据各自的系统资源来配置，master 会轮询由之分配的所有符合条件的任务，我们可以将这个时间设置为，worker 中的最大启动数量 乘以worker 的数量，再除以master 数量，再乘以每个event 被执行完的时间，再乘以2/3 > > I also encountered this situation, but I think this polling time needs to be configured according to their respective system resources. The master will poll all eligible tasks assigned by it. We can set this time as the maximum number of startups in the worker multiplied by the number of workers, divided by the number of masters, multiplied by the time each event is executed, and multiplied by 2/3 hi @fuchanghai 我觉得策略可能稍显复杂，比如这里的event执行完的时间就不好定义，worker最大数量也是个问题。无论如何，默认5ms是非常不合理的。而且statecheck的事件我看下来只有dependtask和subprocess需要，它们的检查应该不需要并发，所以这里我用一个独立的队列并且保证事件不重复去规避这个问题，这样用户无论怎么调整执行间隔都不会导致事件大量提交的问题。 The strategy may be a bit complicated, for example, the time for the event to finish executing is not well defined here, and the maximum number of workers is also a problem. In any case, the default 5ms is very unreasonable. And statecheck event I see down only `dependtask` and `subprocess` need, their checks should not need concurrent, so here I use a separate queue and ensure that the event does not repeat to avoid this problem, so that the user no matter how to adjust the execution interval will not lead to a large number of events submitted to the problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
