kunwp1 commented on PR #4205: URL: https://github.com/apache/texera/pull/4205#issuecomment-3931676037
@Xiao-zhen-Liu Can you do a one more pass of the review? @chenlica and I discussed the design and decided to keep the two control messages independent (one for UI updates, one for persistence) to avoid coupling their intervals together. To address the concern about increased worker query frequency, I added an optimization in `QueryWorkerStatisticsHandler`: when a timer fires, the controller checks whether a full-graph worker query was already completed recently (within `min(status-update-interval, runtime-statistics-persistence-interval)`). If so, it reuses the cached stats from `WorkerExecution` and forwards them to the appropriate sink (UI or persistence) without issuing any worker RPCs. This means the number of `queryStatistics` RPCs sent to workers does not increase compared to before. The faster timer drives all real worker queries and the slower timer always reuses the result. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
