Ranga Reddy created HUDI-8778:
---------------------------------
Summary: Dynamic configuration updates causes metrics reporting to
stop
Key: HUDI-8778
URL: https://issues.apache.org/jira/browse/HUDI-8778
Project: Apache Hudi
Issue Type: Bug
Components: hudi-utilities
Affects Versions: 0.15.0, 0.14.1
Reporter: Ranga Reddy
Fix For: 1.1.0
*Problem Statement:*
When running the Hudi streamer with the
[https://hudi.apache.org/releases/release-0.14.0#dynamic-configuration-updates]
feature configured, the streamer restarted after a configuration change as
expected; however, metrics reporting stopped.
I believe the sequence of events are as follows:
Hudi calls
[streamSync.close()|https://code.8labs.io/third-party/github.com/apache/hudi/-/blob/release-0.14.1-scwx/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/HoodieStreamer.java?ref_type=heads#L733]
which ends up [shutting down metrics
reporting|https://code.8labs.io/third-party/github.com/apache/hudi/-/blob/release-0.14.1-scwx/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/StreamSync.java?ref_type=heads#L1195].
Since the metrics instances are all [cached in a static
field|https://code.8labs.io/third-party/github.com/apache/hudi/-/blob/release-0.14.1-scwx/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/Metrics.java#L47],
when a new {{StreamSync}} instance is created, it creates a new HoodieMetrics
instance which
[reuses|https://code.8labs.io/third-party/github.com/apache/hudi/-/tree/release-0.14.1-scwx/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/HoodieMetrics.java#L95]
the stopped {{Metrics}} instance.
[#10132|https://github.com/apache/hudi/pull/10132] is a fix for a similar
issue, but not related to the streamer. (That patch is already in hudi 0.14.)
*Reproduce Steps:*
Steps to reproduce the behavior:
* Create an implementation of the ConfigurationHotUpdateStrategy class
* Start the hudi streamer with the {{--config-hot-update-strategy-class}}
option configure with your hot update strategy class
* Verify metrics reporting is working
* Change a configuration property so that your
{{ConfigurationHotUpdateStrategy.updateProperties}} method returns updated
properties
* Wait for the streamer to re-initialize with the updated properties
* Observe that metrics reporting no longer works
*Expected behavior*
After the streamer re-initializes metrics reporting continues to work.
*Github Issue:*
* [https://github.com/apache/hudi/issues/12282]
* [https://github.com/apache/hudi/issues/12282#issuecomment-2536237910]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
