hudi-bot opened a new issue, #15020: URL: https://github.com/apache/hudi/issues/15020
Timeline server metrics are pushed to local registry but never going to reporters. Exposing these metrics would greatly improve debugging latency around async processes and timeline server syncs. Metrics are already captured in the [Request Handler|https://github.com/apache/hudi/blob/master/hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/RequestHandler.java#L527-L531] ## JIRA info - Link: https://issues.apache.org/jira/browse/HUDI-3409 - Type: Improvement - Epic: https://issues.apache.org/jira/browse/HUDI-3526 - Fix version(s): - 1.1.0 --- ## Comments 17/Feb/22 02:22;xushiyan;[~jmcordova] [~jcordovam] [~vipul94] [~thanhnd] any of you interested in picking this up?;;; --- 24/Feb/22 21:17;shivnarayan;[~xushiyan] : [~Kalluri] has shown interest in picking this up. btw Rajesh: this is what I am thinking. currently metrics are in hudi-client-common. and timeline-server module does not depend on this module for now. we can probably create a new module called hudi-metrics and move all metrics related classes to this one from hudi-client-module. and make both hudi-client-module and timeline-server-module depend on this new one. Raymond: let me know if you have better idea to go about this. ;;; --- 25/Feb/22 00:28;Kalluri;Looks like this is a somewhat related RFC [https://cwiki.apache.org/confluence/display/HUDI/RFC+-+23+%3A+Hudi+Observability+metrics+collection] ;;; --- 25/Feb/22 16:21;Kalluri;[~xushiyan] [~shivnarayan] I am thinking of a different Jira ticket to seperate out metrics first and make that a prereq for this ticket. Thoughts?;;; --- 26/Feb/22 02:05;shivnarayan;yes, sure makes sense. ;;; --- 27/Feb/22 16:03;Kalluri;As part of this story I am planning to seperate `{*}hoodie-metrics-common{*}` module from {*}`hudi-client-common`{*}. I was originally thinking to move the metrics code into `{*}hudi-common{*}` but after looking at the code and some discussions, it sounded like more of core components. How do we feel about renaming *`hudi-common`* to *`hudi-core`* so that we can use *`hudi-common`* to cater to the common things like metrics and such. I am thinking there will be more things like metrics that will need to be grouped and we will have an explosion of modules if we dont have a *`hudi-common`* to house them under. Please let me know what approach we should take here based on your experience with the codebase. ;;; --- 27/Feb/22 16:54;xushiyan;[~Kalluri] renaming a core module like hudi-common causing too much cascading changes and i don't see a major benefit of renaming it to hudi-core. So the ROI is low. I suggest we should focus on the original intent and keep the first step simple: metrics classes separated out and moved to hudi-metrics. [~shivnarayan] WDYT?;;; --- 27/Mar/22 02:02;Kalluri;[~dswift] can you provide little more detail on how we can utilize these metrics, do you have a usecase where your debugging is hampered by the lack of these metrics "Timeline server metrics are pushed to local registry but never going to reporters. Exposing these metrics would greatly improve debugging latency around async processes and timeline server syncs. " I am not challenging the need for these, but just want a little bit context from you on what made you file this.;;; -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
