yea make sense to keep module-specific metrics classes, like deltastreamer metrics should just reside in hudi-utilities.
On Tue, Jul 28, 2020 at 9:52 AM Vinoth Chandar <[email protected]> wrote: > IMO having metrics within each module is probably more maintainable. > the common metrics interfaces/base classes can just live in hudi-common for > now? > > On Tue, Jul 28, 2020 at 9:06 AM Shiyan Xu <[email protected]> > wrote: > > > +1. It would be very helpful to have more internal > performance/cost-related > > metrics (perhaps optionally enabled). Also it does make sense to move > > metrics classes to common, or even to a separate module (if the scope > gets > > extended a lot further) > > > > On Tue, Jul 28, 2020 at 8:43 AM [email protected] <[email protected]> > > wrote: > > > > > +1. Would love to see observability metrics exposed for file system > RPC > > > calls. This would greatly help in figuring out RPC performance and > > > bottlenecks across varied file-systems that Hudi supports. > > > On Tuesday, July 28, 2020, 08:24:54 AM PDT, Nishith < > > > [email protected]> wrote: > > > > > > +1 > > > > > > Having the metrics flexibly in common will help in building > observability > > > in other modules. > > > > > > Thanks, > > > Nishith > > > > > > > On Jul 28, 2020, at 7:28 AM, Vinoth Chandar <[email protected]> > wrote: > > > > > > > > +1 as well. > > > > > > > > Given we support many reporters now. Could you please further > > > > improve/retain modularity. > > > > > > > >> On Mon, Jul 27, 2020 at 6:30 PM vino yang <[email protected]> > > > wrote: > > > >> > > > >> Hi Modi, > > > >> > > > >> +1 for this proposal. > > > >> > > > >> I agree with your opinion that the metric report should not only > > report > > > the > > > >> client's metrics. > > > >> > > > >> And we should decouple the implementation of metrics from the client > > > module > > > >> so that it could be developed independently. > > > >> > > > >> Best, > > > >> Vino > > > >> > > > >> Abhishek Modi <[email protected]> 于2020年7月28日周二 上午4:17写道： > > > >> > > > >>> Hi Everyone! > > > >>> > > > >>> I'm hoping to have a discussion around adding a lightweight metrics > > > class > > > >>> to Hudi Common. There are parts of Hudi Common that have large > > > >> performance > > > >>> implications, and I think adding metrics to these parts will help > us > > > >> track > > > >>> Hudi's health in production and help us understand the performance > > > >>> implications of changes we make. > > > >>> > > > >>> I've opened a Jira on this topic - > > > >>> https://issues.apache.org/jira/browse/HUDI-1025. This jira > > > >>> specifically suggests adding HoodieWrapperFileSystem as this class > > has > > > >>> performance implications not just for Hudi, but also for the > > underlying > > > >>> DFS. > > > >>> > > > >>> Looking forward to everyone's opinions on this :) > > > >>> > > > >>> Best, > > > >>> Modi > > > >>> > > > >> > > >
