IMO having metrics within each module is probably more maintainable. the common metrics interfaces/base classes can just live in hudi-common for now?
On Tue, Jul 28, 2020 at 9:06 AM Shiyan Xu <[email protected]> wrote: > +1. It would be very helpful to have more internal performance/cost-related > metrics (perhaps optionally enabled). Also it does make sense to move > metrics classes to common, or even to a separate module (if the scope gets > extended a lot further) > > On Tue, Jul 28, 2020 at 8:43 AM [email protected] <[email protected]> > wrote: > > > +1. Would love to see observability metrics exposed for file system RPC > > calls. This would greatly help in figuring out RPC performance and > > bottlenecks across varied file-systems that Hudi supports. > > On Tuesday, July 28, 2020, 08:24:54 AM PDT, Nishith < > > [email protected]> wrote: > > > > +1 > > > > Having the metrics flexibly in common will help in building observability > > in other modules. > > > > Thanks, > > Nishith > > > > > On Jul 28, 2020, at 7:28 AM, Vinoth Chandar <[email protected]> wrote: > > > > > > +1 as well. > > > > > > Given we support many reporters now. Could you please further > > > improve/retain modularity. > > > > > >> On Mon, Jul 27, 2020 at 6:30 PM vino yang <[email protected]> > > wrote: > > >> > > >> Hi Modi, > > >> > > >> +1 for this proposal. > > >> > > >> I agree with your opinion that the metric report should not only > report > > the > > >> client's metrics. > > >> > > >> And we should decouple the implementation of metrics from the client > > module > > >> so that it could be developed independently. > > >> > > >> Best, > > >> Vino > > >> > > >> Abhishek Modi <[email protected]> 于2020年7月28日周二 上午4:17写道： > > >> > > >>> Hi Everyone! > > >>> > > >>> I'm hoping to have a discussion around adding a lightweight metrics > > class > > >>> to Hudi Common. There are parts of Hudi Common that have large > > >> performance > > >>> implications, and I think adding metrics to these parts will help us > > >> track > > >>> Hudi's health in production and help us understand the performance > > >>> implications of changes we make. > > >>> > > >>> I've opened a Jira on this topic - > > >>> https://issues.apache.org/jira/browse/HUDI-1025. This jira > > >>> specifically suggests adding HoodieWrapperFileSystem as this class > has > > >>> performance implications not just for Hudi, but also for the > underlying > > >>> DFS. > > >>> > > >>> Looking forward to everyone's opinions on this :) > > >>> > > >>> Best, > > >>> Modi > > >>> > > >> >
