Hi Bernd, By "metrics report" I mean a way for TwillRunnable to emit metrics (e.g. increment a named counter by n, which basically is Counter) and have way to aggregate across all runnable instances of a given Twill application. Currently there is no such thing in Twill, but I think it is valuable to have it.
Terence On Wed, Jan 22, 2014 at 7:40 AM, Bernd Fondermann <[email protected]> wrote: > On Tue, Jan 21, 2014 at 10:02 PM, Terence Yim <[email protected]> wrote: >> Hi Bernd, >> >> Is the counter concept that you talk about something like metrics >> report? We've thought about it, but not yet implemented. > > I'm talking about the Distributed Counter concept in classic MapReduce > where all MR tasks for the same job can for example count overall bad > records and such. It's like what Curator's Distributed Atomic Long[1] > provides, without the subtract. > > I could not find a reference to "metrics report" in Twill, but I'm > familiar with Hadoop's metrics. > > Bernd > > [1] http://curator.apache.org/curator-recipes/distributed-atomic-long.html > >> Potentially >> it could be done by leveraging the same Kafka instance for logging to >> transport custom metrics from containers as well. I think it's >> something valuable to have in Twill. >> >> Thanks, >> Terence >> >> On Tue, Jan 21, 2014 at 1:23 AM, Bernd Fondermann >> <[email protected]> wrote: >>> Hi, >>> >>> getting more familiar with Twill, questions keep coming up. >>> For example: One of the great things about MapRed is Counters. >>> Is there a similar concept (yet) at Twill? Or do you recommend using >>> Curator? >>> >>> Thanks, >>> >>> Bernd
