As I mentioned in the original message, take a look at HDFS-9114, which is basically making HDFS-8880 operable.
Generating a sink that is not a sink (which is what HDFS-8880 basically does), isn’t particularly practical. 8880 would be much more appealing if it was a sink. If FileSink isn’t fixable, then let's make a Log4jSink which would basically be FileSink with those issues fixed. If not all NN metrics are getting logged, then that’s probably a bug in *all* of the metrics systems, no? Having them show up only in this one off sounds absolutely terrible. On Oct 6, 2015, at 1:11 PM, Arpit Agarwal <aagar...@hortonworks.com> wrote: > There is no property called hadoop.metrics.log.file. Where do you see it? And > there is no property that would show up in the command-line of every process. > namenode.metrics.logger would show up in the NN command-line and even that is > off by default. > > FileSink looks limited for practical use. > - No support for rolling output files. > - The format is hard-coded and not practical to grep. Changing it is > incompatible. > - Won't write to new output files unless I create the files first with > 'touch'. > > - Not all NN metrics are logged. These last two are likely bugs. > > > > > > > On 10/6/15, 11:46 AM, "Allen Wittenauer" <a...@altiscale.com> wrote: > >> Current state is the custom stuff in HDFS-8880 is mostly undocumented except >> for some tuning bits in hdfs-default.xml vs. the javadocs for metrics2. >> Neither of which is ideal. >> >> There’s no doubt the documentation for all of the metrics2 sinks are… >> sparse. Which is likely what led to the duplicate functionality. :( >> >> On Oct 6, 2015, at 11:29 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: >> >>> If it's duplicate we should probably back it out, but taking a step back, >>> is the issue that there isn't good documentation about configuring Metrics2 >>> / FileSync? I see the API docs, but a user-focused guide on how to >>> configure Metrics2 would probably be a welcome addition. >>> >>> HBase has a blog at https://blogs.apache.org/hbase/ this could also be good >>> content for a blog post. >>> >>> Best, >>> Andrew >>> >>> On Tue, Oct 6, 2015 at 11:12 AM, Allen Wittenauer <a...@altiscale.com> >>> wrote: >>> >>>> >>>> Folks, >>>> >>>> I’ve been looking over HDFS-8880 and it’s various follow-on >>>> JIRAs. The intentions are good, but the implementation is >>>> mostly/effectively a duplicate of the FileSink that’s already part of the >>>> Hadoop metrics subsystem. (which therefore means it works with all daemons, >>>> out of the box already). Reading through HDFS-9114, it’s pretty obvious >>>> now that users are going to get *very* confused as just what happens when >>>> they set the “hadoop.metrics.log.file” property. It’s opening a pandora’s >>>> box of work, since that property only partially works with one sub-project, >>>> will show up on the command line of every daemon, and isn’t documented... >>>> >>>> I’d like to see this series of patches reverted (they haven’t >>>> shipped yet, so now is the time!) and effort placed into updating the >>>> metrics2 FileSink to have whatever functionality is missing. >>>> >>>> Thoughts? >> >>
