There is no property called hadoop.metrics.log.file. Where do you see it? And there is no property that would show up in the command-line of every process. namenode.metrics.logger would show up in the NN command-line and even that is off by default.
FileSink looks limited for practical use. - No support for rolling output files. - The format is hard-coded and not practical to grep. Changing it is incompatible. - Won't write to new output files unless I create the files first with 'touch'. - Not all NN metrics are logged. These last two are likely bugs. On 10/6/15, 11:46 AM, "Allen Wittenauer" <a...@altiscale.com> wrote: >Current state is the custom stuff in HDFS-8880 is mostly undocumented except >for some tuning bits in hdfs-default.xml vs. the javadocs for metrics2. >Neither of which is ideal. > >There’s no doubt the documentation for all of the metrics2 sinks are… sparse. >Which is likely what led to the duplicate functionality. :( > >On Oct 6, 2015, at 11:29 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: > >> If it's duplicate we should probably back it out, but taking a step back, >> is the issue that there isn't good documentation about configuring Metrics2 >> / FileSync? I see the API docs, but a user-focused guide on how to >> configure Metrics2 would probably be a welcome addition. >> >> HBase has a blog at https://blogs.apache.org/hbase/ this could also be good >> content for a blog post. >> >> Best, >> Andrew >> >> On Tue, Oct 6, 2015 at 11:12 AM, Allen Wittenauer <a...@altiscale.com> wrote: >> >>> >>> Folks, >>> >>> I’ve been looking over HDFS-8880 and it’s various follow-on >>> JIRAs. The intentions are good, but the implementation is >>> mostly/effectively a duplicate of the FileSink that’s already part of the >>> Hadoop metrics subsystem. (which therefore means it works with all daemons, >>> out of the box already). Reading through HDFS-9114, it’s pretty obvious >>> now that users are going to get *very* confused as just what happens when >>> they set the “hadoop.metrics.log.file” property. It’s opening a pandora’s >>> box of work, since that property only partially works with one sub-project, >>> will show up on the command line of every daemon, and isn’t documented... >>> >>> I’d like to see this series of patches reverted (they haven’t >>> shipped yet, so now is the time!) and effort placed into updating the >>> metrics2 FileSink to have whatever functionality is missing. >>> >>> Thoughts? > >
