[
https://issues.apache.org/jira/browse/SAMZA-350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Riccomini updated SAMZA-350:
----------------------------------
Attachment: SAMZA-350-0.patch
Attaching a patch. RB located at:
https://reviews.apache.org/r/24216/
# Added a new samza-log4j package.
# Reformatted settings.gradle to make it a little cleaner.
# Fixed a TestJmxReporter to use port variable instead of hard-coding the port
number.
# Wrote JmxAppender, which exposes a single getLevel/setLevel API for Log4J.
# Added a log4j.xml to samza-log4j's test resources directory. This shows how
to use the appender, and also is used in the testing.
# Wrote a test to verify that toggling the level actually affects the log level.
I opted for this implementation instead of the other proposed above because it
seems neither intrusive nor magical. It's true that it still requires a config
change, but it's in log4j.xml not the job's config. I like this style better.
My problem with all zero-config on-by-default options was that it inevitably
involved some hacking, trickery, or the user to know what they're doing. Here,
you just set a JmxAppender in your log4j.xml (vs. the job config in some other
proposals) in order to turn the feature it. It keeps it very isolated to being
log4j specific, which I like.
I'll update hello-samza afterwards to have its log4j.xml files use the appender.
> Allow dynamic log level toggling
> --------------------------------
>
> Key: SAMZA-350
> URL: https://issues.apache.org/jira/browse/SAMZA-350
> Project: Samza
> Issue Type: Bug
> Components: container
> Affects Versions: 0.7.0
> Reporter: Chris Riccomini
> Assignee: Chris Riccomini
> Fix For: 0.8.0
>
> Attachments: SAMZA-350-0.patch
>
>
> Samza uses SLF4J for logging. We do not expose any mechanism to dynamically
> change the log level at runtime. I'm not even sure if SLF4J can do this
> because the log level settings are usually implementation-specific.
> Log4J does allow this via the setLevel API. By default in Log4J 2, JMX is
> already enabled. The hello-samza app shows how to use Log4J 1.2.*, and we use
> Log4J 1.2.* at LinkedIn.
> We should figure out how to implement this. Ideally, we'd do it at the SLF4J
> level so we don't break the log-implementation-independence that we get with
> SLF4J. Unfortunately, it doesn't seem that SLF4J supports this. This leaves
> us with log-implementation-specific support.
> I propose in this ticket that we just write Log4J 1.2.* specific
> implementation that calls:
> {code}
> Logger.getRootLogger().setLevel(...);
> {code}
> The second question is how to implement the RPC. There are many ways. We
> could use the ConfigLog (SAMZA-348), some other control stream, an HTTP
> endpoint in the containers, or JMX.
> My preference is to use JMX for this. It's nice and simple. The ConfigLog
> approach seems appealing, but it would either require containers to restart
> (not ideal if you're trying to debug an issue as it's happening) or would
> require all containers to listen to the ConfigLog topic (seems a big
> commitment if we want to keep containers light-weight).
> The third question is where this code should live. This code should be run in
> all containers and the YARN AM. I am favor having this in its own module
> (samza-log4j) just so we don't have to introduce a Log4J dependency to
> samza-core, which we've thus far been able to avoid. A reasonable location
> for this seems to be in a TaskLifecycleListener, which could setup the MBean
> in the beforeInit method. Unfortunately, the YARN AM does not use
> TaskLifecycleListener. We could write a SamzaAppMasterLifecycleListener
> interface, and implement the Log4J MBean for both.
> There might also be a better place for this to live. An argument could be
> made that this should be part of Samza-proper, and that we should just switch
> to Log4J. Another argument could be made that we need a better lifecycle
> interface, or some generic plugin thingy to let us run this kind of logic.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
