Twill should not impose a logging framework on the runnables IMO. I
think slf4j is reasonable and since twill is handling the Kafka hookup,
the runnables themselves need know nothing about logback correct? I
think ultimately, how logging is handled should be pluggable. In our
use case, the kafka tie-in is a nice idea, but for most cases will not
fare so well. For instance:
* The log handlers are only relevant in the process which starts the
application. Once that jvm is gone, the kafka queue is never consumed
again. Our yarn clients are typically established for control purposes
and are ephemeral. I realize that the kafka queue could be consumed by
a new process but it doesn't seem like that is part of the API.
* A different strategy may be appropriate for some
applications/runnables. For example I may want to accumulate my logging
in HDFS or just local to the AM .
* Right now our yarn applications dump logs locally and we use custom
endpoints in the applications to pull logs for analysis. Otherwise they
just accumulate and are then subject to the yarn default aggregation policy.
* We have several output log files to segregate different activity. I'd
like to be able to replicate that in the twill environment.
To sum up, I'm not quite sure how to handle logging yet in the twill
environment, but I'm pretty sure we'll need more flexibility and some
type of pluggability.
Cheers,
Martin
On 02/10/2017 02:22 PM, Henry Saputra wrote:
Ah ok, thanks for clarifying your concern, Martin.
Twill currently do need the logback in the runnables running the YARN to be
able to collect the logs and publish them to the embedded Kafka.
So in your case, you want to use slf4j but backed up by log4j binding
instead for client and runnables?
- Henry
On Fri, Feb 10, 2017 at 9:24 AM, Martin Serrano <mar...@attivio.com> wrote:
Terence,
Correct, I don't want to use logback on the client side. slf4j is okay
-- we ship with the log4j binding. In the twill containers, the AM doesn't
matter for me, but the runnables does. So yes, flexibility for the
runnables is important.
Thanks,
Martin
On 02/10/2017 01:56 AM, Terence Yim wrote:
Hi Martin,
If I understand correctly, your intention is to not using slf4j + logback
on the client side? How about the twill containers (both AM and runnables)?
Is it ok to use logback or you want twill to be more flexible about that?
I understand the failure on the AM that you mentioned, I am just
wondering what's your end goal looks like to shape a better solution for
this.
Terence
Sent from my iPhone
On Feb 9, 2017, at 11:33 AM, Martin Serrano <mar...@attivio.com> wrote:
Terence,
I'm familiar with the logback appender and Kafka code. My point is this:
* the AppMaster depends on logback.
* when the YarnTwillPreparer class calls createTwillJar it is creating
the runtime jar for the AppMaster from the current classpath (or more
accurately from the classloader used by the current thread).
* this means the logback jar will not be within the twill jar unless it
is currently on the classpath of the client. The current dependency code
ignores dependent classes which are not found in the classpath while
walking the dependency graph. This is what leads to the class not found
exception when starting the appmaster. This is why I filed TWILL-215.
* having the logback jar in the current classpath turns on logback
within my twill client code since I use slf4j.
Does that make sense?
-Martin
On 02/09/2017 02:19 PM, Terence Yim wrote:
Hi Martin,
Twill has a logback Appender implementation for capturing logs emitted
via
slf4j api from runnable and publish them to the embedded Kafka running
inside the AM process. If you are using log4j as the API for emitting
logs,
what you can do is to use the log4j-over-slf4j bridge to have logs
emitted
via the log4j API get bridged to slf4j.
I suspect why you are seeing the class missing error is most likely
because
you have the slf4j to log4j bridge (the reverse of the one I mentioned
above, look for a jar with name containing "slf4j-log4j12" in the client
classpath) that comes earlier in the classpath then the logback jars.
Terence
On Thu, Feb 9, 2017 at 10:47 AM, Martin Serrano <mar...@attivio.com>
wrote:
Henry,
I see this behavior deploying with YARN 2.7.1, HDP 2.3. But I'm not
sure
you understood my issue.
* The logback jar dependency is only picked up if it is on the
classpath
when the bundle is created.
* With logback in my twill client classpath, the appmaster starts fine.
However without logback in my client classpath the appmaster will get a
ClassNotFoundException.
* We use log4j and with logback in my client classpath, it takes over
the
slf4j bindings and I lose control of the client logging.
So my question was about whether this is expected or if there is a
well-known procedure for working around it. It seems there should be
a way
to tell the twill system to where to find the appmaster dependencies
without having them in the classpath of the twill client.
Thanks!
-Martin
On 02/08/2017 08:09 PM, Henry Saputra wrote:
But the logback dependency should be included in the jar packaging
that
YARN client sends for Twill ApplicationMaster.
Are you seeing this behavior in deploying Twill app in latest YARN?
- Henry
On Wed, Feb 8, 2017 at 12:30 PM, Martin Serrano <mar...@attivio.com>
wrote:
Hey Devs,
It seems like the twill project goes through some pain to try to
insulate
itself logging frameworks. I see use of the slf4j API. However, the
appmaster code has a dependency on logback via the
org.apache.twill.internal.logging.Loggings class. The appmaster
will
not start up without this dependency present. With the dependency
code
as
it is now, there is no way to include the logback jar in the
generated
bundle without it being on the current classpath. I've created a
ticket
(TWILL-215) to make a missing dependency trigger an exception at
bundle
generation time rather than appmaster execution time.
When the logback jar is on my classpath, my client code picks up
logback
instead of our current logger (log4j). Is this what is expected? Is
there
any known workaround? It seems like there may be a case for
specifying
dependencies of the appmaster that are located outside of the
current jvm
classpath.
Thanks,
Martin
