Terence, W/ Fluo we are currently configuring logback to log locally. The reason being that we were unsure about scalability of logs being sent back to a single node, in the case where we run many Fluo workers.
But that is not what prompted this discussion. Fluo depends on Accumulo and Hadoop. Currently Fluo uses maven to build its complete runtime classpath (w/ maven its easy to exclude things like log4j). This is problematic in the case where the user builds Fluo with version X of hadoop and has version Y running on their cluster. I am looking into making the fluo scripts build the runtime classpath using the installed software, with something like the following. FLUO_CLASSPATH=$FLUO_HOME/lib/*:$ACCUMULO_HOME/lib/*:`hadoop classpath` Using this method `hadoop classpath` brings in log4j and slf4j-log4j which makes slf4j unhappy, because twill brings in logback slf4j bindings. Keith On Tue, Mar 3, 2015 at 3:39 PM, Terence Yim <[email protected]> wrote: > Hi Keith, > > From what you've described, is that you want to not having logback as > the slf4j provider? Does that mean you actually want to turn off the > automatically real-time log collection support from Twill? If that's > the case, I think Twill should provide a way to turn that off when > launching the application and do the exclusion of logback jars from > the runtime classpath. > > Terence > > On Tue, Mar 3, 2015 at 11:55 AM, Keith Turner <[email protected]> wrote: > > Terence > > > > I am still looking into this, but I think the following is problematic > > > > * logback-classic-1.0.13.jar contains the slf4j bindings > > * twill depends on classes in logback-classic-1.0.13.jar > > > > If logback-classic-1.0.13.jar is on classpath, then it will automatically > > set itself up as a "provider" for slf4j. If logback-classic-1.0.13.jar > is > > not on the classpath, then some twill classes can not load. > > > > If there was a way to have logback classes on the classpath, but not have > > logback slf4j binding on the classpath then all would be good. However > > AFAICT this is not easily accomplished. > > > > There is code at [ServiceMain line 144][1] that tries to conditionally > > initialize logback. But the issues mentioned above may derail the > > intentions behind this code. > > > > [1]: > > > https://github.com/apache/incubator-twill/blob/v0.4.0-incubating/twill-yarn/src/main/java/org/apache/twill/internal/ServiceMain.java#L144 > > > > Keith > > > > On Tue, Mar 3, 2015 at 1:33 PM, Terence Yim <[email protected]> wrote: > > > >> Hi Keith, > >> > >> Twill implements logback log appender so that application logs are > >> collected and published to the Kafka (that runs inside AM), hence the > >> compile time dependency from the Twill core code. For a Twill > >> application, however, it doesn't need to have compile time dependency > >> on logback (only runtime) and the application itself can use log4j. If > >> you want logs that logged through log4j api are collected and > >> published to the Kafka as well, I believe you'll need to have the > >> log4j-over-slf4j bridge to do the job. > >> > >> Thanks, > >> Terence > >> > >> On Tue, Mar 3, 2015 at 10:16 AM, Nitin Motgi <[email protected]> wrote: > >> > +1 > >> > > >> > ### > >> > Random auto-corrects and typos are my special gift to you. When I > >> forward they are from others. > >> > > >> >> On Mar 3, 2015, at 10:03 AM, Henry Saputra <[email protected]> > >> wrote: > >> >> > >> >> Sounds like good suggestion to me. > >> >> > >> >> > >> >> - Henry > >> >> > >> >>> On Tue, Mar 3, 2015 at 9:55 AM, Keith Turner <[email protected]> > wrote: > >> >>> Why does twill have compile time bindings on logback? Is there a > way > >> to > >> >>> avoid this, can I use twill to launch something that uses log4j? If > >> not, > >> >>> would it make sense to move twill logback code to an optional twill > >> >>> module/jar? > >> >>> > >> >>> The following is taken from http://www.slf4j.org/codes.html > >> >>> > >> >>> Embedded components such as libraries or frameworks should not > >> declare a > >> >>> dependency on any SLF4J binding but only depend on slf4j-api. When > a > >> >>> library > >> >>> declares a compile-time dependency on a SLF4J binding, it imposes > that > >> >>> binding > >> >>> on the end-user, thus negating SLF4J's purpose. When you come > across > >> an > >> >>> embedded component declaring a compile-time dependency on any SLF4J > >> >>> binding, > >> >>> please take the time to contact the authors of said > component/library > >> and > >> >>> kindly ask them to mend their ways. > >> >>> > >> >>> Thanks > >> >>> > >> >>> Keith > >> >
