For clarity, I meant to say "any of the exhaust logging from the
integration testing framework". The logging from the tests are the good
stuff.

:)

On Fri, Oct 28, 2016 at 3:13 PM, David Lyle <[email protected]> wrote:

> I think you noticed the main problem with turning logging off entirely.
>
> I'd be inclined to have two files: one which defaults to INFO and another
> that defaults to ERROR for Travis. We can give a 
> -Dlog4j.configuration=file:log4j.config.set.to.ERROR.only
> for travis, which I think Otto suggested.
>
> That said, we're still going to have to suppress the
>
> [ERROR|org.apache.curator.framework.imps.CuratorFrameworkImpl|CuratorFrameworkImpl]
> Background retry gave up
>
> type errors if they're not actual errors. Really, any of the exhaust
> logging from the integration tests should be captured and discarded.
>
> -D...
>
>
> On Fri, Oct 28, 2016 at 2:41 PM, Michael Miklavcic <
> [email protected]> wrote:
>
>> To Otto and David's points: We could default
>> src/test/resources/log4j.properties to log4j.rootLogger=OFF
>>
>> Or alternatively, we could programmatically disable logging in a @Before
>> method in the tests. e.g.
>>
>> package org.apache.metron;
>>
>> import org.apache.log4j.Level;
>> import org.apache.log4j.Logger;
>> import org.junit.BeforeClass;
>> import org.junit.Test;
>>
>> public class MikeTest {
>>
>>   @BeforeClass
>>   public static void setup() {
>>     Logger.getRootLogger().setLevel(Level.OFF);
>>   }
>>
>>   @Test
>>   public void noMessage() {
>>     new SomeClass().doSomething();
>>   }
>>
>>   private class SomeClass {
>>     private final Logger LOG = Logger.getLogger(SomeClass.class);
>>
>>     public void doSomething() {
>>       LOG.error("don't want");
>>     }
>>   }
>>
>> }
>>
>> But that would disable logging for every test, regardless of intent. And
>> we might miss valuable messages for tests that unexpectedly fail. I guess
>> we could do this on a per-test basis, but that feels like a lot of work.
>> I'm curious what our biggest offender is currently - is it the zookeeper
>> and other services from the integration tests, as Casey pointed out?
>>
>>
>> On Fri, Oct 28, 2016 at 1:46 PM, Otto Fowler <[email protected]>
>> wrote:
>>
>>> What if we had a set of log4j.properties just for travis?  I think we can
>>> script travis and deploy them then.
>>> We could have full logging for local builds, and travis specific high
>>> level
>>> logging at the same time.
>>>
>>>
>>> On October 27, 2016 at 20:52:46, David Lyle ([email protected])
>>> wrote:
>>>
>>> I'd prefer not to see a file output by default, it's non-standard and
>>> people won't know to look there without additional documentation. If we
>>> default to ERROR logging and suppress ERROR output in the case of
>>> "expected" errors, we should be able to keep the test output under 4MB.
>>> Then, if additional logging is desired, one could enable a different log
>>> level and carry on.
>>>
>>> For the out of order components, we can capture the output and suppress
>>> it.
>>>
>>> -D...
>>>
>>>
>>> On Thu, Oct 27, 2016 at 8:33 PM, Casey Stella <[email protected]>
>>> wrote:
>>>
>>> > I can get behind logging to a file, but can we see the file as an
>>> output
>>> of
>>> > Travis? Sometimes the problem is that it is failing in Travis but not
>>> > locally or it is a situation that you want to give guidance on a PR but
>>> > don't want to pull a branch and rerun tests. If we log to a file, I'd
>>> like
>>> > to make sure we can get that file as part of the Travis build.
>>> >
>>> > On Fri, Oct 28, 2016 at 00:42 Otto Fowler <[email protected]>
>>> wrote:
>>> >
>>> > > I think we should have all the tests log to file verbosely and to
>>> console
>>> > > reduced. We can gather verbose logs on fail and have them part of the
>>> > > clean etc. Error messages from integration components that do not
>>> result
>>> > > in test failures will still be a problem otherwise. I do not think
>>> > > personally that the Travis visible logs need to show more than what
>>> > failed.
>>> > >
>>> > > On October 27, 2016 at 17:05:28, Ryan Merriman ([email protected])
>>> > > wrote:
>>> > >
>>> > > > I definitely agree with Otto that we need to make our build logs
>>> less
>>> > > > verbose. I think the first step is to get control of the log levels
>>> > > across
>>> > > > various components. Are there specific examples of the types of
>>> > messages
>>> > > > we want to suppress? For instance:
>>> > > >
>>> > > > org.apache.metron.profiler.integration.ProfilerIntegrationTest -
>>> > > zookeeper
>>> > > > and curator log levels are set to INFO
>>> > > > org.apache.metron.enrichment.bolt.ThreatIntelJoinBoltTest -
>>> expected
>>> > > > exceptions are printed out when they should be suppressed (to
>>> Dave's
>>> > > > point)
>>> > > > ...
>>> > > >
>>> > > > Maybe building a list of examples will be easier after we take a
>>> pass
>>> > at
>>> > > > setting all the log levels to ERROR. Ideally we only see the test
>>> > results
>>> > > > and an error if something really did fail. Does that sound like a
>>> > > > reasonable approach?
>>> > > >
>>> > > > Ryan
>>> > > >
>>> > > > On Wed, Oct 19, 2016 at 7:44 AM, David Lyle <[email protected]>
>>> > > wrote:
>>> > > >
>>> > > > Hi Otto,
>>> > > >
>>> > > > Good point. One of the practices I'd like to see is to suppress any
>>> > > > exception/error output in tests when it's expected.
>>> > > >
>>> > > > Thanks!
>>> > > >
>>> > > > -D...
>>> > > >
>>> > > >
>>> > > > On Wed, Oct 19, 2016 at 8:18 AM, Otto Fowler <
>>> [email protected]>
>>>
>>> > > > wrote:
>>> > > >
>>> > > > The Metron build logs contain a great deal of information,
>>> including
>>> a
>>> > > > large number of warnings / exceptions that do not result in test
>>> > failures
>>> > > > but still fill the log. This makes troubleshooting test/build
>>> issues
>>> > > > difficult. For example - the travis ci log for PR #276’s recent
>>> failure
>>> > > > doesn’t even have the final errors because the log is too long.
>>> > > >
>>> > > > What are some things that can be done to make the situation better?
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>

Reply via email to