I know it's not directly related, but any general purpose algorithm that
captures to memory buffers needs an overflow to disk mechanism - every time
a ByteArrayOutputStream is used for this, some guy with a *huge* output
from his build gets an OOM.
It's nice you're looking into this issue. As for the actual question of
logger implementations, I prefer trivial problems like trying to determine
that P=NP.
Kristian
2015-05-28 8:39 GMT+02:00 Anders Hammar <and...@hammar.net>:
> I agree with Jason, it would be better to keep this outside of core (the
> core distro).
>
> /Anders
>
> On Thu, May 28, 2015 at 5:21 AM, Jason van Zyl <ja...@takari.io> wrote:
>
> > I honestly don't think an optional feature relying on an optional
> > dependency belongs in the core itself.
> >
> > On May 27, 2015, at 10:34 PM, Igor Fedorenko <i...@ifedorenko.com>
> wrote:
> >
> > > There are three semi-related parts to my implementation
> > >
> > > 1. SLF4J MDC management, basically setting and removing project
> > > attributes in a thread-local map. Truly reliable implementation will
> > > need to be coded in all Builders. Alternatively, it should be possible
> > > to use existing lifecycle callbacks to implement mostly reliable
> lib/ext
> > > extension.
> > >
> > > 2. Install and configure System out/err and java.util.logging bridges.
> > > This can be implemented as lib/ext extension using existing callbacks
> > >
> > > 3. Custom logback appenders and configuration that provide
> "interesting"
> > > behaviour using SLF4J MDC. These too can be implemented of the core.
> > >
> > > Keeping all this in the core will allow marginally more reliable SLF4J
> > > MDC management implementation and I also think will make it more likely
> > > for other developers to help with the implementation. It will also
> serve
> > > as a working example for adding support for other logging framework.
> But
> > > I agree, this does not have to be implemented in the core so I'll try
> to
> > > rework my implementation as lib/ext extension.
> > >
> > > --
> > > Regards,
> > > Igor
> > >
> > > On Wed, May 27, 2015, at 06:40 PM, Jason van Zyl wrote:
> > >>
> > >> On May 27, 2015, at 3:55 PM, Igor Fedorenko <i...@ifedorenko.com>
> > wrote:
> > >>
> > >>> So I went ahead and implemented these changes, including working (but
> > >>> not terribly well tested) logback appenders to buffer-and-group
> project
> > >>> console log messages and create per-project build.log files.
> > >>>
> > >>
> > >> What changes were required in the core?
> > >>
> > >>> Does anyone see a problem if I check in these appenders in maven core
> > >>> source tree or you prefer me to keep them elsewhere?
> > >>>
> > >>
> > >> If it's required to alter the distribution to install logback in order
> > to
> > >> use the appenders then I think elsewhere is more appropriate. As they
> > >> won't be used by the standard version of Maven so I don't think it
> > >> belongs in core along with the other modules. Maven shared is probably
> > >> fine.
> > >>
> > >>> Just to be clear, I do not propose to "hardware" maven to logback
> and I
> > >>> do not propose to include logback support in maven distribution. I
> want
> > >>> to introduce new maven-ext-logback module, which users will be able
> to
> > >>> use to customize their maven distributions very much the same way
> they
> > >>> need to do it now to use any of the advanced logging frameworks.
> > >>>
> > >>> --
> > >>> Regards,
> > >>> Igor
> > >>>
> > >>> On Tue, May 26, 2015, at 03:38 PM, Igor Fedorenko wrote:
> > >>>> I spent some time looking into this, and I think project-level
> logging
> > >>>> will require several semi-related changes.
> > >>>>
> > >>>> * As Ralph pointed out, Maven needs to use SLF4J MDC to associate
> log
> > >>>> messages with individual projects being built. This is required to
> > >>>> enable any project-related logging approach and I plan to submit
> this
> > >>>> change in next couple of days.
> > >>>>
> > >>>> * Another problem is use of non-slf4j logging techniques by Maven
> > >>>> plugins. Direct use of System out/err and java.util.logging are two
> > >>>> obvious problems and I plan to change maven to "bridge" these to
> > slf4j.
> > >>>>
> > >>>> * The old log4j 1.x logging and commons-logging need to be bridged
> to
> > >>>> slf4j too. I can make this change in maven, but this is not strictly
> > >>>> necessary because it can be done from lib/ext extension too.
> > >>>>
> > >>>> The rest really depends on whether we can agree on single "advanced"
> > >>>> logging backend and if we need to support several logging
> > >>>> configurations. I think we've found three viable project-level
> logging
> > >>>> approaches: buffered console output, better logging pattern and my
> > >>>> original target/build.log idea. Unless somebody really wants to
> > restart
> > >>>> logback-vs-log4j discussion, I suggest we postpone this decision and
> > >>>> instead implement project-level logging as out-of-core extensions.
> > >>>>
> > >>>> Does this make sense?
> > >>>>
> > >>>> --
> > >>>> Regards,
> > >>>> Igor
> > >>>>
> > >>>> On Tue, May 26, 2015, at 01:13 PM, Ralph Goers wrote:
> > >>>>> If you use the SLF4J MDC - which is supported by Logback, Log4j 1.x
> > and
> > >>>>> 2.x - you can include anything stored in the MDC on every line of
> log
> > >>>>> output. Just use %X to include all MDC items or %MDC{key} to
> > include the
> > >>>>> specific key. This would require storing the value(s) at the
> > beginning
> > >>>>> of every thread.
> > >>>>>
> > >>>>> If you include %t in the pattern than every log line should include
> > the
> > >>>>> threadId.
> > >>>>>
> > >>>>> If you are saying that the log lines include newlines in them that
> > should
> > >>>>> still be OK. The only way the lines should be mangled is if each
> > thread
> > >>>>> somehow has its own instance of the logging framework and they are
> > all
> > >>>>> configured to write to the same file.
> > >>>>>
> > >>>>> Ralph
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>> On May 25, 2015, at 7:28 AM, Igor Fedorenko <i...@ifedorenko.com>
> > wrote:
> > >>>>>>
> > >>>>>> Yes, thread-id will help to some degree, but maven uses multiline
> > log
> > >>>>>> messages quite often and these will still be mangled and
> unreadable.
> > >>>>>> Per-project build log files is the only solution I found to
> preserve
> > >>>>>> readable logs. Also, each project build is mostly independent from
> > the
> > >>>>>> rest and I find reading self-contain per-project log files much
> > easier.
> > >>>>>>
> > >>>>>> --
> > >>>>>> Regards,
> > >>>>>> Igor
> > >>>>>>
> > >>>>>> On Mon, May 25, 2015, at 10:21 AM, Sean Busbey wrote:
> > >>>>>>> In multithreaded builds we could add a thread ID to each output
> > line.
> > >>>>>>> That
> > >>>>>>> would make it easier to read and filter in different files in
> post
> > >>>>>>> processing.
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Sean
> > >>>>>>> On May 25, 2015 6:30 AM, "Igor Fedorenko" <i...@ifedorenko.com>
> > wrote:
> > >>>>>>>
> > >>>>>>>> I had to troubleshoot a large multithreaded build last week and
> > that
> > >>>>>>>> proved to be rather difficult mostly because build log was a
> > jumble of
> > >>>>>>>> messages produced by concurrently running threads. It was not
> > possible
> > >>>>>>>> to tell which message came from which thread, which made the
> > build log
> > >>>>>>>> more or less useless.
> > >>>>>>>>
> > >>>>>>>> What I ended up doing was to write per-module log message to
> > individual
> > >>>>>>>> ${project.build.directory}/build.log log files.
> > >>>>>>>>
> > >>>>>>>> That was kinda tricky to implement because log files were opened
> > very
> > >>>>>>>> early during module build and were subsequently deleted by
> > >>>>>>>> maven-clean-plugin (I tried on Linux and OSX, and I assume the
> > build
> > >>>>>>>> will fail on Windows). I had to modify maven-clean-plugin
> > configuration
> > >>>>>>>> in the project pom.xml to retain
> > ${project.build.directory}/build.log
> > >>>>>>>> files.
> > >>>>>>>>
> > >>>>>>>> I felt my solution required too my effort and I wonder what
> > others do to
> > >>>>>>>> capture logs during multithreaded builds. Can we come up with a
> > >>>>>>>> "recommended" way of doing this?
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Regards,
> > >>>>>>>> Igor
> > >>>>>>>>
> > >>>>>>>>
> > ---------------------------------------------------------------------
> > >>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > >>>>>>>> For additional commands, e-mail: dev-h...@maven.apache.org
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > ---------------------------------------------------------------------
> > >>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > >>>>>> For additional commands, e-mail: dev-h...@maven.apache.org
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> ---------------------------------------------------------------------
> > >>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > >>>>> For additional commands, e-mail: dev-h...@maven.apache.org
> > >>>>>
> > >>>>
> > >>>>
> ---------------------------------------------------------------------
> > >>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > >>>> For additional commands, e-mail: dev-h...@maven.apache.org
> > >>>>
> > >>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > >>> For additional commands, e-mail: dev-h...@maven.apache.org
> > >>>
> > >>
> > >> Thanks,
> > >>
> > >> Jason
> > >>
> > >> ----------------------------------------------------------
> > >> Jason van Zyl
> > >> Founder, Takari and Apache Maven
> > >> http://twitter.com/jvanzyl
> > >> http://twitter.com/takari_io
> > >> ---------------------------------------------------------
> > >>
> > >> To do two things at once is to do neither.
> > >>
> > >> -- Publilius Syrus, Roman slave, first century B.C.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > >> For additional commands, e-mail: dev-h...@maven.apache.org
> > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > For additional commands, e-mail: dev-h...@maven.apache.org
> > >
> >
> > Thanks,
> >
> > Jason
> >
> > ----------------------------------------------------------
> > Jason van Zyl
> > Founder, Takari and Apache Maven
> > http://twitter.com/jvanzyl
> > http://twitter.com/takari_io
> > ---------------------------------------------------------
> >
> > The modern conservative is engaged in one of man's oldest exercises in
> > moral philosophy; that is,
> > the search for a superior moral justification for selfishness.
> >
> > -- John Kenneth Galbraith
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > For additional commands, e-mail: dev-h...@maven.apache.org
> >
> >
>
