On May 27, 2015, at 3:55 PM, Igor Fedorenko <[email protected]> wrote:
> So I went ahead and implemented these changes, including working (but
> not terribly well tested) logback appenders to buffer-and-group project
> console log messages and create per-project build.log files.
>
What changes were required in the core?
> Does anyone see a problem if I check in these appenders in maven core
> source tree or you prefer me to keep them elsewhere?
>
If it's required to alter the distribution to install logback in order to use
the appenders then I think elsewhere is more appropriate. As they won't be used
by the standard version of Maven so I don't think it belongs in core along with
the other modules. Maven shared is probably fine.
> Just to be clear, I do not propose to "hardware" maven to logback and I
> do not propose to include logback support in maven distribution. I want
> to introduce new maven-ext-logback module, which users will be able to
> use to customize their maven distributions very much the same way they
> need to do it now to use any of the advanced logging frameworks.
>
> --
> Regards,
> Igor
>
> On Tue, May 26, 2015, at 03:38 PM, Igor Fedorenko wrote:
>> I spent some time looking into this, and I think project-level logging
>> will require several semi-related changes.
>>
>> * As Ralph pointed out, Maven needs to use SLF4J MDC to associate log
>> messages with individual projects being built. This is required to
>> enable any project-related logging approach and I plan to submit this
>> change in next couple of days.
>>
>> * Another problem is use of non-slf4j logging techniques by Maven
>> plugins. Direct use of System out/err and java.util.logging are two
>> obvious problems and I plan to change maven to "bridge" these to slf4j.
>>
>> * The old log4j 1.x logging and commons-logging need to be bridged to
>> slf4j too. I can make this change in maven, but this is not strictly
>> necessary because it can be done from lib/ext extension too.
>>
>> The rest really depends on whether we can agree on single "advanced"
>> logging backend and if we need to support several logging
>> configurations. I think we've found three viable project-level logging
>> approaches: buffered console output, better logging pattern and my
>> original target/build.log idea. Unless somebody really wants to restart
>> logback-vs-log4j discussion, I suggest we postpone this decision and
>> instead implement project-level logging as out-of-core extensions.
>>
>> Does this make sense?
>>
>> --
>> Regards,
>> Igor
>>
>> On Tue, May 26, 2015, at 01:13 PM, Ralph Goers wrote:
>>> If you use the SLF4J MDC - which is supported by Logback, Log4j 1.x and
>>> 2.x - you can include anything stored in the MDC on every line of log
>>> output. Just use %X to include all MDC items or %MDC{key} to include the
>>> specific key. This would require storing the value(s) at the beginning
>>> of every thread.
>>>
>>> If you include %t in the pattern than every log line should include the
>>> threadId.
>>>
>>> If you are saying that the log lines include newlines in them that should
>>> still be OK. The only way the lines should be mangled is if each thread
>>> somehow has its own instance of the logging framework and they are all
>>> configured to write to the same file.
>>>
>>> Ralph
>>>
>>>
>>>
>>>> On May 25, 2015, at 7:28 AM, Igor Fedorenko <[email protected]> wrote:
>>>>
>>>> Yes, thread-id will help to some degree, but maven uses multiline log
>>>> messages quite often and these will still be mangled and unreadable.
>>>> Per-project build log files is the only solution I found to preserve
>>>> readable logs. Also, each project build is mostly independent from the
>>>> rest and I find reading self-contain per-project log files much easier.
>>>>
>>>> --
>>>> Regards,
>>>> Igor
>>>>
>>>> On Mon, May 25, 2015, at 10:21 AM, Sean Busbey wrote:
>>>>> In multithreaded builds we could add a thread ID to each output line.
>>>>> That
>>>>> would make it easier to read and filter in different files in post
>>>>> processing.
>>>>>
>>>>> --
>>>>> Sean
>>>>> On May 25, 2015 6:30 AM, "Igor Fedorenko" <[email protected]> wrote:
>>>>>
>>>>>> I had to troubleshoot a large multithreaded build last week and that
>>>>>> proved to be rather difficult mostly because build log was a jumble of
>>>>>> messages produced by concurrently running threads. It was not possible
>>>>>> to tell which message came from which thread, which made the build log
>>>>>> more or less useless.
>>>>>>
>>>>>> What I ended up doing was to write per-module log message to individual
>>>>>> ${project.build.directory}/build.log log files.
>>>>>>
>>>>>> That was kinda tricky to implement because log files were opened very
>>>>>> early during module build and were subsequently deleted by
>>>>>> maven-clean-plugin (I tried on Linux and OSX, and I assume the build
>>>>>> will fail on Windows). I had to modify maven-clean-plugin configuration
>>>>>> in the project pom.xml to retain ${project.build.directory}/build.log
>>>>>> files.
>>>>>>
>>>>>> I felt my solution required too my effort and I wonder what others do to
>>>>>> capture logs during multithreaded builds. Can we come up with a
>>>>>> "recommended" way of doing this?
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Igor
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [email protected]
>>>>>> For additional commands, e-mail: [email protected]
>>>>>>
>>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [email protected]
>>>> For additional commands, e-mail: [email protected]
>>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [email protected]
>>> For additional commands, e-mail: [email protected]
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Takari and Apache Maven
http://twitter.com/jvanzyl
http://twitter.com/takari_io
---------------------------------------------------------
To do two things at once is to do neither.
-- Publilius Syrus, Roman slave, first century B.C.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
