On 22 Dec 07, at 1:02 AM 22 Dec 07, Milos Kleint wrote:
i've played with the DefaultLifecycleExecutor and
DefaultPluginManager a bit and tried to separate the preparation
phase from the actual mojo executions. The preparations seem to
attribute to astonishing 90% of the build execution.
See http://picasaweb.google.com/mkleint/Maven/
photo#5146520463951211778
what have I actually done? (patch attached, I hope it gets through)
in the preparation phase
1. iterate all the projects, figure out the mojo bindings.
2. for each binding, load the asociated plugin
3. for each project figure out what the required maximum project
dependency resolution is and resolve it.
in the execution phase actually just run the mojo's execute method.
(ripped the project dependency resolution from there)
The work i've done is probably not generally useful, but it clearly
shows where space for improvements is.
Why is it not generally useful? I think as part of improving the build
plan and exposing what can be configured and change they the user is
good. How the mojo bindings are put together would definitely affect
the subsequent execution. John should take a look at the patch as
well, but I will take a look but. Did you manage to reduce any
complexity or reduce any code or did you just add different processing?
However I'd like to use this in the Netbeans IDE, to "preload"
various actions as the user edits the files, so that the build/run/
debug cycle gets faster (as I'm dependent on running maven goals on
these actions) and when the user actually triggers "Run" only the
execution phase needs to be run.
Yes, definitely useful to IDE integration. Visualization of the build
plan will be one of the most useful features there will be.
Milos
On Dec 19, 2007 10:34 PM, Jason van Zyl <[EMAIL PROTECTED]> wrote:
On 19 Dec 07, at 12:28 PM 19 Dec 07, Milos Kleint wrote:
> here is the actual screenshot.
>
> http://picasaweb.google.com/mkleint/Maven
>
> more inline..
>
> On Dec 19, 2007 9:19 PM, Jason van Zyl <[EMAIL PROTECTED]> wrote:
>
>>
>> On 19 Dec 07, at 11:10 AM 19 Dec 07, Milos Kleint wrote:
>>
>>> Hello,
>>>
>>> i've started building my project with current 2.1-SNAPSHOT trunk
and
>>> I noticed the build is taking more time than before. I did a bit
of
>>> profiling.
>>> I've executed "mvn install" on a set of 24 projects, all build
>>> before, right before the profiling test.
>>>
>>> What struck me is that half of the time seems to be spent in
>>> DefaultPluginManager.resolveTransitiveDependencies(). See attached
>>> picture from netbeans profiler.
>>> It was executed 82 times. I suppose the root of the problem
will be
>>> somewhere much deeper and I guess there's a way to optimize the
>>> number of calls to the method as well.
>>> After a bit of debugging i figured that for a single project this
>>> method is called multiple times during execution. First the
>>> "compile" scope is resolved, later "runtime" or "test" eventually
>>> (depends on actual plugins bound to the project)
>>> Ideally it should be called 24 times max as far as I understand
the
>>> problem. The BuildPlan should know up front what mojos will be
>>> executed and what is the maximum level or dependency resolution
for
>>> the given project.
>>>
>>> Agreed? Is that something that I should pursue further or am I on
>>> the wrong track?
>>>
>>
>> Sure, take a look but in the plugin manager I started ripping out
all
>> the optimizations after ripping out a bunch of other code. But
narrow
>> it down for one build where
>>
>> 1) You don't have anything downloaded i.e. a clean repository, and
>> 2) Where you have all the dependencies downloaded
>>
>> For one pass it shouldn't be resolving more then once for each
>> plugin.
>> I wouldn't try to start caching anything but if you're seeing more
>> then one call per plugin then something needs to be reworked.
>
>
> well, from what I understand I see 1 resolution per plugin per
> project, if
> the plugin requires dependency resolution.
Yes, this is a reactor this is to account for cases where you may have
the same plugin used, but might have dependencies stated itself which
can change things so you need to track this. The non-optimized version
obviously deals with this but you will end up with a case where
different plugin declarations have slightly different dependency
requirements. John tried to account for everything, but we could still
cache most of the information.
>
> my ''optimization'' involves looking up front what is the maximum
> dependency
> resolution scope and resolve it just once per project. In your case
> 1 it
> means more than necessary gets downloaded upfront probably, but
case 2
> should be faster.
So we can sync up here as I'm remaking the project builder and trying
to setup a listener that can watch for things as projects are being
built. One of the things I'm trying to collect are profiles and
extensions so that we don't have to scan the POMs again after they are
built. We could do the same with plugin declarations so that up-front
you would know what you needed. So you could see that for all the
plugin configurations there were no variations, pick off which ones
need resolution, do that and then fire the build.
So if you could determine what information you needed up front I can
factor that in, and in the short term you can take the approach John
has made with things like the extension scanner. But ultimately we
should not need to read any pom.xml file more then once. Whether it
come in the maven-artifact, the project builder or anything else.
>
>
> Milos
>
>
>>
>>
>>> Milos
>>>
>>> PS: If anyone is interested I can send the actual netbeans
profiler
>>> dump files for browsing (via private email).
>>>
>>>
>>>
>>>
---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>> Thanks,
>>
>> Jason
>>
>> ----------------------------------------------------------
>> Jason van Zyl
>> Founder, Apache Maven
>> jason at sonatype dot com
>> ----------------------------------------------------------
>>
>> believe nothing, no matter where you read it,
>> or who has said it,
>> not even if i have said it,
>> unless it agrees with your own reason
>> and your own common sense.
>>
>> -- Buddha
>>
>>
>>
>>
>>
---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>>
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
What matters is not ideas, but the people who have them. Good people
can fix bad ideas, but good ideas can't save bad people.
-- Paul Graham
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<
lifecycleexecutor
.patch
>---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
We know what we are, but know not what we may be.
-- Shakespeare
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
