On 25/11/2010, at 10:40 PM, Gretar Arnason wrote:
> Ok Dierk and Robert, point accepted :)
>
> Maybe I'm too ANT contaminated, plus I'm almost at the end of porting a huge
> system build from ANT to gradle and was hoping to not have to change the
> whole architecture of things. So I'm struggling with these contextual
> requirements, specially with setting up an assembler/installer (a integration
> project - no specific compiling involved), which has many outputs depending
> on context (depending on customers for example)
> I've not had any problems setting up dependency hierarchies when building
> modules of the system but the picture is totally different when I have this
> integration project (which has a lot of 'macros'). Maybe I will be allotted
> some time at my company to re-make the integration project using gradle
> proper (one can only hope).
>
> Adam,
> to give you a little glimpse of what I was trying to achieve is to basically
> collapse into one task the calling of many tasks (in order specific way) - to
> simplify the usage for developers. So instead of using from command line
> something like (has even been shortened)
> >gradle assembleStuff uploadSomeStuff runSomeExternalStuff
> I would like to be able to do
> >gradle doTheWholeStuff
> which would do the same as the line above and it should run it in the same
> order as is in the command line.
> However I would still want to retain being able to do
> >gradle uploadSomeStuff (here I don't want the assembleStuff)
> I'm still not sure I'd want to connect the tasks explicitly together since
> it depends on the context whether assembleStuff is something I want to
> happen. Plus I might want to reuse assembleStuff in some other context.
From their names, it sounds like those tasks do actually depend on each other.
You can't really upload some stuff until you've assembled some stuff. And you
can't really run some external stuff until you've uploaded some stuff.
If I understand you correctly, the difference between a dependsOn relationship
and what you want to achieve is that you want to be able to reuse something
from an earlier task without re-executing that task. For example, if you've
already assembled some stuff, then just upload some stuff without attempting to
assemble it again. Or, if you've already uploaded it, just run some external
stuff without attempting to assemble or upload it again.
This is exactly what Gradle's incremental build feature does: if something has
already been done, and nothing has changed since then, skip doing it again.
What is missing from incremental build at the moment is that Gradle only
understands about local files when it does its up-to-date checking. We want to
add additional types of resources. For example, remote files are one type of
resources we want to support.
Once Gradle understands about remote files, in your example above, you might
simply do:
task assembleStuff
task uploadSomeStuff(dependsOn: assembleStuff)
task runSomeExternalStuff(dependsOn: uploadSomeStuff)
and Gradle will figure out which of these tasks can be skipped because they
have already been executed. So, if I do:
gradle uploadSomeStuff
gradle runSomeExternalStuff
then the second invocation of gradle will skip assembling and uploading.
>
> If starting from scratch I would probably do this a lot differently but this
> is the thing I have to work with now.
> I'm just having difficulties seeing why ordering the tasks by the order
> received is any worse that ordering alphabetically. I do concur that relying
> on the dependsOn order is prone for errors and maybe it is good in this case
> to just not give the developer the option to screw up. It still nags be that
> we have many examples of technologies trying to restrict developers (to do
> the right thing) just resulting in developers jumping to some other
> technology that answers their demands.
Please don't think we've dismissed your problem. The discussion so far has been
about your proposed solution to the problem, ie dependsOn ordering. I'd much
rather we started with the problem (like your example above), and see what
solution we end up with (maybe we make incremental build better, rather than
use dependsOn ordering).
So, I don't think the question is alphabetical ordering vs. dependsOn ordering.
To me, the question is really about what is missing from the dependency model
which prevents us from explicitly stating what the actual dependency
relationships between tasks are. We've discussed this a few times on the list
already.
There are a few things we plan to do.
The main goal is to move away from declaring dependencies directly on tasks,
and instead mainly declare dependencies on things - files, publications,
deployed services, and so on. Gradle can automatically figure out the task
dependencies from this. There are a few reasons why this good, but the main
ones are 1) to reduce the amount of stuff you need to specify in the build
script, which means a simpler and more maintainable build, and 2) give Gradle
some flexibility in how it builds the artifacts, which means better options for
concurrent and for incremental executions.
However, there will always be situations where you need to wire tasks together
explicitly. The idea is to provide a few low level task graph operations and
some high level constructs which sit on top of the low level stuff.
The low level constructs will be something like:
* Some way to change the dependencies of a task when a given task is scheduled
to be executed. For example, if the 'ciBuild' task is going to be executed in
this build, then the 'check' task dependsOn the 'coverageReport' task. This
might look something like:
ciBuild {
whenScheduled { check.dependsOn coverageReport }
}
* Some way to explicitly schedule a task to be executed when a given task is to
be executed. For example, if the 'test' task is going to be executed in this
build, then also schedule the 'testReport' task to be executed. This might look
like this:
test {
whenScheduled { gradle.taskGraph.add(testReport); testReport.dependsOn test
}
}
* Some way to order task execution without adding dependencies. For example, we
want to be able to run tests as early as possible, and upload tasks as late as
possible. This might look like:
test {
executesBefore tasks.all
}
upload {
executesBefore tasks.none
}
On top of these, we can build a bunch of constructs. For example, it would be
quite easy to write a custom Task implementation whose dependencies are
executed in the order they are added.
> Don't get me wrong, I love gradle, I wouldn't be investing my time here if I
> didn't. It's just that I am leaning toward the mentality that its ok to let
> the developers learn from their mistakes (but of course some people never
> learn - hopefully not me though :) ).
>
> Thanks for a good discussion, I appreciate the time you put into responding.
> regards
> gretar
>
>
>
>
>
> On Wed, Nov 24, 2010 at 10:31 PM, Robert Fischer
> <[email protected]> wrote:
> "dependency-order-by-declaration-sequence is in my eyes one of the features
> that makes so many ANT builds difficult to maintain. You need to know the
> whole
> build to safely make a tiny change in the dependency chain and you
> have to make those
> changes more often. You even have to know the whole build to _use_ it
> correctly!"
>
> This is what I was getting at in saying that dependency ordering
> becomes a mess of fragile bookkeeping. Conflicting dependency
> resolutions orders—argh!
>
> ~~ Robert.
>
> On 24 November 2010 17:20, Dierk König <[email protected]> wrote:
> >> The execution order is defined when the output-input characteristic
> >> requires it.
> >
> >
> > I'm probably not familiar enough with gradle to know exactly what you mean
> > by 'output-input characteristic'
> > could you elaborate further
> >
> > Gradle tasks can define what they take as input and what they produce as
> > output.
> > And many tasks do.
> > If task x produces an output artifact that is consumed as input by task y
> > then a
> > dependency on both x and y is guaranteed to be in executed sequence: first
> > x, second y
> > (unless the output is up to date and x can be skipped).
> > x -> artifact -> y
> > a.dependsOn = [y, x] // declare the 'what', not the 'how'
> > This behavior is one of the distinctively mindful design choices of Gradle.
> > Imagine the sequence of dependsOn declarations would define the order.
> > You can get a conflict with the output-input chain sequence. Now - who wins?
> > That in my opinion is the best example where imposing an
> > order-by-declaration-sequence
> > limits the build system.
> > It is not the main reason, though. The reason - on an architectural level -
> > is that
> > dependencies _have_ no order by nature.
> > Also, with an imposed order, you would care too much about the "how" where
> > you should
> > rather care about than the "what".
> > cheers
> > Dierk
> > P.S. dependency-order-by-declaration-sequence is in my eyes one of the
> > features
> > that makes so many ANT builds difficult to maintain. You need to know the
> > whole
> > build to safely make a tiny change in the dependency chain and you have to
> > make those
> > changes more often. You even have to know the whole build to _use_ it
> > correctly!
> > P.P.S. In case you haven't guessed, yet: I love Gradle!
> > Kudos to Hans and Adam!
> >
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
>
--
Adam Murdoch
Gradle Developer
http://www.gradle.org
CTO, Gradle Inc. - Gradle Training, Support, Consulting
http://www.gradle.biz
