On 02/08/2013, at 9:34 AM, kelemen <attila.keleme...@gmail.com> wrote:
> 2013/8/1 Adam Murdoch [via Gradle] <[hidden email]>
>
> On 30/07/2013, at 6:52 PM, kelemen <[hidden email]> wrote:
>
>> 2013/7/30 Adam Murdoch [via Gradle] <<a
>> href="x-msg://4953/user/SendEmail.jtp?type=node&node=5711554&i=0"
>> target="_top" rel="nofollow" link="external">[hidden email]>
>>
>>
>> On 29/07/2013, at 4:25 PM, Alex Ruiz <[hidden email]> wrote:
>>
>>> Thanks, Adam. I've been looking into (and debugging) how the current API
>>> for retrieving models work, to have a better understanding on how the API
>>> and implementation of this new action would look like.
>>>
>>> Attila: This approach may also benefit your work on Netbeans. Please take a
>>> look.
>>>
>>> This is what I have in mind, based on what I have done this weekend:
>>>
>>> - The action interface can be simple, and so far this is what I think about
>>> its API:
>>>
>>> public interface ConsumerAction<T> extends Serializable {
>>> T execute(Project project);
>>> }
>>>
>>> where Project is a org.gradle.api.Project. My understanding is that by
>>> having a Project we can do everything we need (through
>>> Project#getService()).
>>
>> The action should accept something other than `org.gradle.api.Project`, for
>> a few reasons:
>>
>> 1. `org.gradle.api.Project` is not version-independent. The entry point for
>> the action should be a function of the client's tooling API version, not
>> target Gradle version.
>> 2. Some actions don't need a configured project. For example, an action may
>> just be interested in the project hierarchy or the build environment. Or it
>> may not be interested in the tasks of a project, or the dependencies of a
>> project, or whatever. Instead, we need an interface that implies no
>> configuration is done, except for that which is explicitly requested by the
>> action.
>> 3. It's not going to work well with decoupled project mode (e.g. parallel
>> execution, configure on demand). This will become the default at some point.
>> The plan there is that there will be no way to navigate from one project to
>> another. Instead, we need an interface which provides access to the whole
>> build, but in a way that allows things like parallel configuration and
>> configuration on demand.
>> 4. There's no opportunity to cache models. We'd have to cache the whole
>> project model, which is unlikely to be possible any time soon.
>>
>>
>> It would be very convenient to have access to `org.gradle.api.Project`
>> regardless. Not being version independent is not necessarily a problem,
>> since Gradle has to maintain compatibility due to the build scripts and this
>> compatibility should be enough. Actually, I could provide a lot better
>> experience in NB if this was possible. For example, I could easily display
>> all the source root with proper classpaths including the usual "integTest"
>> sources.
>
> The process of importing Gradle into an IDE has to involve some kind of
> mapping. The Gradle model is much richer that the IDEs and this model has to
> be converted into the simpler model of each IDE. This mapping is lossy and as
> such the mapping needs to be flexible, and the build author and the various
> plugins must be able to have some influence over it. The mapping shouldn't be
> hardcoded in the IDE. This is one of the reasons the tooling API exists.
>
> So, we have:
>
> Gradle model -> transformation -> IDE model
>
> It's a little more detailed than that:
>
> Gradle DSL -> transformation -> canonical model -> transformation -> IDE model
>
> That is, the user configures some stuff through the DSL, this is then
> converted to some internal model that various parts of Gradle use, including
> the IDE mapping. We keep the DSL backwards compatible, but we don't for the
> canonical model. We plan to, but we're not there yet. In other words, the
> stuff you need from the project (i.e. the canonical model) is not backwards
> compatible at all.
>
> So, rather than access `Project` directly, I'd much prefer if you help us
> grow the tooling API models instead. It's easy to add stuff once you know the
> pattern.
>
> These models then form a contract, as they declare what you need from the
> model. We can keep this stable and make it faster over time. They also allow
> us to offer consistent behaviour across IDEs (which is important to us,
> possibly not so for you).
>
>
> There might be some misundestanding between us, so I will try to clarify my
> assumptions and see where it differs from yours:
>
> 1. The proposed `ConsumerAction` runs in the same process where the project
> got evaluated, so I believe that the compatibility of the DSL is sufficient.
> For example: I should be able to iterate over the sourcesets because I can do
> so in the build script.
>
> 2. If projects are evaluated in multiple processes, then I'd prefer to design
> an API which works even in this case but still runs foreign (i.e., foreign to
> Gradle) code in the process where the project got evaluated (in multiple
> processes if necessary).
>
> 3. I assume that DSL compatibility implies that no methods or classes will
> disappear from the http://www.gradle.org/docs/current/javadoc/. I don't
> assume, that I can just implement interfaces (like `Project`).
>
> 4. I'm not sure what you mean by "canonical model". `Project` or something
> else? If it is the `Project`, then I don't understand because the build
> script can access it as well (see 3). If it is something else, then I don't
> need that one.
Except that you do. There are multiple ways I can define, for example, a Java
project. For example, you can use the 'java' plugin, or the 'android' plugin,
or one of the new JVM language plugins or something entirely custom. These are
all different DSLs, but they are all translated to produce a single
DSL-independent model that describes things about how to build the project,
such as where the source is, the language level, the compile dependencies and
so on. This is the canonical model that the IDE should be using as input. It
should not be using the original DSL.
The canonical model has to be further mapped in a lossy way to the simpler view
of the world that the IDEs have. It's not really the IDE's business as to how
this mapping should happen. It's the build's business.
So, we have something like this:
DSL 1 ----------+
|
DSL 2 ----------+--> canonical model --+--> simplified model --> IDE
| |
my custom DSL --+ +--> other plugins
It's the model that pops out the righthand side that the IDEs should be using,
not the models that go in at the lefthand side.
> I just need the `Project` instance right after the buildscript set it up. Of
> course, this limits the IDE integration (those using `Project`) in some
> marginal use-cases. But there are already limitations, I don't think an IDE
> integration should consider all extreme use cases. Build masters should be
> sane enough to write build scripts so that the IDE understands it. By the
> way, I already pass init scripts (optionally, opt-out) when running tasks
> which means that I already access `Project`.
>
> 5. I don't understand what you mean on consistent behaviour.
The consistency is to do with how the mappings at each of the above stages is
done. We don't want Netbeans, Eclipse, IDEA and my-custom-tool all deciding to
use a different way of mapping to the simplified view.
> There are still models like `EclipseProject` and `IdeaProject`. How does this
> help consistency? If you mean deprecating these models and adding common
> models like `JavaProjectModel`, `GroovyProjectModel` and whatever else, then
> this could mean an awful lot of models. Now, I may consider adding models to
> Gradle myself (although I already spend lots of my free time on coding other
> projects).
That would be great. Please give it a try. We'd very much welcome this.
> However, assume that I go mad adding lots models, which you find quality
> enough to accept: Do you plan to support these models "forever" considering
> backward (or worse, forward) compatibility without me?
Absolutely. That's the whole point. It's a collaboration. The models describe
what Netbeans needs to know about the build. We work together to define this.
And then, given this description, we can find the best ways to provide that
information, and that is free to change over time.
This is only really a problem from our point of view if there's lots of stuff
that Netbeans needs to know that doesn't make sense for other IDEs to know, but
I don't think that's the case.
To me, it's about putting the knowledge in a certain place based on where it
belongs, not based on who does the work. The mappings belong in the build, but
that doesn't mean that a Gradle core developer needs to implement it.
> You see, even though it could be my true intent to support them for you, I
> might die tomorrow. This is something you must consider. That is, Gradleware
> cannot truly rely on external contributons for these models.
>
> 6. Let's be rational: I mean no offense but considering the past, you spend a
> small portion of your resources for making models better. I believe that it
> is not rational to believe that the percentage of resources you can and will
> spend on adding new features to the model will be increased in the future. To
> be honest, I think you shouldn't change your current priorities too much
> because other things are very important as well.
It's a good point. We hope to do better at this, and reaching out to the
community is part of this. There are some improvements ready to be included in
the 1.8 release, and we want this to continue. Let's see how we go.
>
> So, given these, I wish to avoid Gradleware spending more resources on
> maintaining compatibility for the new models. I want you to push these
> maintaining work to IDE integration developers (like me).
Right, that's exactly the idea. But that's not not an argument for accessing
Project and each tool integrator re-implementing the same stuff in different
ways. To me, it's an argument for sharing work via the models.
> But you cannot do so by asking me to add new models (see 5) myself. If you
> want me to create models without pushing these models to the Gradle repo,
> then I don't see what is the gain. I still need to build that model somehow
> from `Project`.
>
> What is the gain from accessing the `Project` through the Tooling API (as
> proposed by Alex and me)? In this case, I'm relieved from the burden of
> maintaining a backward (or forward) compatible model. Even if I screw up in a
> version, I can simply fix this in a patch and release a new version of the NB
> plugin independently of Gradle. Speaking of which, independent release from
> Gradle allows me to consider users needs and react to these needs on my own
> schedule. Also, different IDEs might have considerable different view of the
> world and I'm also pretty sure that users of different IDEs have different
> expectations on how things should be done (there is a reason for why they
> choose one IDE over another).
>
> Saying all these, I'd prefer if developers of Idea and Eclipse integration
> would voice their opinion. Also, I believe you should wait for their opinion.
> That is, IDE integration developers are the main users of this API (at least,
> I believe this is where these new changes matter most).
>
>
>>
>> Caching can be done external to Gradle. Actually, I already cache models.
>> Also, there is no way to know in general if reevaluation of a project is
>> necessary, so I believe that Gradle cannot do considerably better than an
>> external process.
>
> There's an awful lot to track when checking for changes in the model. Here
> are some examples:
>
> - Changes in the build script and in one of the ancestor's build scripts.
> - Changes in the settings scripts.
> - Changes in the init scripts passed as command-line args, or in
> ~/.gradle/init.gradle or ~/.gradle/init.d
> - Changes in any local or remote scripts applied by any of the above or any
> init scripts bundled in the distributions.
> - Changes in gradle.properties and ~/.gradle/gradle.properties.
> - Changes to the Gradle version in gradle-wrapper.properties.
> - Changes in the source or dependencies of buildSrc
> - Changes in the classpath used at configuration time, eg local jars.
> - Changes to dependencies in the remote repositories.
> - Changes in the configuration files or remote resources that my custom
> plugin uses to configure the build.
>
> You get the idea. It think good change detection has to be a collaboration
> between Gradle and the IDE.
>
>
>
> Knowing on what the build script depends is impossible. It can be a content
> of a directory or other things (an absurd example would be System.nanoTime %
> 2). Although it might be reasonable to expect the user not to depend on
> anything he or she whishes. However, I have never seen serious need for this
> automatic change detection. People seem to be content with manual reloading.
> Also, it might be even harmful to reload a model just because a build script
> or something else changes because the background work (what Gradle does)
> might interfere with the user (reading from the harddrive might slow down
> things) and actually, users don't really need the new model for every small
> changes. Like adding a println to a file in buildSrc is really not something
> to worry about.
>
> Although in the past I thought similar to you, I now believe that this kind
> of caching is not something the users need. Caching which benefits the user
> is something different and it is important that the user may refresh at will
> (ignoring caching). I still need to implement persistent cache but I believe
> that this is not something you should spend effort on. At least not without
> explicitly polling the users need. But even if it turns that this is
> something important, allowing access to `Project` shouldn't cause a problem.
> Only that you won't cache `Project` instances, so anyone relying it won't
> automatically benefit from your caching. The implementor (and its users) then
> might choose if this is something worth losing the benefits of being able to
> access `Project`.
>
>
>
> View this message in context: Re: Proposal for retrieving multiple types of
> models from a project in a single pass, using the Tooling API
> Sent from the gradle-dev mailing list archive at Nabble.com.
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
