I can see that you are adamant about not giving access to `Project`. Anyway, I have realized that I already have access to the `Project` instance via init scripts *evil smile*. So, I can inject a custom model builder there.
You say that "We don't want Netbeans, Eclipse, IDEA and my-custom-tool all deciding to use a different way of mapping to the simplified view". However, I have a different internal model representation, so I have to map things anyway. The only question is from what model do I do this conversion: DSL or a model created from the DSL. So, the created model (`IdeaProject`) does not make my life easier the slightest. Quite the contrary, I have to undo a lot of things, so your mapping efforts went in vain. This is because in NetBeans, I need something very close to the DSL without further mapping. For example, `SourceSet.getCompileClasspath()` would be a lot better than to have access to configurations like "compile" (which is unreliable to depend on). Actually, knowing "compile" instead of the actual classpath is a lot of pain to me because I have to do some dependency resolution myself (to make matters worse, I don't even have all the info), something which Gradle can already do itself. All in all, I believe that accessing models instead of the DSL doesn't make anything easier. It certainly is no easier for me. I believe that "re-implementing the same stuff in different ways" assumes that I need to same thing as other IDEs which is not the case. As I see, models only structure the information in a certain way but I'm pretty sure that different IDEs need to structure things in a different way. So why is it assumed, that choosing a particular structure of the information (other than the DSL) will help all the IDEs? That is, if I were to implement a model for "JavaProject", I would implement something benefical to NetBeans because I have to choose a structure and why should I choose something else? Also, I don't agree with your example of Android projects: They are not Java projects and should (and do) have a different model anyway. That is, if a build script does not apply the "java" plugin (implicitly or explicitly) then it is not a Java project, therefore needs a different model. 2013/8/13 Adam Murdoch [via Gradle] < ml-node+s1045684n5711697...@n5.nabble.com> > > On 02/08/2013, at 9:34 AM, kelemen <[hidden > email]<http://user/SendEmail.jtp?type=node&node=5711697&i=0>> > wrote: > > 2013/8/1 Adam Murdoch [via Gradle] <<a > href="x-msg://1801/user/SendEmail.jtp?type=node&node=5711582&i=0" > target="_top" rel="nofollow" link="external">[hidden email]> > >> >> On 30/07/2013, at 6:52 PM, kelemen <[hidden >> email]<http://user/SendEmail.jtp?type=node&node=5711581&i=0>> >> wrote: >> >> 2013/7/30 Adam Murdoch [via Gradle] <<a href="<a >> href="x-msg://4953/user/SendEmail.jtp?type=node&amp;node=5711554&amp;i=0">x-msg://4953/user/SendEmail.jtp?type=node&node=5711554&i=0" >> target="_top" rel="nofollow" link="external">[hidden email]> >> >> >>> On 29/07/2013, at 4:25 PM, Alex Ruiz <[hidden >>> email]<http://user/SendEmail.jtp?type=node&node=5711550&i=0>> >>> wrote: >>> >>> Thanks, Adam. I've been looking into (and debugging) how the current API >>> for retrieving models work, to have a better understanding on how the API >>> and implementation of this new action would look like. >>> >>> Attila: This approach may also benefit your work on Netbeans. Please >>> take a look. >>> >>> This is what I have in mind, based on what I have done this weekend: >>> >>> - The action interface can be simple, and so far this is what I think >>> about its API: >>> >>> public interface ConsumerAction<T> extends Serializable { >>> T execute(Project project); >>> } >>> >>> where Project is a org.gradle.api.Project. My understanding is that >>> by having a Project we can do everything we need (through >>> Project#getService()). >>> >>> >>> The action should accept something other than `org.gradle.api.Project`, >>> for a few reasons: >>> >>> 1. `org.gradle.api.Project` is not version-independent. The entry point >>> for the action should be a function of the client's tooling API version, >>> not target Gradle version. >>> 2. Some actions don't need a configured project. For example, an action >>> may just be interested in the project hierarchy or the build environment. >>> Or it may not be interested in the tasks of a project, or the dependencies >>> of a project, or whatever. Instead, we need an interface that implies no >>> configuration is done, except for that which is explicitly requested by the >>> action. >>> 3. It's not going to work well with decoupled project mode (e.g. >>> parallel execution, configure on demand). This will become the default at >>> some point. The plan there is that there will be no way to navigate from >>> one project to another. Instead, we need an interface which provides access >>> to the whole build, but in a way that allows things like parallel >>> configuration and configuration on demand. >>> 4. There's no opportunity to cache models. We'd have to cache the whole >>> project model, which is unlikely to be possible any time soon. >>> >>> >> It would be very convenient to have access to `org.gradle.api.Project` >> regardless. Not being version independent is not necessarily a problem, >> since Gradle has to maintain compatibility due to the build scripts and >> this compatibility should be enough. Actually, I could provide a lot better >> experience in NB if this was possible. For example, I could easily display >> all the source root with proper classpaths including the usual "integTest" >> sources. >> >> >> The process of importing Gradle into an IDE has to involve some kind of >> mapping. The Gradle model is much richer that the IDEs and this model has >> to be converted into the simpler model of each IDE. This mapping is lossy >> and as such the mapping needs to be flexible, and the build author and the >> various plugins must be able to have some influence over it. The mapping >> shouldn't be hardcoded in the IDE. This is one of the reasons the tooling >> API exists. >> >> So, we have: >> >> Gradle model -> transformation -> IDE model >> >> It's a little more detailed than that: >> >> Gradle DSL -> transformation -> canonical model -> transformation -> IDE >> model >> >> That is, the user configures some stuff through the DSL, this is then >> converted to some internal model that various parts of Gradle use, >> including the IDE mapping. We keep the DSL backwards compatible, but we >> don't for the canonical model. We plan to, but we're not there yet. In >> other words, the stuff you need from the project (i.e. the canonical model) >> is not backwards compatible at all. >> >> So, rather than access `Project` directly, I'd much prefer if you help us >> grow the tooling API models instead. It's easy to add stuff once you know >> the pattern. >> >> These models then form a contract, as they declare what you need from the >> model. We can keep this stable and make it faster over time. They also >> allow us to offer consistent behaviour across IDEs (which is important to >> us, possibly not so for you). >> >> > There might be some misundestanding between us, so I will try to clarify > my assumptions and see where it differs from yours: > > 1. The proposed `ConsumerAction` runs in the same process where the > project got evaluated, so I believe that the compatibility of the DSL is > sufficient. For example: I should be able to iterate over the sourcesets > because I can do so in the build script. > > 2. If projects are evaluated in multiple processes, then I'd prefer to > design an API which works even in this case but still runs foreign (i.e., > foreign to Gradle) code in the process where the project got evaluated (in > multiple processes if necessary). > > 3. I assume that DSL compatibility implies that no methods or classes will > disappear from the http://www.gradle.org/docs/current/javadoc/. I don't > assume, that I can just implement interfaces (like `Project`). > > 4. I'm not sure what you mean by "canonical model". `Project` or something > else? If it is the `Project`, then I don't understand because the build > script can access it as well (see 3). If it is something else, then I don't > need that one. > > > Except that you do. There are multiple ways I can define, for example, a > Java project. For example, you can use the 'java' plugin, or the 'android' > plugin, or one of the new JVM language plugins or something entirely > custom. These are all different DSLs, but they are all translated to > produce a single DSL-independent model that describes things about how to > build the project, such as where the source is, the language level, the > compile dependencies and so on. This is the canonical model that the IDE > should be using as input. It should not be using the original DSL. > > The canonical model has to be further mapped in a lossy way to the simpler > view of the world that the IDEs have. It's not really the IDE's business as > to how this mapping should happen. It's the build's business. > > So, we have something like this: > > DSL 1 ----------+ > | > DSL 2 ----------+--> canonical model --+--> simplified model --> IDE > | | > my custom DSL --+ +--> other plugins > > It's the model that pops out the righthand side that the IDEs should be > using, not the models that go in at the lefthand side. > > I just need the `Project` instance right after the buildscript set it up. > Of course, this limits the IDE integration (those using `Project`) in some > marginal use-cases. But there are already limitations, I don't think an IDE > integration should consider all extreme use cases. Build masters should be > sane enough to write build scripts so that the IDE understands it. By the > way, I already pass init scripts (optionally, opt-out) when running tasks > which means that I already access `Project`. > > 5. I don't understand what you mean on consistent behaviour. > > > The consistency is to do with how the mappings at each of the above stages > is done. We don't want Netbeans, Eclipse, IDEA and my-custom-tool all > deciding to use a different way of mapping to the simplified view. > > > There are still models like `EclipseProject` and `IdeaProject`. How does > this help consistency? If you mean deprecating these models and adding > common models like `JavaProjectModel`, `GroovyProjectModel` and whatever > else, then this could mean an awful lot of models. Now, I may consider > adding models to Gradle myself (although I already spend lots of my free > time on coding other projects). > > > That would be great. Please give it a try. We'd very much welcome this. > > > However, assume that I go mad adding lots models, which you find quality > enough to accept: Do you plan to support these models "forever" considering > backward (or worse, forward) compatibility without me? > > > Absolutely. That's the whole point. It's a collaboration. The models > describe what Netbeans needs to know about the build. We work together to > define this. And then, given this description, we can find the best ways to > provide that information, and that is free to change over time. > > This is only really a problem from our point of view if there's lots of > stuff that Netbeans needs to know that doesn't make sense for other IDEs to > know, but I don't think that's the case. > > To me, it's about putting the knowledge in a certain place based on where > it belongs, not based on who does the work. The mappings belong in the > build, but that doesn't mean that a Gradle core developer needs to > implement it. > > > You see, even though it could be my true intent to support them for you, > I might die tomorrow. This is something you must consider. That is, > Gradleware cannot truly rely on external contributons for these models. > > 6. Let's be rational: I mean no offense but considering the past, you > spend a small portion of your resources for making models better. I believe > that it is not rational to believe that the percentage of resources you can > and will spend on adding new features to the model will be increased in the > future. To be honest, I think you shouldn't change your current priorities > too much because other things are very important as well. > > > It's a good point. We hope to do better at this, and reaching out to the > community is part of this. There are some improvements ready to be included > in the 1.8 release, and we want this to continue. Let's see how we go. > > > > So, given these, I wish to avoid Gradleware spending more resources on > maintaining compatibility for the new models. I want you to push these > maintaining work to IDE integration developers (like me). > > > Right, that's exactly the idea. But that's not not an argument for > accessing Project and each tool integrator re-implementing the same stuff > in different ways. To me, it's an argument for sharing work via the models. > > > But you cannot do so by asking me to add new models (see 5) myself. If > you want me to create models without pushing these models to the Gradle > repo, then I don't see what is the gain. I still need to build that model > somehow from `Project`. > > What is the gain from accessing the `Project` through the Tooling API (as > proposed by Alex and me)? In this case, I'm relieved from the burden of > maintaining a backward (or forward) compatible model. Even if I screw up in > a version, I can simply fix this in a patch and release a new version of > the NB plugin independently of Gradle. Speaking of which, independent > release from Gradle allows me to consider users needs and react to these > needs on my own schedule. Also, different IDEs might have considerable > different view of the world and I'm also pretty sure that users of > different IDEs have different expectations on how things should be done > (there is a reason for why they choose one IDE over another). > > Saying all these, I'd prefer if developers of Idea and Eclipse integration > would voice their opinion. Also, I believe you should wait for their > opinion. That is, IDE integration developers are the main users of this API > (at least, I believe this is where these new changes matter most). > > >> >> >> Caching can be done external to Gradle. Actually, I already cache models. >> Also, there is no way to know in general if reevaluation of a project is >> necessary, so I believe that Gradle cannot do considerably better than an >> external process. >> >> >> There's an awful lot to track when checking for changes in the model. >> Here are some examples: >> >> - Changes in the build script and in one of the ancestor's build scripts. >> - Changes in the settings scripts. >> - Changes in the init scripts passed as command-line args, or in >> ~/.gradle/init.gradle or ~/.gradle/init.d >> - Changes in any local or remote scripts applied by any of the above or >> any init scripts bundled in the distributions. >> - Changes in gradle.properties and ~/.gradle/gradle.properties. >> - Changes to the Gradle version in gradle-wrapper.properties. >> - Changes in the source or dependencies of buildSrc >> - Changes in the classpath used at configuration time, eg local jars. >> - Changes to dependencies in the remote repositories. >> - Changes in the configuration files or remote resources that my custom >> plugin uses to configure the build. >> >> You get the idea. It think good change detection has to be a >> collaboration between Gradle and the IDE. >> >> >> > Knowing on what the build script depends is impossible. It can be a > content of a directory or other things (an absurd example would be > System.nanoTime % 2). Although it might be reasonable to expect the user > not to depend on anything he or she whishes. However, I have never seen > serious need for this automatic change detection. People seem to be content > with manual reloading. Also, it might be even harmful to reload a model > just because a build script or something else changes because the > background work (what Gradle does) might interfere with the user (reading > from the harddrive might slow down things) and actually, users don't really > need the new model for every small changes. Like adding a println to a file > in buildSrc is really not something to worry about. > > Although in the past I thought similar to you, I now believe that this > kind of caching is not something the users need. Caching which benefits the > user is something different and it is important that the user may refresh > at will (ignoring caching). I still need to implement persistent cache but > I believe that this is not something you should spend effort on. At least > not without explicitly polling the users need. But even if it turns that > this is something important, allowing access to `Project` shouldn't cause a > problem. Only that you won't cache `Project` instances, so anyone relying > it won't automatically benefit from your caching. The implementor (and its > users) then might choose if this is something worth losing the benefits of > being able to access `Project`. > > > > ------------------------------ > View this message in context: Re: Proposal for retrieving multiple types > of models from a project in a single pass, using the Tooling > API<http://gradle.1045684.n5.nabble.com/Proposal-for-retrieving-multiple-types-of-models-from-a-project-in-a-single-pass-using-the-Tooling-AI-tp5711516p5711582.html> > Sent from the gradle-dev mailing list > archive<http://gradle.1045684.n5.nabble.com/gradle-dev-f1436218.html>at > Nabble.com. > > > > -- > Adam Murdoch > Gradle Co-founder > http://www.gradle.org > VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting > http://www.gradleware.com > > > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://gradle.1045684.n5.nabble.com/Proposal-for-retrieving-multiple-types-of-models-from-a-project-in-a-single-pass-using-the-Tooling-AI-tp5711516p5711697.html > To unsubscribe from Proposal for retrieving multiple types of models from > a project in a single pass, using the Tooling API, click > here<http://gradle.1045684.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5711516&code=YXR0aWxhLmtlbGVtZW44NUBnbWFpbC5jb218NTcxMTUxNnwtMTMxMjM2NTcwMA==> > . > NAML<http://gradle.1045684.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://gradle.1045684.n5.nabble.com/Proposal-for-retrieving-multiple-types-of-models-from-a-project-in-a-single-pass-using-the-Tooling-AI-tp5711516p5711700.html Sent from the gradle-dev mailing list archive at Nabble.com.
