On Tue, Aug 13, 2013 at 11:32 AM, kelemen <attila.keleme...@gmail.com>wrote:
> I can see that you are adamant about not giving access to `Project`. > Anyway, I have realized that I already have access to the `Project` > instance via init scripts *evil smile*. So, I can inject a custom model > builder there. > > You say that "We don't want Netbeans, Eclipse, IDEA and my-custom-tool all > deciding to use a different way of mapping to the simplified view". > However, I have a different internal model representation, so I have to map > things anyway. The only question is from what model do I do this > conversion: DSL or a model created from the DSL. So, the created model > (`IdeaProject`) does not make my life easier the slightest. Quite the > contrary, I have to undo a lot of things, so your mapping efforts went in > vain. This is because in NetBeans, I need something very close to the DSL > without further mapping. For example, `SourceSet.getCompileClasspath()` > would be a lot better than to have access to configurations like "compile" > (which is unreliable to depend on). Actually, knowing "compile" instead of > the actual classpath is a lot of pain to me because I have to do some > dependency resolution myself (to make matters worse, I don't even have all > the info), something which Gradle can already do itself. > > All in all, I believe that accessing models instead of the DSL doesn't > make anything easier. It certainly is no easier for me. I believe that > "re-implementing the same stuff in different ways" assumes that I need to > same thing as other IDEs which is not the case. As I see, models only > structure the information in a certain way but I'm pretty sure that > different IDEs need to structure things in a different way. So why is it > assumed, that choosing a particular structure of the information (other > than the DSL) will help all the IDEs? That is, if I were to implement a > model for "JavaProject", I would implement something benefical to NetBeans > because I have to choose a structure and why should I choose something > else? Also, I don't agree with your example of Android projects: They are > not Java projects and should (and do) have a different model anyway. That > is, if a build script does not apply the "java" plugin (implicitly or > explicitly) then it is not a Java project, therefore needs a different > model. > There is one important aspect relevant for this debate that I did not see mentioned in the last couple of threads. It is essential for us to have a Gradle model that represent the world view of NetBeans, Eclipse, Idea, Jenkins, etc.. As the mapping from the Gradle model to the consuming side is lossy or at least often not 1:1 we can only make a guess of what the user wants. For example if you have a 900 submodule build this guess is to have 900 projects in Eclipse. In this example it would mean terrible performance on the side of Eclipse. The same guessing game you would need to within the IDE if you consume the Gradle models directly. How can we let Gradle users influence on how a project is mapped for example to Eclipse? Having an Eclipse model within Gradle is enabling this. You can hook into the default Gradle-Eclipse mapping process and modify the Eclipse model that is send to Eclipse for configuring the build. In this example you might replace the bulk of the Eclipse project dependencies with external dependencies. Having distinct models gives the Gradle user a powerful mechanism to customize how a Gradle is represented in another tool. That might not be relevant for all elements of the Gradle model, but , definitely for a decent chunk. Hans > > 2013/8/13 Adam Murdoch [via Gradle] <[hidden > email]<http://user/SendEmail.jtp?type=node&node=5711700&i=0> > > > >> >> On 02/08/2013, at 9:34 AM, kelemen <[hidden >> email]<http://user/SendEmail.jtp?type=node&node=5711697&i=0>> >> wrote: >> >> 2013/8/1 Adam Murdoch [via Gradle] <<a >> href="x-msg://1801/user/SendEmail.jtp?type=node&node=5711582&i=0" >> target="_top" rel="nofollow" link="external">[hidden email]> >> >>> >>> On 30/07/2013, at 6:52 PM, kelemen <[hidden >>> email]<http://user/SendEmail.jtp?type=node&node=5711581&i=0>> >>> wrote: >>> >>> 2013/7/30 Adam Murdoch [via Gradle] <<a href="<a >>> href="x-msg://4953/user/SendEmail.jtp?type=node&amp;node=5711554&amp;i=0">x-msg://4953/user/SendEmail.jtp?type=node&node=5711554&i=0" >>> target="_top" rel="nofollow" link="external">[hidden email]> >>> >>> >>>> On 29/07/2013, at 4:25 PM, Alex Ruiz <[hidden >>>> email]<http://user/SendEmail.jtp?type=node&node=5711550&i=0>> >>>> wrote: >>>> >>>> Thanks, Adam. I've been looking into (and debugging) how the current >>>> API for retrieving models work, to have a better understanding on how the >>>> API and implementation of this new action would look like. >>>> >>>> Attila: This approach may also benefit your work on Netbeans. Please >>>> take a look. >>>> >>>> This is what I have in mind, based on what I have done this weekend: >>>> >>>> - The action interface can be simple, and so far this is what I think >>>> about its API: >>>> >>>> public interface ConsumerAction<T> extends Serializable { >>>> T execute(Project project); >>>> } >>>> >>>> where Project is a org.gradle.api.Project. My understanding is that >>>> by having a Project we can do everything we need (through >>>> Project#getService()). >>>> >>>> >>>> The action should accept something other than `org.gradle.api.Project`, >>>> for a few reasons: >>>> >>>> 1. `org.gradle.api.Project` is not version-independent. The entry point >>>> for the action should be a function of the client's tooling API version, >>>> not target Gradle version. >>>> 2. Some actions don't need a configured project. For example, an action >>>> may just be interested in the project hierarchy or the build environment. >>>> Or it may not be interested in the tasks of a project, or the dependencies >>>> of a project, or whatever. Instead, we need an interface that implies no >>>> configuration is done, except for that which is explicitly requested by the >>>> action. >>>> 3. It's not going to work well with decoupled project mode (e.g. >>>> parallel execution, configure on demand). This will become the default at >>>> some point. The plan there is that there will be no way to navigate from >>>> one project to another. Instead, we need an interface which provides access >>>> to the whole build, but in a way that allows things like parallel >>>> configuration and configuration on demand. >>>> 4. There's no opportunity to cache models. We'd have to cache the whole >>>> project model, which is unlikely to be possible any time soon. >>>> >>>> >>> It would be very convenient to have access to `org.gradle.api.Project` >>> regardless. Not being version independent is not necessarily a problem, >>> since Gradle has to maintain compatibility due to the build scripts and >>> this compatibility should be enough. Actually, I could provide a lot better >>> experience in NB if this was possible. For example, I could easily display >>> all the source root with proper classpaths including the usual "integTest" >>> sources. >>> >>> >>> The process of importing Gradle into an IDE has to involve some kind of >>> mapping. The Gradle model is much richer that the IDEs and this model has >>> to be converted into the simpler model of each IDE. This mapping is lossy >>> and as such the mapping needs to be flexible, and the build author and the >>> various plugins must be able to have some influence over it. The mapping >>> shouldn't be hardcoded in the IDE. This is one of the reasons the tooling >>> API exists. >>> >>> So, we have: >>> >>> Gradle model -> transformation -> IDE model >>> >>> It's a little more detailed than that: >>> >>> Gradle DSL -> transformation -> canonical model -> transformation -> IDE >>> model >>> >>> That is, the user configures some stuff through the DSL, this is then >>> converted to some internal model that various parts of Gradle use, >>> including the IDE mapping. We keep the DSL backwards compatible, but we >>> don't for the canonical model. We plan to, but we're not there yet. In >>> other words, the stuff you need from the project (i.e. the canonical model) >>> is not backwards compatible at all. >>> >>> So, rather than access `Project` directly, I'd much prefer if you help >>> us grow the tooling API models instead. It's easy to add stuff once you >>> know the pattern. >>> >>> These models then form a contract, as they declare what you need from >>> the model. We can keep this stable and make it faster over time. They also >>> allow us to offer consistent behaviour across IDEs (which is important to >>> us, possibly not so for you). >>> >>> >> There might be some misundestanding between us, so I will try to clarify >> my assumptions and see where it differs from yours: >> >> 1. The proposed `ConsumerAction` runs in the same process where the >> project got evaluated, so I believe that the compatibility of the DSL is >> sufficient. For example: I should be able to iterate over the sourcesets >> because I can do so in the build script. >> >> 2. If projects are evaluated in multiple processes, then I'd prefer to >> design an API which works even in this case but still runs foreign (i.e., >> foreign to Gradle) code in the process where the project got evaluated (in >> multiple processes if necessary). >> >> 3. I assume that DSL compatibility implies that no methods or classes >> will disappear from the http://www.gradle.org/docs/current/javadoc/. I >> don't assume, that I can just implement interfaces (like `Project`). >> >> 4. I'm not sure what you mean by "canonical model". `Project` or >> something else? If it is the `Project`, then I don't understand because the >> build script can access it as well (see 3). If it is something else, then I >> don't need that one. >> >> >> Except that you do. There are multiple ways I can define, for example, a >> Java project. For example, you can use the 'java' plugin, or the 'android' >> plugin, or one of the new JVM language plugins or something entirely >> custom. These are all different DSLs, but they are all translated to >> produce a single DSL-independent model that describes things about how to >> build the project, such as where the source is, the language level, the >> compile dependencies and so on. This is the canonical model that the IDE >> should be using as input. It should not be using the original DSL. >> >> The canonical model has to be further mapped in a lossy way to the >> simpler view of the world that the IDEs have. It's not really the IDE's >> business as to how this mapping should happen. It's the build's business. >> >> So, we have something like this: >> >> DSL 1 ----------+ >> | >> DSL 2 ----------+--> canonical model --+--> simplified model --> IDE >> | | >> my custom DSL --+ +--> other plugins >> >> It's the model that pops out the righthand side that the IDEs should be >> using, not the models that go in at the lefthand side. >> >> I just need the `Project` instance right after the buildscript set it >> up. Of course, this limits the IDE integration (those using `Project`) in >> some marginal use-cases. But there are already limitations, I don't think >> an IDE integration should consider all extreme use cases. Build masters >> should be sane enough to write build scripts so that the IDE understands >> it. By the way, I already pass init scripts (optionally, opt-out) when >> running tasks which means that I already access `Project`. >> >> 5. I don't understand what you mean on consistent behaviour. >> >> >> The consistency is to do with how the mappings at each of the above >> stages is done. We don't want Netbeans, Eclipse, IDEA and my-custom-tool >> all deciding to use a different way of mapping to the simplified view. >> >> >> There are still models like `EclipseProject` and `IdeaProject`. How does >> this help consistency? If you mean deprecating these models and adding >> common models like `JavaProjectModel`, `GroovyProjectModel` and whatever >> else, then this could mean an awful lot of models. Now, I may consider >> adding models to Gradle myself (although I already spend lots of my free >> time on coding other projects). >> >> >> That would be great. Please give it a try. We'd very much welcome this. >> >> >> However, assume that I go mad adding lots models, which you find quality >> enough to accept: Do you plan to support these models "forever" considering >> backward (or worse, forward) compatibility without me? >> >> >> Absolutely. That's the whole point. It's a collaboration. The models >> describe what Netbeans needs to know about the build. We work together to >> define this. And then, given this description, we can find the best ways to >> provide that information, and that is free to change over time. >> >> This is only really a problem from our point of view if there's lots of >> stuff that Netbeans needs to know that doesn't make sense for other IDEs to >> know, but I don't think that's the case. >> >> To me, it's about putting the knowledge in a certain place based on where >> it belongs, not based on who does the work. The mappings belong in the >> build, but that doesn't mean that a Gradle core developer needs to >> implement it. >> >> >> You see, even though it could be my true intent to support them for you, >> I might die tomorrow. This is something you must consider. That is, >> Gradleware cannot truly rely on external contributons for these models. >> >> 6. Let's be rational: I mean no offense but considering the past, you >> spend a small portion of your resources for making models better. I believe >> that it is not rational to believe that the percentage of resources you can >> and will spend on adding new features to the model will be increased in the >> future. To be honest, I think you shouldn't change your current priorities >> too much because other things are very important as well. >> >> >> It's a good point. We hope to do better at this, and reaching out to the >> community is part of this. There are some improvements ready to be included >> in the 1.8 release, and we want this to continue. Let's see how we go. >> >> >> >> So, given these, I wish to avoid Gradleware spending more resources on >> maintaining compatibility for the new models. I want you to push these >> maintaining work to IDE integration developers (like me). >> >> >> Right, that's exactly the idea. But that's not not an argument for >> accessing Project and each tool integrator re-implementing the same stuff >> in different ways. To me, it's an argument for sharing work via the models. >> >> >> But you cannot do so by asking me to add new models (see 5) myself. If >> you want me to create models without pushing these models to the Gradle >> repo, then I don't see what is the gain. I still need to build that model >> somehow from `Project`. >> >> What is the gain from accessing the `Project` through the Tooling API (as >> proposed by Alex and me)? In this case, I'm relieved from the burden of >> maintaining a backward (or forward) compatible model. Even if I screw up in >> a version, I can simply fix this in a patch and release a new version of >> the NB plugin independently of Gradle. Speaking of which, independent >> release from Gradle allows me to consider users needs and react to these >> needs on my own schedule. Also, different IDEs might have considerable >> different view of the world and I'm also pretty sure that users of >> different IDEs have different expectations on how things should be done >> (there is a reason for why they choose one IDE over another). >> >> Saying all these, I'd prefer if developers of Idea and Eclipse >> integration would voice their opinion. Also, I believe you should wait for >> their opinion. That is, IDE integration developers are the main users of >> this API (at least, I believe this is where these new changes matter most). >> >> >>> >>> >>> Caching can be done external to Gradle. Actually, I already cache >>> models. Also, there is no way to know in general if reevaluation of a >>> project is necessary, so I believe that Gradle cannot do considerably >>> better than an external process. >>> >>> >>> There's an awful lot to track when checking for changes in the model. >>> Here are some examples: >>> >>> - Changes in the build script and in one of the ancestor's build scripts. >>> - Changes in the settings scripts. >>> - Changes in the init scripts passed as command-line args, or in >>> ~/.gradle/init.gradle or ~/.gradle/init.d >>> - Changes in any local or remote scripts applied by any of the above or >>> any init scripts bundled in the distributions. >>> - Changes in gradle.properties and ~/.gradle/gradle.properties. >>> - Changes to the Gradle version in gradle-wrapper.properties. >>> - Changes in the source or dependencies of buildSrc >>> - Changes in the classpath used at configuration time, eg local jars. >>> - Changes to dependencies in the remote repositories. >>> - Changes in the configuration files or remote resources that my custom >>> plugin uses to configure the build. >>> >>> You get the idea. It think good change detection has to be a >>> collaboration between Gradle and the IDE. >>> >>> >>> >> Knowing on what the build script depends is impossible. It can be a >> content of a directory or other things (an absurd example would be >> System.nanoTime % 2). Although it might be reasonable to expect the user >> not to depend on anything he or she whishes. However, I have never seen >> serious need for this automatic change detection. People seem to be content >> with manual reloading. Also, it might be even harmful to reload a model >> just because a build script or something else changes because the >> background work (what Gradle does) might interfere with the user (reading >> from the harddrive might slow down things) and actually, users don't really >> need the new model for every small changes. Like adding a println to a file >> in buildSrc is really not something to worry about. >> >> Although in the past I thought similar to you, I now believe that this >> kind of caching is not something the users need. Caching which benefits the >> user is something different and it is important that the user may refresh >> at will (ignoring caching). I still need to implement persistent cache but >> I believe that this is not something you should spend effort on. At least >> not without explicitly polling the users need. But even if it turns that >> this is something important, allowing access to `Project` shouldn't cause a >> problem. Only that you won't cache `Project` instances, so anyone relying >> it won't automatically benefit from your caching. The implementor (and its >> users) then might choose if this is something worth losing the benefits of >> being able to access `Project`. >> >> >> >> ------------------------------ >> View this message in context: Re: Proposal for retrieving multiple types >> of models from a project in a single pass, using the Tooling >> API<http://gradle.1045684.n5.nabble.com/Proposal-for-retrieving-multiple-types-of-models-from-a-project-in-a-single-pass-using-the-Tooling-AI-tp5711516p5711582.html> >> Sent from the gradle-dev mailing list >> archive<http://gradle.1045684.n5.nabble.com/gradle-dev-f1436218.html>at >> Nabble.com. >> >> >> >> -- >> Adam Murdoch >> Gradle Co-founder >> http://www.gradle.org >> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting >> http://www.gradleware.com >> >> >> >> >> >> ------------------------------ >> If you reply to this email, your message will be added to the >> discussion below: >> >> http://gradle.1045684.n5.nabble.com/Proposal-for-retrieving-multiple-types-of-models-from-a-project-in-a-single-pass-using-the-Tooling-AI-tp5711516p5711697.html >> To unsubscribe from Proposal for retrieving multiple types of models >> from a project in a single pass, using the Tooling API, click here. >> NAML<http://gradle.1045684.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> > > > ------------------------------ > View this message in context: Re: Proposal for retrieving multiple types > of models from a project in a single pass, using the Tooling > API<http://gradle.1045684.n5.nabble.com/Proposal-for-retrieving-multiple-types-of-models-from-a-project-in-a-single-pass-using-the-Tooling-AI-tp5711516p5711700.html> > Sent from the gradle-dev mailing list > archive<http://gradle.1045684.n5.nabble.com/gradle-dev-f1436218.html>at > Nabble.com. >
