I have verified that it is possible to inject a custom model builder (although highly inconvenient). While experimenting with this, I have learned that custom models must be interfaces. Is there any technical reason for that? If not, I would like this restriction to be removed because it is quite inconvenient. That is, I don't need (in this case) any kind of compatibilty and even if I did, I would rather use a serialization proxy with immutable classes.
Also, is there any chance to make it easier to inject custom model builders (without hacking with the init script)? So that I don't need to play around with classpaths as well. 2013/8/13 Kelemen Attila <attila.keleme...@gmail.com> > I can see that you are adamant about not giving access to `Project`. > Anyway, I have realized that I already have access to the `Project` > instance via init scripts *evil smile*. So, I can inject a custom model > builder there. > > You say that "We don't want Netbeans, Eclipse, IDEA and my-custom-tool all > deciding to use a different way of mapping to the simplified view". > However, I have a different internal model representation, so I have to map > things anyway. The only question is from what model do I do this > conversion: DSL or a model created from the DSL. So, the created model > (`IdeaProject`) does not make my life easier the slightest. Quite the > contrary, I have to undo a lot of things, so your mapping efforts went in > vain. This is because in NetBeans, I need something very close to the DSL > without further mapping. For example, `SourceSet.getCompileClasspath()` > would be a lot better than to have access to configurations like "compile" > (which is unreliable to depend on). Actually, knowing "compile" instead of > the actual classpath is a lot of pain to me because I have to do some > dependency resolution myself (to make matters worse, I don't even have all > the info), something which Gradle can already do itself. > > All in all, I believe that accessing models instead of the DSL doesn't > make anything easier. It certainly is no easier for me. I believe that > "re-implementing the same stuff in different ways" assumes that I need to > same thing as other IDEs which is not the case. As I see, models only > structure the information in a certain way but I'm pretty sure that > different IDEs need to structure things in a different way. So why is it > assumed, that choosing a particular structure of the information (other > than the DSL) will help all the IDEs? That is, if I were to implement a > model for "JavaProject", I would implement something benefical to NetBeans > because I have to choose a structure and why should I choose something > else? Also, I don't agree with your example of Android projects: They are > not Java projects and should (and do) have a different model anyway. That > is, if a build script does not apply the "java" plugin (implicitly or > explicitly) then it is not a Java project, therefore needs a different > model. > > > 2013/8/13 Adam Murdoch [via Gradle] < > ml-node+s1045684n5711697...@n5.nabble.com> > > >> On 02/08/2013, at 9:34 AM, kelemen <[hidden >> email]<http://user/SendEmail.jtp?type=node&node=5711697&i=0>> >> wrote: >> >> 2013/8/1 Adam Murdoch [via Gradle] <<a >> href="x-msg://1801/user/SendEmail.jtp?type=node&node=5711582&i=0" >> target="_top" rel="nofollow" link="external">[hidden email]> >> >>> >>> On 30/07/2013, at 6:52 PM, kelemen <[hidden >>> email]<http://user/SendEmail.jtp?type=node&node=5711581&i=0>> >>> wrote: >>> >>> 2013/7/30 Adam Murdoch [via Gradle] <<a href="<a >>> href="x-msg://4953/user/SendEmail.jtp?type=node&amp;node=5711554&amp;i=0">x-msg://4953/user/SendEmail.jtp?type=node&node=5711554&i=0" >>> target="_top" rel="nofollow" link="external">[hidden email]> >>> >>> >>>> On 29/07/2013, at 4:25 PM, Alex Ruiz <[hidden >>>> email]<http://user/SendEmail.jtp?type=node&node=5711550&i=0>> >>>> wrote: >>>> >>>> Thanks, Adam. I've been looking into (and debugging) how the current >>>> API for retrieving models work, to have a better understanding on how the >>>> API and implementation of this new action would look like. >>>> >>>> Attila: This approach may also benefit your work on Netbeans. Please >>>> take a look. >>>> >>>> This is what I have in mind, based on what I have done this weekend: >>>> >>>> - The action interface can be simple, and so far this is what I think >>>> about its API: >>>> >>>> public interface ConsumerAction<T> extends Serializable { >>>> T execute(Project project); >>>> } >>>> >>>> where Project is a org.gradle.api.Project. My understanding is that >>>> by having a Project we can do everything we need (through >>>> Project#getService()). >>>> >>>> >>>> The action should accept something other than `org.gradle.api.Project`, >>>> for a few reasons: >>>> >>>> 1. `org.gradle.api.Project` is not version-independent. The entry point >>>> for the action should be a function of the client's tooling API version, >>>> not target Gradle version. >>>> 2. Some actions don't need a configured project. For example, an action >>>> may just be interested in the project hierarchy or the build environment. >>>> Or it may not be interested in the tasks of a project, or the dependencies >>>> of a project, or whatever. Instead, we need an interface that implies no >>>> configuration is done, except for that which is explicitly requested by the >>>> action. >>>> 3. It's not going to work well with decoupled project mode (e.g. >>>> parallel execution, configure on demand). This will become the default at >>>> some point. The plan there is that there will be no way to navigate from >>>> one project to another. Instead, we need an interface which provides access >>>> to the whole build, but in a way that allows things like parallel >>>> configuration and configuration on demand. >>>> 4. There's no opportunity to cache models. We'd have to cache the whole >>>> project model, which is unlikely to be possible any time soon. >>>> >>>> >>> It would be very convenient to have access to `org.gradle.api.Project` >>> regardless. Not being version independent is not necessarily a problem, >>> since Gradle has to maintain compatibility due to the build scripts and >>> this compatibility should be enough. Actually, I could provide a lot better >>> experience in NB if this was possible. For example, I could easily display >>> all the source root with proper classpaths including the usual "integTest" >>> sources. >>> >>> >>> The process of importing Gradle into an IDE has to involve some kind of >>> mapping. The Gradle model is much richer that the IDEs and this model has >>> to be converted into the simpler model of each IDE. This mapping is lossy >>> and as such the mapping needs to be flexible, and the build author and the >>> various plugins must be able to have some influence over it. The mapping >>> shouldn't be hardcoded in the IDE. This is one of the reasons the tooling >>> API exists. >>> >>> So, we have: >>> >>> Gradle model -> transformation -> IDE model >>> >>> It's a little more detailed than that: >>> >>> Gradle DSL -> transformation -> canonical model -> transformation -> IDE >>> model >>> >>> That is, the user configures some stuff through the DSL, this is then >>> converted to some internal model that various parts of Gradle use, >>> including the IDE mapping. We keep the DSL backwards compatible, but we >>> don't for the canonical model. We plan to, but we're not there yet. In >>> other words, the stuff you need from the project (i.e. the canonical model) >>> is not backwards compatible at all. >>> >>> So, rather than access `Project` directly, I'd much prefer if you help >>> us grow the tooling API models instead. It's easy to add stuff once you >>> know the pattern. >>> >>> These models then form a contract, as they declare what you need from >>> the model. We can keep this stable and make it faster over time. They also >>> allow us to offer consistent behaviour across IDEs (which is important to >>> us, possibly not so for you). >>> >>> >> There might be some misundestanding between us, so I will try to clarify >> my assumptions and see where it differs from yours: >> >> 1. The proposed `ConsumerAction` runs in the same process where the >> project got evaluated, so I believe that the compatibility of the DSL is >> sufficient. For example: I should be able to iterate over the sourcesets >> because I can do so in the build script. >> >> 2. If projects are evaluated in multiple processes, then I'd prefer to >> design an API which works even in this case but still runs foreign (i.e., >> foreign to Gradle) code in the process where the project got evaluated (in >> multiple processes if necessary). >> >> 3. I assume that DSL compatibility implies that no methods or classes >> will disappear from the http://www.gradle.org/docs/current/javadoc/. I >> don't assume, that I can just implement interfaces (like `Project`). >> >> 4. I'm not sure what you mean by "canonical model". `Project` or >> something else? If it is the `Project`, then I don't understand because the >> build script can access it as well (see 3). If it is something else, then I >> don't need that one. >> >> >> Except that you do. There are multiple ways I can define, for example, a >> Java project. For example, you can use the 'java' plugin, or the 'android' >> plugin, or one of the new JVM language plugins or something entirely >> custom. These are all different DSLs, but they are all translated to >> produce a single DSL-independent model that describes things about how to >> build the project, such as where the source is, the language level, the >> compile dependencies and so on. This is the canonical model that the IDE >> should be using as input. It should not be using the original DSL. >> >> The canonical model has to be further mapped in a lossy way to the >> simpler view of the world that the IDEs have. It's not really the IDE's >> business as to how this mapping should happen. It's the build's business. >> >> So, we have something like this: >> >> DSL 1 ----------+ >> | >> DSL 2 ----------+--> canonical model --+--> simplified model --> IDE >> | | >> my custom DSL --+ +--> other plugins >> >> It's the model that pops out the righthand side that the IDEs should be >> using, not the models that go in at the lefthand side. >> >> I just need the `Project` instance right after the buildscript set it >> up. Of course, this limits the IDE integration (those using `Project`) in >> some marginal use-cases. But there are already limitations, I don't think >> an IDE integration should consider all extreme use cases. Build masters >> should be sane enough to write build scripts so that the IDE understands >> it. By the way, I already pass init scripts (optionally, opt-out) when >> running tasks which means that I already access `Project`. >> >> 5. I don't understand what you mean on consistent behaviour. >> >> >> The consistency is to do with how the mappings at each of the above >> stages is done. We don't want Netbeans, Eclipse, IDEA and my-custom-tool >> all deciding to use a different way of mapping to the simplified view. >> >> >> There are still models like `EclipseProject` and `IdeaProject`. How does >> this help consistency? If you mean deprecating these models and adding >> common models like `JavaProjectModel`, `GroovyProjectModel` and whatever >> else, then this could mean an awful lot of models. Now, I may consider >> adding models to Gradle myself (although I already spend lots of my free >> time on coding other projects). >> >> >> That would be great. Please give it a try. We'd very much welcome this. >> >> >> However, assume that I go mad adding lots models, which you find quality >> enough to accept: Do you plan to support these models "forever" considering >> backward (or worse, forward) compatibility without me? >> >> >> Absolutely. That's the whole point. It's a collaboration. The models >> describe what Netbeans needs to know about the build. We work together to >> define this. And then, given this description, we can find the best ways to >> provide that information, and that is free to change over time. >> >> This is only really a problem from our point of view if there's lots of >> stuff that Netbeans needs to know that doesn't make sense for other IDEs to >> know, but I don't think that's the case. >> >> To me, it's about putting the knowledge in a certain place based on where >> it belongs, not based on who does the work. The mappings belong in the >> build, but that doesn't mean that a Gradle core developer needs to >> implement it. >> >> >> You see, even though it could be my true intent to support them for you, >> I might die tomorrow. This is something you must consider. That is, >> Gradleware cannot truly rely on external contributons for these models. >> >> 6. Let's be rational: I mean no offense but considering the past, you >> spend a small portion of your resources for making models better. I believe >> that it is not rational to believe that the percentage of resources you can >> and will spend on adding new features to the model will be increased in the >> future. To be honest, I think you shouldn't change your current priorities >> too much because other things are very important as well. >> >> >> It's a good point. We hope to do better at this, and reaching out to the >> community is part of this. There are some improvements ready to be included >> in the 1.8 release, and we want this to continue. Let's see how we go. >> >> >> >> So, given these, I wish to avoid Gradleware spending more resources on >> maintaining compatibility for the new models. I want you to push these >> maintaining work to IDE integration developers (like me). >> >> >> Right, that's exactly the idea. But that's not not an argument for >> accessing Project and each tool integrator re-implementing the same stuff >> in different ways. To me, it's an argument for sharing work via the models. >> >> >> But you cannot do so by asking me to add new models (see 5) myself. If >> you want me to create models without pushing these models to the Gradle >> repo, then I don't see what is the gain. I still need to build that model >> somehow from `Project`. >> >> What is the gain from accessing the `Project` through the Tooling API (as >> proposed by Alex and me)? In this case, I'm relieved from the burden of >> maintaining a backward (or forward) compatible model. Even if I screw up in >> a version, I can simply fix this in a patch and release a new version of >> the NB plugin independently of Gradle. Speaking of which, independent >> release from Gradle allows me to consider users needs and react to these >> needs on my own schedule. Also, different IDEs might have considerable >> different view of the world and I'm also pretty sure that users of >> different IDEs have different expectations on how things should be done >> (there is a reason for why they choose one IDE over another). >> >> Saying all these, I'd prefer if developers of Idea and Eclipse >> integration would voice their opinion. Also, I believe you should wait for >> their opinion. That is, IDE integration developers are the main users of >> this API (at least, I believe this is where these new changes matter most). >> >> >>> >>> >>> Caching can be done external to Gradle. Actually, I already cache >>> models. Also, there is no way to know in general if reevaluation of a >>> project is necessary, so I believe that Gradle cannot do considerably >>> better than an external process. >>> >>> >>> There's an awful lot to track when checking for changes in the model. >>> Here are some examples: >>> >>> - Changes in the build script and in one of the ancestor's build scripts. >>> - Changes in the settings scripts. >>> - Changes in the init scripts passed as command-line args, or in >>> ~/.gradle/init.gradle or ~/.gradle/init.d >>> - Changes in any local or remote scripts applied by any of the above or >>> any init scripts bundled in the distributions. >>> - Changes in gradle.properties and ~/.gradle/gradle.properties. >>> - Changes to the Gradle version in gradle-wrapper.properties. >>> - Changes in the source or dependencies of buildSrc >>> - Changes in the classpath used at configuration time, eg local jars. >>> - Changes to dependencies in the remote repositories. >>> - Changes in the configuration files or remote resources that my custom >>> plugin uses to configure the build. >>> >>> You get the idea. It think good change detection has to be a >>> collaboration between Gradle and the IDE. >>> >>> >>> >> Knowing on what the build script depends is impossible. It can be a >> content of a directory or other things (an absurd example would be >> System.nanoTime % 2). Although it might be reasonable to expect the user >> not to depend on anything he or she whishes. However, I have never seen >> serious need for this automatic change detection. People seem to be content >> with manual reloading. Also, it might be even harmful to reload a model >> just because a build script or something else changes because the >> background work (what Gradle does) might interfere with the user (reading >> from the harddrive might slow down things) and actually, users don't really >> need the new model for every small changes. Like adding a println to a file >> in buildSrc is really not something to worry about. >> >> Although in the past I thought similar to you, I now believe that this >> kind of caching is not something the users need. Caching which benefits the >> user is something different and it is important that the user may refresh >> at will (ignoring caching). I still need to implement persistent cache but >> I believe that this is not something you should spend effort on. At least >> not without explicitly polling the users need. But even if it turns that >> this is something important, allowing access to `Project` shouldn't cause a >> problem. Only that you won't cache `Project` instances, so anyone relying >> it won't automatically benefit from your caching. The implementor (and its >> users) then might choose if this is something worth losing the benefits of >> being able to access `Project`. >> >> >> >> ------------------------------ >> View this message in context: Re: Proposal for retrieving multiple types >> of models from a project in a single pass, using the Tooling >> API<http://gradle.1045684.n5.nabble.com/Proposal-for-retrieving-multiple-types-of-models-from-a-project-in-a-single-pass-using-the-Tooling-AI-tp5711516p5711582.html> >> Sent from the gradle-dev mailing list >> archive<http://gradle.1045684.n5.nabble.com/gradle-dev-f1436218.html>at >> Nabble.com. >> >> >> >> -- >> Adam Murdoch >> Gradle Co-founder >> http://www.gradle.org >> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting >> http://www.gradleware.com >> >> >> >> >> >> ------------------------------ >> If you reply to this email, your message will be added to the >> discussion below: >> >> http://gradle.1045684.n5.nabble.com/Proposal-for-retrieving-multiple-types-of-models-from-a-project-in-a-single-pass-using-the-Tooling-AI-tp5711516p5711697.html >> To unsubscribe from Proposal for retrieving multiple types of models >> from a project in a single pass, using the Tooling API, click >> here<http://gradle.1045684.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5711516&code=YXR0aWxhLmtlbGVtZW44NUBnbWFpbC5jb218NTcxMTUxNnwtMTMxMjM2NTcwMA==> >> . >> NAML<http://gradle.1045684.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> > > -- View this message in context: http://gradle.1045684.n5.nabble.com/Proposal-for-retrieving-multiple-types-of-models-from-a-project-in-a-single-pass-using-the-Tooling-AI-tp5711516p5711708.html Sent from the gradle-dev mailing list archive at Nabble.com.
