On 30 Jun 2014, at 1:51 am, Daniel Lacasse <daniel.lacass...@gmail.com> wrote:
> I have been using the native extension for Gradle inside an actual project > for a couple months. The biggest limitation I'm seeing is the inflexibility > of the compile task when compared to other build tool in the industry. The > --parallel switch works great for concurrent execution of tasks. When it > comes to the compile task, all files are compiled one after the other. The > native compiler are quite slow especially when it comes to compiling C++ > templates. As a comparison, my dev box is mostly idle while compiling with > Gradle as oppose to a fairly important load when compiling Visual Studio. > > The main use case of this feature is the speed up of the compilation process > for the native extension. To highlight how this feature is important, I will > point out that some company where I previously worked at use system like > Incredibuild to perform parallel distributed compilation. Even with such > system, the compilation was still pretty time consuming. In it's present > form, Gradle is not suitable in term of speed for those scenario. I talked to > a couple Gradleware engineers during Gradle Summit 2014 and some insane > features are planed to address this problem. Unfortunately, a quicker > solution is needed in order to speed up the adoption of Gradle as a native > build tool. I also want this feature to be in accordance to the long term > Gradle road map. > > I would like to start the discussion for contributing this into Gradle. That would be great. > From my limiting knowledge of Gradle here are a couple open issues I have. I > hope some brilliant minds from Gradleware can share there wisdom on where to > move forward with this feature. > > Open issues > - What is the current road map for such feature. I think there are 3 potential steps we could take: 1. The compilation tasks do something specific, where they fork n concurrent compilations, and n is just some setting on the compilation tasks. 2. Then, we introduce some general service that tasks can use for coordinating concurrent work. This would be integrated with —parallel-threads. The compilation tasks, the test tasks, and the task executor would all use this service to ensure that an appropriate amount of concurrent work happens at any given time. This would be a public service that any task implementation could use. 3. Then later, we add more capabilities to this so that the work can treated more like tasks - with their own up-to-date checks, dependencies, and so on. We could start with #1 and later extract #2, or we could jump straight to #2. In some ways, it might be nice to start with #2. > - This could also be used in any language which require compilation such > as Java. > - Allowing custom implementation of this feature could allow a company to > plug Gradle in there current Incredibuild infrastructure or any other > distributed framework. > - How this feature fits with the --parallel flag? For every build, there should be two settings that you can tweak: 1. The maximum amount of parallel work that can be performed by Gradle. 2. Whether or not tasks should be executed in parallel. That is, Gradle will be able to do stuff in parallel even if the tasks aren’t executed concurrently. It already does this with test execution. I would change —parallel-threads to control the maximum work but to not enable parallel task execution. It would default to some ‘reasonable’ value - the number of cores, say, at least to start with. > - How the number of parallel compilation unit will be configure aka number > of files that can be compiled in parallel? As above. > - Should this feature be always on by default or have a toggle flag? > Always on, I think. Why would you turn it off? -- Adam Murdoch Gradle Co-founder http://www.gradle.org CTO Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com
