*"clean build" with no changes = 3h1m53.88s* Top 15 tasks (roughly the same for each project) :projectA:preDexRelease 1m26.29s :projectA:preDexDebug 1m25.14s :projectA:processDebugManifest 18.452s :projectA:processReleaseManifest 17.705s :projectA:lint 10.699s :projectA:mergeReleaseResources 8.264s :projectA:dexDebug 7.742s :projectA:dexRelease 7.449s :projectA:mergeDebugResources 4.303s :projectA:compileDebugJava 1.672s :projectA:processReleaseResources 1.662s :projectA:processDebugResources 1.325s :projectA:compileReleaseJava 1.219s :projectA:packageDebug 1.179s :projectA:packageRelease 0.923s
*clean assembleDebug with pre-dex turned off = 58m58.29s* Top 15 tasks (roughly the same for each project) :projectA:dexDebug 56.567s :projectA:mergeDebugResources 5.135s :projectA:processDebugManifest 3.517s :projectA:compileDebugJava 1.579s :projectA:packageDebug 1.174s :projectA:processDebugResources 0.972s :projectA:processDebugJavaRes 0.373s UP-TO-DATE :projectA:prepareLibrary1UnspecifiedLibrary 0.340s :projectA:prepareLibrary2UnspecifiedLibrary 0.218s :projectA:compileDebugAidl 0.057s :projectA:compileDebugRenderscript 0.046s :projectA:prepareLibrary3UnspecifiedLibrary 0.045s :projectA:prepareLibrary4UnspecifiedLibrary 0.043s :projectA:prepareLibrary5UnspecifiedLibrary 0.029s :projectA:prepareLibrary6UnspecifiedLibrary 0.028s I can not find my data for the parallel run. I will do another one of those runs and post the data. Let me know if this is helpful for you. Thanks On Thursday, January 16, 2014 1:46:45 PM UTC-6, Ashton Cummings wrote: > > I have been using --profile to figure out the bottlenecks. However, due to > the nature of our project, i can not share that information. =/ > > I can give you times and stuff, just not project names and details. So, > unless i modify the data substantially, i can not share the profile info. > Sorry. > > However, i will give you the build time improvements and where the > bottlenecks are. If you need any specifics, let me know. I will try and get > you details that i can. > > I will get the info to you as soon as i can. I have so much profile data > now, i am going to have to run through it and see if i can find the correct > data. Might just do the runs again to make sure to get you the correct data. > > On Thursday, January 16, 2014 12:21:46 PM UTC-6, Xavier Ducrohet wrote: >> >> Do you remember the time gain you had when disabling pre-dexing? >> >> If you wouldn't mind. I'd love to get the output of running gradle with >> --profile. It will create a report in the build folder of the main project >> which will show me build time for all projects and all tasks in all >> projects. >> >> >> On Thu, Jan 16, 2014 at 10:06 AM, Ashton Cummings <[email protected]>wrote: >> >>> I hope that we can. Our alternatives that i am testing are Maven and >>> ANT. Right now our maven approach is showing times that are comparable to >>> our ANT. I personally want to stick with gradle, however, the decision is >>> not up to me. I just give the higher ups my results and conclusions. =) >>> >>> I will keep an eye out for those performance improvements and will keep >>> our people informed. >>> >>> Thank you for your time and assistance. If you figure out anything else >>> or have any other suggestions, please continue to contact me. I really hope >>> to convince them to move to gradle. >>> >>> >>> On Thursday, January 16, 2014 11:13:32 AM UTC-6, Xavier Ducrohet wrote: >>> >>>> BTW, we are still a ways off from 1.0. I totally understand not wanting >>>> to switch to Gradle right now (especially given your build times), but if >>>> you decide to not switch right now, please don't plan to stick with Ant >>>> for >>>> 2+ years (or some other random long duration). >>>> The Gradle build system will see a lot of changes in the next 6 months. >>>> I would recommend revisiting it mid-year at least :) >>>> >>>> >>>> On Thu, Jan 16, 2014 at 9:10 AM, Xavier Ducrohet <[email protected]>wrote: >>>> >>>>> http://tools.android.com/tech-docs/new-build-system contains our >>>>> changelog. >>>>> >>>>> We generally don't include Gradle specific changes in it even when we >>>>> start requiring newer versions of Gradle, but performance fixes will >>>>> probably be indicated in it because we know it's a pain point. >>>>> >>>>> >>>>> On Thu, Jan 16, 2014 at 8:54 AM, Ashton Cummings >>>>> <[email protected]>wrote: >>>>> >>>>>> Xavier, >>>>>> >>>>>> Thanks for the information. We are coming to a point now where the >>>>>> decision to switch our build process is coming to an end. We would >>>>>> really >>>>>> like to move to gradle, hopefully you guys are able to get these >>>>>> optimizations in soon. Once we make the decision, we are going to be >>>>>> stuck >>>>>> with it for a while. >>>>>> >>>>>> Where can i keep up to date with the exact changes that are going >>>>>> into each build of the gradle plugin? I need to know when these changes >>>>>> are >>>>>> in place as soon as possible. >>>>>> >>>>>> Thanks >>>>>> >>>>>> >>>>>> On Thursday, January 16, 2014 10:48:52 AM UTC-6, Xavier Ducrohet >>>>>> wrote: >>>>>> >>>>>>> Right now each project will pre-dex its dependencies on its own. >>>>>>> This means 2 components depending on the same library will both run >>>>>>> pre-dex >>>>>>> on that library's classes.jar which is silly. We're looking at fixing >>>>>>> this. >>>>>>> >>>>>>> Another thing we are looking at is parallelization at the task >>>>>>> level. Right now Gradle only supports it at the projects level. It'll >>>>>>> build >>>>>>> 2 sub projects in parallel (if there are no dependencies between the >>>>>>> two), >>>>>>> but in a single project it won't parallelize its task. We are working >>>>>>> with >>>>>>> the Gradle developers to change this. >>>>>>> >>>>>>> Of course the tasks themselves can do their work in parallel, and >>>>>>> most of our tasks do already. Pre-dexing doesn't though, so if a >>>>>>> project >>>>>>> has more than one dependencies it'll run pre-dex one by one. This is >>>>>>> fixed >>>>>>> in the next Gradle version. >>>>>>> >>>>>>> We do take performance very seriously, but we also wanted to provide >>>>>>> a lot of features and flexibility in our build. This means we do some >>>>>>> extra >>>>>>> work. For instance our resource merger is extra work that aapt does in >>>>>>> a >>>>>>> way but differently (more efficiently in some ways, less in others, but >>>>>>> with much less flexibility). We are working to do more to optimize each >>>>>>> task and, as I mentioned above, provide generic performance >>>>>>> improvements as >>>>>>> well. >>>>>>> >>>>>>> I'm a bit surprised your build went from 1:20 to only 0:58 when >>>>>>> running assembleDebug vs build. 'build' does both debug and release but >>>>>>> also runs lint. It should be a lot more work than just assembleDebug. >>>>>>> Also, >>>>>>> do you have flavors? >>>>>>> >>>>>>> One other thing to consider is incremental build. Pre-dexing for >>>>>>> instance makes clean build clearly slower but incremental builds >>>>>>> faster. >>>>>>> The rest of the build is also very good at only doing what needs to be >>>>>>> done >>>>>>> and nothing extra, which Ant wasn't very good at. >>>>>>> >>>>>>> Disabling pre-dexing on all projects without touching each project's >>>>>>> build.gradle doesn't seem to be easy. The issue is that doing (in the >>>>>>> main >>>>>>> build.gradle): >>>>>>> >>>>>>> subprojects { >>>>>>> /// some config >>>>>>> } >>>>>>> >>>>>>> this gets applied before all the other projects' own build.gradle >>>>>>> are applied. So unless you can apply the 'android' or 'android-library' >>>>>>> in >>>>>>> there, you won't be able to use our DSL. I need to look into this. >>>>>>> >>>>>>> >>>>>>> On Thu, Jan 16, 2014 at 6:51 AM, Ashton Cummings <[email protected] >>>>>>> > wrote: >>>>>>> >>>>>>>> My Stack post <http://stackoverflow.com/q/21125302/427763> >>>>>>>> >>>>>>>> Using gradle 1.9-rc-3 >>>>>>>> When building with gradle on a multi-project setup containing >>>>>>>> roughly 140 projects/libraries, the build time took 1 hour and 22 >>>>>>>> minutes. >>>>>>>> And i was using "--parallel". And our ANT build takes less than 20 >>>>>>>> minutes without parallel building. >>>>>>>> >>>>>>>> Here is what i was initially doing: >>>>>>>> >>>>>>>> ./gradlew clean >>>>>>>>> ./gradlew build --parallel >>>>>>>>> >>>>>>>> >>>>>>>> This took 1 hour and 22 minutes, roughly. I did a little testing it >>>>>>>> seems like the dexing is taking the longest amount of time. Is there a >>>>>>>> way >>>>>>>> to get the gradle process to re-use the stuff it has already dexed? If >>>>>>>> the >>>>>>>> libraries have already been built, it should re-use the already dexed >>>>>>>> libraries. >>>>>>>> >>>>>>>> I saw the option --no-rebuild, but when i run with that option it >>>>>>>> says the following: >>>>>>>> >>>>>>>>> File '/path/to/project/build/libs/project.aar' specified for property >>>>>>>>> 'bundle' does not exist. >>>>>>>>> >>>>>>>>> I replaced the path and the project name with generic stuff. >>>>>>>> >>>>>>>> *** >>>>>>>> >>>>>>>> After some assistance i added "preDexLibraries = false" to all of >>>>>>>> my "build.gradle" files. However, i still would like to know a >>>>>>>> centralize >>>>>>>> place that i can put that entry and it affect all the other >>>>>>>> "build.gradle" >>>>>>>> files. Because having to add that to all of the "build.gradle" files >>>>>>>> is >>>>>>>> annoying and if i ever have to remove it, it will be more annoying. >>>>>>>> This >>>>>>>> did not reduce the time to what we are needing. >>>>>>>> >>>>>>>> After some more assistance, i ran "assembleDebug" instead of >>>>>>>> "build". This way it will only do debug builds instead of both release >>>>>>>> and >>>>>>>> debug. This still did not bring us down to the time we are getting >>>>>>>> with >>>>>>>> ANT. With all these changes we are only down to 58 minutes. And our >>>>>>>> ANT >>>>>>>> process is less than 20 minutes. Our ANT process is what android >>>>>>>> provides >>>>>>>> only modified to support large multi-project setups and to re-use >>>>>>>> dexed >>>>>>>> libraries. The dex process in the gradle process is taking 1+ minute >>>>>>>> for >>>>>>>> each project. It is by far the largest time consumer. >>>>>>>> >>>>>>>> *** >>>>>>>> >>>>>>>> So, how can we speed the process up more? Is gradle re-using the >>>>>>>> dexed libraries after they have been dexed? What are your suggestions? >>>>>>>> For >>>>>>>> us to move over to gradle, it has to be able to compete with the ANT >>>>>>>> process. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "adt-dev" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> >>>>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Xavier Ducrohet >>>>>>> Android SDK Tech Lead >>>>>>> Google Inc. >>>>>>> http://developer.android.com | http://tools.android.com >>>>>>> >>>>>>> Please do not send me questions directly. Thanks! >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "adt-dev" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Xavier Ducrohet >>>>> Android SDK Tech Lead >>>>> Google Inc. >>>>> http://developer.android.com | http://tools.android.com >>>>> >>>>> Please do not send me questions directly. Thanks! >>>>> >>>> >>>> >>>> >>>> -- >>>> Xavier Ducrohet >>>> Android SDK Tech Lead >>>> Google Inc. >>>> http://developer.android.com | http://tools.android.com >>>> >>>> Please do not send me questions directly. Thanks! >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "adt-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >> >> >> -- >> Xavier Ducrohet >> Android SDK Tech Lead >> Google Inc. >> http://developer.android.com | http://tools.android.com >> >> Please do not send me questions directly. Thanks! >> > -- You received this message because you are subscribed to the Google Groups "adt-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
