+1 also minor bump - Carlos @csantanapr
> On Apr 27, 2016, at 5:24 PM, Jason Ginchereau <jason...@microsoft.com> wrote: > > OK, I think there is a solution that will make everyone happy, while > maintaining compatibility with existing after_prepare hooks: An incremental > prepare can copy both files that are newer AND FILES THAT ARE OLDER than the > target files in the platform's www directory, while skipping any files which > have an identical last-modified time. That way, any after_prepare hook that > modifies a prepared file will still see the file refreshed on every prepare, > as it does today, while all the unchanged files will be skipped, greatly > speeding up the prepare. > > With this approach, I think there is no need to put the incremental prepare > behind a --incremental option; it can just be the default behavior. (I am > still adding prepared files to cordova clean also.) And since this shouldn't > break anything, it won't require a major version bump. But I suppose it > should be a minor version bump, since it's new functionality. > > Jason > > > -----Original Message----- > From: Jason Ginchereau [mailto:jason...@microsoft.com] > Sent: Tuesday, April 26, 2016 9:39 AM > To: dev@cordova.apache.org > Subject: RE: [DISCUSS] Faster incremental builds > > I don't understand how adding a file list to the hook parameters will help > solve this problem. Any hook that currently fails after an incremental > prepare would need to be updated either way. It would be simpler to just > update it to correctly handle the case when a file wasn't updated by prepare. > > We still don't have any real examples of hooks where this is a problem, do > we? But it still seems safest to save this change for a major version update, > unfortunately. > > Jason > > > -----Original Message----- > From: Carlos Santana [mailto:csantan...@gmail.com] > Sent: Friday, April 22, 2016 5:03 PM > To: dev@cordova.apache.org > Subject: Re: [DISCUSS] Faster incremental builds > > Why your saying is more complicated? > Complicated for hook authors? > for the hooks that don't need they don't have to do anything just ignore the > argument in context. > > "cordova prepare --clean" == "cordova clean && cordova prepare" > > > > - Carlos > @csantanapr > >> On Apr 22, 2016, at 7:45 PM, Jason Ginchereau <jason...@microsoft.com> wrote: >> >> I don't think making the after_prepare hook API more complicated is a good >> design. Instead can we just document that after_prepare hooks should be >> designed to handle the files being copied only sometimes? Most of them >> probably are anyway, because Cordova never made an explicit guarantee about >> the files being fresh every time the hook is called. Anyway I guess it could >> still be considered a breaking change so maybe it should trigger a >> major-version bump? >> >> I don't understand what you are suggesting with "cordova prepare --clean". >> >> -----Original Message----- >> From: Carlos Santana [mailto:csantan...@gmail.com] >> Sent: Thursday, April 21, 2016 17:16 >> To: dev@cordova.apache.org >> Subject: Re: [DISCUSS] Faster incremental builds >> >> I guess it's fine to have default to be faster, we just need to pass the >> files to be copied to feed the hook to be aware. It's kind of a break API on >> CLI so maybe CLI 7.x but not sure. >> >> Also for folks that want the conversation approach with one command we can >> also have a flag --clean "cordova prepare --clean" >> >> >> - Carlos >> @csantanapr >> >>> On Apr 21, 2016, at 7:42 PM, Jason Ginchereau <jason...@microsoft.com> >>> wrote: >>> >>> Thanks Carlos, that's a good point. >>> >>> The Ionic add_platform_class hook >>> (https://github.com/driftyco/ionic-package-hooks) actually would be fine; >>> it doesn't do anything if the CSS class to be added is already there. >>> >>> But I guess it's possible some other after_prepare hooks might not be as >>> smart. I'm not sure what to do about that. >>> >>> >>> -----Original Message----- >>> From: Carlos Santana [mailto:csantan...@gmail.com] >>> Sent: Thursday, April 21, 2016 2:25 PM >>> To: dev@cordova.apache.org >>> Subject: Re: [DISCUSS] Faster incremental builds >>> >>> The only thing to watch out is folks modified files during after_prepare >>> event, then they will be broken if the file have not changed and they >>> modified/add content to the file a second time. >>> >>> To make this work the hook event will need to inform batch/node and plugins >>> hooks on before_prepare and after_prepare the list of files that are going >>> to be copy, so the hook owner can know not to modified the file if their >>> file is not I the list. >>> >>> One real use case is Ionic and I think MobileFirst. >>> For MobileForst I think we don't modified I think we just recreate >>> new file ok every prepare >>> >>> For Ionic they edit the index.html to a class to to the body element >>> to for platform identification >>> >>> - Carlos >>> @csantanapr >>> >>>> On Apr 21, 2016, at 5:12 PM, Jason Ginchereau <jason...@microsoft.com> >>>> wrote: >>>> >>>> If "cordova clean" would also take care of deleting the files copied by >>>> prepare, then I'd be confident in making prepare incremental by default. >>>> Then if anyone was ever concerned about whether the incremental prepare >>>> was doing the right thing, "cordova clean" would be a fairly obvious way >>>> to reset the build. I still can't think of a realistic developer scenario >>>> where that would be necessary, but it seems like the right thing for a >>>> "clean" command to do regardless. >>>> >>>> So I will work on enhancing the "clean" function for each platform at the >>>> same time I enable the incremental prepare. >>>> >>>> Jason >>>> >>>> -----Original Message----- >>>> From: Jason Ginchereau [mailto:jason...@microsoft.com] >>>> Sent: Wednesday, April 20, 2016 10:00 AM >>>> To: dev@cordova.apache.org >>>> Subject: RE: [DISCUSS] Faster incremental builds >>>> >>>> My concern with making this the default behavior is that Cordova >>>> prepare doesn't have a corresponding "clean" operation. While there >>>> is a "cordova clean" command, it only does a clean for the >>>> native-code project (e.g. gradle clean), which doesn't affect the >>>> files copied during prepare. I guess this would only be a problem if >>>> you did something to give your source files older last-modified >>>> times... then a cordova prepare/build/run that was incremental would >>>> not pick up the older files, and "cordova clean" wouldn't fix it. I >>>> was thinking a git checkout could do that, but upon investigation it >>>> actually doesn't reset files' last-modified times. (I guess any >>>> source control tools shouldn't, as it would mess up any kind of >>>> incremental build system.) >>>> >>>> Maybe "cordova clean" should also delete the files that are copied by >>>> prepare? Or maybe there could be something like a "cordova prepare >>>> --force" flag for the rare case when a non-incremental prepare is needed? >>>> Between those two I'd prefer making "cordova clean" more thorough; I >>>> didn't think of that until now. >>>> >>>> Jason >>>> >>>> -----Original Message----- >>>> From: Steven Gill [mailto:stevengil...@gmail.com] >>>> Sent: Wednesday, April 20, 2016 9:18 AM >>>> To: dev@cordova.apache.org >>>> Subject: Re: [DISCUSS] Faster incremental builds >>>> >>>> Thanks for sharing the stats! Looks like a great improvement! >>>> >>>> I'm fine with this going in sans flag if we test it bit more first. >>>> >>>> Have your tried it with the --browserify flag? >>>>> On Apr 20, 2016 7:18 AM, "Victor Sosa" <sosah.vic...@gmail.com> wrote: >>>>> >>>>> This is an amazing improvement in Cordova performance. Thanks Jason >>>>> +1!! >>>>> >>>>>> On Wed, Apr 20, 2016 at 9:14 AM Shazron <shaz...@gmail.com> wrote: >>>>>> >>>>>> +1 what Jesse said >>>>>> >>>>>>> On Tuesday, April 19, 2016, Jesse <purplecabb...@gmail.com> wrote: >>>>>>> >>>>>>> This sounds great! >>>>>>> Personally, I don't think this belongs behind a flag, why would >>>>>>> anyone >>>>>> not >>>>>>> want to build faster? >>>>>>> I think we just need to test more to make sure it functions as >>>>>>> expected everywhere, and then it should just make it's way in >>>>>>> directly, without >>>>>> the >>>>>>> overhead of addition flag code, or documentation ... >>>>>>> >>>>>>> >>>>>>> @purplecabbage >>>>>>> risingj.com >>>>>>> >>>>>>> On Tue, Apr 19, 2016 at 5:58 PM, Jason Ginchereau < >>>>>> jason...@microsoft.com >>>>>>> <javascript:;>> >>>>>>> wrote: >>>>>>> >>>>>>>> I collected data for 4 projects with varying numbers and sizes >>>>>>>> of >>>>>> source >>>>>>>> files (mostly .js and .png): >>>>>>>> Tiny : 17 files, 0.24 MB (A new Cordova project created from >>>>>> the >>>>>>>> VS template) >>>>>>>> Small : 55 files, 7.7 MB >>>>>>>> Medium: 242 files, 44 MB >>>>>>>> Large : 12,923 files, 134 MB >>>>>>>> >>>>>>>> Following are full -> incremental times for 'cordova prepare android' >>>>>>>> commands, without any source file changes in between, averaged >>>>>>>> over 5 >>>>>>> warm >>>>>>>> runs on my medium-spec Windows desktop machine with SSD: >>>>>>>> >>>>>>>> Tiny : 1.98 -> 1.84 s >>>>>>>> Small : 2.40 -> 1.94 s >>>>>>>> Medium: 4.22 -> 2.02 s >>>>>>>> Large : 50.24 -> 5.81 s >>>>>>>> >>>>>>>> As you can see, larger projects see greatly improved prepare >>>>>>>> times >>>>> when >>>>>>>> using the incremental option. >>>>>>>> >>>>>>>> Also, I confirmed that without using the --incremental option >>>>>>>> with >>>>> this >>>>>>>> change, times are not measurably different compared to the >>>>>>>> released >>>>>>> version >>>>>>>> 6.1.1 of Cordova. So there doesn't seem to be any regression >>>>>>>> compared >>>>>> to >>>>>>>> the previous use of shelljs.cp('-R') to blindly copy files when >>>>>>> preparing. >>>>>>>> >>>>>>>> Jason >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Steven Gill [mailto:stevengil...@gmail.com <javascript:;>] >>>>>>>> Sent: Tuesday, April 19, 2016 3:14 PM >>>>>>>> To: dev@cordova.apache.org <javascript:;> >>>>>>>> Subject: Re: [DISCUSS] Faster incremental builds >>>>>>>> >>>>>>>> Sounds like a worthy cause. Do you have any stats on how much >>>>>>>> time is saved? >>>>>>>> >>>>>>>> Definitely put it behind the --incremental flag to start. >>>>>>>> >>>>>>>> On Tue, Apr 19, 2016 at 2:43 PM, Jason Ginchereau < >>>>>>> jason...@microsoft.com <javascript:;>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> We've had a few customers complain that the dev inner loop for >>>>>> Cordova >>>>>>>>> apps is slow compared to native app development. So recently >>>>>>>>> I've >>>>>> been >>>>>>>>> looking at ways to optimize it. The two largest pieces of a >>>>>>>>> Cordova build are "prepare" and "compile" phases. While there's >>>>>>>>> not much we can realistically do to speed up the native-code >>>>>>>>> compilation (in >>>>>> which >>>>>>>>> gradle/xcodebuild/etc is invoked), it's clear that the prepare >>>>> phase >>>>>>>>> is not nearly as efficient as it could be. So I opened a JIRA >>>>>>>>> for that, and I have a solution in the works that I want to get >>>>> feedback >>>>>>> on. >>>>>>>>> >>>>>>>>> CB-11117: Preparing platforms should skip copying files which >>>>> haven't >>>>>>>>> changed< >>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f >>>>>>>>> %2fissues.apache.org >>>>>> %2fjira%2fbrowse%2fCB-11117&data=01%7c01%7cjasongi >>>>>>>>> n%40microsoft.com >>>>>> %7c297f714a60a94c3f258108d3689ff306%7c72f988bf86f141a >>>>>> f91ab2d7cd011db47%7c1&sdata=E2Ww9LusEmVkUgjXPPGnxcGz%2fMH8fWruqGWU >>>>>> r >>>>>> L >>>>>> Xq >>>>>>>>> KRk%3d> Many cordova CLI commands include a "prepare" >>>>>>>>> operation, including 'cordova build', 'cordova run', 'cordova >>>>>>>>> plugin add', and more. Every time each of those commands runs, >>>>>>>>> the target platform >>>>> is >>>>>>>>> "prepared", which involves copying all files from >>>>>>>>> [<project>/www, <project>/platforms/<platform>/platform_www, >>>>>>>>> <project>/merges/<platform>] to the platform's target www >>>>>>>>> folder, >>>>> as >>>>>>>>> well as copying a bunch of icons and splash screens to >>>>>>> platform-specific >>>>>>>> locations. >>>>>>>>> >>>>>>>>> For the very first prepare of a platform, all that file copying >>>>>>>>> is necessary. But most of the time after that most of the files >>>>>>>>> being copied have not changed and therefore don't really need >>>>>>>>> to be >>>>> copied >>>>>>>>> again. So the typical developer inner loop (edit a few source >>>>> files, >>>>>>>>> build and run the app, repeat) is a lot slower than it could be >>>>> for a >>>>>>>>> Cordova project, especially one that includes a significant >>>>>>>>> number >>>>> of >>>>>>>>> source files or resources. >>>>>>>>> >>>>>>>>> Instead, Cordova should be smart enough to skip copying of >>>>>>>>> files >>>>> that >>>>>>>>> haven't changed, based on their last-modified timestamp. But >>>>>>>>> there should still be a way to force a >>>>>>>>> clean/full/non-incremental copy if desired. To preserve >>>>>>>>> compatibility with all possible existing workflows, I'm leaning >>>>>>>>> toward keeping the full copy as the default behavior, and >>>>>>>>> enabling the optimization only with a new >>>>> --incremental >>>>>>>>> flag, at least initially. If we find the incremental option is >>>>>> working >>>>>>>>> well for everyone, we can promote that behavior to default in a >>>>>> future >>>>>>>> major version update. >>>>>>>>> >>>>>>>>> See PRs here for implementation of this idea... >>>>>>>>> https://github.com/apache/cordova-lib/pull/429: Add FileUpdater >>>>>> module >>>>>>>>> to cordova-common >>>>>>>>> https://github.com/apache/cordova-android/pull/295: Use >>>>> FileUpdater >>>>>> to >>>>>>>>> optimize prepare for android platform I'll submit PRs for >>>>> cordova-ios >>>>>>>>> and cordova-windows platforms soon, as well as a cordova-cli >>>>>>>>> change >>>>>> to >>>>>>>>> enable the optimization via a new --incremental flag (assuming >>>>>>>>> we don't want to make it the default behavior). >>>>>>>>> >>>>>>>>> Jason >>>> >>>> -------------------------------------------------------------------- >>>> - To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org >>>> For additional commands, e-mail: dev-h...@cordova.apache.org >>>> ÃÂâÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂà>>>> ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂà>>>> à >>>> ÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂà >>>> ÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂà >>>>  >>>> à>>>> ÃÂÃÂÃÂ¥Fò >>>> Vç7V'67&ÃÂ&RÃÂRÃÂÃÂÃÂÃÂâFWbÃÂVç7V'67&ÃÂ& >>>> T6÷&F÷fæ6ÃÂRæà >>>> ·&pÃÂäf÷"FFÃÂFÃÂöæ >>>> ÃÂ6öÃÂÃÂæG2ÃÂRÃÂÃÂÃÂÃÂâFWbÃÂÃÂVà>>>> 6÷&F÷fæ6ÃÂRæ÷&pà>>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org >>> For additional commands, e-mail: dev-h...@cordova.apache.org >>> >>> BÂKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK >>> KK >>> KCBÂà [ÂÃXÂÃÃÂXÂKK[XZ[ >> Â]Â][ÂÃXÂÃÃÂXÂPÃÃÂÃÂKÂ\XÃKÂÃÂÃBÂÂÃÂY >>> ][ÃÂ[ÃÃ[X[ ÃK[XZ[ >> Â]ÂZ[ ÃÃÂÃÂKÂ\XÃKÂÃÂÃB >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org >> For additional commands, e-mail: dev-h...@cordova.apache.org >> >> BKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK >> KCBÈ [ÝXØÜXKK[XZ[ > ]][ÝXØÜXPÛÜÝK\XÚKÜÃBÜY >> ][Û[ÛÛ[X[ ËK[XZ[ > ]Z[ ÛÜÝK\XÚKÜÃB > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org > For additional commands, e-mail: dev-h...@cordova.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org > For additional commands, e-mail: dev-h...@cordova.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org For additional commands, e-mail: dev-h...@cordova.apache.org
