2009/4/23 Steve Appling <[email protected]> > > > Tom Eyckmans wrote: > >> >> >> 2009/4/23 Steve Appling <[email protected] <mailto: >> [email protected]>> >> >> >> >> >> Tom Eyckmans wrote: >> >> Hi Steve, >> >> The previous message got rejected because of it's size so I have >> put the test jar here http://www.box.net/shared/rjfyath7lb hope >> that works. >> >> clipped ... >> >> >> True. Could you perform a speed test with the jar file in >> attachment? >> >> just execute it like so: >> >> java -jar changedetection-0.1 dirOne dirTwo dirThree >> Do you really need to include the file content, or >> can >> you trust >> the OS to update last modified date? >> Not sure, but I know there is a force method on randomaccessfile >> that allows to only force the content or both content and >> metadata to be flushed. Thats why I included the content because >> this allows the modified date not to be updated even when the >> content has changed. >> >> >> I know it is possible to do this, but unusual in normal development. >> >> >> >> Seems better to just use absolute file path + last >> modified date. >> >> >From our main product branch I ran it on our java source and >> webapp content. Performance seems to depend HEAVILY on what Windows >> has cached. >> >> First time: >> >> C:\Working\main\dev>java -jar \temp\changedetection-0.1.jar source >> webroot >> C:\Working\main\dev\source : e48543ab1d4bd97990d70f52000b9595d0c75a88 >> C:\Working\main\dev\webroot : 0b23e572ac55209733d2564058fc3e553dfa95c9 >> time taken: PT258.716S >> >> Second time: >> C:\Working\main\dev>java -jar \temp\changedetection-0.1.jar source >> webroot >> C:\Working\main\dev\source : e48543ab1d4bd97990d70f52000b9595d0c75a88 >> C:\Working\main\dev\webroot : 0b23e572ac55209733d2564058fc3e553dfa95c9 >> time taken: PT8.312S >> >> This is a big project (8,447 files under source and 15,670 files under >> webroot), but it still seems like including the content of the files is >> prohibitively expensive (at least at first). It would be >> interesting to see how this works without the file content. I'll be >> glad to test that for you too. >> >> I've noticed this aswell, very hard to create something that works >> consistently every single time. >> New jar that doesn't include the file content, on Gradle source this takes >> 0.2 seconds. Could you test this against your code base ? >> >> Link: >> http://www.box.net/shared/b53oc00c3r >> > > Much better - > > First time: 24.283S > Second time: 2.674S > > 24 seconds is acceptable to me for this size project for a first time test. > 2.6 seconds rocks for 24k+ files. I'm sure performance would be even > better under any OS other than windows :)
Cool, this is also just a single threaded test so I probably can squeeze this down some more. > > > I'm curious exactly what you plan to do with this. Copy tasks should > probably just do their own freshness check to avoid having to traverse the > directory structure twice if something does need to be done. I'm going to make the logic for change detection available in a generic way so that tasks can use it and only get called back for the files that have changed, whether the task effectively performs it's action when a callback happens or justs keeps a list for later processing is a task implementation detail. How the generic change detection I'm currently having the following in mind. SHA calculation would work bottom-up keeping the SHA of every file / directory. Directories would get the SHA calculated on the SHA of their contents + SHA of itself. Detecting the actual changes would work top top-down. > > > -- > Steve Appling > Automated Logic Research Team > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > >
