On 9 Sep 2013 13:52, "Adam Murdoch" <adam.murd...@gradleware.com> wrote: > > > On 10/09/2013, at 5:32 AM, Daz DeBoer <darrell.deb...@gradleware.com> wrote: > >> On Sun, Sep 8, 2013 at 3:34 PM, Adam Murdoch <adam.murd...@gradleware.com> wrote: >>> >>> >>> On 09/09/2013, at 5:21 AM, Daz DeBoer <darrell.deb...@gradleware.com> wrote: >>> >>>> G'day >>>> Since we're currently changing the way we manage locks in the Gradle cache, it would be great if we could separate the 'filestore' part of the cache (that hasn't changed in structure since 1.0) from the binary metadata part (which changes quite frequently). >>>> >>>> What this means is that we'd have these in separate directories: >>>> ~/.gradle/caches/metadata-1 >>>> ~/.gradle/caches/filestore-1 >>>> >>>> The 'filestore' would simply contain the content that is currently in caches/artifacts-n/filestore. >>>> >>>> The benefit of this change is that the actual downloaded files would not need to be copied into a new location every time we change the metadata file format. >>>> >>>> The only complication I see is that we'd need a separate (cross-version) lock for the filestore since it would be shared by different Gradle versions. We'd need to bump the filestore version whenever the locking mechanism changes, rather than every time the metadata store format changes. >>> >>> >>> I think having two locks would make it easy to deadlock when there are 2 processes (or threads) resolving concurrently. We could address this by enforcing a certain lock order (eg always lock the files if you need to lock the metadata). >>> >>> Another option would be to have a single lock for both the files and metadata, versioned on the locking protocol, something like: >>> >>> ~/.gradle/caches/files-1 >>> ~/.gradle/caches/files-1/store-1 >>> ~/.gradle/caches/files-1/metadata-1 >>> >> >> It could even look at bit more intentional if we used eg: >> >> caches/modules-1 >> caches/modules-1/artifacts-1.0 >> caches/modules-1/metadata-1.3 > > > This isn't really going to work. If a format change is backwards compatible, then we should not change the directory name, otherwise we have two sets of processes that are using different stores even though they can share them. So we wouldn't use the minor version in file names, only major versions. >
I don't really understand how my proposal is any different, except in the actual numbers used. When the lock file format changes, we need to create a new top level directory with new file store and metadata. When the file store or metadata formats change we bump that version only. The actual numbers uses are arbitrary. Not at all important, though. >> >> This would be a cosmetic difference, but I think it would look clearer to users. >> >> Either way, any chance we can slip this into the cache locking changes that are currently in progress? Or add it as a story to be done soon? > > > It's a separate story, but we can try to do it soon. I was waiting for the next meta-data format change. > > > -- > Adam Murdoch > Gradle Co-founder > http://www.gradle.org > VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting > http://www.gradleware.com > > Join us at the Gradle eXchange 2013, Oct 28th in London, UK: http://skillsmatter.com/event/java-jee/gradle-exchange-2013 > > >
