On 12/05/2013, at 3:03 AM, Kyle Mahan <kyle.ma...@gmail.com> wrote:
> Thank you for the response Luke. I read the doc and did a little more
> investigation on my use case, and I see that the problem is more complicated
> than I'd thought. For example, my plan to naively skip duplicated
> META-INF/LICENSE files would have violated the license terms of those
> libraries... For some file types it may be appropriate to rename duplicates
> (eg, LICENSE-apache-activemq-5.8.0.jar, LICENSE-google-guava-1.0.14.jar) but
> perhaps that sort of thing is a better job for a function in a build script
> rather than something baked into the tool itself.
>
> In any case, the real issue for me was apparently that jarsigner won't sign
> jars with duplicate entries, and it was better not to use a single monolithic
> jar anyway. I'm still happy to help if I can, but I think I will take a look
> at the list of issues for community newbs :)
The problem isn't quite as complicated as the spec would have you believe. Or,
more precisely, there are two problems here:
1. Add some mechanism to allow you to deal with duplicates in copy and archive
tasks.
2. Change the default behaviours in various places to work better, so that you
need to do this less often.
In other words, make it possible, then make it better.
Most of the complexity around this issue is in #2, not in #1. Furthermore, we
can't change the defaults until Gradle 2.0, so the whole of #2 can pretty much
be ignored for now (in the sense that we don't need to solve #2 before we start
on #1).
As far as #1 goes, we can chop this up into a couple of bite-sized pieces for
you to implement, if you're still interested in helping out.
One issue we need to solve is that different policies need to be applied to
different types of files in the archive/copy output. I would deal with this by
adding some stuff to `FileCopyDetails`, to be used with `eachFile { … }`.
We can start with a `duplicatesStrategy` option. Values would be something
like `copy`, `exclude`, `fail`, `warn`. Given this, you can pretty much
implement whatever policy you like:
tar.eachFile { details -> details.duplicatesStrategy = 'fail' }
zip.eachFile { details ->
details.duplicatesStrategy =
details.path.startsWith('/META-INF/services') ? 'copy' : 'fail'
}
On top of this, I would also add a couple of conveniences:
- A `duplicatesStrategy` on CopySpec, which is used as the default for all
files in the copy spec. Given that the copy tasks all implement CopySpec, this
would also be available at the task level:
tar.duplicatesStrategy = 'fail'
zip {
duplicatesStrategy = 'fail'
eachFile { details ->
if (details.path.startsWith('/META-INF/services')) {
duplicatesStrategy = 'copy' }
}
}
- A simpler way of using `eachFile` over a set of files, perhaps something like
this:
zip.matching('/META-INF/services/**') { duplicatesStrategy = 'copy' }
zip.notMatching('/META-INF/services/**') { duplicatesStrategy = 'fail' }
This will also make it easier to do other fine-grained tweaks to each file,
such as filtering content for certain files, or defining the input and output
character encodings to use, or the unix file permissions, and so on.
(I'm sure we can come up with better names for the 'matching' and 'notMatching'
methods).
Let us know if you're still interested in working on this and we can update the
spec to add in some more details. I would love to see this implemented, and I
know there are many, many people who would really appreciate having this
feature.
>
> Thanks,
> Kyle
>
>
> On Fri, May 10, 2013 at 2:04 AM, Luke Daley <luke.da...@gradleware.com> wrote:
>
> On 10/05/2013, at 8:26 AM, Kyle Mahan <kyle.ma...@gmail.com> wrote:
>
> > I'm interested in helping out on this issue if possible:
> > http://issues.gradle.org/browse/GRADLE-2171 ("Add an option to avoid
> > duplicate entries when creating a zip file"). This comes up assembling a
> > fat jar containing all of an application's dependencies smushed together in
> > one jar (a lot of jars tend to have eg, META-INF/LICENSE).
> >
> > From what I could tell, it was still an open question whether to support
> > the Ant configuration options "add"/"preserve"/"fail", and what the default
> > should be. Ant defaults to "preserve", which will put multiple files with
> > the same name in the zip. I assume the vast majority (all?) of cases would
> > want one of the other two behaviors: either a) skip duplicates or b) throw
> > an exception. Personally I'd be ok with (a) only.
> >
> > In case it helps the discussion, here's my "sketch" of a feature to skip
> > duplicates.
> >
> > https://github.com/kylewm/gradle/commit/34bfcd06e16fc3124627668d69ea1533d5f1c048
>
> I've not really been across this issue, but we do have the start of a spec
> for this:
> https://github.com/gradle/gradle/blob/master/design-docs/duplicate-entries-in-archives.md
>
> Thanks for your interest. Hopefully this gets the ball rolling again.
>
> --
> Luke Daley
> Principal Engineer, Gradleware
> http://gradleware.com
>
> Join me at the Gradle Summit 2013, June 13th and 14th in Santa Clara, CA:
> http://www.gradlesummit.com
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
>
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
Join us at the Gradle Summit 2013, June 13th and 14th in Santa Clara, CA:
http://www.gradlesummit.com
