Shading requires two pieces of information:
1) Which dependencies should be part of the shaded jar (controlled by
includes/excludes)
2) How to relocate code within those dependencies (controlled by
relocations)
The reason why the exclude(".*") exists is because typically it is an error
to produce a shaded package with dependencies which are not relocated. When
libraries do this, it causes a lot of NoClassFound/NoMethodFound errors for
users since a user can't know which version of a dependency they are
actually getting (the one that was bundled part of your jar or the one they
depend on as a library). Only applications should ever really do this,
libraries should always repackage all their code to prevent such errors.
Note that in the SQL package, you can provide your own shadowClosure to the
applyJavaNature() which means that the default won't apply. For example:
https://github.com/apache/beam/blob/a3ba6a0e8de3ae72b8fc6fc6038eb9dc725f092e/sdks/java/harness/build.gradle#L20
and remove the 'DEFAULT_SHADOW_CLOSURE <<'
On Mon, May 21, 2018 at 10:26 AM Andrew Pilloud <apill...@google.com> wrote:
> The issue SQL is seeing is caused by a default dependency of exclude(".*")
> added in build_rules.gradle. This breaks the normal method of building
> shadow jars as everything must be explicitly included. SQL explicitly added
> calcite to the jar, but not calcite's dependencies. I've been told this is
> the desired behavior as we want to ensure everything included is relocated.
>
> I don't know much about gradle, but this seems fragile. Is it possible to
> have all dependencies automatically relocated so we don't need the
> exclude(".*") rule?
>
> Andrew
>
> On Thu, May 17, 2018 at 7:41 PM Andrew Pilloud <apill...@google.com>
> wrote:
>
>> Yep, I added the issue as a blocker.
>> https://issues.apache.org/jira/projects/BEAM/issues/BEAM-4357
>>
>> On Thu, May 17, 2018, 6:05 PM Kenneth Knowles <k...@google.com> wrote:
>>
>>> This sounds like a release blocker. Can you add it to the list? (Assign
>>> fix version on jira)
>>>
>>> Kenn
>>>
>>> On Thu, May 17, 2018, 17:30 Lukasz Cwik <lc...@google.com> wrote:
>>>
>>>> Typically we have a test block which uses a configuration that has the
>>>> shadow/shadowTest configurations on the classpath instead of the
>>>> compile/testCompile configurations. The most common examples are validates
>>>> runner/integration tests for example:
>>>>
>>>> https://github.com/apache/beam/blob/0c5ebc449554a02cae5e4fd01afb07ecdb0bbaea/runners/direct-java/build.gradle#L84
>>>>
>>>> On Thu, May 17, 2018 at 3:59 PM Andrew Pilloud <apill...@google.com>
>>>> wrote:
>>>>
>>>>> I decided to try our new JDBC support with sqlline and discovered that
>>>>> our SQL shaded jar is completely broken. As
>>>>> in java.lang.NoClassDefFoundError all over the place. How are we testing
>>>>> the output jars from other beam packages? Is there an example I can follow
>>>>> to make our integration tests run against the release artifacts?
>>>>>
>>>>> Andrew
>>>>>
>>>>
