Hi,
We have seen an issue with different versions of
org.apache.httpcomponents:httpclient jar getting included in classpath when
apps are launched on newer versions of Hadoop. This is because we have an
explicit dependency in Apex to version 4.3.5, the version of hadoop we
build against (2.2 for b
Forgot to add that 4.3.5 is a later version that what newer versions of
hadoop are using (4.2.5) and we need it for kerberos SPNEGO security
related functionality.
Thanks
On Fri, Jul 8, 2016 at 11:28 AM, Pramod Immaneni
wrote:
> Hi,
>
> We have seen an issue with different versions of
> org.apa
+1 Flink and Spark shade a lot of their dependencies to avoid problems like
this, they both even shade guava. Don't see any down side to doing it.
On Fri, Jul 8, 2016 at 11:31 AM, Pramod Immaneni
wrote:
> Forgot to add that 4.3.5 is a later version that what newer versions of
> hadoop are using
+1 for the change. This would fix dependency conflicts between httpclient
4.2.5 and 4.3.5.
I think the question is whether we should publish a separate shaded
artifact (like we did for ning-ahc) for the shaded httpclient 4.3.5, or
just shading it internally in apex-engine. The former allows applica
Good question. Even though it would be useful for somebody wanting to use
SPNEGO in their operator for example I don't see any immediate use. We can
just go with it being internal and create a separate artifact in future if
need arises. What do you think?
Thanks
On Fri, Jul 8, 2016 at 11:40 AM, D
I'm good with shading it internally in apex-engine for now until there is
an actual need in a real use case. Thanks!
David
On Fri, Jul 8, 2016 at 4:42 PM, Pramod Immaneni
wrote:
> Good question. Even though it would be useful for somebody wanting to use
> SPNEGO in their operator for example I
Keeping shaded dependencies as separate artifacts simplifies the build and
allows others to use them as well. Why shade a dependency on every build
invocation when it never changes?
We could also consider building a single shaded artifact for multiple
dependencies in case we are adding more in the
We could go with a separate artifact although I think we should have a
stronger reason than builds to create one, such as the likelihood of others
needing it. If it were shaded internally, in every build the mapping would
still remain the same isn't it, so there would be no changes to things like
e
