[
https://issues.apache.org/jira/browse/SENTRY-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Towles updated SENTRY-2074:
---------------------------------
Description:
Using the maven dependency plugin to analyze the dependency usage for each
module and put all directly used libraries in the pom. Clean-up the unused ones
and adjust the scope of libraries only used for tests and provided libraries
for plugins.
The one of the primary motivations for this patch is to help on its way to
cleaning up the distribution. Currently the dist module reads all dependencies
no mater what scope they are and drops them into the distributed libs. This
causes things like junit and ant to be pushed into the libs that are being
distributed. With the changes to have direct dependencies always defined it
allows us to take compile and runtime scopes only into account when dropping
the libs needed.
As well this identifies which libraries are provided already by environments
where the plugins/bindings are going into. For example in the hive bindings,
the hive and hadoop libraries need only be defined with "provided" scope, since
with those application we want to use the hadoop and hive libraries that the
applications already provide.
This makes it a lot easier for shading and package shifting of the binding and
plugins for libraries and versions of those libraries that are needed by the
binding and might conflict with versions already in the application which the
binding or plugin is going into. Guava is a major issue with this. Doing this
short of shading based on the cleanup would allow us to rev Guava and use newer
Guava features while not conflicting with the Guava version the application is
using. By having the directly used dependency defined it gives us control over
the exact version we are using and not be dependent on and having conflicts
with the transitive dependencies of the application being embedded in.
This patch will not really make the development process harder since the
analysis of the dependencies needed automatically runs as part of the build and
a failure occurs telling you which "used but undefined" and which "defined but
unused" libraries are missing or in the pom. There is even an xml dump of the
dependencies part need to put right into the pom. No additional runs or
dependency analysis needs to take place.
was:Using the maven dependency plugin to analyze the dependency usage for
each module and put all directly used libraries in the pom. Clean-up the unused
ones and adjust the scope of libraries only used for tests and provided
libraries for plugins.
> Fix maven dependencies to have all directly used libraries defined
> ------------------------------------------------------------------
>
> Key: SENTRY-2074
> URL: https://issues.apache.org/jira/browse/SENTRY-2074
> Project: Sentry
> Issue Type: Improvement
> Components: Build
> Affects Versions: 2.0.0
> Reporter: Brian Towles
> Assignee: Brian Towles
> Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-2074.01.patch, SENTRY-2074.02.patch
>
>
> Using the maven dependency plugin to analyze the dependency usage for each
> module and put all directly used libraries in the pom. Clean-up the unused
> ones and adjust the scope of libraries only used for tests and provided
> libraries for plugins.
>
> The one of the primary motivations for this patch is to help on its way to
> cleaning up the distribution. Currently the dist module reads all
> dependencies no mater what scope they are and drops them into the distributed
> libs. This causes things like junit and ant to be pushed into the libs that
> are being distributed. With the changes to have direct dependencies always
> defined it allows us to take compile and runtime scopes only into account
> when dropping the libs needed.
> As well this identifies which libraries are provided already by environments
> where the plugins/bindings are going into. For example in the hive bindings,
> the hive and hadoop libraries need only be defined with "provided" scope,
> since with those application we want to use the hadoop and hive libraries
> that the applications already provide.
> This makes it a lot easier for shading and package shifting of the binding
> and plugins for libraries and versions of those libraries that are needed by
> the binding and might conflict with versions already in the application which
> the binding or plugin is going into. Guava is a major issue with this. Doing
> this short of shading based on the cleanup would allow us to rev Guava and
> use newer Guava features while not conflicting with the Guava version the
> application is using. By having the directly used dependency defined it gives
> us control over the exact version we are using and not be dependent on and
> having conflicts with the transitive dependencies of the application being
> embedded in.
> This patch will not really make the development process harder since the
> analysis of the dependencies needed automatically runs as part of the build
> and a failure occurs telling you which "used but undefined" and which
> "defined but unused" libraries are missing or in the pom. There is even an
> xml dump of the dependencies part need to put right into the pom. No
> additional runs or dependency analysis needs to take place.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
