[
https://issues.apache.org/jira/browse/RANGER-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Clément Lucas updated RANGER-4309:
----------------------------------
Description:
To start simplifying the work of people who want to build the project using the
standard build process, we would like to propose an update of the README.md
with the minimum versions necessary for the building of Apache ranger with the
plugins and dependencies on them.
For the moment, 4 prerequisites would be indicated:
* JDK 1.8+ taken from pom.xml
* Apache Maven 3.6.3+ taken from pom.xml
* Hadoop 3.3.0+ taken from tests
* Hive 3.0.0+ taken from tests
h3. Hadoop 3.3.0+
The method *CheckPermissionWithContext* was not implemented before hadoop 3.3.0.
Here is the file in which this method is implemented since 3.3.0 and it's
possible to see that it wasn't at the release 3.2.4 :
[https://github.com/apache/hadoop/blob/release-3.2.4-RC0/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeAttributeProvider.java]
It is necessary to build the latest version, if we try to build with the
hadoop.version flag pointing to 3.2.4 version we get the following error:
{quote}[ERROR]
/usr/hdp/ranger/hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java:[229,2]
error: method does not override or implement a method from a supertype
{quote}
h3. Hive 3.0.0+
At the beginning, we wanted to build Apache Ranger with hive.version=2.3.3.
{quote}[ERROR] Failed to execute goal on project ranger-hive-plugin: Could not
resolve dependencies for project
org.apache.ranger:ranger-hive-plugin:jar:3.0.0-SNAPSHOT: Failed to collect
dependencies at org.apache.hive:hive-service:jar:2.3.3 ->
org.apache.hive:hive-exec:jar:2.3.3 ->
org.apache.calcite:calcite-core:jar:1.10.0 ->
org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde
{quote}
The approach we have tried to fix this was to add the following to both
{{ranger-hive-plugin-shim/pom.xml}} and {{hive-agent/pom.xml}} for both
hive-exec and hive-service artifacts inside <exclusions></exclusions>:
{code:java}
<exclusion>
<groupId>org.pentaho</groupId>
<artifactId>*</artifactId>
</exclusion>{code}
We got the following error (for example with hive.version=2.8.1):
{quote}[ERROR] Failed to execute goal on project ranger-hive-plugin: Could not
resolve dependencies for project
org.apache.ranger:ranger-hive-plugin:jar:3.0.0-SNAPSHOT: The following
artifacts could not be resolved: org.apache.hive:hive-common:jar:2.8.1
(absent), org.apache.hive:hive-service:jar:2.8.1 (absent),
org.apache.hive:hive-exec:jar:2.8.1 (absent),
org.apache.hive:hive-metastore:jar:2.8.1 (absent),
org.apache.hive:hive-jdbc:jar:2.8.1 (absent):
org.apache.hive:hive-common:jar:2.8.1 was not found in
[https://repository.apache.org/content/repositories/snapshots] during a
previous attempt. This failure was cached in the local repository and
resolution is not reattempted until the update interval of
apache.snapshots.https has elapsed or updates are forced
{quote}
We had the same errors for all versions < 3.0.0
Tested by [[email protected]|mailto:[email protected]] and
[[email protected]|mailto:[email protected]].
was:
To start simplifying the work of people who want to build the project using the
standard build process, we would like to propose an update of the README.md
with the minimum versions necessary for the building of Apache ranger with the
plugins and dependencies on them.
For the moment, 4 prerequisites would be indicated:
* JDK 1.8+ taken from pom.xml
* Apache Maven 3.6.3+ taken from pom.xml
* Hadoop 3.3.0+ taken from tests
* Hive 3.0.0+ taken from tests
h3. Hadoop 3.3.0+
The method *CheckPermissionWithContext* was not implemented before hadoop 3.3.0.
Here is the file in which this method is implemented since 3.3.0 and it's
possible to see that it wasn't at the release 3.2.4 :
[https://github.com/apache/hadoop/blob/release-3.2.4-RC0/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeAttributeProvider.java]
It is necessary to build the latest version, if we try to build with the
hadoop.version flag pointing to 3.2.4 version we get the following error:
{quote}[ERROR]
/usr/hdp/ranger/hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java:[229,2]
error: method does not override or implement a method from a supertype
{quote}
h3. Hive 3.0.0+
At the beginning, we wanted to build Apache Ranger with hive.version=2.3.3.
{quote}[ERROR] Failed to execute goal on project ranger-hive-plugin: Could not
resolve dependencies for project
org.apache.ranger:ranger-hive-plugin:jar:3.0.0-SNAPSHOT: Failed to collect
dependencies at org.apache.hive:hive-service:jar:2.3.3 ->
org.apache.hive:hive-exec:jar:2.3.3 ->
org.apache.calcite:calcite-core:jar:1.10.0 ->
org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde
{quote}
The approach we have tried to fix this was to add the following to both
{{ranger-hive-plugin-shim/pom.xml}} and {{hive-agent/pom.xml}} for both
hive-exec and hive-service artifacts inside <exclusions></exclusions>:
{{}}
{code:java}
<exclusion>
<groupId>org.pentaho</groupId>
<artifactId>*</artifactId>
</exclusion>{code}
{{}}
We got the following error (for example with hive.version=2.8.1):
{quote}[ERROR] Failed to execute goal on project ranger-hive-plugin: Could not
resolve dependencies for project
org.apache.ranger:ranger-hive-plugin:jar:3.0.0-SNAPSHOT: The following
artifacts could not be resolved: org.apache.hive:hive-common:jar:2.8.1
(absent), org.apache.hive:hive-service:jar:2.8.1 (absent),
org.apache.hive:hive-exec:jar:2.8.1 (absent),
org.apache.hive:hive-metastore:jar:2.8.1 (absent),
org.apache.hive:hive-jdbc:jar:2.8.1 (absent):
org.apache.hive:hive-common:jar:2.8.1 was not found in
[https://repository.apache.org/content/repositories/snapshots] during a
previous attempt. This failure was cached in the local repository and
resolution is not reattempted until the update interval of
apache.snapshots.https has elapsed or updates are forced
{quote}
We had the same errors for all versions < 3.0.0
Tested by [[email protected]|mailto:[email protected]] and
[[email protected]|mailto:[email protected]].
> Add prerequisites with minimal plugin versions for the regular build process
> ----------------------------------------------------------------------------
>
> Key: RANGER-4309
> URL: https://issues.apache.org/jira/browse/RANGER-4309
> Project: Ranger
> Issue Type: Improvement
> Components: documentation, plugins
> Affects Versions: 2.4.0
> Environment: CentOS 7.9 with JDK-8 and JDK-11, Apache Maven 3.9.3
> Reporter: Clément Lucas
> Priority: Minor
>
> To start simplifying the work of people who want to build the project using
> the standard build process, we would like to propose an update of the
> README.md with the minimum versions necessary for the building of Apache
> ranger with the plugins and dependencies on them.
> For the moment, 4 prerequisites would be indicated:
> * JDK 1.8+ taken from pom.xml
> * Apache Maven 3.6.3+ taken from pom.xml
> * Hadoop 3.3.0+ taken from tests
> * Hive 3.0.0+ taken from tests
> h3. Hadoop 3.3.0+
> The method *CheckPermissionWithContext* was not implemented before hadoop
> 3.3.0.
> Here is the file in which this method is implemented since 3.3.0 and it's
> possible to see that it wasn't at the release 3.2.4 :
> [https://github.com/apache/hadoop/blob/release-3.2.4-RC0/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeAttributeProvider.java]
> It is necessary to build the latest version, if we try to build with the
> hadoop.version flag pointing to 3.2.4 version we get the following error:
> {quote}[ERROR]
> /usr/hdp/ranger/hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java:[229,2]
> error: method does not override or implement a method from a supertype
> {quote}
> h3. Hive 3.0.0+
> At the beginning, we wanted to build Apache Ranger with hive.version=2.3.3.
> {quote}[ERROR] Failed to execute goal on project ranger-hive-plugin: Could
> not resolve dependencies for project
> org.apache.ranger:ranger-hive-plugin:jar:3.0.0-SNAPSHOT: Failed to collect
> dependencies at org.apache.hive:hive-service:jar:2.3.3 ->
> org.apache.hive:hive-exec:jar:2.3.3 ->
> org.apache.calcite:calcite-core:jar:1.10.0 ->
> org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde
> {quote}
> The approach we have tried to fix this was to add the following to both
> {{ranger-hive-plugin-shim/pom.xml}} and {{hive-agent/pom.xml}} for both
> hive-exec and hive-service artifacts inside <exclusions></exclusions>:
> {code:java}
> <exclusion>
> <groupId>org.pentaho</groupId>
> <artifactId>*</artifactId>
> </exclusion>{code}
> We got the following error (for example with hive.version=2.8.1):
> {quote}[ERROR] Failed to execute goal on project ranger-hive-plugin: Could
> not resolve dependencies for project
> org.apache.ranger:ranger-hive-plugin:jar:3.0.0-SNAPSHOT: The following
> artifacts could not be resolved: org.apache.hive:hive-common:jar:2.8.1
> (absent), org.apache.hive:hive-service:jar:2.8.1 (absent),
> org.apache.hive:hive-exec:jar:2.8.1 (absent),
> org.apache.hive:hive-metastore:jar:2.8.1 (absent),
> org.apache.hive:hive-jdbc:jar:2.8.1 (absent):
> org.apache.hive:hive-common:jar:2.8.1 was not found in
> [https://repository.apache.org/content/repositories/snapshots] during a
> previous attempt. This failure was cached in the local repository and
> resolution is not reattempted until the update interval of
> apache.snapshots.https has elapsed or updates are forced
> {quote}
> We had the same errors for all versions < 3.0.0
>
> Tested by [[email protected]|mailto:[email protected]] and
> [[email protected]|mailto:[email protected]].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
