[
https://issues.apache.org/jira/browse/CONNECTORS-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piergiorgio Lucidi updated CONNECTORS-1751:
-------------------------------------------
Description:
Thanks to the ManifoldCF SDK, I found another issue with the Maven build.
Steps to reproduce the issue with the Manifold SDK:
1. Clone the ManifoldCF SDK project from the following URL:
git clone [https://github.com/OpenPj/manifoldcf-sdk.git]
2. Remove row 27 from the {{run.sh}} script, this line is including the hotfix
for the Hadoop version upgrading it to the latest version 3.3.6.
3. Run the following command in order to download the MCF source code and run
the Ant and Maven build process in a Docker container (Maven 3.9.5 and JDK
Temurin 11).
This in order to install locally all the Maven dependencies needed to implement
custom extensions / connectors. The Docker Volume includes the entire Maven
repo (around 1GB) used by the ManifoldCF build process and it will be copied in
the SDK target folder and configured in order to compile your Java Custom code:
./run.sh init 2.26 ga
The current version of the ManifoldCF SDK 0.0.3 includes all the sep
instructions to fix all the build problems described here including the Hadoop
one:
https://issues.apache.org/jira/browse/CONNECTORS-1750
These issues are also resolved in ManifoldCF main trunk, except the Hadoop one,
but these are still present in the latest release packages (source code
packages).
I have to confess that I don't know if those issue are also included in other
releases.
The SDK is returning the following error that should be related to the Hadoop
Annotations dependencies of Hadoop 2.6.0, an old version of Hadoop that
includes a JDK 1.6 dependency: {{{{}} jdk.tools:jdk.tools:jar:1.6{}}}.
{code:java}
[ERROR] Failed to execute goal on project mcf-hdfs-connector: Could not resolve
dependencies for project org.apache.manifoldcf:mcf-hdfs-connector:jar:2.26: The
following artifacts could not be resolved: jdk.tools:jdk.tools:jar:1.6: Could
not find artifact jdk.tools:jdk.tools:jar:1.6 at specified path
/opt/java/openjdk/../lib/tools.jar -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
on project mcf-hdfs-connector: Could not resolve dependencies for project
org.apache.manifoldcf:mcf-hdfs-connector:jar:2.26: The following artifacts
could not be resolved: jdk.tools:jdk.tools:jar:1.6: Could not find artifact
jdk.tools:jdk.tools:jar:1.6 at specified path /opt/java/openjdk/../lib/tools.jar
at
org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
(LifecycleDependencyResolver.java:243)
at
org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
(LifecycleDependencyResolver.java:136)
at
org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
(MojoExecutor.java:355)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute
(MojoExecutor.java:313)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:174)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000
(MojoExecutor.java:75)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run
(MojoExecutor.java:162)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute
(DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:159)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
(LifecycleModuleBuilder.java:105)
at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
(MultiThreadedBuilder.java:193)
at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
(MultiThreadedBuilder.java:180)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:628)
at java.lang.Thread.run (Thread.java:829)
. . .
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not
find artifact jdk.tools:jdk.tools:jar:1.6 at specified path
/opt/java/openjdk/../lib/tools.jar
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve
(DefaultArtifactResolver.java:305)
at
org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts
(DefaultArtifactResolver.java:261)
at
org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies
(DefaultRepositorySystem.java:353)
at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve
(DefaultProjectDependenciesResolver.java:182)
at
org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
(LifecycleDependencyResolver.java:224)
at
org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
(LifecycleDependencyResolver.java:136)
at
org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
(MojoExecutor.java:355)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute
(MojoExecutor.java:313)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:174)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000
(MojoExecutor.java:75)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run
(MojoExecutor.java:162)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute
(DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:159)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
(LifecycleModuleBuilder.java:105)
at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
(MultiThreadedBuilder.java:193)
at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
(MultiThreadedBuilder.java:180)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:628)
at java.lang.Thread.run (Thread.java:829){code}
Trying to use the latest version of Hadoop 3.3.6, reverting to the original
run.sh file, it solves this issue.
I would like to use the updated version because even if we found a way to
exclude this old dependency, it sounds strange to use very old libraries that
requires JDK 1.6.
was:
Thanks to the ManifoldCF SDK, I found another issue with the Maven build.
Steps to reproduce the issue with the Manifold SDK:
1. Clone the ManifoldCF SDK project from the following URL:
git clone [https://github.com/OpenPj/manifoldcf-sdk.git]
2. Remove row 27 from the {{run.sh}} script, this line is including the hotfix
for the Hadoop version upgrading it to the latest version 3.3.6.
3. Run the following command in order to download the MCF source code and run
the Ant and Maven build process in a Docker container (Maven 3.9.5 and JDK
Temurin 11).
This in order to install locally all the Maven dependencies needed to implement
custom extensions / connectors. The Docker Volume includes the entire Maven
repo (around 1GB) used by the ManifoldCF build process and it will be copied in
the SDK target folder and configured in order to compile your Java Custom code:
./run.sh init 2.26 ga
The current version of the [ManifoldCF SDK
(0.0.2-Beta)|https://github.com/OpenPj/manifoldcf-sdk/releases/tag/v0.0.2-Beta]
includes all the sep instructions to fix all the build problems described here
including the Hadoop one:
https://issues.apache.org/jira/browse/CONNECTORS-1750
These issues are also resolved in ManifoldCF main trunk, except the Hadoop one,
but these are still present in the latest release packages (source code
packages).
I have to confess that I don't know if those issue are also included in other
releases.
The SDK is returning the following error that should be related to the Hadoop
Annotations dependencies of Hadoop 2.6.0, an old version of Hadoop that
includes a JDK 1.6 dependency: {{{{}} jdk.tools:jdk.tools:jar:1.6{}}}.
{code:java}
[ERROR] Failed to execute goal on project mcf-hdfs-connector: Could not resolve
dependencies for project org.apache.manifoldcf:mcf-hdfs-connector:jar:2.26: The
following artifacts could not be resolved: jdk.tools:jdk.tools:jar:1.6: Could
not find artifact jdk.tools:jdk.tools:jar:1.6 at specified path
/opt/java/openjdk/../lib/tools.jar -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
on project mcf-hdfs-connector: Could not resolve dependencies for project
org.apache.manifoldcf:mcf-hdfs-connector:jar:2.26: The following artifacts
could not be resolved: jdk.tools:jdk.tools:jar:1.6: Could not find artifact
jdk.tools:jdk.tools:jar:1.6 at specified path /opt/java/openjdk/../lib/tools.jar
at
org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
(LifecycleDependencyResolver.java:243)
at
org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
(LifecycleDependencyResolver.java:136)
at
org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
(MojoExecutor.java:355)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute
(MojoExecutor.java:313)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:174)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000
(MojoExecutor.java:75)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run
(MojoExecutor.java:162)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute
(DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:159)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
(LifecycleModuleBuilder.java:105)
at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
(MultiThreadedBuilder.java:193)
at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
(MultiThreadedBuilder.java:180)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:628)
at java.lang.Thread.run (Thread.java:829)
. . .
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not
find artifact jdk.tools:jdk.tools:jar:1.6 at specified path
/opt/java/openjdk/../lib/tools.jar
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve
(DefaultArtifactResolver.java:305)
at
org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts
(DefaultArtifactResolver.java:261)
at
org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies
(DefaultRepositorySystem.java:353)
at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve
(DefaultProjectDependenciesResolver.java:182)
at
org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
(LifecycleDependencyResolver.java:224)
at
org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
(LifecycleDependencyResolver.java:136)
at
org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
(MojoExecutor.java:355)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute
(MojoExecutor.java:313)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:174)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000
(MojoExecutor.java:75)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run
(MojoExecutor.java:162)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute
(DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:159)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
(LifecycleModuleBuilder.java:105)
at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
(MultiThreadedBuilder.java:193)
at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
(MultiThreadedBuilder.java:180)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:628)
at java.lang.Thread.run (Thread.java:829){code}
Trying to use the latest version of Hadoop 3.3.6, reverting to the original
run.sh file, it solves this issue.
I would like to use the updated version because even if we found a way to
exclude this old dependency, it sounds strange to use very old libraries that
requires JDK 1.6.
> HDFS Connector - Maven build issue
> ----------------------------------
>
> Key: CONNECTORS-1751
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1751
> Project: ManifoldCF
> Issue Type: Bug
> Components: HDFS connector
> Affects Versions: ManifoldCF 2.26
> Reporter: Piergiorgio Lucidi
> Priority: Major
> Fix For: ManifoldCF next
>
>
> Thanks to the ManifoldCF SDK, I found another issue with the Maven build.
> Steps to reproduce the issue with the Manifold SDK:
>
> 1. Clone the ManifoldCF SDK project from the following URL:
> git clone [https://github.com/OpenPj/manifoldcf-sdk.git]
>
> 2. Remove row 27 from the {{run.sh}} script, this line is including the
> hotfix for the Hadoop version upgrading it to the latest version 3.3.6.
>
> 3. Run the following command in order to download the MCF source code and run
> the Ant and Maven build process in a Docker container (Maven 3.9.5 and JDK
> Temurin 11).
>
> This in order to install locally all the Maven dependencies needed to
> implement custom extensions / connectors. The Docker Volume includes the
> entire Maven repo (around 1GB) used by the ManifoldCF build process and it
> will be copied in the SDK target folder and configured in order to compile
> your Java Custom code:
>
> ./run.sh init 2.26 ga
>
> The current version of the ManifoldCF SDK 0.0.3 includes all the sep
> instructions to fix all the build problems described here including the
> Hadoop one:
> https://issues.apache.org/jira/browse/CONNECTORS-1750
>
> These issues are also resolved in ManifoldCF main trunk, except the Hadoop
> one, but these are still present in the latest release packages (source code
> packages).
> I have to confess that I don't know if those issue are also included in other
> releases.
>
> The SDK is returning the following error that should be related to the Hadoop
> Annotations dependencies of Hadoop 2.6.0, an old version of Hadoop that
> includes a JDK 1.6 dependency: {{{{}} jdk.tools:jdk.tools:jar:1.6{}}}.
>
> {code:java}
> [ERROR] Failed to execute goal on project mcf-hdfs-connector: Could not
> resolve dependencies for project
> org.apache.manifoldcf:mcf-hdfs-connector:jar:2.26: The following artifacts
> could not be resolved: jdk.tools:jdk.tools:jar:1.6: Could not find artifact
> jdk.tools:jdk.tools:jar:1.6 at specified path
> /opt/java/openjdk/../lib/tools.jar -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project mcf-hdfs-connector: Could not resolve dependencies for
> project org.apache.manifoldcf:mcf-hdfs-connector:jar:2.26: The following
> artifacts could not be resolved: jdk.tools:jdk.tools:jar:1.6: Could not find
> artifact jdk.tools:jdk.tools:jar:1.6 at specified path
> /opt/java/openjdk/../lib/tools.jar
> at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
> (LifecycleDependencyResolver.java:243)
> at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
> (LifecycleDependencyResolver.java:136)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
> (MojoExecutor.java:355)
> at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute
> (MojoExecutor.java:313)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:212)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:174)
> at org.apache.maven.lifecycle.internal.MojoExecutor.access$000
> (MojoExecutor.java:75)
> at org.apache.maven.lifecycle.internal.MojoExecutor$1.run
> (MojoExecutor.java:162)
> at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute
> (DefaultMojosExecutionStrategy.java:39)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:159)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:105)
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
> (MultiThreadedBuilder.java:193)
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
> (MultiThreadedBuilder.java:180)
> at java.util.concurrent.FutureTask.run (FutureTask.java:264)
> at java.util.concurrent.Executors$RunnableAdapter.call
> (Executors.java:515)
> at java.util.concurrent.FutureTask.run (FutureTask.java:264)
> at java.util.concurrent.ThreadPoolExecutor.runWorker
> (ThreadPoolExecutor.java:1128)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run
> (ThreadPoolExecutor.java:628)
> at java.lang.Thread.run (Thread.java:829)
> . . .
> Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not
> find artifact jdk.tools:jdk.tools:jar:1.6 at specified path
> /opt/java/openjdk/../lib/tools.jar
> at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve
> (DefaultArtifactResolver.java:305)
> at
> org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts
> (DefaultArtifactResolver.java:261)
> at
> org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies
> (DefaultRepositorySystem.java:353)
> at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve
> (DefaultProjectDependenciesResolver.java:182)
> at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
> (LifecycleDependencyResolver.java:224)
> at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
> (LifecycleDependencyResolver.java:136)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
> (MojoExecutor.java:355)
> at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute
> (MojoExecutor.java:313)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:212)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:174)
> at org.apache.maven.lifecycle.internal.MojoExecutor.access$000
> (MojoExecutor.java:75)
> at org.apache.maven.lifecycle.internal.MojoExecutor$1.run
> (MojoExecutor.java:162)
> at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute
> (DefaultMojosExecutionStrategy.java:39)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:159)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:105)
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
> (MultiThreadedBuilder.java:193)
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
> (MultiThreadedBuilder.java:180)
> at java.util.concurrent.FutureTask.run (FutureTask.java:264)
> at java.util.concurrent.Executors$RunnableAdapter.call
> (Executors.java:515)
> at java.util.concurrent.FutureTask.run (FutureTask.java:264)
> at java.util.concurrent.ThreadPoolExecutor.runWorker
> (ThreadPoolExecutor.java:1128)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run
> (ThreadPoolExecutor.java:628)
> at java.lang.Thread.run (Thread.java:829){code}
>
> Trying to use the latest version of Hadoop 3.3.6, reverting to the original
> run.sh file, it solves this issue.
>
> I would like to use the updated version because even if we found a way to
> exclude this old dependency, it sounds strange to use very old libraries that
> requires JDK 1.6.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
