Yes, there is no hadoop-1.0.3 profile. You can look at the parent
pom.xml to see the profiles that exist. It doesn't mean 1.0.3 doesn't
work, just that there is nothing specific to activate for this version
range. I don't think the docs suggest this profile exists. You don't
need any extra profile.

YARN's versioning was different from Hadoop's for a while. There is no
YARN 1.0.3 and you have to set yarn.version separately.

But if you don't set profiles like -Pyarn it won't be built anyway and
it's irrelevant.

However I think there's an unfortunate interaction with the
hadoop-provided profile, which will also generate references to YARN
artifacts. You can set yarn.version to anything like 2.2.0 and I
imagine it will get over that error.


On Mon, Oct 13, 2014 at 10:39 PM, mildebrandt <[email protected]> wrote:
> Hello,
>
> After reading the following pages:
> https://spark.apache.org/docs/latest/building-with-maven.html
> http://spark.apache.org/docs/latest/hadoop-third-party-distributions.html
>
> I came up with this build command:
> mvn -Dhadoop.version=1.0.3 -DskipTests -Pdeb -Phadoop-1.0.3 -Phive
> -Phadoop-provided clean package
>
> This fails with the following error:
> [WARNING] The requested profile "hadoop-1.0.3" could not be activated
> because it does not exist.
> [ERROR] Failed to execute goal on project spark-parent: Could not resolve
> dependencies for project org.apache.spark:spark-parent:pom:1.1.0: The
> following artifacts could not be resolved:
> org.apache.hadoop:hadoop-yarn-api:jar:1.0.3,
> org.apache.hadoop:hadoop-yarn-common:jar:1.0.3,
> org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:1.0.3,
> org.apache.hadoop:hadoop-yarn-client:jar:1.0.3: Failure to find
> org.apache.hadoop:hadoop-yarn-api:jar:1.0.3 in
> https://repo1.maven.org/maven2 was cached in the local repository,
> resolution will not be reattempted until the update interval of central has
> elapsed or updates are forced -> [Help 1]
>
> If I remove the hadoop-provided profile, it works fine. So my questions:
>  - Why isn't there a hadoop-1.0.3 profile? The
> hadoop-third-party-distributions.html page seems to suggest that it would
> exist.
>  - It's true, I don't see a hadoop-yarn-api:jar:1.0.3 in the maven
> repository. I'd like to build without yarn support. And ideas why it's
> pulling yarn? Is it due to the fact that the hadoop-1.0.3 profile doesn't
> exist?
>  - Since running without the hadoop-provided profile works, is there any
> issue running that version of Spark against my Apache 1.0.3 cluster?
>
> Thanks,
> -Chris
>
>
>
> --
> View this message in context: 
> http://apache-spark-user-list.1001560.n3.nabble.com/Problems-building-Spark-for-Hadoop-1-0-3-tp16324.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to