[jira] [Commented] (SPARK-32008) 3.0.0 release build fails
[
https://issues.apache.org/jira/browse/SPARK-32008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17138249#comment-17138249
]
Philipp Dallig commented on SPARK-32008:
Hi [~shivaram]
Thanks for your fast response. I was able to build Spark by myself with ubuntu
20.04 as a base image.
ubuntu 20.04 delivers with R version >= 3.6
I'll edit the Dockerfile in my bug description.
> 3.0.0 release build fails
> -
>
> Key: SPARK-32008
> URL: https://issues.apache.org/jira/browse/SPARK-32008
> Project: Spark
> Issue Type: Bug
> Components: Build, Documentation
>Affects Versions: 3.0.0
>Reporter: Philipp Dallig
>Priority: Major
>
> Hi,
> I try to build the spark release 3.0.0 by myself.
> I got the following error.
> {code}
> 20/06/16 15:20:49 WARN PrefixSpan: Input data is not cached.
> 20/06/16 15:20:50 WARN Instrumentation: [b307b568] regParam is zero, which
> might cause numerical instability and overfitting.
> Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics:
> 'vignetteInfo' is not an exported object from 'namespace:tools'
> Execution halted
> {code}
> I can reproduce this error with a small Dockerfile.
> {code}
> FROM ubuntu:18.04 as builder
> ENV MVN_VERSION=3.6.3 \
> M2_HOME=/opt/apache-maven \
> MAVEN_HOME=/opt/apache-maven \
> MVN_HOME=/opt/apache-maven \
>
> MVN_SHA512=c35a1803a6e70a126e80b2b3ae33eed961f83ed74d18fcd16909b2d44d7dada3203f1ffe726c17ef8dcca2dcaa9fca676987befeadc9b9f759967a8cb77181c0
> \
> MAVEN_OPTS="-Xmx3g -XX:ReservedCodeCacheSize=1g" \
> R_HOME=/usr/lib/R \
> GIT_REPO=https://github.com/apache/spark.git \
> GIT_BRANCH=v3.0.0 \
> SPARK_DISTRO_NAME=hadoop3.2 \
> SPARK_LOCAL_HOSTNAME=localhost
> # Preparation
> RUN /usr/bin/apt-get update && \
> # APT
> INSTALL_PKGS="openjdk-8-jdk-headless git wget python3 python3-pip
> python3-setuptools r-base r-base-dev pandoc pandoc-citeproc
> libcurl4-openssl-dev libssl-dev libxml2-dev texlive qpdf language-pack-en" &&
> \
> DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -y install
> --no-install-recommends $INSTALL_PKGS && \
> rm -rf /var/lib/apt/lists/* && \
> Rscript -e "install.packages(c('knitr', 'rmarkdown', 'devtools',
> 'testthat', 'e1071', 'survival'), repos='https://cloud.r-project.org/')" && \
> # Maven
> /usr/bin/wget -nv -O apache-maven.tar.gz
> "https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download=maven/maven-3/${MVN_VERSION}/binaries/apache-maven-${MVN_VERSION}-bin.tar.gz;
> && \
> echo "${MVN_SHA512} apache-maven.tar.gz" > apache-maven.sha512 && \
> sha512sum --strict -c apache-maven.sha512 && \
> tar -xvzf apache-maven.tar.gz -C /opt && \
> rm -v apache-maven.sha512 apache-maven.tar.gz && \
> /bin/ln -vs /opt/apache-maven-${MVN_VERSION} /opt/apache-maven && \
> /bin/ln -vs /opt/apache-maven/bin/mvn /usr/bin/mvn
> # Spark Distribution Build
> RUN mkdir -p /workspace && \
> cd /workspace && \
> git clone --branch ${GIT_BRANCH} ${GIT_REPO} && \
> cd /workspace/spark && \
> ./dev/make-distribution.sh --name ${SPARK_DISTRO_NAME} --pip --r --tgz
> -Psparkr -Phadoop-3.2 -Phive-2.3 -Phive-thriftserver -Pyarn -Pkubernetes
> {code}
> I am very grateful to all helpers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-32008) 3.0.0 release build fails
[
https://issues.apache.org/jira/browse/SPARK-32008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17137937#comment-17137937
]
Shivaram Venkataraman commented on SPARK-32008:
---
It looks like the R vignette build failed and looking at the error message this
seems related to https://github.com/rstudio/rmarkdown/issues/1831 -- I think it
should work fine if you try to use R version >= 3.6
> 3.0.0 release build fails
> -
>
> Key: SPARK-32008
> URL: https://issues.apache.org/jira/browse/SPARK-32008
> Project: Spark
> Issue Type: Bug
> Components: Build, Documentation
>Affects Versions: 3.0.0
>Reporter: Philipp Dallig
>Priority: Major
>
> Hi,
> I try to build the spark release 3.0.0 by myself.
> I got the following error.
> {code}
> 20/06/16 15:20:49 WARN PrefixSpan: Input data is not cached.
> 20/06/16 15:20:50 WARN Instrumentation: [b307b568] regParam is zero, which
> might cause numerical instability and overfitting.
> Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics:
> 'vignetteInfo' is not an exported object from 'namespace:tools'
> Execution halted
> {code}
> I can reproduce this error with a small Dockerfile.
> {code}
> FROM ubuntu:18.04 as builder
> ENV MVN_VERSION=3.6.3 \
> M2_HOME=/opt/apache-maven \
> MAVEN_HOME=/opt/apache-maven \
> MVN_HOME=/opt/apache-maven \
>
> MVN_SHA512=c35a1803a6e70a126e80b2b3ae33eed961f83ed74d18fcd16909b2d44d7dada3203f1ffe726c17ef8dcca2dcaa9fca676987befeadc9b9f759967a8cb77181c0
> \
> MAVEN_OPTS="-Xmx3g -XX:ReservedCodeCacheSize=1g" \
> R_HOME=/usr/lib/R \
> GIT_REPO=https://github.com/apache/spark.git \
> GIT_BRANCH=v3.0.0 \
> SPARK_DISTRO_NAME=hadoop3.2 \
> SPARK_LOCAL_HOSTNAME=localhost
> # Preparation
> RUN /usr/bin/apt-get update && \
> # APT
> INSTALL_PKGS="openjdk-8-jdk-headless git wget python3 python3-pip
> python3-setuptools r-base r-base-dev pandoc pandoc-citeproc
> libcurl4-openssl-dev libssl-dev libxml2-dev texlive qpdf language-pack-en" &&
> \
> DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -y install
> --no-install-recommends $INSTALL_PKGS && \
> rm -rf /var/lib/apt/lists/* && \
> Rscript -e "install.packages(c('knitr', 'rmarkdown', 'devtools',
> 'testthat', 'e1071', 'survival'), repos='https://cloud.r-project.org/')" && \
> # Maven
> /usr/bin/wget -nv -O apache-maven.tar.gz
> "https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download=maven/maven-3/${MVN_VERSION}/binaries/apache-maven-${MVN_VERSION}-bin.tar.gz;
> && \
> echo "${MVN_SHA512} apache-maven.tar.gz" > apache-maven.sha512 && \
> sha512sum --strict -c apache-maven.sha512 && \
> tar -xvzf apache-maven.tar.gz -C /opt && \
> rm -v apache-maven.sha512 apache-maven.tar.gz && \
> /bin/ln -vs /opt/apache-maven-${MVN_VERSION} /opt/apache-maven && \
> /bin/ln -vs /opt/apache-maven/bin/mvn /usr/bin/mvn
> # Spark Distribution Build
> RUN mkdir -p /workspace && \
> cd /workspace && \
> git clone --branch ${GIT_BRANCH} ${GIT_REPO} && \
> cd /workspace/spark && \
> ./dev/make-distribution.sh --name ${SPARK_DISTRO_NAME} --pip --r --tgz
> -Psparkr -Phadoop-3.2 -Phive-2.3 -Phive-thriftserver -Pyarn -Pkubernetes
> {code}
> I am very grateful to all helpers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
