[
https://issues.apache.org/jira/browse/SPARK-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14268313#comment-14268313
]
Josh Rosen commented on SPARK-5053:
-----------------------------------
It looks like nearly all of these new builds are failing for various reasons,
so I could use some help fixing them.
One issue is that several of the PySpark tests are failing with
{code}
OK (skipped=1)
Traceback (most recent call last):
File "pyspark/mllib/_common.py", line 20, in <module>
import numpy
File "/usr/lib64/python2.6/site-packages/numpy/__init__.py", line 170, in
<module>
from . import add_newdocs
File "/usr/lib64/python2.6/site-packages/numpy/add_newdocs.py", line 13, in
<module>
from numpy.lib import add_newdoc
File "/usr/lib64/python2.6/site-packages/numpy/lib/__init__.py", line 8, in
<module>
from .type_check import *
File "/usr/lib64/python2.6/site-packages/numpy/lib/type_check.py", line 11,
in <module>
import numpy.core.numeric as _nx
File "/usr/lib64/python2.6/site-packages/numpy/core/__init__.py", line 46, in
<module>
from numpy.testing import Tester
File "/usr/lib64/python2.6/site-packages/numpy/testing/__init__.py", line 13,
in <module>
from .utils import *
File "/usr/lib64/python2.6/site-packages/numpy/testing/utils.py", line 15, in
<module>
from tempfile import mkdtemp
File "/usr/lib64/python2.6/tempfile.py", line 34, in <module>
from random import Random as _Random
File
"/home/jenkins/workspace/Spark-1.1-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/python/pyspark/mllib/random.py",
line 23, in <module>
from pyspark.rdd import RDD
File
"/home/jenkins/workspace/Spark-1.1-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/python/pyspark/__init__.py",
line 63, in <module>
from pyspark.context import SparkContext
File
"/home/jenkins/workspace/Spark-1.1-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/python/pyspark/context.py",
line 22, in <module>
from tempfile import NamedTemporaryFile
ImportError: cannot import name NamedTemporaryFile
{code}
Some of the other failures might just be due to flaky tests exposed by higher
Jenkins loads; let's see if they persist after rebuilds.
> Test maintenance branches on Jenkins using SBT
> ----------------------------------------------
>
> Key: SPARK-5053
> URL: https://issues.apache.org/jira/browse/SPARK-5053
> Project: Spark
> Issue Type: New Feature
> Components: Project Infra
> Reporter: Josh Rosen
> Priority: Blocker
>
> We need to create Jenkins jobs to test maintenance branches using SBT. The
> current Maven jobs for backport branches do not run the same checks that the
> pull request builder / SBT builds do (e.g. MiMa checks, PySpark, RAT, etc.)
> which means that cherry-picking backports can silently break things and we'll
> only discover it once PRs that are explicitly opened against those branches
> fail tests; this long delay between introducing test failures and detecting
> them is a huge productivity issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
