btw, seems we also have the same skipping logic for pyarrow. But, I have not looked into if tests related to pyarrow are skipped or not.
On Wed, Jan 31, 2018 at 8:15 PM, Yin Huai <yh...@databricks.com> wrote: > Hello, > > I was running python tests and found that pyspark.sql.tests. > GroupbyAggPandasUDFTests.test_unsupported_types > <https://github.com/apache/spark/blob/52e00f70663a87b5837235bdf72a3e6f84e11411/python/pyspark/sql/tests.py#L4528-L4548> > does > not run with Python 2 because the test uses "assertRaisesRegex" (supported > by Python 3) instead of "assertRaisesRegexp" (supported by Python 2). > However, spark jenkins does not fail because of this issue (see run history > at here > <https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-master-test-sbt-hadoop-2.7/>). > After looking into this issue, seems test script will skip tests related > to pandas if pandas is not installed > <https://github.com/apache/spark/blob/2ac895be909de7e58e1051dc2a1bba98a25bf4be/python/pyspark/sql/tests.py#L51-L63>, > which means that jenkins does not have pandas installed. > > @Shane, can you help us check if jenkins workers have pandas installed? > > Thanks, > > Yin >
