I created https://issues.apache.org/jira/browse/SPARK-23292 for this issue.
On Wed, Jan 31, 2018 at 8:17 PM, Yin Huai <yh...@databricks.com> wrote: > btw, seems we also have the same skipping logic for pyarrow. But, I have > not looked into if tests related to pyarrow are skipped or not. > > On Wed, Jan 31, 2018 at 8:15 PM, Yin Huai <yh...@databricks.com> wrote: > >> Hello, >> >> I was running python tests and found that pyspark.sql.tests.Groupby >> AggPandasUDFTests.test_unsupported_types >> <https://github.com/apache/spark/blob/52e00f70663a87b5837235bdf72a3e6f84e11411/python/pyspark/sql/tests.py#L4528-L4548> >> does >> not run with Python 2 because the test uses "assertRaisesRegex" (supported >> by Python 3) instead of "assertRaisesRegexp" (supported by Python 2). >> However, spark jenkins does not fail because of this issue (see run history >> at here >> <https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-master-test-sbt-hadoop-2.7/>). >> After looking into this issue, seems test script will skip tests related >> to pandas if pandas is not installed >> <https://github.com/apache/spark/blob/2ac895be909de7e58e1051dc2a1bba98a25bf4be/python/pyspark/sql/tests.py#L51-L63>, >> which means that jenkins does not have pandas installed. >> >> @Shane, can you help us check if jenkins workers have pandas installed? >> >> Thanks, >> >> Yin >> > >
