I want to caution that in testing a build from this morning's branch-2.1 we found that Hive partition pruning was not working. We found that Spark SQL was fetching all Hive table partitions for a very simple query whereas in a build from several weeks ago it was fetching only the required partitions. I cannot currently think of a reason for the regression outside of some difference between branch-2.1 from our previous build and branch-2.1 from this morning.
That's all I know right now. We are actively investigating to find the root cause of this problem, and specifically whether this is a problem in the Spark codebase or not. I will report back when I have an answer to that question. Michael > On Apr 18, 2017, at 11:59 AM, Michael Armbrust <mich...@databricks.com> wrote: > > Please vote on releasing the following candidate as Apache Spark version > 2.1.1. The vote is open until Fri, April 21st, 2018 at 13:00 PST and passes > if a majority of at least 3 +1 PMC votes are cast. > > [ ] +1 Release this package as Apache Spark 2.1.1 > [ ] -1 Do not release this package because ... > > > To learn more about Apache Spark, please see http://spark.apache.org/ > <http://spark.apache.org/> > > The tag to be voted on is v2.1.1-rc3 > <https://github.com/apache/spark/tree/v2.1.1-rc3> > (2ed19cff2f6ab79a718526e5d16633412d8c4dd4) > > List of JIRA tickets resolved can be found with this filter > <https://issues.apache.org/jira/browse/SPARK-20134?jql=project%20%3D%20SPARK%20AND%20fixVersion%20%3D%202.1.1>. > > The release files, including signatures, digests, etc. can be found at: > http://home.apache.org/~pwendell/spark-releases/spark-2.1.1-rc3-bin/ > <http://home.apache.org/~pwendell/spark-releases/spark-2.1.1-rc3-bin/> > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/pwendell.asc > <https://people.apache.org/keys/committer/pwendell.asc> > > The staging repository for this release can be found at: > https://repository.apache.org/content/repositories/orgapachespark-1230/ > <https://repository.apache.org/content/repositories/orgapachespark-1230/> > > The documentation corresponding to this release can be found at: > http://people.apache.org/~pwendell/spark-releases/spark-2.1.1-rc3-docs/ > <http://people.apache.org/~pwendell/spark-releases/spark-2.1.1-rc3-docs/> > > > FAQ > > How can I help test this release? > > If you are a Spark user, you can help us test this release by taking an > existing Spark workload and running on this release candidate, then reporting > any regressions. > > What should happen to JIRA tickets still targeting 2.1.1? > > Committers should look at those and triage. Extremely important bug fixes, > documentation, and API tweaks that impact compatibility should be worked on > immediately. Everything else please retarget to 2.1.2 or 2.2.0. > > But my bug isn't fixed!??! > > In order to make timely releases, we will typically not hold the release > unless the bug in question is a regression from 2.1.0. > > What happened to RC1? > > There were issues with the release packaging and as a result was skipped.
