Another two correctness bug fixes were merged to 2.3 today: https://issues.apache.org/jira/browse/SPARK-24934 https://issues.apache.org/jira/browse/SPARK-24957
On Mon, Jul 30, 2018 at 1:19 PM Xiao Li <gatorsm...@gmail.com> wrote: > Sounds good to me. Thanks! Today, we merged another correctness fix > https://github.com/apache/spark/pull/21772. > > Xiao > > 2018-07-29 18:31 GMT-07:00 Saisai Shao <sai.sai.s...@gmail.com>: > >> Sure, I will do a next RC. I'm still waiting for a CVE fix, if this can >> be done in this two days, I will also include that one. >> >> Xiao Li <gatorsm...@gmail.com> 于2018年7月28日周六 上午12:05写道： >> >>> The following blocker/important fixes have been merged to Spark 2.3 >>> branch: >>> >>> https://issues.apache.org/jira/browse/SPARK-24927 >>> https://issues.apache.org/jira/browse/SPARK-24867 >>> https://issues.apache.org/jira/browse/SPARK-24891 >>> >>> *Saisai*, could you start the next RC? >>> >>> Thanks, >>> >>> Xiao >>> >>> >>> 2018-07-20 14:21 GMT-07:00 Tom Graves <tgraves...@yahoo.com>: >>> >>>> fyi, I merged in a couple jira that were critical (and I thought would >>>> be good to include in the next release) that if we spin another RC will get >>>> included, we should update the jira SPARK-24755 >>>> <https://github.com/apache/spark/commit/d0280ab818391fd11662647459f1e9e683b2bc8e> >>>> and SPARK-24677 >>>> <https://github.com/apache/spark/commit/7be70e29dd92de36dbb30ce39623d588f48e4cac>, >>>> if anyone disagrees we could back those out but I think they would be good >>>> to include. >>>> >>>> Tom >>>> >>>> On Thursday, July 19, 2018, 8:13:23 PM CDT, Saisai Shao < >>>> sai.sai.s...@gmail.com> wrote: >>>> >>>> >>>> Sure, I can wait for this and create another RC then. >>>> >>>> Thanks, >>>> Saisai >>>> >>>> Xiao Li <gatorsm...@gmail.com> 于2018年7月20日周五 上午9:11写道： >>>> >>>> Yes. https://issues.apache.org/jira/browse/SPARK-24867 is the one I >>>> created. The PR has been created. Since this is not rare, let us merge it >>>> to 2.3.2? >>>> >>>> Reynold' PR is to get rid of AnalysisBarrier. That is better than >>>> multiple patches we added for AnalysisBarrier after 2.3.0 release. We can >>>> target it to 2.4. >>>> >>>> Thanks, >>>> >>>> Xiao >>>> >>>> 2018-07-19 17:48 GMT-07:00 Saisai Shao <sai.sai.s...@gmail.com>: >>>> >>>> I see, thanks Reynold. >>>> >>>> Reynold Xin <r...@databricks.com> 于2018年7月20日周五 上午8:46写道： >>>> >>>> Looking at the list of pull requests it looks like this is the ticket: >>>> https://issues.apache.org/jira/browse/SPARK-24867 >>>> >>>> >>>> >>>> On Thu, Jul 19, 2018 at 5:25 PM Reynold Xin <r...@databricks.com> >>>> wrote: >>>> >>>> I don't think my ticket should block this release. It's a big general >>>> refactoring. >>>> >>>> Xiao do you have a ticket for the bug you found? >>>> >>>> >>>> On Thu, Jul 19, 2018 at 5:24 PM Saisai Shao <sai.sai.s...@gmail.com> >>>> wrote: >>>> >>>> Hi Xiao, >>>> >>>> Are you referring to this JIRA ( >>>> https://issues.apache.org/jira/browse/SPARK-24865)? >>>> >>>> Xiao Li <gatorsm...@gmail.com> 于2018年7月20日周五 上午2:41写道： >>>> >>>> dfWithUDF.cache() >>>> dfWithUDF.write.saveAsTable("t") >>>> dfWithUDF.write.saveAsTable("t1") >>>> >>>> >>>> Cached data is not being used. It causes a big performance regression. >>>> >>>> >>>> >>>> >>>> 2018-07-19 11:32 GMT-07:00 Sean Owen <sro...@gmail.com>: >>>> >>>> What regression are you referring to here? A -1 vote really needs a >>>> rationale. >>>> >>>> On Thu, Jul 19, 2018 at 1:27 PM Xiao Li <gatorsm...@gmail.com> wrote: >>>> >>>> I would first vote -1. >>>> >>>> I might find another regression caused by the analysis barrier. Will >>>> keep you posted. >>>> >>>> >>>> >>>> >>> >
