At some point we should celebrate having the larger RC number ever in Spark ...
On Mon, Mar 25, 2019 at 9:44 PM, DB Tsai < dbt...@dbtsai.com.invalid > wrote: > > > > RC9 was just cut. Will send out another thread once the build is finished. > > > > > Sincerely, > > > > DB Tsai > ---------------------------------------------------------- Web: https:/ / www. > dbtsai. com ( https://www.dbtsai.com/ ) > PGP Key ID: 42E5B25A8F7A82C1 > > > > On Mon, Mar 25, 2019 at 5:10 PM Sean Owen < srowen@ apache. org ( > sro...@apache.org ) > wrote: > > >> >> >> That's all merged now. I think you're clear to start an RC. >> >> >> >> On Mon, Mar 25, 2019 at 4:06 PM DB Tsai < dbtsai@ dbtsai. com. invalid ( >> dbt...@dbtsai.com.invalid ) > wrote: >> >> >>> >>> >>> I am going to cut a 2.4.1 rc9 soon tonight. Besides SPARK-26961 https:/ / >>> github. >>> com/ apache/ spark/ pull/ 24126 ( >>> https://github.com/apache/spark/pull/24126 ) , anything critical that we >>> have to wait for 2.4.1 release? Thanks! >>> >>> >>> >>> Sincerely, >>> >>> >>> >>> DB Tsai >>> ---------------------------------------------------------- Web: https:/ / >>> www. >>> dbtsai. com ( https://www.dbtsai.com/ ) >>> PGP Key ID: 42E5B25A8F7A82C1 >>> >>> >>> >>> On Sun, Mar 24, 2019 at 8:19 PM Sean Owen < srowen@ apache. org ( >>> sro...@apache.org ) > wrote: >>> >>> >>>> >>>> >>>> Still waiting on a successful test - hope this one works. >>>> >>>> >>>> >>>> On Sun, Mar 24, 2019, 10:13 PM DB Tsai < dbtsai@ dbtsai. com ( >>>> dbt...@dbtsai.com ) > wrote: >>>> >>>> >>>>> >>>>> >>>>> Hello Sean, >>>>> >>>>> >>>>> >>>>> By looking at SPARK-26961 PR, seems it's ready to go. Do you think we can >>>>> merge it into 2.4 branch soon? >>>>> >>>>> >>>>> >>>>> Sincerely, >>>>> >>>>> >>>>> >>>>> DB Tsai >>>>> ---------------------------------------------------------- Web: https:/ / >>>>> www. >>>>> dbtsai. com ( https://www.dbtsai.com/ ) >>>>> PGP Key ID: 42E5B25A8F7A82C1 >>>>> >>>>> >>>>> >>>>> On Sat, Mar 23, 2019 at 12:04 PM Sean Owen < srowen@ apache. org ( >>>>> sro...@apache.org ) > wrote: >>>>> >>>>> >>>>>> >>>>>> >>>>>> I think we can/should get in SPARK-26961 too; it's all but ready to >>>>>> commit. >>>>>> >>>>>> >>>>>> >>>>>> On Sat, Mar 23, 2019 at 2:02 PM DB Tsai < dbtsai@ dbtsai. com ( >>>>>> dbt...@dbtsai.com ) > wrote: >>>>>> >>>>>> >>>>>>> >>>>>>> >>>>>>> -1 >>>>>>> >>>>>>> >>>>>>> >>>>>>> I will fail RC8, and cut another RC9 on Monday to include SPARK-27160, >>>>>>> SPARK-27178, SPARK-27112. Please let me know if there is any critical PR >>>>>>> that has to be back-ported into branch-2.4. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Sincerely, >>>>>>> >>>>>>> >>>>>>> >>>>>>> DB Tsai >>>>>>> ---------------------------------------------------------- Web: https:/ >>>>>>> / www. >>>>>>> dbtsai. com ( https://www.dbtsai.com/ ) >>>>>>> PGP Key ID: 42E5B25A8F7A82C1 >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Mar 22, 2019 at 12:28 AM DB Tsai < dbtsai@ dbtsai. com ( >>>>>>> dbt...@dbtsai.com ) > wrote: >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Since we have couple concerns and hesitations to release rc8, how >>>>>>>> about we >>>>>>>> give it couple days, and have another vote on March 25, Monday? In this >>>>>>>> case, I will cut another rc9 in the Monday morning. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Darcy, as Dongjoon mentioned, >>>>>>>> https:/ / github. com/ apache/ spark/ pull/ 24092 ( >>>>>>>> https://github.com/apache/spark/pull/24092 ) is conflict against >>>>>>>> branch-2.4, can you make anther PR against branch-2.4 so we can include >>>>>>>> the ORC fix in 2.4.1? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Sincerely, >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> DB Tsai >>>>>>>> ---------------------------------------------------------- Web: >>>>>>>> https:/ / www. >>>>>>>> dbtsai. com ( https://www.dbtsai.com/ ) >>>>>>>> PGP Key ID: 42E5B25A8F7A82C1 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Mar 20, 2019 at 9:11 PM Felix Cheung < felixcheung_m@ hotmail. >>>>>>>> com >>>>>>>> ( felixcheun...@hotmail.com ) > wrote: >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Reposting for shane here >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> [SPARK-27178] >>>>>>>>> https:/ / github. com/ apache/ spark/ commit/ >>>>>>>>> 342e91fdfa4e6ce5cc3a0da085d1fe723184021b >>>>>>>>> ( >>>>>>>>> https://github.com/apache/spark/commit/342e91fdfa4e6ce5cc3a0da085d1fe723184021b >>>>>>>>> ) >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Is problematic too and it’s not in the rc8 cut >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> https:/ / github. com/ apache/ spark/ commits/ branch-2. 4 ( >>>>>>>>> https://github.com/apache/spark/commits/branch-2.4 ) >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> (Personally I don’t want to delay 2.4.1 either..) >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> ________________________________ >>>>>>>>> From: Sean Owen < srowen@ gmail. com ( sro...@gmail.com ) > >>>>>>>>> Sent: Wednesday, March 20, 2019 11:18 AM >>>>>>>>> To: DB Tsai >>>>>>>>> Cc: dev >>>>>>>>> Subject: Re: [VOTE] Release Apache Spark 2.4.1 (RC8) >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> +1 for this RC. The tag is correct, licenses and sigs check out, >>>>>>>>> tests of >>>>>>>>> the source with most profiles enabled works for me. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Mar 19, 2019 at 5:28 PM DB Tsai < d_tsai@ apple. com. invalid >>>>>>>>> ( >>>>>>>>> d_t...@apple.com.invalid ) > wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Please vote on releasing the following candidate as Apache Spark >>>>>>>>>> version >>>>>>>>>> 2.4.1. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The vote is open until March 23 PST and passes if a majority +1 PMC >>>>>>>>>> votes >>>>>>>>>> are cast, with a minimum of 3 +1 votes. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [ ] +1 Release this package as Apache Spark 2.4.1 >>>>>>>>>> [ ] -1 Do not release this package because ... >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> To learn more about Apache Spark, please see http:/ / spark. apache. >>>>>>>>>> org/ ( >>>>>>>>>> http://spark.apache.org/ ) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The tag to be voted on is v2.4.1-rc8 (commit >>>>>>>>>> 746b3ddee6f7ad3464e326228ea226f5b1f39a41): https:/ / github. com/ >>>>>>>>>> apache/ spark/ >>>>>>>>>> tree/ v2. 4. 1-rc8 ( https://github.com/apache/spark/tree/v2.4.1-rc8 >>>>>>>>>> ) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The release files, including signatures, digests, etc. can be found >>>>>>>>>> at: https:/ >>>>>>>>>> / dist. apache. org/ repos/ dist/ dev/ spark/ v2. 4. 1-rc8-bin/ ( >>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v2.4.1-rc8-bin/ ) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Signatures used for Spark RCs can be found in this file: https:/ / >>>>>>>>>> dist. apache. >>>>>>>>>> org/ repos/ dist/ dev/ spark/ KEYS ( >>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS ) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The staging repository for this release can be found at: https:/ / >>>>>>>>>> repository. >>>>>>>>>> apache. org/ content/ repositories/ orgapachespark-1318/ ( >>>>>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1318/ >>>>>>>>>> ) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The documentation corresponding to this release can be found at: >>>>>>>>>> https:/ / >>>>>>>>>> dist. apache. org/ repos/ dist/ dev/ spark/ v2. 4. 1-rc8-docs/ ( >>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v2.4.1-rc8-docs/ ) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The list of bug fixes going into 2.4.1 can be found at the following >>>>>>>>>> URL: https:/ >>>>>>>>>> / issues. apache. org/ jira/ projects/ SPARK/ versions/ 2. 4. 1 ( >>>>>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/2.4.1 ) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> FAQ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ========================= >>>>>>>>>> How can I help test this release? >>>>>>>>>> ========================= >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> If you are a Spark user, you can help us test this release by taking >>>>>>>>>> an >>>>>>>>>> existing Spark workload and running on this release candidate, then >>>>>>>>>> reporting any regressions. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> If you're working in PySpark you can set up a virtual env and >>>>>>>>>> install the >>>>>>>>>> current RC and see if anything important breaks, in the Java/Scala >>>>>>>>>> you can >>>>>>>>>> add the staging repository to your projects resolvers and test with >>>>>>>>>> the RC >>>>>>>>>> (make sure to clean up the artifact cache before/after so you don't >>>>>>>>>> end up >>>>>>>>>> building with a out of date RC going forward). >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> =========================================== >>>>>>>>>> What should happen to JIRA tickets still targeting 2.4.1? >>>>>>>>>> =========================================== >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The current list of open tickets targeted at 2.4.1 can be found at: >>>>>>>>>> https:/ >>>>>>>>>> / issues. apache. org/ jira/ projects/ SPARK ( >>>>>>>>>> https://issues.apache.org/jira/projects/SPARK ) and search for >>>>>>>>>> "Target >>>>>>>>>> Version/s" = 2.4.1 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Committers should look at those and triage. Extremely important bug >>>>>>>>>> fixes, >>>>>>>>>> documentation, and API tweaks that impact compatibility should be >>>>>>>>>> worked >>>>>>>>>> on immediately. Everything else please retarget to an appropriate >>>>>>>>>> release. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ================== >>>>>>>>>> But my bug isn't fixed? >>>>>>>>>> ================== >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> In order to make timely releases, we will typically not hold the >>>>>>>>>> release >>>>>>>>>> unless the bug in question is a regression from the previous >>>>>>>>>> release. That >>>>>>>>>> being said, if there is something which is a regression that has not >>>>>>>>>> been >>>>>>>>>> correctly targeted please ping me or a committer to help target the >>>>>>>>>> issue. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> DB Tsai | Siri Open Source Technologies [not a contribution] | >>>>>>>>>> Apple, >>>>>>>>>> Inc >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>> To >>>>>>>>>> unsubscribe e-mail: dev-unsubscribe@ spark. apache. org ( >>>>>>>>>> dev-unsubscr...@spark.apache.org ) >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> To >>>>>>>>> unsubscribe e-mail: dev-unsubscribe@ spark. apache. org ( >>>>>>>>> dev-unsubscr...@spark.apache.org ) >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- To >>>>>> unsubscribe e-mail: dev-unsubscribe@ spark. apache. org ( >>>>>> dev-unsubscr...@spark.apache.org ) >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>> >>> >>> >>> --------------------------------------------------------------------- To >>> unsubscribe e-mail: dev-unsubscribe@ spark. apache. org ( >>> dev-unsubscr...@spark.apache.org ) >>> >>> >> >> >> >> --------------------------------------------------------------------- To >> unsubscribe e-mail: dev-unsubscribe@ spark. apache. org ( >> dev-unsubscr...@spark.apache.org ) >> >> > > > > --------------------------------------------------------------------- To > unsubscribe e-mail: dev-unsubscribe@ spark. apache. org ( > dev-unsubscr...@spark.apache.org ) > > >
