Can we delay the next RC cut until after Labor Day? On Sat, Sep 2, 2023 at 9:59 PM Yuanjian Li <xyliyuanj...@gmail.com> wrote:
> Thank you for all the reports! > The vote has failed. I plan to cut RC4 in two days. > > @Dipayan Dev <dev.dipaya...@gmail.com> I quickly skimmed through the > corresponding ticket, and it doesn't seem to be a regression introduced in > 3.5. Additionally, someone is asking if this is the same issue as > SPARK-35279. > @Yuming Wang <yumw...@apache.org> I will check the signature for RC4 > @Jungtaek Lim <kabhwan.opensou...@gmail.com> I will follow-up with you > regarding SPARK-45045 <https://issues.apache.org/jira/browse/SPARK-45045> > @Wenchen Fan <cloud0...@gmail.com> Agree, we should include the > correctness fix in 3.5 > > Jungtaek Lim <kabhwan.opensou...@gmail.com> 于2023年8月31日周四 23:45写道： > >> My apologies, I have to add another ticket for a blocker, SPARK-45045 >> <https://issues.apache.org/jira/browse/SPARK-45045>. That said, I'm -1 >> (non-binding). >> >> SPARK-43183 <https://issues.apache.org/jira/browse/SPARK-43183> made a >> behavioral change regarding the StreamingQueryListener as well as >> StreamingQuery API as a side-effect, while the intention was more about >> introducing the change in the former one. I just got some reports that the >> behavioral change for StreamingQuery API broke various tests in 3rd party >> data sources. To help 3rd party ecosystems to adopt 3.5 without hassle, I'd >> like to see this be fixed in 3.5.0. >> >> There is no fix yet but I'm working on it. I'll give an update here. >> Maybe we could lower down priority and let the release go with describing >> this as a "known issue", if I couldn't make progress in a couple of days. >> I'm sorry about that. >> >> Thanks, >> Jungtaek Lim >> >> On Fri, Sep 1, 2023 at 12:12 PM Wenchen Fan <cloud0...@gmail.com> wrote: >> >>> Sorry for the last-minute bug report, but we found a regression in 3.5: >>> the SQL INSERT command without a column list fills missing columns with >>> NULL while Spark 3.4 does not allow it. According to the SQL standard, this >>> shouldn't be allowed and thus a regression in 3.5. >>> >>> The fix has been merged but one day after the RC3 cut: >>> https://github.com/apache/spark/pull/42393 . I'm -1 and let's include >>> this fix in 3.5. >>> >>> Thanks, >>> Wenchen >>> >>> On Thu, Aug 31, 2023 at 9:09 PM Ian Manning <ian.a.mann...@gmail.com> >>> wrote: >>> >>>> +1 (non-binding) >>>> >>>> Using Spark Core, Spark SQL, Structured Streaming. >>>> >>>> On Tue, Aug 29, 2023 at 8:12 PM Yuanjian Li <xyliyuanj...@gmail.com> >>>> wrote: >>>> >>>>> Please vote on releasing the following candidate(RC3) as Apache Spark >>>>> version 3.5.0. >>>>> >>>>> The vote is open until 11:59pm Pacific time Aug 31st and passes if a >>>>> majority +1 PMC votes are cast, with a minimum of 3 +1 votes. >>>>> >>>>> [ ] +1 Release this package as Apache Spark 3.5.0 >>>>> >>>>> [ ] -1 Do not release this package because ... >>>>> >>>>> To learn more about Apache Spark, please see http://spark.apache.org/ >>>>> >>>>> The tag to be voted on is v3.5.0-rc3 (commit >>>>> 9f137aa4dc43398aafa0c3e035ed3174182d7d6c): >>>>> >>>>> https://github.com/apache/spark/tree/v3.5.0-rc3 >>>>> >>>>> The release files, including signatures, digests, etc. can be found at: >>>>> >>>>> https://dist.apache.org/repos/dist/dev/spark/v3.5.0-rc3-bin/ >>>>> >>>>> Signatures used for Spark RCs can be found in this file: >>>>> >>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS >>>>> >>>>> The staging repository for this release can be found at: >>>>> >>>>> https://repository.apache.org/content/repositories/orgapachespark-1447 >>>>> >>>>> The documentation corresponding to this release can be found at: >>>>> >>>>> https://dist.apache.org/repos/dist/dev/spark/v3.5.0-rc3-docs/ >>>>> >>>>> The list of bug fixes going into 3.5.0 can be found at the following >>>>> URL: >>>>> >>>>> https://issues.apache.org/jira/projects/SPARK/versions/12352848 >>>>> >>>>> This release is using the release script of the tag v3.5.0-rc3. >>>>> >>>>> >>>>> FAQ >>>>> >>>>> ========================= >>>>> >>>>> How can I help test this release? >>>>> >>>>> ========================= >>>>> >>>>> If you are a Spark user, you can help us test this release by taking >>>>> >>>>> an existing Spark workload and running on this release candidate, then >>>>> >>>>> reporting any regressions. >>>>> >>>>> If you're working in PySpark you can set up a virtual env and install >>>>> >>>>> the current RC and see if anything important breaks, in the Java/Scala >>>>> >>>>> you can add the staging repository to your projects resolvers and test >>>>> >>>>> with the RC (make sure to clean up the artifact cache before/after so >>>>> >>>>> you don't end up building with an out of date RC going forward). >>>>> >>>>> =========================================== >>>>> >>>>> What should happen to JIRA tickets still targeting 3.5.0? >>>>> >>>>> =========================================== >>>>> >>>>> The current list of open tickets targeted at 3.5.0 can be found at: >>>>> >>>>> https://issues.apache.org/jira/projects/SPARK and search for "Target >>>>> Version/s" = 3.5.0 >>>>> >>>>> Committers should look at those and triage. Extremely important bug >>>>> >>>>> fixes, documentation, and API tweaks that impact compatibility should >>>>> >>>>> be worked on immediately. Everything else please retarget to an >>>>> >>>>> appropriate release. >>>>> >>>>> ================== >>>>> >>>>> But my bug isn't fixed? >>>>> >>>>> ================== >>>>> >>>>> In order to make timely releases, we will typically not hold the >>>>> >>>>> release unless the bug in question is a regression from the previous >>>>> >>>>> release. That being said, if there is something which is a regression >>>>> >>>>> that has not been correctly targeted please ping me or a committer to >>>>> >>>>> help target the issue. >>>>> >>>>> Thanks, >>>>> >>>>> Yuanjian Li >>>>> >>>>
