I’d be in favor of a back porting with the idea its a bug fix for a language (admittedly not a version we’ve supported before)
On Mon, Feb 13, 2023 at 9:19 AM L. C. Hsieh <vii...@gmail.com> wrote: > If it is not supported in Spark 3.3.x, it looks like an improvement at > Spark 3.4. > For such cases we usually do not back port. I think this is also why > the PR did not back port when it was merged. > > I'm okay if there is consensus to back port it. > > On Mon, Feb 13, 2023 at 9:08 AM Sean Owen <sro...@gmail.com> wrote: > > > > Does that change change the result for Spark 3.3.x? > > It looks like we do not support Python 3.11 in Spark 3.3.x, which is one > answer to whether this should be changed now. > > But if that's the only change that matters for Python 3.11 and makes it > work, sure I think we should back-port. It doesn't necessarily block a > release but if that's the case, it seems OK to include to me in a next RC. > > > > On Mon, Feb 13, 2023 at 10:53 AM Bjørn Jørgensen < > bjornjorgen...@gmail.com> wrote: > >> > >> There is a fix for python 3.11 > https://github.com/apache/spark/pull/38987 > >> We should have this in more branches. > >> > >> man. 13. feb. 2023 kl. 09:39 skrev Bjørn Jørgensen < > bjornjorgen...@gmail.com>: > >>> > >>> On manjaro it is Python 3.10.9 > >>> > >>> On ubuntu it is Python 3.11.1 > >>> > >>> man. 13. feb. 2023 kl. 03:24 skrev yangjie01 <yangji...@baidu.com>: > >>>> > >>>> Which Python version do you use for testing? When I use the latest > Python 3.11, I can reproduce similar test failures (43 tests of sql module > fail), but when I use python 3.10, they will succeed > >>>> > >>>> > >>>> > >>>> YangJie > >>>> > >>>> > >>>> > >>>> 发件人: Bjørn Jørgensen <bjornjorgen...@gmail.com> > >>>> 日期: 2023年2月13日 星期一 05:09 > >>>> 收件人: Sean Owen <sro...@gmail.com> > >>>> 抄送: "L. C. Hsieh" <vii...@gmail.com>, Spark dev list < > dev@spark.apache.org> > >>>> 主题: Re: [VOTE] Release Spark 3.3.2 (RC1) > >>>> > >>>> > >>>> > >>>> Tried it one more time and the same result. > >>>> > >>>> > >>>> > >>>> On another box with Manjaro > >>>> > >>>> > ------------------------------------------------------------------------ > >>>> [INFO] Reactor Summary for Spark Project Parent POM 3.3.2: > >>>> [INFO] > >>>> [INFO] Spark Project Parent POM ........................... SUCCESS > [01:50 min] > >>>> [INFO] Spark Project Tags ................................. SUCCESS [ > 17.359 s] > >>>> [INFO] Spark Project Sketch ............................... SUCCESS [ > 12.517 s] > >>>> [INFO] Spark Project Local DB ............................. SUCCESS [ > 14.463 s] > >>>> [INFO] Spark Project Networking ........................... SUCCESS > [01:07 min] > >>>> [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS > [ 9.013 s] > >>>> [INFO] Spark Project Unsafe ............................... SUCCESS > [ 8.184 s] > >>>> [INFO] Spark Project Launcher ............................. SUCCESS [ > 10.454 s] > >>>> [INFO] Spark Project Core ................................. SUCCESS > [23:58 min] > >>>> [INFO] Spark Project ML Local Library ..................... SUCCESS [ > 21.218 s] > >>>> [INFO] Spark Project GraphX ............................... SUCCESS > [01:24 min] > >>>> [INFO] Spark Project Streaming ............................ SUCCESS > [04:57 min] > >>>> [INFO] Spark Project Catalyst ............................. SUCCESS > [08:00 min] > >>>> [INFO] Spark Project SQL .................................. SUCCESS > [ 01:02 h] > >>>> [INFO] Spark Project ML Library ........................... SUCCESS > [14:38 min] > >>>> [INFO] Spark Project Tools ................................ SUCCESS > [ 4.394 s] > >>>> [INFO] Spark Project Hive ................................. SUCCESS > [53:43 min] > >>>> [INFO] Spark Project REPL ................................. SUCCESS > [01:16 min] > >>>> [INFO] Spark Project Assembly ............................. SUCCESS > [ 2.186 s] > >>>> [INFO] Kafka 0.10+ Token Provider for Streaming ........... SUCCESS [ > 16.150 s] > >>>> [INFO] Spark Integration for Kafka 0.10 ................... SUCCESS > [01:34 min] > >>>> [INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS > [32:55 min] > >>>> [INFO] Spark Project Examples ............................. SUCCESS [ > 23.800 s] > >>>> [INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS > [ 7.301 s] > >>>> [INFO] Spark Avro ......................................... SUCCESS > [01:19 min] > >>>> [INFO] > ------------------------------------------------------------------------ > >>>> [INFO] BUILD SUCCESS > >>>> [INFO] > ------------------------------------------------------------------------ > >>>> [INFO] Total time: 03:31 h > >>>> [INFO] Finished at: 2023-02-12T21:54:20+01:00 > >>>> [INFO] > ------------------------------------------------------------------------ > >>>> [bjorn@amd7g spark-3.3.2]$ java -version > >>>> openjdk version "17.0.6" 2023-01-17 > >>>> OpenJDK Runtime Environment (build 17.0.6+10) > >>>> OpenJDK 64-Bit Server VM (build 17.0.6+10, mixed mode) > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> :) > >>>> > >>>> > >>>> > >>>> So I'm +1 > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> søn. 12. feb. 2023 kl. 12:53 skrev Bjørn Jørgensen < > bjornjorgen...@gmail.com>: > >>>> > >>>> I use ubuntu rolling > >>>> > >>>> $ java -version > >>>> openjdk version "17.0.6" 2023-01-17 > >>>> OpenJDK Runtime Environment (build 17.0.6+10-Ubuntu-0ubuntu1) > >>>> OpenJDK 64-Bit Server VM (build 17.0.6+10-Ubuntu-0ubuntu1, mixed > mode, sharing) > >>>> > >>>> > >>>> > >>>> I have reboot now and restart ./build/mvn clean package > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> søn. 12. feb. 2023 kl. 04:47 skrev Sean Owen <sro...@gmail.com>: > >>>> > >>>> +1 The tests and all results were the same as ever for me (Java 11, > Scala 2.13, Ubuntu 22.04) > >>>> > >>>> I also didn't see that issue ... maybe somehow locale related? which > could still be a bug. > >>>> > >>>> > >>>> > >>>> On Sat, Feb 11, 2023 at 8:49 PM L. C. Hsieh <vii...@gmail.com> wrote: > >>>> > >>>> Thank you for testing it. > >>>> > >>>> I was going to run it again but still didn't see any errors. > >>>> > >>>> I also checked CI (and looked again now) on branch-3.3 before cutting > RC. > >>>> > >>>> BTW, I didn't find an actual test failure (i.e. "- test_name *** > >>>> FAILED ***") in the log file. > >>>> > >>>> Maybe it is due to the dev env? What dev env you're using to run the > test? > >>>> > >>>> > >>>> On Sat, Feb 11, 2023 at 8:58 AM Bjørn Jørgensen > >>>> <bjornjorgen...@gmail.com> wrote: > >>>> > > >>>> > > >>>> > ./build/mvn clean package > >>>> > > >>>> > Run completed in 1 hour, 18 minutes, 29 seconds. > >>>> > Total number of tests run: 11652 > >>>> > Suites: completed 516, aborted 0 > >>>> > Tests: succeeded 11609, failed 43, canceled 8, ignored 57, pending 0 > >>>> > *** 43 TESTS FAILED *** > >>>> > [INFO] > ------------------------------------------------------------------------ > >>>> > [INFO] Reactor Summary for Spark Project Parent POM 3.3.2: > >>>> > [INFO] > >>>> > [INFO] Spark Project Parent POM ........................... SUCCESS > [ 3.418 s] > >>>> > [INFO] Spark Project Tags ................................. SUCCESS > [ 17.845 s] > >>>> > [INFO] Spark Project Sketch ............................... SUCCESS > [ 20.791 s] > >>>> > [INFO] Spark Project Local DB ............................. SUCCESS > [ 16.527 s] > >>>> > [INFO] Spark Project Networking ........................... SUCCESS > [01:03 min] > >>>> > [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS > [ 9.914 s] > >>>> > [INFO] Spark Project Unsafe ............................... SUCCESS > [ 12.007 s] > >>>> > [INFO] Spark Project Launcher ............................. SUCCESS > [ 7.620 s] > >>>> > [INFO] Spark Project Core ................................. SUCCESS > [40:04 min] > >>>> > [INFO] Spark Project ML Local Library ..................... SUCCESS > [ 29.997 s] > >>>> > [INFO] Spark Project GraphX ............................... SUCCESS > [02:33 min] > >>>> > [INFO] Spark Project Streaming ............................ SUCCESS > [05:51 min] > >>>> > [INFO] Spark Project Catalyst ............................. SUCCESS > [13:29 min] > >>>> > [INFO] Spark Project SQL .................................. FAILURE > [ 01:25 h] > >>>> > [INFO] Spark Project ML Library ........................... SKIPPED > >>>> > [INFO] Spark Project Tools ................................ SKIPPED > >>>> > [INFO] Spark Project Hive ................................. SKIPPED > >>>> > [INFO] Spark Project REPL ................................. SKIPPED > >>>> > [INFO] Spark Project Assembly ............................. SKIPPED > >>>> > [INFO] Kafka 0.10+ Token Provider for Streaming ........... SKIPPED > >>>> > [INFO] Spark Integration for Kafka 0.10 ................... SKIPPED > >>>> > [INFO] Kafka 0.10+ Source for Structured Streaming ........ SKIPPED > >>>> > [INFO] Spark Project Examples ............................. SKIPPED > >>>> > [INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED > >>>> > [INFO] Spark Avro ......................................... SKIPPED > >>>> > [INFO] > ------------------------------------------------------------------------ > >>>> > [INFO] BUILD FAILURE > >>>> > [INFO] > ------------------------------------------------------------------------ > >>>> > [INFO] Total time: 02:30 h > >>>> > [INFO] Finished at: 2023-02-11T17:32:45+01:00 > >>>> > > >>>> > lør. 11. feb. 2023 kl. 06:01 skrev L. C. Hsieh <vii...@gmail.com>: > >>>> >> > >>>> >> Please vote on releasing the following candidate as Apache Spark > version 3.3.2. > >>>> >> > >>>> >> The vote is open until Feb 15th 9AM (PST) and passes if a majority > +1 > >>>> >> PMC votes are cast, with a minimum of 3 +1 votes. > >>>> >> > >>>> >> [ ] +1 Release this package as Apache Spark 3.3.2 > >>>> >> [ ] -1 Do not release this package because ... > >>>> >> > >>>> >> To learn more about Apache Spark, please see > https://spark.apache.org/ > >>>> >> > >>>> >> The tag to be voted on is v3.3.2-rc1 (commit > >>>> >> 5103e00c4ce5fcc4264ca9c4df12295d42557af6): > >>>> >> https://github.com/apache/spark/tree/v3.3.2-rc1 > >>>> >> > >>>> >> The release files, including signatures, digests, etc. can be > found at: > >>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.3.2-rc1-bin/ > >>>> >> > >>>> >> Signatures used for Spark RCs can be found in this file: > >>>> >> https://dist.apache.org/repos/dist/dev/spark/KEYS > >>>> >> > >>>> >> The staging repository for this release can be found at: > >>>> >> > https://repository.apache.org/content/repositories/orgapachespark-1433/ > >>>> >> > >>>> >> The documentation corresponding to this release can be found at: > >>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.3.2-rc1-docs/ > >>>> >> > >>>> >> The list of bug fixes going into 3.3.2 can be found at the > following URL: > >>>> >> https://issues.apache.org/jira/projects/SPARK/versions/12352299 > >>>> >> > >>>> >> This release is using the release script of the tag v3.3.2-rc1. > >>>> >> > >>>> >> FAQ > >>>> >> > >>>> >> ========================= > >>>> >> How can I help test this release? > >>>> >> ========================= > >>>> >> > >>>> >> If you are a Spark user, you can help us test this release by > taking > >>>> >> an existing Spark workload and running on this release candidate, > then > >>>> >> reporting any regressions. > >>>> >> > >>>> >> If you're working in PySpark you can set up a virtual env and > install > >>>> >> the current RC and see if anything important breaks, in the > Java/Scala > >>>> >> you can add the staging repository to your projects resolvers and > test > >>>> >> with the RC (make sure to clean up the artifact cache before/after > so > >>>> >> you don't end up building with a out of date RC going forward). > >>>> >> > >>>> >> =========================================== > >>>> >> What should happen to JIRA tickets > <https://www.google.com/maps/search/n+to+JIRA+tickets?entry=gmail&source=g> > still targeting 3.3.2? > >>>> >> =========================================== > >>>> >> > >>>> >> The current list of open tickets targeted at 3.3.2 can be found at: > >>>> >> https://issues.apache.org/jira/projects/SPARK and search for > "Target > >>>> >> Version/s" = 3.3.2 > >>>> >> > >>>> >> Committers should look at those and triage. Extremely important bug > >>>> >> fixes, documentation, and API tweaks that impact compatibility > should > >>>> >> be worked on immediat > <https://www.google.com/maps/search/orked+on+immediat?entry=gmail&source=g>ely. > Everything else please retarget to an > >>>> >> appropriate release. > >>>> >> > >>>> >> ================== > >>>> >> But my bug isn't fixed? > >>>> >> ================== > >>>> >> > >>>> >> In order to make timely releases, we will typically not hold the > >>>> >> release unless the bug in question is a regression from the > previous > >>>> >> release. That being said, if there is something which is a > regression > >>>> >> that has not been correctly targeted please ping me or a committer > to > >>>> >> help target the issue. > >>>> >> > >>>> >> > --------------------------------------------------------------------- > >>>> >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >>>> >> > >>>> > > >>>> > > >>>> > -- > >>>> > Bjørn Jørgensen > >>>> > Vestre Aspehaug 4, 6010 Ålesund > >>>> > Norge > >>>> > > >>>> > +47 480 94 297 > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> > >>>> Bjørn Jørgensen > >>>> Vestre Aspehaug 4, 6010 Ålesund > >>>> Norge > >>>> > >>>> +47 480 94 297 > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> > >>>> Bjørn Jørgensen > >>>> Vestre Aspehaug 4, 6010 Ålesund > >>>> Norge > >>>> > >>>> +47 480 94 297 > >>> > >>> > >>> > >>> -- > >>> Bjørn Jørgensen > >>> Vestre Aspehaug 4, 6010 Ålesund > >>> Norge > >>> > >>> +47 480 94 297 > >> > >> > >> > >> -- > >> Bjørn Jørgensen > >> Vestre Aspehaug 4, 6010 Ålesund > >> Norge > >> > >> +47 480 94 297 > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > -- Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
