-1 from my side as well, found this today. While testing Apache iceberg with 3.3 found this bug where a table with partitions with null values we get a NPE on partition discovery, earlier we use to get `DEFAULT_PARTITION_NAME`
Please look into : https://issues.apache.org/jira/browse/SPARK-39417 for more details Regards, Prashant Singh On Wed, Jun 8, 2022 at 10:27 PM Jerry Peng <jerry.boyang.p...@gmail.com> wrote: > > > I agree with Jungtaek, -1 from me because of the issue of Kafka source > throwing an error with an incorrect error message that was introduced > recently. This may mislead users and cause unnecessary confusion. > > On Wed, Jun 8, 2022 at 12:04 AM Jungtaek Lim <kabhwan.opensou...@gmail.com> > wrote: > >> Apologize for late participation. >> >> I'm sorry, but -1 (non-binding) from me. >> >> Unfortunately I found a major user-facing issue which hurts UX seriously >> on Kafka data source usage. >> >> In some cases, Kafka data source can throw IllegalStateException for the >> case of failOnDataLoss=true which condition is bound to the state of Kafka >> topic (not Spark's issue). With the recent change of Spark, >> IllegalStateException is now bound to the "internal error", and Spark gives >> incorrect guidance to the end users, telling to end users that Spark has a >> bug and they are encouraged to file a JIRA ticket which is simply wrong. >> >> Previously, Kafka data source provided the error message with the context >> why it failed, and how to workaround it. I feel this is a serious >> regression on UX. >> >> Please look into https://issues.apache.org/jira/browse/SPARK-39412 for >> more details. >> >> >> On Wed, Jun 8, 2022 at 3:40 PM Hyukjin Kwon <gurwls...@gmail.com> wrote: >> >>> Okay. Thankfully the binary release is fine per >>> https://github.com/apache/spark/blob/v3.3.0-rc5/dev/create-release/release-build.sh#L268 >>> . >>> The source package (and GitHub tag) has 3.3.0.dev0, and the binary >>> package has 3.3.0. Technically this is not a blocker now because PyPI >>> upload will be able to be made correctly. >>> I lowered the priority to critical. I switch my -1 to 0. >>> >>> On Wed, 8 Jun 2022 at 15:17, Hyukjin Kwon <gurwls...@gmail.com> wrote: >>> >>>> Arrrgh .. I am very sorry that I found this problem late. >>>> RC 5 does not have the correct version of PySpark, see >>>> https://github.com/apache/spark/blob/v3.3.0-rc5/python/pyspark/version.py#L19 >>>> I think the release script was broken because the version now has 'str' >>>> type, see >>>> https://github.com/apache/spark/blob/v3.3.0-rc5/dev/create-release/release-tag.sh#L88 >>>> I filed a JIRA at https://issues.apache.org/jira/browse/SPARK-39411 >>>> >>>> -1 from me >>>> >>>> >>>> >>>> On Wed, 8 Jun 2022 at 13:16, Cheng Pan <pan3...@gmail.com> wrote: >>>> >>>>> +1 (non-binding) >>>>> >>>>> * Verified SPARK-39313 has been address[1] >>>>> * Passed integration test w/ Apache Kyuubi (Incubating)[2] >>>>> >>>>> [1] https://github.com/housepower/spark-clickhouse-connector/pull/123 >>>>> [2] https://github.com/apache/incubator-kyuubi/pull/2817 >>>>> >>>>> Thanks, >>>>> Cheng Pan >>>>> >>>>> On Wed, Jun 8, 2022 at 7:04 AM Chris Nauroth <cnaur...@apache.org> >>>>> wrote: >>>>> > >>>>> > +1 (non-binding) >>>>> > >>>>> > * Verified all checksums. >>>>> > * Verified all signatures. >>>>> > * Built from source, with multiple profiles, to full success, for >>>>> Java 11 and Scala 2.13: >>>>> > * build/mvn -Phadoop-3 -Phadoop-cloud -Phive-thriftserver >>>>> -Pkubernetes -Pscala-2.13 -Psparkr -Pyarn -DskipTests clean package >>>>> > * Tests passed. >>>>> > * Ran several examples successfully: >>>>> > * bin/spark-submit --class org.apache.spark.examples.SparkPi >>>>> examples/jars/spark-examples_2.12-3.3.0.jar >>>>> > * bin/spark-submit --class >>>>> org.apache.spark.examples.sql.hive.SparkHiveExample >>>>> examples/jars/spark-examples_2.12-3.3.0.jar >>>>> > * bin/spark-submit >>>>> examples/src/main/python/streaming/network_wordcount.py localhost 9999 >>>>> > * Tested some of the issues that blocked prior release candidates: >>>>> > * bin/spark-sql -e 'SELECT (SELECT IF(x, 1, 0)) AS a FROM >>>>> (SELECT true) t(x) UNION SELECT 1 AS a;' >>>>> > * bin/spark-sql -e "select date '2018-11-17' > 1" >>>>> > * SPARK-39293 ArrayAggregate fix >>>>> > >>>>> > Chris Nauroth >>>>> > >>>>> > >>>>> > On Tue, Jun 7, 2022 at 1:30 PM Cheng Su <chen...@fb.com.invalid> >>>>> wrote: >>>>> >> >>>>> >> +1 (non-binding). Built and ran some internal test for Spark SQL. >>>>> >> >>>>> >> >>>>> >> >>>>> >> Thanks, >>>>> >> >>>>> >> Cheng Su >>>>> >> >>>>> >> >>>>> >> >>>>> >> From: L. C. Hsieh <vii...@gmail.com> >>>>> >> Date: Tuesday, June 7, 2022 at 1:23 PM >>>>> >> To: dev <dev@spark.apache.org> >>>>> >> Subject: Re: [VOTE] Release Spark 3.3.0 (RC5) >>>>> >> >>>>> >> +1 >>>>> >> >>>>> >> Liang-Chi >>>>> >> >>>>> >> On Tue, Jun 7, 2022 at 1:03 PM Gengliang Wang <ltn...@gmail.com> >>>>> wrote: >>>>> >> > >>>>> >> > +1 (non-binding) >>>>> >> > >>>>> >> > Gengliang >>>>> >> > >>>>> >> > On Tue, Jun 7, 2022 at 12:24 PM Thomas Graves < >>>>> tgraves...@gmail.com> wrote: >>>>> >> >> >>>>> >> >> +1 >>>>> >> >> >>>>> >> >> Tom Graves >>>>> >> >> >>>>> >> >> On Sat, Jun 4, 2022 at 9:50 AM Maxim Gekk >>>>> >> >> <maxim.g...@databricks.com.invalid> wrote: >>>>> >> >> > >>>>> >> >> > Please vote on releasing the following candidate as Apache >>>>> Spark version 3.3.0. >>>>> >> >> > >>>>> >> >> > The vote is open until 11:59pm Pacific time June 8th and >>>>> passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. >>>>> >> >> > >>>>> >> >> > [ ] +1 Release this package as Apache Spark 3.3.0 >>>>> >> >> > [ ] -1 Do not release this package because ... >>>>> >> >> > >>>>> >> >> > To learn more about Apache Spark, please see >>>>> http://spark.apache.org/ >>>>> >> >> > >>>>> >> >> > The tag to be voted on is v3.3.0-rc5 (commit >>>>> 7cf29705272ab8e8c70e8885a3664ad8ae3cd5e9): >>>>> >> >> > https://github.com/apache/spark/tree/v3.3.0-rc5 >>>>> >> >> > >>>>> >> >> > The release files, including signatures, digests, etc. can be >>>>> found at: >>>>> >> >> > https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc5-bin/ >>>>> >> >> > >>>>> >> >> > Signatures used for Spark RCs can be found in this file: >>>>> >> >> > https://dist.apache.org/repos/dist/dev/spark/KEYS >>>>> >> >> > >>>>> >> >> > The staging repository for this release can be found at: >>>>> >> >> > >>>>> https://repository.apache.org/content/repositories/orgapachespark-1406 >>>>> >> >> > >>>>> >> >> > The documentation corresponding to this release can be found >>>>> at: >>>>> >> >> > https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc5-docs/ >>>>> >> >> > >>>>> >> >> > The list of bug fixes going into 3.3.0 can be found at the >>>>> following URL: >>>>> >> >> > >>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369 >>>>> >> >> > >>>>> >> >> > This release is using the release script of the tag v3.3.0-rc5. >>>>> >> >> > >>>>> >> >> > >>>>> >> >> > FAQ >>>>> >> >> > >>>>> >> >> > ========================= >>>>> >> >> > How can I help test this release? >>>>> >> >> > ========================= >>>>> >> >> > If you are a Spark user, you can help us test this release by >>>>> taking >>>>> >> >> > an existing Spark workload and running on this release >>>>> candidate, then >>>>> >> >> > reporting any regressions. >>>>> >> >> > >>>>> >> >> > If you're working in PySpark you can set up a virtual env and >>>>> install >>>>> >> >> > the current RC and see if anything important breaks, in the >>>>> Java/Scala >>>>> >> >> > you can add the staging repository to your projects resolvers >>>>> and test >>>>> >> >> > with the RC (make sure to clean up the artifact cache >>>>> before/after so >>>>> >> >> > you don't end up building with a out of date RC going forward). >>>>> >> >> > >>>>> >> >> > =========================================== >>>>> >> >> > What should happen to JIRA tickets still targeting 3.3.0? >>>>> >> >> > =========================================== >>>>> >> >> > The current list of open tickets targeted at 3.3.0 can be >>>>> found at: >>>>> >> >> > https://issues.apache.org/jira/projects/SPARK and search for >>>>> "Target Version/s" = 3.3.0 >>>>> >> >> > >>>>> >> >> > Committers should look at those and triage. Extremely >>>>> important bug >>>>> >> >> > fixes, documentation, and API tweaks that impact compatibility >>>>> should >>>>> >> >> > be worked on immediately. Everything else please retarget to an >>>>> >> >> > appropriate release. >>>>> >> >> > >>>>> >> >> > ================== >>>>> >> >> > But my bug isn't fixed? >>>>> >> >> > ================== >>>>> >> >> > In order to make timely releases, we will typically not hold >>>>> the >>>>> >> >> > release unless the bug in question is a regression from the >>>>> previous >>>>> >> >> > release. That being said, if there is something which is a >>>>> regression >>>>> >> >> > that has not been correctly targeted please ping me or a >>>>> committer to >>>>> >> >> > help target the issue. >>>>> >> >> > >>>>> >> >> > Maxim Gekk >>>>> >> >> > >>>>> >> >> > Software Engineer >>>>> >> >> > >>>>> >> >> > Databricks, Inc. >>>>> >> >> >>>>> >> >> >>>>> --------------------------------------------------------------------- >>>>> >> >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>>> >> >> >>>>> >> >>>>> >> >>>>> --------------------------------------------------------------------- >>>>> >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>>> >>>>>
