[ANNOUNCE] Apache Spark 3.3.3 released

[Yuming Wang]

We are happy to announce the availability of Apache Spark 3.3.3!

Spark 3.3.3 is a maintenance release containing stability fixes. This
release is based on the branch-3.3 maintenance branch of Spark. We strongly
recommend all 3.3 users to upgrade to this stable release.

To download Spark 3.3.3, head over to the download page:
https://spark.apache.org/downloads.html

To view the release notes:
https://spark.apache.org/releases/spark-release-3-3-3.html

We would like to acknowledge all community members for contributing to this
release. This release would not have been possible without you.

[ANNOUNCE] Apache Spark 3.3.1 released

[Yuming Wang]

We are happy to announce the availability of Apache Spark 3.3.1!

Spark 3.3.1 is a maintenance release containing stability fixes. This
release is based on the branch-3.3 maintenance branch of Spark. We strongly
recommend all 3.3 users to upgrade to this stable release.

To download Spark 3.3.1, head over to the download page:
https://spark.apache.org/downloads.html

To view the release notes:
https://spark.apache.org/releases/spark-release-3-3-1.html

We would like to acknowledge all community members for contributing to this
release. This release would not have been possible without you.

Re: [ANNOUNCE] Apache Spark 3.2.1 released

[Yuming Wang]

Thank you Huaxin.

On Sat, Jan 29, 2022 at 9:08 AM huaxin gao  wrote:

> We are happy to announce the availability of Spark 3.2.1!
>
> Spark 3.2.1 is a maintenance release containing stability fixes. This
> release is based on the branch-3.2 maintenance branch of Spark. We strongly
> recommend all 3.2 users to upgrade to this stable release.
>
> To download Spark 3.2.1, head over to the download page:
> https://spark.apache.org/downloads.html
>
> To view the release notes:
> https://spark.apache.org/releases/spark-release-3-2-1.html
>
> We would like to acknowledge all community members for contributing to this
> release. This release would not have been possible without you.
>
> Huaxin Gao
>

Re: [ANNOUNCE] Apache Spark 3.2.0

[Yuming Wang]

Congrats and thanks!

On Tue, Oct 19, 2021 at 10:17 PM Gengliang Wang  wrote:

> Hi all,
>
> Apache Spark 3.2.0 is the third release of the 3.x line. With tremendous
> contribution from the open-source community, this release managed to
> resolve in excess of 1,700 Jira tickets.
>
> We'd like to thank our contributors and users for their contributions and
> early feedback to this release. This release would not have been possible
> without you.
>
> To download Spark 3.2.0, head over to the download page:
> https://spark.apache.org/downloads.html
>
> To view the release notes:
> https://spark.apache.org/releases/spark-release-3-2-0.html
>

Re: beeline spark thrift server issue

[Yuming Wang]

Unable to access log(https://pastebin.com/G5Mwaw7E).

On Thu, May 13, 2021 at 11:23 PM Suryansh Agnihotri <
sagnihotri2...@gmail.com> wrote:

> Hi
> I was trying to access spark sql through jdbc but facing some error. I am
> trying to run beeline
>
> ! /usr/lib/spark/bin/beeline -u
> 'jdbc:hive2://host:10016/default;transportMode=binary'  -e '' 2>&1| awk
> '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL' -e 'Error: Could
> not open'
>
> Error: Could not open client transport with JDBC Uri:
> host:10016/default;transportMode=binary: java.net.ConnectException:
> Connection refused (Connection refused) (state=08S01,code=0)
>
> hive.server2.thrift.port=10016 and mode is binary.
> I verified the process is running on this port.
> I checked the spark thrift server logs https://pastebin.com/G5Mwaw7E
> It says "java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient"
> (pasted the logs in above link).
>
> *I am using spark version 3.1.0 , hive 3.1.2 and hadoop 3.1.2.*
> Following this guide
>
> https://spark.apache.org/docs/latest/sql-data-sources-hive-tables.html#interacting-with-different-versions-of-hive-metastore
>
> I set "spark.sql.hive.metastore.version" to 3.1.2 and set jars to point
> to hivemetastore 3.1.2 jars but still getting the same error.
>
> From logs I also got suspicious about a mismatch in the version of
> datanucleus core in hive and spark but both use the same version.
> https://github.com/apache/hive/blob/branch-3.1/pom.xml#L129
> https://github.com/apache/spark/blob/branch-3.1/pom.xml#L184 (edited)
>
> Is this a known issue , how should this be fixed. Let me know if anything
> else is required.
> Thanks
>

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

[Yuming Wang]

Great work, Hyukjin!

On Wed, Mar 3, 2021 at 9:50 AM Hyukjin Kwon  wrote:

> We are excited to announce Spark 3.1.1 today.
>
> Apache Spark 3.1.1 is the second release of the 3.x line. This release adds
> Python type annotations and Python dependency management support as part
> of Project Zen.
> Other major updates include improved ANSI SQL compliance support, history
> server support
> in structured streaming, the general availability (GA) of Kubernetes and
> node decommissioning
> in Kubernetes and Standalone. In addition, this release continues to focus
> on usability, stability,
> and polish while resolving around 1500 tickets.
>
> We'd like to thank our contributors and users for their contributions and
> early feedback to
> this release. This release would not have been possible without you.
>
> To download Spark 3.1.1, head over to the download page:
> http://spark.apache.org/downloads.html
>
> To view the release notes:
> https://spark.apache.org/releases/spark-release-3-1-1.html
>
>

[ANNOUNCE] Announcing Apache Spark 3.0.0-preview2

[Yuming Wang]

Hi all,

To enable wide-scale community testing of the upcoming Spark 3.0 release,
the Apache Spark community has posted a new preview release of Spark 3.0.
This preview is *not a stable release in terms of either API or
functionality*, but it is meant to give the community early access to try
the code that will become Spark 3.0. If you would like to test the release,
please download it, and send feedback using either the mailing lists
 or JIRA

.

There are a lot of exciting new features added to Spark 3.0, including
Dynamic Partition Pruning, Adaptive Query Execution, Accelerator-aware
Scheduling, Data Source API with Catalog Supports, Vectorization in SparkR,
support of Hadoop 3/JDK 11/Scala 2.12, and many more. For a full list of
major features and changes in Spark 3.0.0-preview2, please check the thread(
http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-3-0-preview-release-feature-list-and-major-changes-td28050.html
 and
http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-3-0-preview-release-2-td28491.html
).

We'd like to thank our contributors and users for their contributions and
early feedback to this release. This release would not have been possible
without you.

To download Spark 3.0.0-preview2, head over to the download page:
https://archive.apache.org/dist/spark/spark-3.0.0-preview2

Happy Holidays.

Yuming