[
https://issues.apache.org/jira/browse/SPARK-34128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kent Yao updated SPARK-34128:
-----------------------------
Description:
Since Spark 3.0， the `libthrift` has been bumped up from 0.9.3 to 0.12.0.
Due to THRIFT-4805, The SparkThrift Server will print annoying TExceptions. For
example, the current thrift server module test in Github action workflow
outputs more than 200MB of data for this error only, while the total size of
the test log only about 1GB.
I checked the latest `hive-service-rpc` module in the maven center,
[https://mvnrepository.com/artifact/org.apache.hive/hive-service-rpc/3.1.2.]
It still uses the 0.9.3 version.
Due to THRIFT-5274 , It looks like we need to wait for thrift 0.14.0 to release
or downgrade to 0.9.3 to fix this issue if any of them is appropriate
was:
Since Spark 3.0， the `libthrift` has been bumped up from 0.9.3 to 0.12.0.
Due to THRIFT-4805, The SparkThrift Server will print annoying TExceptions. For
example, the current thrift server module test in Github action workflow
outputs more than 200MB of data for this error only, while the total size of
the test log only about 1GB.
I checked the latest `hive-service-rpc` module in the maven center,
[https://mvnrepository.com/artifact/org.apache.hive/hive-service-rpc/3.1.2.]
It still use the 0.9.3 version.
Due to THRIFT-5274 , It looks like we need to
> Suppress excessive logging of TTransportExceptions in Spark ThriftServer
> -------------------------------------------------------------------------
>
> Key: SPARK-34128
> URL: https://issues.apache.org/jira/browse/SPARK-34128
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.1, 3.1.0
> Reporter: Kent Yao
> Priority: Major
>
> Since Spark 3.0， the `libthrift` has been bumped up from 0.9.3 to 0.12.0.
> Due to THRIFT-4805, The SparkThrift Server will print annoying TExceptions.
> For example, the current thrift server module test in Github action workflow
> outputs more than 200MB of data for this error only, while the total size of
> the test log only about 1GB.
>
> I checked the latest `hive-service-rpc` module in the maven center,
> [https://mvnrepository.com/artifact/org.apache.hive/hive-service-rpc/3.1.2.]
> It still uses the 0.9.3 version.
>
> Due to THRIFT-5274 , It looks like we need to wait for thrift 0.14.0 to
> release or downgrade to 0.9.3 to fix this issue if any of them is appropriate
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
