[
https://issues.apache.org/jira/browse/THRIFT-5645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jens Geyer updated THRIFT-5645:
-------------------------------
Summary: libthrift 0.14.0 onwards doesn't work with Spark 3.3.0 (was:
libthrift 0.14.0 onwards doesn't works with Spark 3.3.0)
> libthrift 0.14.0 onwards doesn't work with Spark 3.3.0
> ------------------------------------------------------
>
> Key: THRIFT-5645
> URL: https://issues.apache.org/jira/browse/THRIFT-5645
> Project: Thrift
> Issue Type: Bug
> Components: Java - Library
> Affects Versions: 0.14.0, 0.15.0, 0.14.1, 0.14.2, 0.16.0, 0.17.0
> Reporter: Pratik Malani
> Priority: Major
> Fix For: 0.18.0
>
> Attachments: image-2022-10-03-14-45-30-224.png
>
>
> libthrift:0.13.0 is affected with
> [CVE-2020-13949|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13949]
> Currently I am using Spark 3.3.0 and Hive 2.3.9 with Hadoop 3.3.4.
> When we do an upgrade to use libthrift:0.14.0 and above jar, below exception
> is thrown while starting the Spark Thriftserver.
> {noformat}
> org.apache.hive.service.ServiceException: Failed to Start HiveServer2
> at
> org.apache.hive.service.CompositeService.start(CompositeService.java:79)
> at
> org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:104)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.start(HiveThriftServer2.scala:154)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
> at
> org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> Caused by: java.lang.NoSuchMethodError:
> org.apache.thrift.server.TThreadPoolServer$Args.requestTimeout(I)Lorg/apache/thrift/server/TThreadPoolServer$Args;
> at
> org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.initializeServer(ThriftBinaryCLIService.java:101)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.start(ThriftCLIService.java:176)
> at
> org.apache.hive.service.CompositeService.start(CompositeService.java:69)
> at
> org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:104)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.start(HiveThriftServer2.scala:154)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
> at
> org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {noformat}
> After detailed investigation, found out that since 0.14.0 version, in class
> org.apache.thrift.server.TThreadPoolServer, requestTimeout property has been
> removed.
>
> Even in latest hive release (3.1.3), it is still referencing to the
> requestTimeout attribute has been removed.
> [https://jar-download.com/artifacts/org.apache.hive/hive-service/3.1.3/source-code/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java]
> !image-2022-10-03-14-45-30-224.png|width=722,height=92!
> Wanted to know as part of what change and why this was removed? Is there any
> alternative to use?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
