[
https://issues.apache.org/jira/browse/THRIFT-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jens Geyer updated THRIFT-5435:
-------------------------------
Component/s: Java - Library
> Hive Server 2 crashes when Thrift library encounters particular security error
> ------------------------------------------------------------------------------
>
> Key: THRIFT-5435
> URL: https://issues.apache.org/jira/browse/THRIFT-5435
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Reporter: Matt McCline
> Priority: Major
>
> Portion of a blog called "NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN
> HIVESERVER2 LOG" by Eric Liu are below. NOTE: The web link causes the Apache
> bot to remove it. You will need to Google it yourself.
> It addresses HIve Server 2 crashes our customers have seen and filed customer
> service complaints on. The Apache Thrift library raises a fatal
> RuntimeException when in my opinion it should raise a non-fatal Exception
> instead.
> Portion of the blog:
> -----------------------------------------------------------------------------------------------------
> "I have seen lots of users complain about seeing lots of “No data or no sasl
> data in the stream” errors in the HiveServer2 server log, yet they have not
> noticed any performance impact nor query failure for Hive. So I think it
> would be good to write a blog about the possible reason behind this to
> clarify and remove the concerns that users have. The following shows the full
> error message and stacktrace taken from HiveServer2 log:
>
> ERROR org.apache.thrift.server.TThreadPoolServer: [HiveServer2-Handler-Pool:
> Thread-533556]: Error occurred during processing of message.
> java.lang.RuntimeException:
> org.apache.thrift.transport.TSaslTransportException: No data or no sasl data
> in the stream
> at
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:765)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:762)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:360)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1687)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:762)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift.transport.TSaslTransportException: No data or
> no sasl data in the stream
> The likely cause is below:
> # You have kerberos enabled
> # You have multiple HiveServer2 hosts
> # You have Load Balancer enabled in front of all HS2 servers that have such
> errors
> If you have above setup, the error message you saw in HiveServer2 is harmless
> and can be safely ignored. This just indicated that SASL negotiation failed
> for one particular Hive client, which in this case would be the Load Balancer
> who pings regularly to those HiveServer2’s to check for connectivity. Those
> pings from LB were trying with PLAIN TCP connection, hence those messages.
> -----------------------------------------------------------------------------------------------------
> This is similar exception information from Hive Server 2 that crashed on our
> customer's machine:
> ExceptionClassName
> java.lang.RuntimeException
> ExceptionStackTrace
> java.lang.RuntimeException:
> org.apache.thrift.transport.TSaslTransportException: No data or no sasl data
> in the stream
> at
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:360)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no
> sasl data in the stream
> at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326)
> at
> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
> at
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
> ... 10 more
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
