Sean Roberts created SPARK-23089:
------------------------------------
Summary: "Unable to create operation log session directory" when
parent directory not present
Key: SPARK-23089
URL: https://issues.apache.org/jira/browse/SPARK-23089
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.2.0
Environment: When creating a session directory, Thrift should create
the parent directory _(i.e. /tmp/hive/operation_logs)_ if it is not present.
It's common for operators to clean-up old and empty directories in /tmp, or to
have tools (systemd-tmpfiles or tmpwatch) that do it automatically.
This was fixed in HIVE-12262 but not in Spark Thrift as seen by this:
{code}18/01/15 14:22:49 WARN HiveSessionImpl: Unable to create operation log
session directory:
/tmp/hive/operation_logs/683a6318-adc4-42c4-b665-11dad14d7ec7{code}
Resolved by manually creating /tmp/hive/operation_logs/
We are experiencing this with Spark Thrift provided by:
/usr/hdp/2.6.3.0-235/spark2/jars/spark-hive-thriftserver_2.11-2.2.0.2.6.3.0-235.jar
Reporter: Sean Roberts
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
