Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/20864 i take a look at [https://github.com/apache/spark/pull/18666]. i find it can not clean all the ***_resources directories. because when we start HiveThriftServer2, we created two resource directories: `8/03/21 11:23:33 INFO **SessionState: Created local directory: /data1/zdh/spark/hive/tmp/616f66c9-fa4e-4a0c-a63a-10ff97e5019c_resources** 18/03/21 11:23:33 INFO SessionState: Created HDFS directory: /spark-tmp/scratchdir/root/616f66c9-fa4e-4a0c-a63a-10ff97e5019c 18/03/21 11:23:33 INFO SessionState: Created local directory: /data1/zdh/spark/hive/tmp/616f66c9-fa4e-4a0c-a63a-10ff97e5019c 18/03/21 11:23:33 INFO SessionState: Created HDFS directory: /spark-tmp/scratchdir/root/616f66c9-fa4e-4a0c-a63a-10ff97e5019c/_tmp_space.db 18/03/21 11:23:33 INFO HiveClientImpl: Warehouse location for Hive client (version 1.2.2) is file:/media/A/gitspace/spark/dist/sbin/spark-warehouse 18/03/21 11:23:33 INFO HiveMetaStore: 0: get_database: default 18/03/21 11:23:33 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 18/03/21 11:23:33 INFO StateStoreCoordinatorRef: Registered StateStoreCoordinator endpoint 18/03/21 11:23:33 INFO HiveUtils: Initializing execution hive, version 1.2.1 18/03/21 11:23:34 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 18/03/21 11:23:34 INFO ObjectStore: ObjectStore, initialize called 18/03/21 11:23:34 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored 18/03/21 11:23:34 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored 18/03/21 11:23:36 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 18/03/21 11:23:36 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 18/03/21 11:23:36 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 18/03/21 11:23:37 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 18/03/21 11:23:37 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 18/03/21 11:23:37 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 18/03/21 11:23:37 INFO ObjectStore: Initialized ObjectStore 18/03/21 11:23:37 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0 18/03/21 11:23:38 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException 18/03/21 11:23:38 INFO HiveMetaStore: Added admin role in metastore 18/03/21 11:23:38 INFO HiveMetaStore: Added public role in metastore 18/03/21 11:23:38 INFO HiveMetaStore: No user is added in admin role, since config is empty 18/03/21 11:23:38 INFO HiveMetaStore: 0: get_all_databases 18/03/21 11:23:38 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_all_databases 18/03/21 11:23:38 INFO HiveMetaStore: 0: get_functions: db=default pat=* 18/03/21 11:23:38 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_functions: db=default pat=* 18/03/21 11:23:38 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table. 18/03/21 11:23:38 INFO **SessionState: Created local directory: /data1/zdh/spark/hive/tmp/16aa5bb9-33e4-43e6-8bdb-8e0318ab175e_resources** 18/03/21 11:23:38 INFO SessionState: Created HDFS directory: /spark-tmp/scratchdir/root/16aa5bb9-33e4-43e6-8bdb-8e0318ab175e 18/03/21 11:23:38 INFO SessionState: Created local directory: /data1/zdh/spark/hive/tmp/16aa5bb9-33e4-43e6-8bdb-8e0318ab175e 18/03/21 11:23:38 INFO SessionState: Created HDFS directory: /spark-tmp/scratchdir/root/16aa5bb9-33e4-43e6-8bdb-8e0318ab175e/_tmp_space.db 18/03/21 11:23:38 INFO HiveClientImpl: Warehouse location for Hive client (version 1.2.2) is file:/media/A/gitspace/spark/dist/sbin/spark-warehouse` but when stop just remove only one resource directory which is current: `public void close() throws IOException { registry.clear(); if (txnMgr != null) txnMgr.closeTxnManager(); JavaUtils.closeClassLoadersTo(conf.getClassLoader(), parentLoader); **File resourceDir = new File(getConf().getVar(HiveConf.ConfVars.DOWNLOADED_RESOURCES_DIR))**; LOG.debug("Removing resource dir " + resourceDir);`
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org