Karthik Manamcheri created HIVE-21028: -----------------------------------------
Summary: get_table_meta should use a fetch plan Key: HIVE-21028 URL: https://issues.apache.org/jira/browse/HIVE-21028 Project: Hive Issue Type: Improvement Reporter: Karthik Manamcheri The {{getTableMeta}} call retrieves the tables, loops through the tables and during this loop it retrieves the database object to get the containing database name. DataNuclues does a lazy retrieval and so, when the first call to get all the tables is done, it does not retrieve the database objects. When this query is executed {code}query = pm.newQuery(MTable.class, filterBuilder.toString()); {code} it loads all the tables, and when you do {code} table.getDatabase().getName() {code} it then goes and retrieves the database object. *However*, there could be another thread which actually has deleted the database!! If this happens, we end up with exceptions such as {code} 2018-12-04 22:25:06,525 INFO DataNucleus.Datastore.Retrieve: [pool-7-thread-191]: Object with id "6930391[OID]org.apache.hadoop.hive.metastore.model.MTable" not found ! 2018-12-04 22:25:06,527 WARN DataNucleus.Persistence: [pool-7-thread-191]: Exception thrown by StateManager.isLoaded No such database row org.datanucleus.exceptions.NucleusObjectNotFoundException: No such database row {code} We see this happen especially with calls which retrieve all the tables in all the databases (basically a call to get_table_meta with dbNames="\*" and tableNames="\*"). To avoid this, we can define a custom fetch plan and activate it only for the get_table_meta query. This fetch plan would fetch the database object along with the MTable object. We would first create a fetch plan on the pmf {code} pmf.getFetchGroup(MTable.class, "mtable_db_fetch_group").addMember("database"); {code} Then we use it just before calling the query {code} pm.getFetchPlan().addGroup("mtable_db_fetch_group"); query = pm.newQuery(MTable.class, filterBuilder.toString()); Collection<MTable> tables = (Collection<MTable>) query.executeWithArray(...); ... {code} Before the API call ends, we can remove the fetch plan by {code} pm.getFetchPlan().removeGroup("mtable_db_fetch_group"); {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)