[jira] [Comment Edited] (HIVE-23437) Concurrent partition creation requests cause underlying HDFS folder to be deleted
[ https://issues.apache.org/jira/browse/HIVE-23437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105218#comment-17105218 ] Marc Demierre edited comment on HIVE-23437 at 8/24/21, 8:54 AM: We tried a workaround on the client side to ensure the calls are not simultaneous by delaying them. It didn't solve the issue, only made it rarer. We also observed a second instance of the problem which is slightly different: * T1: ** R1 creates the directory, then is paused/waiting * T2: ** R2 arrives, does not create the directory as it exists ** R2 creates the partition (wins the race on DB) and completes * T3: ** R1 resumes, sees that it failed the DB transaction, deletes the folder Relevant logs (R1=2558, R2=2556): {code:java} 2020-05-11 20:00:00,944 INFO [pool-7-thread-2558]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2558: append_partition_by_name: db=myproj_dev_autodump tbl=myproj_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:00,945 INFO [pool-7-thread-2558]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=kafka-d...@platform.mydomain.net ip=10.222.76.2 cmd=append_partition_by_name: db=myproj_dev_autodump tbl=myproj_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,311 INFO [pool-7-thread-2556]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2556: append_partition_by_name: db=myproj_dev_autodump tbl=myproj_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,311 INFO [pool-7-thread-2556]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=kafka-d...@platform.mydomain.net ip=10.222.76.2 cmd=append_partition_by_name: db=myproj_dev_autodump tbl=myproj_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,481 INFO [pool-7-thread-2558]: common.FileUtils (FileUtils.java:mkdir(573)) - Creating directory if it doesn't exist: hdfs://platform/data/myproj/dev/myproj.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,521 WARN [pool-7-thread-2556]: hive.log (MetaStoreUtils.java:updatePartitionStatsFast(352)) - Updating partition stats fast for: myproj_dev_debug_hive_4 2020-05-11 20:00:01,537 WARN [pool-7-thread-2556]: hive.log (MetaStoreUtils.java:updatePartitionStatsFast(355)) - Updated size to 0 2020-05-11 20:00:01,764 INFO [pool-7-thread-2558]: metastore.hivemetastoressimpl (HiveMetaStoreFsImpl.java:deleteDir(41)) - deleting hdfs://platform/data/myproj/dev/myproj.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,787 INFO [pool-7-thread-2558]: fs.TrashPolicyDefault (TrashPolicyDefault.java:moveToTrash(168)) - Moved: 'hdfs://platform/data/myproj/dev/myproj.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18' to trash at: hdfs://platform/user/kafka-dump/.Trash/Current/data/myproj/dev/myproj.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,787 INFO [pool-7-thread-2558]: metastore.hivemetastoressimpl (HiveMetaStoreFsImpl.java:deleteDir(48)) - Moved to trash: hdfs://platform/data/myproj/dev/myproj.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,788 ERROR [pool-7-thread-2558]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(217)) - Retrying HMSHandler after 2000 ms (attempt 1 of 10) with error: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MPartition@3254e57d" using statement "INSERT INTO "PARTITIONS" ("PART_ID","CREATE_TIME","LAST_ACCESS_TIME","PART_NAME","SD_ID","TBL_ID") VALUES (?,?,?,?,?,?)" failed : ERROR: duplicate key value violates unique constraint "UNIQUEPARTITION" 2020-05-11 20:00:03,788 INFO [pool-7-thread-2558]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2558: append_partition_by_name: db=myproj_dev_autodump tbl=myproj_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:03,788 INFO [pool-7-thread-2558]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=kafka-d...@platform.mydomain.net ip=10.222.76.2 cmd=append_partition_by_name: db=myproj_dev_autodump tbl=myproj_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:03,869 ERROR [pool-7-thread-2558]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(203)) - AlreadyExistsException(message:Partition already exists:Partition(values:[ingestion, hourly, 2020-05-11, 18], dbName:myproj_dev_autodump, tableName:myproj_dev_debug_hive_4, createTime:0, lastAccessTime:0, sd:StorageDescriptor(cols:[FieldSchema(name:name, type:string, comment:null), FieldSchema(name:age, type:int,
[jira] [Comment Edited] (HIVE-23437) Concurrent partition creation requests cause underlying HDFS folder to be deleted
[ https://issues.apache.org/jira/browse/HIVE-23437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105218#comment-17105218 ] Marc Demierre edited comment on HIVE-23437 at 5/12/20, 8:44 AM: We tried a workaround on the client side to ensure the calls are not simultaneous by delaying them. It didn't solve the issue, only made it rarer. We also observed a second instance of the problem which is slightly different: * T1: ** R1 creates the directory, then is paused/waiting * T2: ** R2 arrives, does not create the directory as it exists ** R2 creates the partition (wins the race on DB) and completes * T3: ** R1 resumes, sees that it failed the DB transaction, deletes the folder Relevant logs (R1=2558, R2=2556): {code:java} 2020-05-11 20:00:00,944 INFO [pool-7-thread-2558]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2558: append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:00,945 INFO [pool-7-thread-2558]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=sbd-fh-connect-autodumph...@sbd.corproot.net ip=10.222.76.2 cmd=append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,311 INFO [pool-7-thread-2556]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2556: append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,311 INFO [pool-7-thread-2556]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=sbd-fh-connect-autodumph...@sbd.corproot.net ip=10.222.76.2 cmd=append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,481 INFO [pool-7-thread-2558]: common.FileUtils (FileUtils.java:mkdir(573)) - Creating directory if it doesn't exist: hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,521 WARN [pool-7-thread-2556]: hive.log (MetaStoreUtils.java:updatePartitionStatsFast(352)) - Updating partition stats fast for: sbd_dev_debug_hive_4 2020-05-11 20:00:01,537 WARN [pool-7-thread-2556]: hive.log (MetaStoreUtils.java:updatePartitionStatsFast(355)) - Updated size to 0 2020-05-11 20:00:01,764 INFO [pool-7-thread-2558]: metastore.hivemetastoressimpl (HiveMetaStoreFsImpl.java:deleteDir(41)) - deleting hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,787 INFO [pool-7-thread-2558]: fs.TrashPolicyDefault (TrashPolicyDefault.java:moveToTrash(168)) - Moved: 'hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18' to trash at: hdfs://sbd/user/sbd-fh-connect-autodumphdfs/.Trash/Current/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,787 INFO [pool-7-thread-2558]: metastore.hivemetastoressimpl (HiveMetaStoreFsImpl.java:deleteDir(48)) - Moved to trash: hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,788 ERROR [pool-7-thread-2558]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(217)) - Retrying HMSHandler after 2000 ms (attempt 1 of 10) with error: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MPartition@3254e57d" using statement "INSERT INTO "PARTITIONS" ("PART_ID","CREATE_TIME","LAST_ACCESS_TIME","PART_NAME","SD_ID","TBL_ID") VALUES (?,?,?,?,?,?)" failed : ERROR: duplicate key value violates unique constraint "UNIQUEPARTITION" 2020-05-11 20:00:03,788 INFO [pool-7-thread-2558]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2558: append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:03,788 INFO [pool-7-thread-2558]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=sbd-fh-connect-autodumph...@sbd.corproot.net ip=10.222.76.2 cmd=append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:03,869 ERROR [pool-7-thread-2558]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(203)) - AlreadyExistsException(message:Partition already exists:Partition(values:[ingestion, hourly, 2020-05-11, 18], dbName:sbd_dev_autodump, tableName:sbd_dev_debug_hive_4, createTime:0, lastAccessTime:0, sd:StorageDescriptor(cols:[FieldSchema(name:name, type:string, comment:null), FieldSchema(name:age, type:int, comment:null), FieldSchema(name:salary, type:int,
[jira] [Comment Edited] (HIVE-23437) Concurrent partition creation requests cause underlying HDFS folder to be deleted
[ https://issues.apache.org/jira/browse/HIVE-23437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105218#comment-17105218 ] Marc Demierre edited comment on HIVE-23437 at 5/12/20, 8:25 AM: After trying a workaround on the client side to ensure the calls are not simultaneous by delaying them. It didn't solve the issue, only made it rarer. We also observed a second instance of the problem: * T1: ** R1 creates the directory, then is paused/waiting * T2: ** R2 arrives, does not create the directory as it exists ** R2 creates the partition (wins the race on DB) and completes * T3: ** R1 resumes, sees that it failed the DB transaction, deletes the folder Relevant logs (R1=2558, R2=2556): {code:java} 2020-05-11 20:00:00,944 INFO [pool-7-thread-2558]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2558: append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:00,945 INFO [pool-7-thread-2558]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=sbd-fh-connect-autodumph...@sbd.corproot.net ip=10.222.76.2 cmd=append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,311 INFO [pool-7-thread-2556]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2556: append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,311 INFO [pool-7-thread-2556]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=sbd-fh-connect-autodumph...@sbd.corproot.net ip=10.222.76.2 cmd=append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,481 INFO [pool-7-thread-2558]: common.FileUtils (FileUtils.java:mkdir(573)) - Creating directory if it doesn't exist: hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,521 WARN [pool-7-thread-2556]: hive.log (MetaStoreUtils.java:updatePartitionStatsFast(352)) - Updating partition stats fast for: sbd_dev_debug_hive_4 2020-05-11 20:00:01,537 WARN [pool-7-thread-2556]: hive.log (MetaStoreUtils.java:updatePartitionStatsFast(355)) - Updated size to 0 2020-05-11 20:00:01,764 INFO [pool-7-thread-2558]: metastore.hivemetastoressimpl (HiveMetaStoreFsImpl.java:deleteDir(41)) - deleting hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,787 INFO [pool-7-thread-2558]: fs.TrashPolicyDefault (TrashPolicyDefault.java:moveToTrash(168)) - Moved: 'hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18' to trash at: hdfs://sbd/user/sbd-fh-connect-autodumphdfs/.Trash/Current/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,787 INFO [pool-7-thread-2558]: metastore.hivemetastoressimpl (HiveMetaStoreFsImpl.java:deleteDir(48)) - Moved to trash: hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,788 ERROR [pool-7-thread-2558]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(217)) - Retrying HMSHandler after 2000 ms (attempt 1 of 10) with error: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MPartition@3254e57d" using statement "INSERT INTO "PARTITIONS" ("PART_ID","CREATE_TIME","LAST_ACCESS_TIME","PART_NAME","SD_ID","TBL_ID") VALUES (?,?,?,?,?,?)" failed : ERROR: duplicate key value violates unique constraint "UNIQUEPARTITION" 2020-05-11 20:00:03,788 INFO [pool-7-thread-2558]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2558: append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:03,788 INFO [pool-7-thread-2558]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=sbd-fh-connect-autodumph...@sbd.corproot.net ip=10.222.76.2 cmd=append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:03,869 ERROR [pool-7-thread-2558]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(203)) - AlreadyExistsException(message:Partition already exists:Partition(values:[ingestion, hourly, 2020-05-11, 18], dbName:sbd_dev_autodump, tableName:sbd_dev_debug_hive_4, createTime:0, lastAccessTime:0, sd:StorageDescriptor(cols:[FieldSchema(name:name, type:string, comment:null), FieldSchema(name:age, type:int, comment:null), FieldSchema(name:salary, type:int, comment:null)],
[jira] [Comment Edited] (HIVE-23437) Concurrent partition creation requests cause underlying HDFS folder to be deleted
[ https://issues.apache.org/jira/browse/HIVE-23437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105218#comment-17105218 ] Marc Demierre edited comment on HIVE-23437 at 5/12/20, 8:25 AM: We tried a workaround on the client side to ensure the calls are not simultaneous by delaying them. It didn't solve the issue, only made it rarer. We also observed a second instance of the problem: * T1: ** R1 creates the directory, then is paused/waiting * T2: ** R2 arrives, does not create the directory as it exists ** R2 creates the partition (wins the race on DB) and completes * T3: ** R1 resumes, sees that it failed the DB transaction, deletes the folder Relevant logs (R1=2558, R2=2556): {code:java} 2020-05-11 20:00:00,944 INFO [pool-7-thread-2558]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2558: append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:00,945 INFO [pool-7-thread-2558]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=sbd-fh-connect-autodumph...@sbd.corproot.net ip=10.222.76.2 cmd=append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,311 INFO [pool-7-thread-2556]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2556: append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,311 INFO [pool-7-thread-2556]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=sbd-fh-connect-autodumph...@sbd.corproot.net ip=10.222.76.2 cmd=append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,481 INFO [pool-7-thread-2558]: common.FileUtils (FileUtils.java:mkdir(573)) - Creating directory if it doesn't exist: hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,521 WARN [pool-7-thread-2556]: hive.log (MetaStoreUtils.java:updatePartitionStatsFast(352)) - Updating partition stats fast for: sbd_dev_debug_hive_4 2020-05-11 20:00:01,537 WARN [pool-7-thread-2556]: hive.log (MetaStoreUtils.java:updatePartitionStatsFast(355)) - Updated size to 0 2020-05-11 20:00:01,764 INFO [pool-7-thread-2558]: metastore.hivemetastoressimpl (HiveMetaStoreFsImpl.java:deleteDir(41)) - deleting hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,787 INFO [pool-7-thread-2558]: fs.TrashPolicyDefault (TrashPolicyDefault.java:moveToTrash(168)) - Moved: 'hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18' to trash at: hdfs://sbd/user/sbd-fh-connect-autodumphdfs/.Trash/Current/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,787 INFO [pool-7-thread-2558]: metastore.hivemetastoressimpl (HiveMetaStoreFsImpl.java:deleteDir(48)) - Moved to trash: hdfs://sbd/data/sbd/dev/sbd.dev.debug-hive-4/time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:01,788 ERROR [pool-7-thread-2558]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(217)) - Retrying HMSHandler after 2000 ms (attempt 1 of 10) with error: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MPartition@3254e57d" using statement "INSERT INTO "PARTITIONS" ("PART_ID","CREATE_TIME","LAST_ACCESS_TIME","PART_NAME","SD_ID","TBL_ID") VALUES (?,?,?,?,?,?)" failed : ERROR: duplicate key value violates unique constraint "UNIQUEPARTITION" 2020-05-11 20:00:03,788 INFO [pool-7-thread-2558]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(775)) - 2558: append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:03,788 INFO [pool-7-thread-2558]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=sbd-fh-connect-autodumph...@sbd.corproot.net ip=10.222.76.2 cmd=append_partition_by_name: db=sbd_dev_autodump tbl=sbd_dev_debug_hive_4 part=time=ingestion/bucket=hourly/date=2020-05-11/hour=18 2020-05-11 20:00:03,869 ERROR [pool-7-thread-2558]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(203)) - AlreadyExistsException(message:Partition already exists:Partition(values:[ingestion, hourly, 2020-05-11, 18], dbName:sbd_dev_autodump, tableName:sbd_dev_debug_hive_4, createTime:0, lastAccessTime:0, sd:StorageDescriptor(cols:[FieldSchema(name:name, type:string, comment:null), FieldSchema(name:age, type:int, comment:null), FieldSchema(name:salary, type:int, comment:null)],