[ https://issues.apache.org/jira/browse/IMPALA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17201678#comment-17201678 ]
Tim Armstrong commented on IMPALA-9779: --------------------------------------- I think for both of these we need to a) make sure that the state of the Partition object is cleanly set without a reload (including directly relevant fields and dependent fields like HdfsTable::hasAvroData_ and 2) avoid setting markDirty via bulkAlterPartitions. > Unnecessarily reloading file metadata in some DDLs > -------------------------------------------------- > > Key: IMPALA-9779 > URL: https://issues.apache.org/jira/browse/IMPALA-9779 > Project: IMPALA > Issue Type: Improvement > Components: Catalog > Affects Versions: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala > 2.11.0, Impala 3.0, Impala 2.12.0, Impala 3.1.0, Impala 3.2.0, Impala 3.3.0, > Impala 3.4.0 > Reporter: Quanlong Huang > Assignee: Tim Armstrong > Priority: Critical > > Some DDLs are not modifying the actual table data. We don't need to reload > file meta for them. These DDLs include: > * Compute (incremental) stats > * Drop stats > * Alter table set row format > * Alter table set file format > Code paths of them both call CatalogOpExecutor.bulkAlterPartitions(). The > related partitions are marked as "dirty" anyway. Dirty partitions will be > dropped and reloaded at the end of > CatalogOpExecutor.alterTable(TAlterTableParams, TDdlExecResponse). See the > details in HdfsTable.updatePartitionsFromHms(). > We can consider not marking related partitions as "dirty" in these DDLs. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org