(hive) 01/02: HIVE-28144: Remove overly verbose debug messages from MetastoreDirectSqlUtils (Stamatis Zampetakis reviewed by Butao Zhang)

[zabetak]

This is an automated email from the ASF dual-hosted git repository.

zabetak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git

commit e6e6b12d5fe13e8cbca99e8f3a5d5f13438b2400
Author: Stamatis Zampetakis <zabe...@gmail.com>
AuthorDate: Fri Mar 22 11:05:08 2024 +0100

    HIVE-28144: Remove overly verbose debug messages from 
MetastoreDirectSqlUtils (Stamatis Zampetakis reviewed by Butao Zhang)
    
    When BITVECTOR or KLL stats are disabled/not present in the metastore the 
following message may appear way too often in the HMS logs.
    ```
    2024-03-22T01:50:57,849 DEBUG [CachedStore-CacheUpdateService: Thread-240] 
metastore.MetastoreDirectSqlUtils: Expected blob type but got java.lang.String
    ```
    In fact in some cases, the message appears more than once for every single 
partition that is present in the table(s) being queried. When the number of 
partitions is important it can easily clog the logs with redundant and useless 
information.
    
    To put things in perspective while running the cbo_query10.q on the 
statistics of TPC-DS30TB dataset the message occupies more than 50% (26MB) of 
the total log file (46MB).
    
    The presence of the message does not tells us much on its own. In 
conjunction with the code we can infer that we are not fetching BITVECTOR/KLL 
stats from the metastore but this could be done in a different place without 
having to print the same message 170K times.
    
    Removing this message saves disk space, avoids frequent log rotation, and 
improves the overall readability of the log file.
    
    There is another redundant message which appears when transforming a 
database value to Boolean. The message is redundant since it is followed 
directly by an exception so there is no reason to have both. This message may 
not appear as often as the previous one but given that it doesn't add much 
value it can also be removed.
    
    Close apache/hive#5159
---
 .../org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java    | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git 
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java
 
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java
index 067e415d725..8a608a030ee 100644
--- 
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java
+++ 
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java
@@ -554,7 +554,6 @@ class MetastoreDirectSqlUtils {
         return true;
       }
     }
-    LOG.debug("Value is of type {}", value.getClass());
     throw new MetaException("Cannot extract boolean from column value " + 
value);
   }
 
@@ -590,8 +589,8 @@ class MetastoreDirectSqlUtils {
       return (byte[]) value;
     }
        else {
-      // this may happen when enablebitvector is false
-      LOG.debug("Expected blob type but got " + value.getClass().getName());
+      // 
org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getStatsList(enableBitVector,enableKll)
+      // We get here when enableBitvector or enableKll is false
       return null;
     }
   }