[ https://issues.apache.org/jira/browse/IMPALA-12003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700472#comment-17700472 ]
Quanlong Huang commented on IMPALA-12003: ----------------------------------------- Take the e2e tests in local-catalog mode as an example: [https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/7134/] {noformat} # wc -l catalogd.58a20e90a469.invalid-user.log.INFO.20230314-020629.1 315395 catalogd.58a20e90a469.invalid-user.log.INFO.20230314-020629.1 # grep getPartialCatalogObject catalogd.58a20e90a469.invalid-user.log.INFO.20230314-020629.1 | wc -l 111060{noformat} 1/3 of the logs are about getPartialCatalogObject requests. Most of the getPartialCatalogObject requests have empty "Time spent" (meaning 0ms). Few of them are longer than 10ms. {noformat} # grep getPartialCatalogObject catalogd.58a20e90a469.invalid-user.log.INFO.20230314-020629.1 | grep -e 'Time spent' | wc -l 55528 # grep getPartialCatalogObject catalogd.58a20e90a469.invalid-user.log.INFO.20230314-020629.1 | grep -e 'Time spent: $' | wc -l 46700 # grep getPartialCatalogObject catalogd.58a20e90a469.invalid-user.log.INFO.20230314-020629.1 | grep -e 'Time spent: [0-9]ms' | wc -l 6007{noformat} > Request logs of getPartialCatalogObject are too verbose > ------------------------------------------------------- > > Key: IMPALA-12003 > URL: https://issues.apache.org/jira/browse/IMPALA-12003 > Project: IMPALA > Issue Type: Bug > Components: Catalog > Reporter: Quanlong Huang > Priority: Major > > When local-catalog mode is enabled, there are lots of trivial logs for > getPartialCatalogObject requests: > {noformat} > I0314 02:06:33.130956 104 JniUtil.java:162] getPartialCatalogObject > request: Getting partial catalog object of CATALOG_SERVICE_ID > I0314 02:06:33.136337 104 JniUtil.java:169] Finished > getPartialCatalogObject request: Getting partial catalog object of > CATALOG_SERVICE_ID. Time spent: 6ms > I0314 02:06:33.158417 104 JniUtil.java:162] getPartialCatalogObject > request: Getting partial catalog object of DATABASE:default > I0314 02:06:33.161764 104 JniUtil.java:169] Finished > getPartialCatalogObject request: Getting partial catalog object of > DATABASE:default. Time spent: 3ms > I0314 02:06:33.173619 104 JniUtil.java:162] getPartialCatalogObject > request: Getting partial catalog object of DATABASE:functional > I0314 02:06:33.173954 104 JniUtil.java:169] Finished > getPartialCatalogObject request: Getting partial catalog object of > DATABASE:functional. Time spent: > I0314 02:06:33.176931 104 JniUtil.java:162] getPartialCatalogObject > request: Getting partial catalog object of DATABASE:functional_avro > I0314 02:06:33.177160 104 JniUtil.java:169] Finished > getPartialCatalogObject request: Getting partial catalog object of > DATABASE:functional_avro. Time spent: 1ms > I0314 02:06:33.178148 104 JniUtil.java:162] getPartialCatalogObject > request: Getting partial catalog object of DATABASE:functional_avro_def > I0314 02:06:33.178380 104 JniUtil.java:169] Finished > getPartialCatalogObject request: Getting partial catalog object of > DATABASE:functional_avro_def. Time spent: > I0314 02:06:33.179805 104 JniUtil.java:162] getPartialCatalogObject > request: Getting partial catalog object of DATABASE:functional_avro_snap > I0314 02:06:33.180004 104 JniUtil.java:169] Finished > getPartialCatalogObject request: Getting partial catalog object of > DATABASE:functional_avro_snap. Time spent:{noformat} > To start the minicluster in local-catalog mode in the dev env: > {code:java} > bin/start-impala-cluster.py --catalogd_args="--catalog_topic_mode=minimal" > --impalad_args="--use_local_catalog" {code} > We'd better lower the log lever to TRACE. They don't really help in > supportability. CC [~prozsa] -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org