k.venureddy2...@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/20507 )
Change subject: IMPALA-12460: Add lag and histogram of event processing in the log ...................................................................... Patch Set 5: Code-Review+1 (1 comment) just a suggestion http://gerrit.cloudera.org:8080/#/c/20507/5/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java: http://gerrit.cloudera.org:8080/#/c/20507/5/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1102 PS5, Line 1102: eventList.sort(Map.Entry.comparingByValue()); Can there be too many events and sorting all map entries consumes more memory and time ? If so, how about we use a priorityqueue(minheap/natural ordering) with custom comparator to compare map values. As we build the pq and if size exceeds 10, poll an element so that size is 10 again. something like below. In the end, we have top elements in the pq those can be polled one by one and update the report. loop to add map elements { pq.offer(i); if (pq.size() > 10) pq.poll(); } -- To view, visit http://gerrit.cloudera.org:8080/20507 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9421b5e26bfa2324217ec9695fbd81636727d22 Gerrit-Change-Number: 20507 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Anonymous Coward <k.venureddy2...@gmail.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Sai Hemanth Gantasala <saihema...@cloudera.com> Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com> Gerrit-Comment-Date: Fri, 29 Sep 2023 06:19:02 +0000 Gerrit-HasComments: Yes
