[GitHub] [hadoop] mukund-thakur commented on a diff in pull request #5144: HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references

2022-12-12 Thread GitBox 


mukund-thakur commented on code in PR #5144:
URL: https://github.com/apache/hadoop/pull/5144#discussion_r1046453289


##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:
##
@@ -459,6 +458,13 @@ public void initialize(URI name, Configuration 
originalConf)
 AuditSpan span = null;
 try {
   LOG.debug("Initializing S3AFileSystem for {}", bucket);
+  if (LOG.isTraceEnabled()) {
+// log a full trace for deep diagnostics of where an object is created,
+// for tracking down memory leak issues.
+LOG.trace("Filesystem for {} created; fs.s3a.impl.disable.cache = {}",
+name, originalConf.getBoolean("fs.s3a.impl.disable.cache", false),
+new RuntimeException(super.toString()));

Review Comment:
   Why not just print it? I mean I don't understand the reason behind wrapping 
in RuntimeEx. 
   Also base FileStystem doesn't implement toString() so there won't be 
anything. Why not use this.tpString()?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] mukund-thakur commented on a diff in pull request #5144: HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references

2022-12-08 Thread GitBox 


mukund-thakur commented on code in PR #5144:
URL: https://github.com/apache/hadoop/pull/5144#discussion_r1043859890


##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:
##
@@ -3999,22 +4005,18 @@ public void close() throws IOException {
 }
 isClosed = true;
 LOG.debug("Filesystem {} is closed", uri);
-if (getConf() != null) {
-  String iostatisticsLoggingLevel =
-  getConf().getTrimmed(IOSTATISTICS_LOGGING_LEVEL,
-  IOSTATISTICS_LOGGING_LEVEL_DEFAULT);
-  logIOStatisticsAtLevel(LOG, iostatisticsLoggingLevel, getIOStatistics());
-}
 try {
   super.close();
 } finally {
   stopAllServices();

Review Comment:
   instrumentation is not set to null after closing. Same with auditManager. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] mukund-thakur commented on a diff in pull request #5144: HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references

2022-12-07 Thread GitBox 


mukund-thakur commented on code in PR #5144:
URL: https://github.com/apache/hadoop/pull/5144#discussion_r1042617624


##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:
##
@@ -459,6 +458,13 @@ public void initialize(URI name, Configuration 
originalConf)
 AuditSpan span = null;
 try {
   LOG.debug("Initializing S3AFileSystem for {}", bucket);
+  if (LOG.isTraceEnabled()) {
+// log a full trace for deep diagnostics of where an object is created,
+// for tracking down memory leak issues.
+LOG.trace("Filesystem for {} created; fs.s3a.impl.disable.cache = {}",
+name, originalConf.getBoolean("fs.s3a.impl.disable.cache", false),
+new RuntimeException(super.toString()));

Review Comment:
   Why RTE?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org