[
https://issues.apache.org/jira/browse/HADOOP-17758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361023#comment-17361023
]
yikf edited comment on HADOOP-17758 at 6/10/21, 3:21 PM:
---------------------------------------------------------
gentle ping [~Jim_Brennan]
Thanks for your reporting, We change it beacuse that It will wait forerver if
we have't the timeout and the RerferenceQueue is empty;
eg (Suppose we hav't the timeout):
# we call the RerferenceQueue.remove()
# if we poll is empty from the queue, RerferenceQueue will
lock.wait(0)<waitting forever, because we dont call the lock.notify>
# cleanUp cant work due to the DeadLock
>From above, we nee the timeout to prevent deadLock.
reference JDK ReferenctQueue:
{code:java}
public Reference<? extends T> remove(long timeout)
throws IllegalArgumentException, InterruptedException
{
if (timeout < 0) {
throw new IllegalArgumentException("Negative timeout value");
}
synchronized (lock) {
Reference<? extends T> r = reallyPoll();
if (r != null) return r;
long start = (timeout == 0) ? 0 : System.nanoTime();
for (;;) {
lock.wait(timeout);
r = reallyPoll();
if (r != null) return r;
if (timeout != 0) {
long end = System.nanoTime();
timeout -= (end - start) / 1000_000;
if (timeout <= 0) return null;
start = end;
}
}
}
}
{code}
But i ignore it that the return of method of the remove may be is null, i will
fix the issue, Thanks~
was (Author: kaifeiyi):
Thanks for your reporting, We change it beacuse that It will wait forerver if
we have't the timeout and the RerferenceQueue is empty;
eg (Suppose we hav't the timeout):
# we call the RerferenceQueue.remove()
# if we poll is empty from the queue, RerferenceQueue will
lock.wait(0)<waitting forever, because we dont call the lock.notify>
# cleanUp cant work due to the DeadLock
>From above, we nee the timeout to prevent deadLock.
reference JDK ReferenctQueue:
{code:java}
public Reference<? extends T> remove(long timeout)
throws IllegalArgumentException, InterruptedException
{
if (timeout < 0) {
throw new IllegalArgumentException("Negative timeout value");
}
synchronized (lock) {
Reference<? extends T> r = reallyPoll();
if (r != null) return r;
long start = (timeout == 0) ? 0 : System.nanoTime();
for (;;) {
lock.wait(timeout);
r = reallyPoll();
if (r != null) return r;
if (timeout != 0) {
long end = System.nanoTime();
timeout -= (end - start) / 1000_000;
if (timeout <= 0) return null;
start = end;
}
}
}
}
{code}
But i ignore it that the return of method of the remove may be is null, i will
fix the issue, Thanks~
> NPE and excessive warnings after HADOOP-17728
> ---------------------------------------------
>
> Key: HADOOP-17758
> URL: https://issues.apache.org/jira/browse/HADOOP-17758
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
> Affects Versions: 3.4.0
> Reporter: Jim Brennan
> Priority: Major
>
> I'm noticing these warnings and NPE's when just running a simple pi test on a
> one node cluster:
> {noformat}
> 2021-06-09 21:51:12,334 WARN
> [org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner]
> fs.FileSystem (FileSystem.java:run(4025)) - Exception in the cleaner thread
> but it will continue to run
> java.lang.NullPointerException
> at
> org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:4020)
> at java.lang.Thread.run(Thread.java:748){noformat}
> This appears to be due to [HADOOP-17728].
> I'm not sure I understand why that change was made? Wasn't it by design that
> the remove should wait until something is queued?
> [~kaifeiYi] can you please investigate?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
