[jira] [Commented] (HADOOP-16716) slf4j-log4j12 Should Not Be Included as Dependency
[ https://issues.apache.org/jira/browse/HADOOP-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433940#comment-17433940 ] David Mollitor commented on HADOOP-16716: - [~zabetak] - there was concern raised from [~ste...@apache.org] and I was unable to successfully make the case for this change. > slf4j-log4j12 Should Not Be Included as Dependency > -- > > Key: HADOOP-16716 > URL: https://issues.apache.org/jira/browse/HADOOP-16716 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16716.1.patch > > > The project should be using 'slf4j-api' to log to the SLF4J framwork, but the > end-project, the project that includes hadoop-commons should be specifying > the specific binding to use. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17779) Lock File System Creator Semaphore Uninterruptibly
[
https://issues.apache.org/jira/browse/HADOOP-17779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372796#comment-17372796
]
David Mollitor commented on HADOOP-17779:
-
[~ste...@apache.org] Thanks for the feedback.
I ran into this issue, as reported, here:
https://issues.apache.org/jira/browse/HIVE-24484?focusedCommentId=17371600=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17371600
So ya, we had to put in some hackiness to workaround this change as previously
relying on the standard {{InterruptedException}} was enough, now there needs to
handle {{InterruptedIOException}} as well, which I would argue is a breaking
change.
Anything related to networking, etc, should have a timeout to kill the
connection, not to mention, the scenario you described is currently the case
now. If a thread has the semaphore and is stuck on a Socket connection,
interrupting that thread does not kill the connection. The thread will only
unblock from the socket operation if the socket is closed, the socket is not
connected, or the socket input has been shutdown using {{shutdownInput()}}.
> Lock File System Creator Semaphore Uninterruptibly
> --
>
> Key: HADOOP-17779
> URL: https://issues.apache.org/jira/browse/HADOOP-17779
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 3.3.1
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{FileSystem}} "creator permits" are acquired in an interruptable way. This
> changed the behavior of the call because previous callers were handling the
> IOException as a critical error. An interrupt was handled in the typical
> {{InterruptedException}} way. Lastly, there was no documentation of this new
> event so again, callers are not prepared.
> Restore the previous behavior and lock the semaphore Uninterruptibly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17779) Lock File System Creator Semaphore Uninterruptibly
[
https://issues.apache.org/jira/browse/HADOOP-17779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-17779:
Affects Version/s: 3.3.1
> Lock File System Creator Semaphore Uninterruptibly
> --
>
> Key: HADOOP-17779
> URL: https://issues.apache.org/jira/browse/HADOOP-17779
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 3.3.1
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{FileSystem}} "creator permits" are acquired in an interruptable way. This
> changed the behavior of the call because previous callers were handling the
> IOException as a critical error. An interrupt was handled in the typical
> {{InterruptedException}} way. Lastly, there was no documentation of this new
> event so again, callers are not prepared.
> Restore the previous behavior and lock the semaphore Uninterruptibly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17779) Lock File System Creator Semaphore Uninterruptibly
[
https://issues.apache.org/jira/browse/HADOOP-17779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-17779:
Component/s: fs
> Lock File System Creator Semaphore Uninterruptibly
> --
>
> Key: HADOOP-17779
> URL: https://issues.apache.org/jira/browse/HADOOP-17779
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{FileSystem}} "creator permits" are acquired in an interruptable way. This
> changed the behavior of the call because previous callers were handling the
> IOException as a critical error. An interrupt was handled in the typical
> {{InterruptedException}} way. Lastly, there was no documentation of this new
> event so again, callers are not prepared.
> Restore the previous behavior and lock the semaphore Uninterruptibly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17779) Lock File System Creator Semaphore Uninterruptibly
[
https://issues.apache.org/jira/browse/HADOOP-17779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-17779:
Description:
{{FileSystem}} "creator permits" are acquired in an interruptable way. This
changed the behavior of the call because previous callers were handling the
IOException as a critical error. An interrupt was handled in the typical
{{InterruptedException}} way. Lastly, there was no documentation of this new
event so again, callers are not prepared.
Restore the previous behavior and lock the semaphore Uninterruptibly.
was:{{FileSystem}} "creator permits" are acquired in an interruptable way.
This changed the behavior of the call because previous callers were handling
the IOException as a critical error. An interrupt was handled in the typical
{{InterruptedException}} way. Lastly, there was no documentation of this new
event so again, callers are not prepared.
> Lock File System Creator Semaphore Uninterruptibly
> --
>
> Key: HADOOP-17779
> URL: https://issues.apache.org/jira/browse/HADOOP-17779
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
>
> {{FileSystem}} "creator permits" are acquired in an interruptable way. This
> changed the behavior of the call because previous callers were handling the
> IOException as a critical error. An interrupt was handled in the typical
> {{InterruptedException}} way. Lastly, there was no documentation of this new
> event so again, callers are not prepared.
> Restore the previous behavior and lock the semaphore Uninterruptibly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17779) Lock File System Creator Semaphore Uninterruptibly
David Mollitor created HADOOP-17779:
---
Summary: Lock File System Creator Semaphore Uninterruptibly
Key: HADOOP-17779
URL: https://issues.apache.org/jira/browse/HADOOP-17779
Project: Hadoop Common
Issue Type: Improvement
Reporter: David Mollitor
Assignee: David Mollitor
{{FileSystem}} "creator permits" are acquired in an interruptable way. This
changed the behavior of the call because previous callers were handling the
IOException as a critical error. An interrupt was handled in the typical
{{InterruptedException}} way. Lastly, there was no documentation of this new
event so again, callers are not prepared.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (HADOOP-16963) HADOOP-16582 changed mkdirs() behavior
[
https://issues.apache.org/jira/browse/HADOOP-16963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-16963:
Comment: was deleted
(was: Can you please provide (and link) the corresponding Hive Jira?)
> HADOOP-16582 changed mkdirs() behavior
> --
>
> Key: HADOOP-16963
> URL: https://issues.apache.org/jira/browse/HADOOP-16963
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.10.0, 3.3.0, 2.8.6, 2.9.3, 3.1.3, 3.2.2
>Reporter: Wei-Chiu Chuang
>Priority: Critical
>
> HADOOP-16582 changed behavior of {{mkdirs()}}
> Some Hive tests depend on the old behavior and they fail miserably.
> {quote}
> earlier:
> all plain mkdirs(somePath) were fast-tracked to FileSystem.mkdirs which have
> rerouted them to mkdirs(somePath, somePerm) method with some defaults (which
> were static)
> an implementation of FileSystem have only needed implement "mkdirs(somePath,
> somePerm)" - because the other was not neccessarily called if it was always
> in a FilterFileSystem or something like that
> now:
> especially FilterFileSystem forwards the call of mkdirs(p) to the actual fs
> implementation...which may skip overriden mkdirs(somPath,somePerm) methods
> ...and could cause issues for existing FileSystem implementations
> {quote}
> File this jira to address this problem.
> [~kgyrtkirk] [~ste...@apache.org] [~kihwal]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16963) HADOOP-16582 changed mkdirs() behavior
[
https://issues.apache.org/jira/browse/HADOOP-16963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368118#comment-17368118
]
David Mollitor commented on HADOOP-16963:
-
Can you please provide (and link) the corresponding Hive Jira?
> HADOOP-16582 changed mkdirs() behavior
> --
>
> Key: HADOOP-16963
> URL: https://issues.apache.org/jira/browse/HADOOP-16963
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.10.0, 3.3.0, 2.8.6, 2.9.3, 3.1.3, 3.2.2
>Reporter: Wei-Chiu Chuang
>Priority: Critical
>
> HADOOP-16582 changed behavior of {{mkdirs()}}
> Some Hive tests depend on the old behavior and they fail miserably.
> {quote}
> earlier:
> all plain mkdirs(somePath) were fast-tracked to FileSystem.mkdirs which have
> rerouted them to mkdirs(somePath, somePerm) method with some defaults (which
> were static)
> an implementation of FileSystem have only needed implement "mkdirs(somePath,
> somePerm)" - because the other was not neccessarily called if it was always
> in a FilterFileSystem or something like that
> now:
> especially FilterFileSystem forwards the call of mkdirs(p) to the actual fs
> implementation...which may skip overriden mkdirs(somPath,somePerm) methods
> ...and could cause issues for existing FileSystem implementations
> {quote}
> File this jira to address this problem.
> [~kgyrtkirk] [~ste...@apache.org] [~kihwal]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17482) Remove Commons Logger from FileSystem Class
[ https://issues.apache.org/jira/browse/HADOOP-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17273142#comment-17273142 ] David Mollitor commented on HADOOP-17482: - [~elgoiri] Do you have any cycles to take a look at this request? The PR is fighting many flaky tests. Not sure how many times you want me to run this to get them to pass. > Remove Commons Logger from FileSystem Class > --- > > Key: HADOOP-17482 > URL: https://issues.apache.org/jira/browse/HADOOP-17482 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Labels: pull-request-available > Time Spent: 3.5h > Remaining Estimate: 0h > > Remove reference to Commons Logger in FileSystem, it already has SLF4J, so > it's a bit weird to be mixing and matching and interweaving loggers in this > way. Also, my hope is to eventually migrate everything to SLF4J to simplify > things for downstream consumers of the common library. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17494) Make ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist
David Mollitor created HADOOP-17494: --- Summary: Make ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist Key: HADOOP-17494 URL: https://issues.apache.org/jira/browse/HADOOP-17494 Project: Hadoop Common Issue Type: Improvement Reporter: David Mollitor Assignee: David Mollitor Changing from Protobuf 2 to Protobuf 3 broke some stuff in Apache Hive project. This was not an awesome thing to do between minor versions in regards to backwards compatibility for downstream projects. Additionally, these two frameworks are not drop-in replacements, they have some differences. Also, Protobuf 2 is not deprecated or anything so let us have both protocols available at the same time. In Hadoop 4.x Protobuf 2 support can be dropped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17482) Remove Commons Logger from FileSystem Class
David Mollitor created HADOOP-17482: --- Summary: Remove Commons Logger from FileSystem Class Key: HADOOP-17482 URL: https://issues.apache.org/jira/browse/HADOOP-17482 Project: Hadoop Common Issue Type: Improvement Components: common Reporter: David Mollitor Assignee: David Mollitor Remove reference to Commons Logger in FileSystem, it already has SLF4J, so it's a bit weird to be mixing and matching and interweaving loggers in this way. Also, my hope is to eventually migrate everything to SLF4J to simplify things for downstream consumers of the common library. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17462) Hadoop Client getRpcResponse May Return Wrong Result
[
https://issues.apache.org/jira/browse/HADOOP-17462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor resolved HADOOP-17462.
-
Resolution: Not A Problem
Thanks [~sjlee0] for pointing outing that the {{call}} object is synchronized
on.
It is implemented a bit differently in the reference docs. This functionality
is implemented within the same class itself instead of synchronizing and
accessing the variable from an external class, but I'm thinking it should be
the same.
https://docs.oracle.com/javase/tutorial/essential/concurrency/guardmeth.html
I'll test more in depth and if I can prove it definitively, I'll re-open this
ticket.
> Hadoop Client getRpcResponse May Return Wrong Result
>
>
> Key: HADOOP-17462
> URL: https://issues.apache.org/jira/browse/HADOOP-17462
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java|Title=Client.java}
> /** @return the rpc response or, in case of timeout, null. */
> private Writable getRpcResponse(final Call call, final Connection
> connection,
> final long timeout, final TimeUnit unit) throws IOException {
> synchronized (call) {
> while (!call.done) {
> try {
> AsyncGet.Util.wait(call, timeout, unit);
> if (timeout >= 0 && !call.done) {
> return null;
> }
> } catch (InterruptedException ie) {
> Thread.currentThread().interrupt();
> throw new InterruptedIOException("Call interrupted");
> }
> }
> */
> static class Call {
> final int id; // call id
> final int retry; // retry count
> ...
> boolean done; // true when call is done
> ...
> }
> {code}
> The {{done}} variable is not marked as {{volatile}} so the thread which is
> checking its status is free to cache the value and never reload it even
> though it is expected to change by a different thread. The while loop may be
> stuck waiting for the change, but is always looking at a cached value. If
> that happens, timeout will occur and then return 'null'.
> In previous versions of Hadoop, there was no time-out at this level, so it
> would cause endless loop. Really tough error to track down if it happens.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17462) Hadoop Client getRpcResponse May Return Wrong Result
[
https://issues.apache.org/jira/browse/HADOOP-17462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261741#comment-17261741
]
David Mollitor commented on HADOOP-17462:
-
Let me review [~sjlee0]'s comments before pushing into the project.
> Hadoop Client getRpcResponse May Return Wrong Result
>
>
> Key: HADOOP-17462
> URL: https://issues.apache.org/jira/browse/HADOOP-17462
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java|Title=Client.java}
> /** @return the rpc response or, in case of timeout, null. */
> private Writable getRpcResponse(final Call call, final Connection
> connection,
> final long timeout, final TimeUnit unit) throws IOException {
> synchronized (call) {
> while (!call.done) {
> try {
> AsyncGet.Util.wait(call, timeout, unit);
> if (timeout >= 0 && !call.done) {
> return null;
> }
> } catch (InterruptedException ie) {
> Thread.currentThread().interrupt();
> throw new InterruptedIOException("Call interrupted");
> }
> }
> */
> static class Call {
> final int id; // call id
> final int retry; // retry count
> ...
> boolean done; // true when call is done
> ...
> }
> {code}
> The {{done}} variable is not marked as {{volatile}} so the thread which is
> checking its status is free to cache the value and never reload it even
> though it is expected to change by a different thread. The while loop may be
> stuck waiting for the change, but is always looking at a cached value. If
> that happens, timeout will occur and then return 'null'.
> In previous versions of Hadoop, there was no time-out at this level, so it
> would cause endless loop. Really tough error to track down if it happens.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-17462) Hadoop Client getRpcResponse May Return Wrong Result
[
https://issues.apache.org/jira/browse/HADOOP-17462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261538#comment-17261538
]
David Mollitor edited comment on HADOOP-17462 at 1/8/21, 7:22 PM:
--
[~elgoiri] Hello old friend :) Happy new year. It's been a while.
I am looking at a Hive scenario where the server jstack revealed many hundreds
of threads were stuck on this code. I don't know 100% that the code was stuck
in an endless loop, it could possibly be that HDFS services are slow to respond
and therefore requests are backing up for Hive, but it is my understanding that
the end-user is testing HDFS with other tooling at the same time the see this
issue in Hive and the other tools do not seem to be stuck in the same way as
Hive.
I saw lots of threads blocking here, I noticed this classic issue, just
speculating at this point.
was (Author: belugabehr):
[~elgoiri] Hello old friend :) Happy new year. It's been a while.
I am looking at a Hive scenario where the server jstack revealed many hundreds
of threads were stuck on this code. I don't know 100% that the code was stuck
in an endless loop, it could possibly be that HDFS services are slow to respond
and therefore requests are backing up for Hive, but it is my understanding that
the end-user is testing HDFS with other tooling at the same time the see this
issue in Hive and the other tools do not seem to be stuck in the same way as
Hive.
I saw lots to threads blocking here, I noticed this classic issue, just
speculating at this point.
> Hadoop Client getRpcResponse May Return Wrong Result
>
>
> Key: HADOOP-17462
> URL: https://issues.apache.org/jira/browse/HADOOP-17462
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:java|Title=Client.java}
> /** @return the rpc response or, in case of timeout, null. */
> private Writable getRpcResponse(final Call call, final Connection
> connection,
> final long timeout, final TimeUnit unit) throws IOException {
> synchronized (call) {
> while (!call.done) {
> try {
> AsyncGet.Util.wait(call, timeout, unit);
> if (timeout >= 0 && !call.done) {
> return null;
> }
> } catch (InterruptedException ie) {
> Thread.currentThread().interrupt();
> throw new InterruptedIOException("Call interrupted");
> }
> }
> */
> static class Call {
> final int id; // call id
> final int retry; // retry count
> ...
> boolean done; // true when call is done
> ...
> }
> {code}
> The {{done}} variable is not marked as {{volatile}} so the thread which is
> checking its status is free to cache the value and never reload it even
> though it is expected to change by a different thread. The while loop may be
> stuck waiting for the change, but is always looking at a cached value. If
> that happens, timeout will occur and then return 'null'.
> In previous versions of Hadoop, there was no time-out at this level, so it
> would cause endless loop. Really tough error to track down if it happens.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17462) Hadoop Client getRpcResponse May Return Wrong Result
[
https://issues.apache.org/jira/browse/HADOOP-17462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261538#comment-17261538
]
David Mollitor commented on HADOOP-17462:
-
[~elgoiri] Hello old friend :) Happy new year. It's been a while.
I am looking at a Hive scenario where the server jstack revealed many hundreds
of threads were stuck on this code. I don't know 100% that the code was stuck
in an endless loop, it could possibly be that HDFS services are slow to respond
and therefore requests are backing up for Hive, but it is my understanding that
the end-user is testing HDFS with other tooling at the same time the see this
issue in Hive and the other tools do not seem to be stuck in the same way as
Hive.
I saw lots to threads blocking here, I noticed this classic issue, just
speculating at this point.
> Hadoop Client getRpcResponse May Return Wrong Result
>
>
> Key: HADOOP-17462
> URL: https://issues.apache.org/jira/browse/HADOOP-17462
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:java|Title=Client.java}
> /** @return the rpc response or, in case of timeout, null. */
> private Writable getRpcResponse(final Call call, final Connection
> connection,
> final long timeout, final TimeUnit unit) throws IOException {
> synchronized (call) {
> while (!call.done) {
> try {
> AsyncGet.Util.wait(call, timeout, unit);
> if (timeout >= 0 && !call.done) {
> return null;
> }
> } catch (InterruptedException ie) {
> Thread.currentThread().interrupt();
> throw new InterruptedIOException("Call interrupted");
> }
> }
> */
> static class Call {
> final int id; // call id
> final int retry; // retry count
> ...
> boolean done; // true when call is done
> ...
> }
> {code}
> The {{done}} variable is not marked as {{volatile}} so the thread which is
> checking its status is free to cache the value and never reload it even
> though it is expected to change by a different thread. The while loop may be
> stuck waiting for the change, but is always looking at a cached value. If
> that happens, timeout will occur and then return 'null'.
> In previous versions of Hadoop, there was no time-out at this level, so it
> would cause endless loop. Really tough error to track down if it happens.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17462) Hadoop Client getRpcResponse May Return Wrong Result
[
https://issues.apache.org/jira/browse/HADOOP-17462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-17462:
Description:
{code:java|Title=Client.java}
/** @return the rpc response or, in case of timeout, null. */
private Writable getRpcResponse(final Call call, final Connection connection,
final long timeout, final TimeUnit unit) throws IOException {
synchronized (call) {
while (!call.done) {
try {
AsyncGet.Util.wait(call, timeout, unit);
if (timeout >= 0 && !call.done) {
return null;
}
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
throw new InterruptedIOException("Call interrupted");
}
}
*/
static class Call {
final int id; // call id
final int retry; // retry count
...
boolean done; // true when call is done
...
}
{code}
The {{done}} variable is not marked as {{volatile}} so the thread which is
checking its status is free to cache the value and never reload it even though
it is expected to change by a different thread. The while loop may be stuck
waiting for the change, but is always looking at a cached value. If that
happens, timeout will occur and then return 'null'.
In previous versions of Hadoop, there was no time-out at this level, so it
would cause endless loop. Really tough error to track down if it happens.
was:
{code:java|Title=Client.java}
/** @return the rpc response or, in case of timeout, null. */
private Writable getRpcResponse(final Call call, final Connection connection,
final long timeout, final TimeUnit unit) throws IOException {
synchronized (call) {
while (!call.done) {
try {
AsyncGet.Util.wait(call, timeout, unit);
if (timeout >= 0 && !call.done) {
return null;
}
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
throw new InterruptedIOException("Call interrupted");
}
}
*/
static class Call {
final int id; // call id
final int retry; // retry count
...
boolean done; // true when call is done
...
}
{code}
The {{done}} variable is not marked as {{volatile}} so the thread which is
checking its status is free to cache the value and never reload it even though
it is expected to change by a different thread. The while loop may be stuck
waiting for the change, but is always looking at a cached value.
In previous versions of Hadoop, there was no time-out at this level, so it
would cause endless loop. Really tough error to track down if it happens.
> Hadoop Client getRpcResponse May Return Wrong Result
>
>
> Key: HADOOP-17462
> URL: https://issues.apache.org/jira/browse/HADOOP-17462
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
>
> {code:java|Title=Client.java}
> /** @return the rpc response or, in case of timeout, null. */
> private Writable getRpcResponse(final Call call, final Connection
> connection,
> final long timeout, final TimeUnit unit) throws IOException {
> synchronized (call) {
> while (!call.done) {
> try {
> AsyncGet.Util.wait(call, timeout, unit);
> if (timeout >= 0 && !call.done) {
> return null;
> }
> } catch (InterruptedException ie) {
> Thread.currentThread().interrupt();
> throw new InterruptedIOException("Call interrupted");
> }
> }
> */
> static class Call {
> final int id; // call id
> final int retry; // retry count
> ...
> boolean done; // true when call is done
> ...
> }
> {code}
> The {{done}} variable is not marked as {{volatile}} so the thread which is
> checking its status is free to cache the value and never reload it even
> though it is expected to change by a different thread. The while loop may be
> stuck waiting for the change, but is always looking at a cached value. If
> that happens, timeout will occur and then return 'null'.
> In previous versions of Hadoop, there was no time-out at this level, so it
> would cause endless loop. Really tough error to track down if it happens.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17462) Hadoop Client getRpcResponse May Return Wrong Result
David Mollitor created HADOOP-17462:
---
Summary: Hadoop Client getRpcResponse May Return Wrong Result
Key: HADOOP-17462
URL: https://issues.apache.org/jira/browse/HADOOP-17462
Project: Hadoop Common
Issue Type: Improvement
Components: common
Reporter: David Mollitor
Assignee: David Mollitor
{code:java|Title=Client.java}
/** @return the rpc response or, in case of timeout, null. */
private Writable getRpcResponse(final Call call, final Connection connection,
final long timeout, final TimeUnit unit) throws IOException {
synchronized (call) {
while (!call.done) {
try {
AsyncGet.Util.wait(call, timeout, unit);
if (timeout >= 0 && !call.done) {
return null;
}
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
throw new InterruptedIOException("Call interrupted");
}
}
*/
static class Call {
final int id; // call id
final int retry; // retry count
...
boolean done; // true when call is done
...
}
{code}
The {{done}} variable is not marked as {{volatile}} so the thread which is
checking its status is free to cache the value and never reload it even though
it is expected to change by a different thread. The while loop may be stuck
waiting for the change, but is always looking at a cached value.
In previous versions of Hadoop, there was no time-out at this level, so it
would cause endless loop. Really tough error to track down if it happens.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17443) Improve Hive Class Logging/Error Handling
David Mollitor created HADOOP-17443: --- Summary: Improve Hive Class Logging/Error Handling Key: HADOOP-17443 URL: https://issues.apache.org/jira/browse/HADOOP-17443 Project: Hadoop Common Issue Type: Improvement Reporter: David Mollitor Assignee: David Mollitor * Do not log-and-throw * Pass relevant error message * Do not pass the message from the caught message up the chain, it will be passed up as part of the 'caused by' chain * StringBuffer/StringBuilder * Use anchors in DEBUG logging -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17430) There is no way to clear Text bytes now
[
https://issues.apache.org/jira/browse/HADOOP-17430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252918#comment-17252918
]
David Mollitor commented on HADOOP-17430:
-
Left a comment on the PR. I took a look through the history and this has been
incorrect a long while.
> There is no way to clear Text bytes now
> ---
>
> Key: HADOOP-17430
> URL: https://issues.apache.org/jira/browse/HADOOP-17430
> Project: Hadoop Common
> Issue Type: Wish
> Components: common
>Reporter: gaozhan ding
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> In org.apache.hadoop.io.Text:clear() method, the comments show that we can
> free the bytes by call set(new byte[0]), but it's not going to work now.
> Maybe we can follow this comments.
>
>
> {code:java}
> // org.apache.hadoop.io.Text
> /**
> * Clear the string to empty.
> *
> * Note: For performance reasons, this call does not clear the
> * underlying byte array that is retrievable via {@link #getBytes()}.
> * In order to free the byte-array memory, call {@link #set(byte[])}
> * with an empty byte array (For example, new byte[0]).
> */
> public void clear() {
> length = 0;
> textLength = -1;
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17141) Add Capability To Get Text Length
[
https://issues.apache.org/jira/browse/HADOOP-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-17141:
Description:
The Hadoop {{Text}} class contains an array of byte which contain a UTF-8
encoded string. However, there is no way to quickly get the length of that
string. One can get the number of bytes in the byte array, but to figure out
the length of the String, it needs to be decoded first. In this simple
example, sorting the {{Text}} objects by String length, the String needs to be
decoded from the byte array repeatedly. This was brought to my attention based
on [HIVE-23870].
{code:java}
public static void main(String[] args) {
List list = Arrays.asList(new Text("1"), new Text("22"), new
Text("333"));
list.sort((Text t1, Text t2) -> t1.toString().length() -
t2.toString().length());
}
{code}
Also helpful if I want to check the last letter in the {{Text}} object
repeatedly:
{code:java}
Text t = new Text("");
System.out.println(t.charAt(t.toString().length() - 1));
{code}
was:
The Hadoop {{Text}} class contains an array of byte which contain a UTF-8
encoded string. However, there is no way to quickly get the length of that
string. One can get the number of bytes in the byte array, but to figure out
the length of the String, it needs to be decoded first. In this simple
example, sorting the {{Text}} objects by String length, the String needs to be
decoded from the byte array repeatedly. This was brought to my attention based
on [HIVE-23870].
{code:java}
public static void main(String[] args) {
List list = Arrays.asList(new Text("1"), new Text("22"), new
Text("333"));
list.sort((Text t1, Text t2) -> t1.toString().length() -
t2.toString().length());
}
{code}
Also helpful if I want to check the last letter in the {{Text}} object
repeatedly:
{code:java}
Text t = new Text("");
System.out.println(t.charAt(t.toString().length()));
{code}
> Add Capability To Get Text Length
> -
>
> Key: HADOOP-17141
> URL: https://issues.apache.org/jira/browse/HADOOP-17141
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
>
> The Hadoop {{Text}} class contains an array of byte which contain a UTF-8
> encoded string. However, there is no way to quickly get the length of that
> string. One can get the number of bytes in the byte array, but to figure out
> the length of the String, it needs to be decoded first. In this simple
> example, sorting the {{Text}} objects by String length, the String needs to
> be decoded from the byte array repeatedly. This was brought to my attention
> based on [HIVE-23870].
> {code:java}
> public static void main(String[] args) {
> List list = Arrays.asList(new Text("1"), new Text("22"), new
> Text("333"));
> list.sort((Text t1, Text t2) -> t1.toString().length() -
> t2.toString().length());
> }
> {code}
> Also helpful if I want to check the last letter in the {{Text}} object
> repeatedly:
> {code:java}
> Text t = new Text("");
> System.out.println(t.charAt(t.toString().length() - 1));
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17141) Add Capability To Get Text Length
[
https://issues.apache.org/jira/browse/HADOOP-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-17141:
Description:
The Hadoop {{Text}} class contains an array of byte which contain a UTF-8
encoded string. However, there is no way to quickly get the length of that
string. One can get the number of bytes in the byte array, but to figure out
the length of the String, it needs to be decoded first. In this simple
example, sorting the {{Text}} objects by String length, the String needs to be
decoded from the byte array repeatedly. This was brought to my attention based
on [HIVE-23870].
{code:java}
public static void main(String[] args) {
List list = Arrays.asList(new Text("1"), new Text("22"), new
Text("333"));
list.sort((Text t1, Text t2) -> t1.toString().length() -
t2.toString().length());
}
{code}
Also helpful if I want to check the last letter in the {{Text}} object
repeatedly:
{code:java}
Text t = new Text("");
System.out.println(t.charAt(t.toString().length()));
{code}
was:
The Hadoop {{Text}} class contains an array of byte which contain a UTF-8
encoded string. However, there is no way to quickly get the length of that
string. One can get the number of bytes in the byte array, but to figure out
the length of the String, it needs to be decoded first. In this simple
example, sorting the {{Text}} objects by String length, the String needs to be
decoded from the byte array repeatedly. This was brought to my attention based
on [HIVE-23870].
{code:java}
public static void main(String[] args) {
List list = Arrays.asList(new Text("1"), new Text("22"), new
Text("333"));
list.sort((Text t1, Text t2) -> t1.toString().length() -
t2.toString().length());
}
{code}
> Add Capability To Get Text Length
> -
>
> Key: HADOOP-17141
> URL: https://issues.apache.org/jira/browse/HADOOP-17141
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
>
> The Hadoop {{Text}} class contains an array of byte which contain a UTF-8
> encoded string. However, there is no way to quickly get the length of that
> string. One can get the number of bytes in the byte array, but to figure out
> the length of the String, it needs to be decoded first. In this simple
> example, sorting the {{Text}} objects by String length, the String needs to
> be decoded from the byte array repeatedly. This was brought to my attention
> based on [HIVE-23870].
> {code:java}
> public static void main(String[] args) {
> List list = Arrays.asList(new Text("1"), new Text("22"), new
> Text("333"));
> list.sort((Text t1, Text t2) -> t1.toString().length() -
> t2.toString().length());
> }
> {code}
> Also helpful if I want to check the last letter in the {{Text}} object
> repeatedly:
> {code:java}
> Text t = new Text("");
> System.out.println(t.charAt(t.toString().length()));
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17141) Add Capability To Get Text Length
[
https://issues.apache.org/jira/browse/HADOOP-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-17141:
Description:
The Hadoop {{Text}} class contains an array of byte which contain a UTF-8
encoded string. However, there is no way to quickly get the length of that
string. One can get the number of bytes in the byte array, but to figure out
the length of the String, it needs to be decoded first. In this simple
example, sorting the {{Text}} objects by String length, the String needs to be
decoded from the byte array repeatedly. This was brought to my attention based
on [HIVE-23870].
{code:java}
public static void main(String[] args) {
List list = Arrays.asList(new Text("1"), new Text("22"), new
Text("333"));
list.sort((Text t1, Text t2) -> t1.toString().length() -
t2.toString().length());
}
{code}
> Add Capability To Get Text Length
> -
>
> Key: HADOOP-17141
> URL: https://issues.apache.org/jira/browse/HADOOP-17141
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
>
> The Hadoop {{Text}} class contains an array of byte which contain a UTF-8
> encoded string. However, there is no way to quickly get the length of that
> string. One can get the number of bytes in the byte array, but to figure out
> the length of the String, it needs to be decoded first. In this simple
> example, sorting the {{Text}} objects by String length, the String needs to
> be decoded from the byte array repeatedly. This was brought to my attention
> based on [HIVE-23870].
> {code:java}
> public static void main(String[] args) {
> List list = Arrays.asList(new Text("1"), new Text("22"), new
> Text("333"));
> list.sort((Text t1, Text t2) -> t1.toString().length() -
> t2.toString().length());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17141) Add Capability To Get Text Length
[ https://issues.apache.org/jira/browse/HADOOP-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-17141: Component/s: common > Add Capability To Get Text Length > - > > Key: HADOOP-17141 > URL: https://issues.apache.org/jira/browse/HADOOP-17141 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17141) Add Capability To Get Text Length
David Mollitor created HADOOP-17141: --- Summary: Add Capability To Get Text Length Key: HADOOP-17141 URL: https://issues.apache.org/jira/browse/HADOOP-17141 Project: Hadoop Common Issue Type: Improvement Reporter: David Mollitor Assignee: David Mollitor -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16716) slf4j-log4j12 Should Not Be Included as Dependency
[ https://issues.apache.org/jira/browse/HADOOP-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17101878#comment-17101878 ] David Mollitor commented on HADOOP-16716: - [~ste...@apache.org] Can I trouble you to review this issue once more? > slf4j-log4j12 Should Not Be Included as Dependency > -- > > Key: HADOOP-16716 > URL: https://issues.apache.org/jira/browse/HADOOP-16716 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16716.1.patch > > > The project should be using 'slf4j-api' to log to the SLF4J framwork, but the > end-project, the project that includes hadoop-commons should be specifying > the specific binding to use. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17009) Embrace Immutability of Java Collections
[
https://issues.apache.org/jira/browse/HADOOP-17009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17090793#comment-17090793
]
David Mollitor commented on HADOOP-17009:
-
{code:none}
[INFO] Executed tasks
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:copy-dependencies (package) @
hadoop-cos ---
[INFO] Copying junit-4.12.jar to
/home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1974/src/hadoop-cloud-storage-project/hadoop-cos/target/lib/junit-4.12.jar
[INFO] Copying hamcrest-core-1.3.jar to
/home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1974/src/hadoop-cloud-storage-project/hadoop-cos/target/lib/hamcrest-core-1.3.jar
[INFO] Copying cos_api-bundle-5.6.19.jar to
/home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1974/src/hadoop-cloud-storage-project/hadoop-cos/target/lib/cos_api-bundle-5.6.19.jar
[INFO] Copying classes to
/home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1974/src/hadoop-cloud-storage-project/hadoop-cos/target/lib/hadoop-common-3.4.0-SNAPSHOT.jar
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main . SUCCESS [ 0.324 s]
[INFO] Apache Hadoop Build Tools .. SUCCESS [ 2.147 s]
[INFO] Apache Hadoop Project POM .. SUCCESS [ 0.565 s]
[INFO] Apache Hadoop Annotations .. SUCCESS [ 1.540 s]
[INFO] Apache Hadoop Project Dist POM . SUCCESS [ 0.109 s]
[INFO] Apache Hadoop Assemblies ... SUCCESS [ 0.169 s]
[INFO] Apache Hadoop Maven Plugins SUCCESS [ 4.428 s]
[INFO] Apache Hadoop MiniKDC .. SUCCESS [ 4.027 s]
[INFO] Apache Hadoop Auth . SUCCESS [ 7.716 s]
[INFO] Apache Hadoop Auth Examples SUCCESS [ 1.370 s]
[INFO] Apache Hadoop Common ... SUCCESS [ 41.696 s]
[INFO] Apache Hadoop NFS .. SUCCESS [ 6.527 s]
[INFO] Apache Hadoop KMS .. SUCCESS [ 6.748 s]
[INFO] Apache Hadoop Registry . SUCCESS [ 6.126 s]
[INFO] Apache Hadoop Common Project ... SUCCESS [ 0.045 s]
[INFO] Apache Hadoop HDFS Client .. SUCCESS [ 32.543 s]
[INFO] Apache Hadoop HDFS . SUCCESS [ 51.249 s]
[INFO] Apache Hadoop HDFS Native Client ... SUCCESS [01:45 min]
[INFO] Apache Hadoop HttpFS ... SUCCESS [ 9.273 s]
[INFO] Apache Hadoop HDFS-NFS . SUCCESS [ 5.737 s]
[INFO] Apache Hadoop HDFS-RBF . SUCCESS [ 17.993 s]
[INFO] Apache Hadoop HDFS Project . SUCCESS [ 0.055 s]
[INFO] Apache Hadoop YARN . SUCCESS [ 0.042 s]
[INFO] Apache Hadoop YARN API . SUCCESS [ 22.817 s]
[INFO] Apache Hadoop YARN Common .. SUCCESS [ 26.990 s]
[INFO] Apache Hadoop YARN Server .. SUCCESS [ 0.041 s]
[INFO] Apache Hadoop YARN Server Common ... SUCCESS [ 17.056 s]
[INFO] Apache Hadoop YARN NodeManager . SUCCESS [ 49.390 s]
[INFO] Apache Hadoop YARN Web Proxy ... SUCCESS [ 5.383 s]
[INFO] Apache Hadoop YARN ApplicationHistoryService ... SUCCESS [ 9.187 s]
[INFO] Apache Hadoop YARN Timeline Service SUCCESS [ 7.032 s]
[INFO] Apache Hadoop YARN ResourceManager . SUCCESS [ 30.968 s]
[INFO] Apache Hadoop YARN Server Tests SUCCESS [ 5.771 s]
[INFO] Apache Hadoop YARN Client .. SUCCESS [ 11.756 s]
[INFO] Apache Hadoop YARN SharedCacheManager .. SUCCESS [ 5.344 s]
[INFO] Apache Hadoop YARN Timeline Plugin Storage . SUCCESS [ 5.899 s]
[INFO] Apache Hadoop YARN TimelineService HBase Backend ... SUCCESS [ 0.038 s]
[INFO] Apache Hadoop YARN TimelineService HBase Common SUCCESS [ 6.171 s]
[INFO] Apache Hadoop YARN TimelineService HBase Client SUCCESS [ 6.749 s]
[INFO] Apache Hadoop YARN TimelineService HBase Servers ... SUCCESS [ 0.033 s]
[INFO] Apache Hadoop YARN TimelineService HBase Server 1.2 SUCCESS [ 3.271 s]
[INFO] Apache Hadoop YARN TimelineService HBase tests . SUCCESS [ 7.757 s]
[INFO] Apache Hadoop YARN Router .. SUCCESS [ 7.290 s]
[INFO] Apache Hadoop YARN TimelineService DocumentStore ... SUCCESS [ 4.418 s]
[INFO] Apache Hadoop YARN Applications SUCCESS [ 0.038 s]
[INFO] Apache Hadoop YARN DistributedShell SUCCESS [ 5.828 s]
[INFO] Apache Hadoop YARN Unmanaged Am Launcher ... SUCCESS [ 3.660 s]
[INFO] Apache Hadoop
[jira] [Created] (HADOOP-17009) Embrace Immutability of Java Collections
David Mollitor created HADOOP-17009: --- Summary: Embrace Immutability of Java Collections Key: HADOOP-17009 URL: https://issues.apache.org/jira/browse/HADOOP-17009 Project: Hadoop Common Issue Type: Improvement Components: common Reporter: David Mollitor Assignee: David Mollitor -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16951) Tidy Up Text and ByteWritables Classes
[
https://issues.apache.org/jira/browse/HADOOP-16951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-16951:
Description:
# Remove superfluous code
# Remove superfluous comments
# Checkstyle fixes
# Remove methods that simply call {{super}}.method()
# Use Java 8 facilities to streamline code where applicable
# Simplify and unify some of the constructs between the two classes
The one meaningful change is that I am suggesting that the expanding of the
arrays be 1.5x instead of 2x per expansion. I pulled this idea from open JDK.
was:
# Remove superfluous code
# Remove superfluous comments
# Checkstyle fixes
# Remove methods that simply call {{super}}.method()
# Use Java 8 facilities to streamline code where applicable
# Simplify and unify some of the constructs between the two classes
> Tidy Up Text and ByteWritables Classes
> --
>
> Key: HADOOP-16951
> URL: https://issues.apache.org/jira/browse/HADOOP-16951
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
>
> # Remove superfluous code
> # Remove superfluous comments
> # Checkstyle fixes
> # Remove methods that simply call {{super}}.method()
> # Use Java 8 facilities to streamline code where applicable
> # Simplify and unify some of the constructs between the two classes
>
> The one meaningful change is that I am suggesting that the expanding of the
> arrays be 1.5x instead of 2x per expansion. I pulled this idea from open JDK.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16951) Tidy Up Text and ByteWritables Classes
[
https://issues.apache.org/jira/browse/HADOOP-16951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085772#comment-17085772
]
David Mollitor commented on HADOOP-16951:
-
[~elgoiri] Hello old friend :)
I was recently added to the Hadoop committers. Thank you so much for all your
help.
I created this Jira as a bit of tidying up a core class, but also as a first
push for me. Are you able to review?
Also, what is the process for taking a Github PR and merging it into trunk?
> Tidy Up Text and ByteWritables Classes
> --
>
> Key: HADOOP-16951
> URL: https://issues.apache.org/jira/browse/HADOOP-16951
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
>
> # Remove superfluous code
> # Remove superfluous comments
> # Checkstyle fixes
> # Remove methods that simply call {{super}}.method()
> # Use Java 8 facilities to streamline code where applicable
> # Simplify and unify some of the constructs between the two classes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16951) Tidy Up Text and ByteWritables Classes
[
https://issues.apache.org/jira/browse/HADOOP-16951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-16951:
Summary: Tidy Up Text and ByteWritables Classes (was: Tidy Up Text and
ByteWritables Class)
> Tidy Up Text and ByteWritables Classes
> --
>
> Key: HADOOP-16951
> URL: https://issues.apache.org/jira/browse/HADOOP-16951
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
>
> # Remove superfluous code
> # Remove superfluous comments
> # Checkstyle fixes
> # Remove methods that simply call {{super}}.method()
> # Use Java 8 facilities to streamline code where applicable
> # Simplify and unify some of the constructs between the two classes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16951) Tidy Up Text and ByteWritables Class
David Mollitor created HADOOP-16951:
---
Summary: Tidy Up Text and ByteWritables Class
Key: HADOOP-16951
URL: https://issues.apache.org/jira/browse/HADOOP-16951
Project: Hadoop Common
Issue Type: Improvement
Components: common
Reporter: David Mollitor
Assignee: David Mollitor
# Remove superfluous code
# Remove superfluous comments
# Checkstyle fixes
# Remove methods that simply call {{super}}.method()
# Use Java 8 facilities to streamline code where applicable
# Simplify and unify some of the constructs between the two classes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16817) Remove Superfluous Dependency on Zookeeper Recipes
[ https://issues.apache.org/jira/browse/HADOOP-16817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor resolved HADOOP-16817. - Resolution: Not A Problem I was able to build with the modified POM, but I do see that it is included at places. Hmm. Sorry. > Remove Superfluous Dependency on Zookeeper Recipes > --- > > Key: HADOOP-16817 > URL: https://issues.apache.org/jira/browse/HADOOP-16817 > Project: Hadoop Common > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Fix For: 3.2.0 > > > ZooKeeper Recipes library is a dependency of hadoop-common even though it's > not being used. I noticed it when an upstream library was shading > hadooo-common and also suspiciously shading ZK recipes library. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16817) Remove Superfluous Dependency on Zookeeper Recipes
David Mollitor created HADOOP-16817: --- Summary: Remove Superfluous Dependency on Zookeeper Recipes Key: HADOOP-16817 URL: https://issues.apache.org/jira/browse/HADOOP-16817 Project: Hadoop Common Issue Type: Improvement Reporter: David Mollitor Assignee: David Mollitor Fix For: 3.2.0 ZooKeeper Recipes library is a dependency of hadoop-common even though it's not being used. I noticed it when an upstream library was shading hadooo-common and also suspiciously shading ZK recipes library. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16811) Use JUnit TemporaryFolder Rule in TestFileUtils
[ https://issues.apache.org/jira/browse/HADOOP-16811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019760#comment-17019760 ] David Mollitor commented on HADOOP-16811: - [~ste...@apache.org] Green light on YETUS. > Use JUnit TemporaryFolder Rule in TestFileUtils > --- > > Key: HADOOP-16811 > URL: https://issues.apache.org/jira/browse/HADOOP-16811 > Project: Hadoop Common > Issue Type: Improvement > Components: common, test >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Fix For: 3.3.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16811) Use JUnit TemporaryFolder Rule in TestFileUtils
[ https://issues.apache.org/jira/browse/HADOOP-16811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16811: Fix Version/s: 3.3.0 > Use JUnit TemporaryFolder Rule in TestFileUtils > --- > > Key: HADOOP-16811 > URL: https://issues.apache.org/jira/browse/HADOOP-16811 > Project: Hadoop Common > Issue Type: Improvement > Components: common, test >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Fix For: 3.3.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16811) Use JUnit TemporaryFolder Rule in TestFileUtils
[ https://issues.apache.org/jira/browse/HADOOP-16811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16811: Affects Version/s: 3.2 > Use JUnit TemporaryFolder Rule in TestFileUtils > --- > > Key: HADOOP-16811 > URL: https://issues.apache.org/jira/browse/HADOOP-16811 > Project: Hadoop Common > Issue Type: Improvement > Components: common, test >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Fix For: 3.3.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16811) Use JUnit TemporaryFolder Rule in TestFileUtils
[ https://issues.apache.org/jira/browse/HADOOP-16811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16811: Component/s: test > Use JUnit TemporaryFolder Rule in TestFileUtils > --- > > Key: HADOOP-16811 > URL: https://issues.apache.org/jira/browse/HADOOP-16811 > Project: Hadoop Common > Issue Type: Improvement > Components: common, test >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16811) Use JUnit TemporaryFolder Rule in TestFileUtils
David Mollitor created HADOOP-16811: --- Summary: Use JUnit TemporaryFolder Rule in TestFileUtils Key: HADOOP-16811 URL: https://issues.apache.org/jira/browse/HADOOP-16811 Project: Hadoop Common Issue Type: Improvement Components: common Reporter: David Mollitor Assignee: David Mollitor -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16790) Add Write Convenience Methods
David Mollitor created HADOOP-16790: --- Summary: Add Write Convenience Methods Key: HADOOP-16790 URL: https://issues.apache.org/jira/browse/HADOOP-16790 Project: Hadoop Common Issue Type: New Feature Reporter: David Mollitor Assignee: David Mollitor Was working on something in HBase. I just wanted to write a simple string to a HDFS file and I was surprised how tricky it was. It inspired me to add some write convenience methods to HDFS common. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16715) Remove Compile Scope and Version ID From POM
[ https://issues.apache.org/jira/browse/HADOOP-16715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994732#comment-16994732 ] David Mollitor commented on HADOOP-16715: - [~ste...@apache.org] Any thoughts on this one? > Remove Compile Scope and Version ID From POM > > > Key: HADOOP-16715 > URL: https://issues.apache.org/jira/browse/HADOOP-16715 > Project: Hadoop Common > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > this is the default behavior. No need to explicitly specify. > Do not need to specify the version ID either. Will inherit from parent POM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16716) slf4j-log4j12 Should Not Be Included as Dependency
[ https://issues.apache.org/jira/browse/HADOOP-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16988181#comment-16988181 ] David Mollitor commented on HADOOP-16716: - [~aajisaka] Would you mind reviewing this patch? > slf4j-log4j12 Should Not Be Included as Dependency > -- > > Key: HADOOP-16716 > URL: https://issues.apache.org/jira/browse/HADOOP-16716 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16716.1.patch > > > The project should be using 'slf4j-api' to log to the SLF4J framwork, but the > end-project, the project that includes hadoop-commons should be specifying > the specific binding to use. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16716) slf4j-log4j12 Should Not Be Included as Dependency
[ https://issues.apache.org/jira/browse/HADOOP-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16716: Status: Patch Available (was: Open) > slf4j-log4j12 Should Not Be Included as Dependency > -- > > Key: HADOOP-16716 > URL: https://issues.apache.org/jira/browse/HADOOP-16716 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16716.1.patch > > > The project should be using 'slf4j-api' to log to the SLF4J framwork, but the > end-project, the project that includes hadoop-commons should be specifying > the specific binding to use. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16716) slf4j-log4j12 Should Not Be Included as Dependency
[ https://issues.apache.org/jira/browse/HADOOP-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16716: Attachment: HADOOP-16716.1.patch > slf4j-log4j12 Should Not Be Included as Dependency > -- > > Key: HADOOP-16716 > URL: https://issues.apache.org/jira/browse/HADOOP-16716 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16716.1.patch > > > The project should be using 'slf4j-api' to log to the SLF4J framwork, but the > end-project, the project that includes hadoop-commons should be specifying > the specific binding to use. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16717) Remove GenericsUtil isLog4jLogger dependency on Log4jLoggerAdapter
[
https://issues.apache.org/jira/browse/HADOOP-16717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981670#comment-16981670
]
David Mollitor commented on HADOOP-16717:
-
[~aajisaka] Can you also tackle [HADOOP-16716] ?
> Remove GenericsUtil isLog4jLogger dependency on Log4jLoggerAdapter
> --
>
> Key: HADOOP-16717
> URL: https://issues.apache.org/jira/browse/HADOOP-16717
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: Xieming Li
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HADOOP-16717.001.patch, HADOOP-16717.002.patch,
> HADOOP-16717.003.patch
>
>
> Remove this method:
> {code:java}
> /**
>* Determine whether the log of clazz is Log4j implementation.
>* @param clazz a class to be determined
>* @return true if the log of clazz is Log4j implementation.
>*/
> public static boolean isLog4jLogger(Class clazz) {
> if (clazz == null) {
> return false;
> }
> Logger log = LoggerFactory.getLogger(clazz);
> return log instanceof Log4jLoggerAdapter;
> }
> {code}
> This creates a dependency on Log4jLoggerAdapter (slf4j-log4j12) which means
> that any project which depends on hadoop-commons needs to carry this
> dependency as well. Such a simple use case and such a heavy dependency. The
> commons library should not depend on any specific implementation of SLF4J
> binding
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16716) slf4j-log4j12 Should Not Be Included as Dependency
[
https://issues.apache.org/jira/browse/HADOOP-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976227#comment-16976227
]
David Mollitor commented on HADOOP-16716:
-
Just to clarify, so we are all on the same page here:
Bridge = Takes code base written for a particular logging framework (Log4J for
example; log4j-over-slf4j) and makes it SLF4J compatible
Binding = Allows logging framework to plug into a code base written to work
with the SLF4J framework (slf4j-log4j12)
I'm talking about slf4j-log4j12 which is a binding. For the 'common' project,
I highly recommend making it {{test}} scope so that there is no dependence on
it for project that include hadoop-common in their list of dependencies.
Anything that includes commons should decide what logging framework to use, not
inherit it from this project.
> slf4j-log4j12 Should Not Be Included as Dependency
> --
>
> Key: HADOOP-16716
> URL: https://issues.apache.org/jira/browse/HADOOP-16716
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
>
> The project should be using 'slf4j-api' to log to the SLF4J framwork, but the
> end-project, the project that includes hadoop-commons should be specifying
> the specific binding to use.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16716) slf4j-log4j12 Should Not Be Included as Dependency
[ https://issues.apache.org/jira/browse/HADOOP-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976089#comment-16976089 ] David Mollitor commented on HADOOP-16716: - [~ste...@apache.org] Thank you for the feedback. I understand what you're saying about the ability to exclude the dependency. In fact, what lead me to create this JIRA is that I noticed it was being explicitly excluded in a project I was working on. I was thinking to myself "why would this 'common' library ship a binding?" and "how would anyone know to manually exclude this?" and "why does every project need to know to do this?" I would wager that including this dependency ends up adding multiple bindings to the classpath for upstream projects and causes a lot of warnings and confusion. The entire point of SLF4J is that the bindings are pluggable. I think it's much cleaner to have the upstream projects produce a warning that no logger exists. This forces the user to make a decision about which logging framework they want to use, if by some chance they don't already have a logger included. If this was for the HDFS NameNode, HDFS DataNode, YARN services, etc. I get it. That is appropriate to include a logging framework. But it's not so clear to me that a utility package called 'commons' would automatically introduce a logging framework. > slf4j-log4j12 Should Not Be Included as Dependency > -- > > Key: HADOOP-16716 > URL: https://issues.apache.org/jira/browse/HADOOP-16716 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > > The project should be using 'slf4j-api' to log to the SLF4J framwork, but the > end-project, the project that includes hadoop-commons should be specifying > the specific binding to use. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16716) slf4j-log4j12 Should Not Be Included as Dependency
[ https://issues.apache.org/jira/browse/HADOOP-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16716: Affects Version/s: 3.2 > slf4j-log4j12 Should Not Be Included as Dependency > -- > > Key: HADOOP-16716 > URL: https://issues.apache.org/jira/browse/HADOOP-16716 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > > The project should be using 'slf4j-api' to log to the SLF4J framwork, but the > end-project, the project that includes hadoop-commons should be specifying > the specific binding to use. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16716) slf4j-log4j12 Should Not Be Included as Dependency
[ https://issues.apache.org/jira/browse/HADOOP-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16716: Component/s: common > slf4j-log4j12 Should Not Be Included as Dependency > -- > > Key: HADOOP-16716 > URL: https://issues.apache.org/jira/browse/HADOOP-16716 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > > The project should be using 'slf4j-api' to log to the SLF4J framwork, but the > end-project, the project that includes hadoop-commons should be specifying > the specific binding to use. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16717) Remove GenericsUtil isLog4jLogger
David Mollitor created HADOOP-16717:
---
Summary: Remove GenericsUtil isLog4jLogger
Key: HADOOP-16717
URL: https://issues.apache.org/jira/browse/HADOOP-16717
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 3.2
Reporter: David Mollitor
Remove this method:
{code:java}
/**
* Determine whether the log of clazz is Log4j implementation.
* @param clazz a class to be determined
* @return true if the log of clazz is Log4j implementation.
*/
public static boolean isLog4jLogger(Class clazz) {
if (clazz == null) {
return false;
}
Logger log = LoggerFactory.getLogger(clazz);
return log instanceof Log4jLoggerAdapter;
}
{code}
This creates a dependency on Log4jLoggerAdapter (slf4j-log4j12) which means
that any project which depends on hadoop-commons needs to carry this dependency
as well. Such a simple use case and such a heavy dependency. The commons
library should not depend on any specific implementation of SLF4J binding
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16715) Remove Compile Scope and Version ID From POM
[ https://issues.apache.org/jira/browse/HADOOP-16715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16715: Description: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html this is the default behavior. No need to explicitly specify. Do not need to specify the version ID either. Will inherit from parent POM. was: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html this is the default behavior. No need to explicitly specify. > Remove Compile Scope and Version ID From POM > > > Key: HADOOP-16715 > URL: https://issues.apache.org/jira/browse/HADOOP-16715 > Project: Hadoop Common > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > this is the default behavior. No need to explicitly specify. > Do not need to specify the version ID either. Will inherit from parent POM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16715) Remove Compile Scope and Version ID From POM
[ https://issues.apache.org/jira/browse/HADOOP-16715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16715: Summary: Remove Compile Scope and Version ID From POM (was: Remove Compile Scope From POM) > Remove Compile Scope and Version ID From POM > > > Key: HADOOP-16715 > URL: https://issues.apache.org/jira/browse/HADOOP-16715 > Project: Hadoop Common > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > this is the default behavior. No need to explicitly specify. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16716) slf4j-log4j12 Should Not Be Included as Dependency
David Mollitor created HADOOP-16716: --- Summary: slf4j-log4j12 Should Not Be Included as Dependency Key: HADOOP-16716 URL: https://issues.apache.org/jira/browse/HADOOP-16716 Project: Hadoop Common Issue Type: Improvement Reporter: David Mollitor Assignee: David Mollitor The project should be using 'slf4j-api' to log to the SLF4J framwork, but the end-project, the project that includes hadoop-commons should be specifying the specific binding to use. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16715) Remove Compile Scope From POM
David Mollitor created HADOOP-16715: --- Summary: Remove Compile Scope From POM Key: HADOOP-16715 URL: https://issues.apache.org/jira/browse/HADOOP-16715 Project: Hadoop Common Issue Type: Improvement Reporter: David Mollitor Assignee: David Mollitor https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html this is the default behavior. No need to explicitly specify. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16708) HadoopExecutors cleanup to only log at debug
[
https://issues.apache.org/jira/browse/HADOOP-16708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973676#comment-16973676
]
David Mollitor commented on HADOOP-16708:
-
[~ste...@apache.org] bq. Proposed: downgrade the logs to info.
I assume you mean: downgrade to _debug_
> HadoopExecutors cleanup to only log at debug
>
>
> Key: HADOOP-16708
> URL: https://issues.apache.org/jira/browse/HADOOP-16708
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: util
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: David Mollitor
>Priority: Minor
>
> HadoopExecutors cleanup to only log at debug
> HADOOP-16181 means that executor shutdown now logs @ info
> this means every single hadoop fs command against s3 has four lines of
> information which is entirely irrelevant to all users.
> {code}
> 2019-11-13 17:32:58,654 [shutdown-hook-0] DEBUG s3a.S3AFileSystem
> (S3AFileSystem.java:close(3126)) - Filesystem s3a://hwdev-steve-ireland-new
> is closed
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(118)) - Gracefully shutting down executor
> service. Waiting max 30 SECONDS
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(129)) - Succesfully shutdown executor service
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(118)) - Gracefully shutting down executor
> service. Waiting max 30 SECONDS
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(129)) - Succesfully shutdown executor service
> {code}
> We don't need to inflict this on anyone. Proposed: downgrade the logs to
> info. No need for tests
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16708) HadoopExecutors cleanup to only log at debug
[
https://issues.apache.org/jira/browse/HADOOP-16708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973676#comment-16973676
]
David Mollitor edited comment on HADOOP-16708 at 11/13/19 8:39 PM:
---
[~ste...@apache.org]
bq. Proposed: downgrade the logs to info.
I assume you mean: downgrade to _debug_
was (Author: belugabehr):
[~ste...@apache.org] bq. Proposed: downgrade the logs to info.
I assume you mean: downgrade to _debug_
> HadoopExecutors cleanup to only log at debug
>
>
> Key: HADOOP-16708
> URL: https://issues.apache.org/jira/browse/HADOOP-16708
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: util
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: David Mollitor
>Priority: Minor
>
> HadoopExecutors cleanup to only log at debug
> HADOOP-16181 means that executor shutdown now logs @ info
> this means every single hadoop fs command against s3 has four lines of
> information which is entirely irrelevant to all users.
> {code}
> 2019-11-13 17:32:58,654 [shutdown-hook-0] DEBUG s3a.S3AFileSystem
> (S3AFileSystem.java:close(3126)) - Filesystem s3a://hwdev-steve-ireland-new
> is closed
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(118)) - Gracefully shutting down executor
> service. Waiting max 30 SECONDS
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(129)) - Succesfully shutdown executor service
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(118)) - Gracefully shutting down executor
> service. Waiting max 30 SECONDS
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(129)) - Succesfully shutdown executor service
> {code}
> We don't need to inflict this on anyone. Proposed: downgrade the logs to
> info. No need for tests
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16694) Use Objects requireNonNull Where Appropriate
[
https://issues.apache.org/jira/browse/HADOOP-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972603#comment-16972603
]
David Mollitor commented on HADOOP-16694:
-
[~ste...@apache.org] Are you still good with replacing the current crop of {{if
(o == null) throw new NullPointerException();}} ?
> Use Objects requireNonNull Where Appropriate
>
>
> Key: HADOOP-16694
> URL: https://issues.apache.org/jira/browse/HADOOP-16694
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HADOOP-16694.1.patch, HADOOP-16694.2.patch,
> HADOOP-16694.3.patch
>
>
> https://docs.oracle.com/javase/8/docs/api/java/util/Objects.html#requireNonNull-T-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16692) UserGroupInformation Sets kerberosMinSecondsBeforeRelogin as Millis
[ https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972529#comment-16972529 ] David Mollitor commented on HADOOP-16692: - [~ste...@apache.org] PR Submitted. Thanks! > UserGroupInformation Sets kerberosMinSecondsBeforeRelogin as Millis > --- > > Key: HADOOP-16692 > URL: https://issues.apache.org/jira/browse/HADOOP-16692 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-16692.1.patch, HADOOP-16692.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16692) UserGroupInformation Sets kerberosMinSecondsBeforeRelogin as Millis
[ https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16692: Summary: UserGroupInformation Sets kerberosMinSecondsBeforeRelogin as Millis (was: UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis) > UserGroupInformation Sets kerberosMinSecondsBeforeRelogin as Millis > --- > > Key: HADOOP-16692 > URL: https://issues.apache.org/jira/browse/HADOOP-16692 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-16692.1.patch, HADOOP-16692.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15852) Refactor QuotaUsage
[
https://issues.apache.org/jira/browse/HADOOP-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-15852:
Status: Patch Available (was: Reopened)
I still can't figure out how these changes produced a regression in
{{TestQuota}} but I made a fresh patch and {{TestQuota}} runs successful
locally... finally time to close this one out.
> Refactor QuotaUsage
> ---
>
> Key: HADOOP-15852
> URL: https://issues.apache.org/jira/browse/HADOOP-15852
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-15852.1.patch, HADOOP-15852.2.patch,
> HADOOP-15852.3.patch
>
>
> My new mission is to remove instances of {{StringBuffer}} in favor of
> {{StringBuilder}}.
> * Simplify Code
> * Use Eclipse to generate hashcode/equals
> * User StringBuilder instead of StringBuffer
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15852) Refactor QuotaUsage
[
https://issues.apache.org/jira/browse/HADOOP-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-15852:
Attachment: HADOOP-15852.3.patch
> Refactor QuotaUsage
> ---
>
> Key: HADOOP-15852
> URL: https://issues.apache.org/jira/browse/HADOOP-15852
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-15852.1.patch, HADOOP-15852.2.patch,
> HADOOP-15852.3.patch
>
>
> My new mission is to remove instances of {{StringBuffer}} in favor of
> {{StringBuilder}}.
> * Simplify Code
> * Use Eclipse to generate hashcode/equals
> * User StringBuilder instead of StringBuffer
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15829) Review of NetgroupCache
[ https://issues.apache.org/jira/browse/HADOOP-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-15829: Status: Patch Available (was: Open) > Review of NetgroupCache > --- > > Key: HADOOP-15829 > URL: https://issues.apache.org/jira/browse/HADOOP-15829 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-15829.2.patch, HDFS-13971.1.patch > > > * Simplify code and performance by using Guava Multimap -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15829) Review of NetgroupCache
[ https://issues.apache.org/jira/browse/HADOOP-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-15829: Attachment: HADOOP-15829.2.patch > Review of NetgroupCache > --- > > Key: HADOOP-15829 > URL: https://issues.apache.org/jira/browse/HADOOP-15829 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-15829.2.patch, HDFS-13971.1.patch > > > * Simplify code and performance by using Guava Multimap -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15829) Review of NetgroupCache
[ https://issues.apache.org/jira/browse/HADOOP-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-15829: Status: Open (was: Patch Available) > Review of NetgroupCache > --- > > Key: HADOOP-15829 > URL: https://issues.apache.org/jira/browse/HADOOP-15829 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HDFS-13971.1.patch > > > * Simplify code and performance by using Guava Multimap -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16693) Review InterruptedException Handling
[ https://issues.apache.org/jira/browse/HADOOP-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970486#comment-16970486 ] David Mollitor commented on HADOOP-16693: - [~ste...@apache.org] Please review when you get a chance > Review InterruptedException Handling > > > Key: HADOOP-16693 > URL: https://issues.apache.org/jira/browse/HADOOP-16693 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16693.1.patch > > > Difficult to do well. I hopefully improved it some. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16694) Use Objects requireNonNull Where Appropriate
[ https://issues.apache.org/jira/browse/HADOOP-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16694: Attachment: HADOOP-16694.3.patch > Use Objects requireNonNull Where Appropriate > > > Key: HADOOP-16694 > URL: https://issues.apache.org/jira/browse/HADOOP-16694 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16694.1.patch, HADOOP-16694.2.patch, > HADOOP-16694.3.patch > > > https://docs.oracle.com/javase/8/docs/api/java/util/Objects.html#requireNonNull-T- -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16694) Use Objects requireNull Where Appropriate
[ https://issues.apache.org/jira/browse/HADOOP-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970393#comment-16970393 ] David Mollitor commented on HADOOP-16694: - [~xkrogen] Good idea. Let me take a look. > Use Objects requireNull Where Appropriate > - > > Key: HADOOP-16694 > URL: https://issues.apache.org/jira/browse/HADOOP-16694 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16694.1.patch, HADOOP-16694.2.patch > > > https://docs.oracle.com/javase/8/docs/api/java/util/Objects.html#requireNonNull-T- -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16692) UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis
[ https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16692: Status: Patch Available (was: Open) > UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis > - > > Key: HADOOP-16692 > URL: https://issues.apache.org/jira/browse/HADOOP-16692 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-16692.1.patch, HADOOP-16692.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16693) Review InterruptedException Handling
[ https://issues.apache.org/jira/browse/HADOOP-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970152#comment-16970152 ] David Mollitor commented on HADOOP-16693: - [~ste...@apache.org] Done! Thanks > Review InterruptedException Handling > > > Key: HADOOP-16693 > URL: https://issues.apache.org/jira/browse/HADOOP-16693 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16693.1.patch > > > Difficult to do well. I hopefully improved it some. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16694) Use Objects requireNull Where Appropriate
[ https://issues.apache.org/jira/browse/HADOOP-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16694: Status: Patch Available (was: Open) > Use Objects requireNull Where Appropriate > - > > Key: HADOOP-16694 > URL: https://issues.apache.org/jira/browse/HADOOP-16694 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16694.1.patch, HADOOP-16694.2.patch > > > https://docs.oracle.com/javase/8/docs/api/java/util/Objects.html#requireNonNull-T- -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16694) Use Objects requireNull Where Appropriate
[ https://issues.apache.org/jira/browse/HADOOP-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16694: Status: Open (was: Patch Available) > Use Objects requireNull Where Appropriate > - > > Key: HADOOP-16694 > URL: https://issues.apache.org/jira/browse/HADOOP-16694 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16694.1.patch, HADOOP-16694.2.patch > > > https://docs.oracle.com/javase/8/docs/api/java/util/Objects.html#requireNonNull-T- -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16694) Use Objects requireNull Where Appropriate
[ https://issues.apache.org/jira/browse/HADOOP-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16694: Attachment: HADOOP-16694.2.patch > Use Objects requireNull Where Appropriate > - > > Key: HADOOP-16694 > URL: https://issues.apache.org/jira/browse/HADOOP-16694 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16694.1.patch, HADOOP-16694.2.patch > > > https://docs.oracle.com/javase/8/docs/api/java/util/Objects.html#requireNonNull-T- -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16694) Use Objects requireNull Where Appropriate
[ https://issues.apache.org/jira/browse/HADOOP-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16694: Attachment: HADOOP-16694.1.patch > Use Objects requireNull Where Appropriate > - > > Key: HADOOP-16694 > URL: https://issues.apache.org/jira/browse/HADOOP-16694 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16694.1.patch > > > https://docs.oracle.com/javase/8/docs/api/java/util/Objects.html#requireNonNull-T- -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16694) Use Objects requireNull Where Appropriate
David Mollitor created HADOOP-16694: --- Summary: Use Objects requireNull Where Appropriate Key: HADOOP-16694 URL: https://issues.apache.org/jira/browse/HADOOP-16694 Project: Hadoop Common Issue Type: Improvement Components: common Affects Versions: 3.2 Reporter: David Mollitor Assignee: David Mollitor Attachments: HADOOP-16694.1.patch https://docs.oracle.com/javase/8/docs/api/java/util/Objects.html#requireNonNull-T- -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16694) Use Objects requireNull Where Appropriate
[ https://issues.apache.org/jira/browse/HADOOP-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16694: Status: Patch Available (was: Open) > Use Objects requireNull Where Appropriate > - > > Key: HADOOP-16694 > URL: https://issues.apache.org/jira/browse/HADOOP-16694 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16694.1.patch > > > https://docs.oracle.com/javase/8/docs/api/java/util/Objects.html#requireNonNull-T- -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16692) UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis
[ https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16692: Attachment: HADOOP-16692.2.patch > UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis > - > > Key: HADOOP-16692 > URL: https://issues.apache.org/jira/browse/HADOOP-16692 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-16692.1.patch, HADOOP-16692.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16692) UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis
[ https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16692: Status: Open (was: Patch Available) > UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis > - > > Key: HADOOP-16692 > URL: https://issues.apache.org/jira/browse/HADOOP-16692 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-16692.1.patch, HADOOP-16692.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16692) UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis
[ https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969776#comment-16969776 ] David Mollitor commented on HADOOP-16692: - [~aengineer] I just put up a new patch to keep the name, to keep it consistent with the configuration, but do the transformations as appropriate. > UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis > - > > Key: HADOOP-16692 > URL: https://issues.apache.org/jira/browse/HADOOP-16692 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-16692.1.patch, HADOOP-16692.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16693) Review InterruptedException Handling
[ https://issues.apache.org/jira/browse/HADOOP-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16693: Attachment: HADOOP-16693.1.patch > Review InterruptedException Handling > > > Key: HADOOP-16693 > URL: https://issues.apache.org/jira/browse/HADOOP-16693 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16693.1.patch > > > Difficult to do well. I hopefully improved it some. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16693) Review InterruptedException Handling
David Mollitor created HADOOP-16693: --- Summary: Review InterruptedException Handling Key: HADOOP-16693 URL: https://issues.apache.org/jira/browse/HADOOP-16693 Project: Hadoop Common Issue Type: Improvement Components: common Affects Versions: 3.2 Reporter: David Mollitor Assignee: David Mollitor Attachments: HADOOP-16693.1.patch Difficult to do well. I hopefully improved it some. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16693) Review InterruptedException Handling
[ https://issues.apache.org/jira/browse/HADOOP-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16693: Status: Patch Available (was: Open) > Review InterruptedException Handling > > > Key: HADOOP-16693 > URL: https://issues.apache.org/jira/browse/HADOOP-16693 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HADOOP-16693.1.patch > > > Difficult to do well. I hopefully improved it some. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16692) UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis
[
https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969733#comment-16969733
]
David Mollitor commented on HADOOP-16692:
-
[~aengineer] Ack. -1
The variable is called kerberosMinSecondsBeforeRelogin, but really it is
millis. I'll take another look at it, probably just change the name of the
variable.
{code:java}
/** Min time (in seconds) before relogin for Kerberos */
private static long kerberosMinSecondsBeforeRelogin;
...
kerberosMinSecondsBeforeRelogin = 1000L * conf.getLong(
HADOOP_KERBEROS_MIN_SECONDS_BEFORE_RELOGIN,
HADOOP_KERBEROS_MIN_SECONDS_BEFORE_RELOGIN_DEFAULT);
{code}
> UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis
> -
>
> Key: HADOOP-16692
> URL: https://issues.apache.org/jira/browse/HADOOP-16692
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16692.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16692) UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis
[ https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16692: Summary: UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis (was: UserGroupInformation Treats kerberosMinSecondsBeforeRelogin, as Millis) > UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis > - > > Key: HADOOP-16692 > URL: https://issues.apache.org/jira/browse/HADOOP-16692 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-16692.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16692) UserGroupInformation Treats kerberosMinSecondsBeforeRelogin, as Millis
[ https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16692: Status: Patch Available (was: Open) > UserGroupInformation Treats kerberosMinSecondsBeforeRelogin, as Millis > -- > > Key: HADOOP-16692 > URL: https://issues.apache.org/jira/browse/HADOOP-16692 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-16692.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16692) UserGroupInformation Treats kerberosMinSecondsBeforeRelogin, as Millis
[ https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16692: Attachment: HADOOP-16692.1.patch > UserGroupInformation Treats kerberosMinSecondsBeforeRelogin, as Millis > -- > > Key: HADOOP-16692 > URL: https://issues.apache.org/jira/browse/HADOOP-16692 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-16692.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16692) UserGroupInformation Treats kerberosMinSecondsBeforeRelogin, as Millis
David Mollitor created HADOOP-16692: --- Summary: UserGroupInformation Treats kerberosMinSecondsBeforeRelogin, as Millis Key: HADOOP-16692 URL: https://issues.apache.org/jira/browse/HADOOP-16692 Project: Hadoop Common Issue Type: Bug Components: common Affects Versions: 3.2 Reporter: David Mollitor Assignee: David Mollitor Attachments: HADOOP-16692.1.patch -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16691) Unify Logging in UserGroupInformation
[
https://issues.apache.org/jira/browse/HADOOP-16691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-16691:
Attachment: HADOOP-16691.1.patch
> Unify Logging in UserGroupInformation
> -
>
> Key: HADOOP-16691
> URL: https://issues.apache.org/jira/browse/HADOOP-16691
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16691.1.patch
>
>
> Unify logging style in {{UserGroupInformation}} class.
> # Remove superfluous logging guards
> # Remove superfluous String.format
> # Simplify
> # Other
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16691) Unify Logging in UserGroupInformation
[
https://issues.apache.org/jira/browse/HADOOP-16691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-16691:
Status: Patch Available (was: Open)
> Unify Logging in UserGroupInformation
> -
>
> Key: HADOOP-16691
> URL: https://issues.apache.org/jira/browse/HADOOP-16691
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16691.1.patch
>
>
> Unify logging style in {{UserGroupInformation}} class.
> # Remove superfluous logging guards
> # Remove superfluous String.format
> # Simplify
> # Other
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16691) Unify Logging in UserGroupInformation
David Mollitor created HADOOP-16691:
---
Summary: Unify Logging in UserGroupInformation
Key: HADOOP-16691
URL: https://issues.apache.org/jira/browse/HADOOP-16691
Project: Hadoop Common
Issue Type: Improvement
Components: common
Affects Versions: 3.2.0
Reporter: David Mollitor
Assignee: David Mollitor
Unify logging style in {{UserGroupInformation}} class.
# Remove superfluous logging guards
# Remove superfluous String.format
# Simplify
# Other
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969441#comment-16969441
]
David Mollitor commented on HADOOP-16686:
-
[~elgoiri] OK. Figured it out...
{code:none}
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java:44:
public void set(long value) {:24: 'value' hides a field. [HiddenField]
{code}
The method is not being recognized as a proper Setter method because it should
be called 'setValue', but I'm not going to change the API here. Please
disregard these 'HiddenField' warnings.
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch,
> HADOOP-16686.3.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969423#comment-16969423
]
David Mollitor edited comment on HADOOP-16686 at 11/7/19 5:25 PM:
--
[~elgoiri] Maybe the 'HiddenField' check should be modified to allow the
current syntax. It is very common in setters to use the current format.
[https://stackoverflow.com/questions/7776046/checkstyle-how-to-resolve-hidden-field-error]
I'll address the other ones.
Edit: The Hadoop checkstyle configuration is already configured in this way.
I'll have to figure out why it's hitting here.
was (Author: belugabehr):
[~elgoiri] Maybe the 'HiddenField' check should be modified to allow the
current syntax. It is very common in setters to use the current format.
[https://stackoverflow.com/questions/7776046/checkstyle-how-to-resolve-hidden-field-error]
I'll address the other ones.
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch,
> HADOOP-16686.3.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-16686:
Status: Patch Available (was: Open)
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch,
> HADOOP-16686.3.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-16686:
Attachment: HADOOP-16686.3.patch
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch,
> HADOOP-16686.3.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-16686:
Status: Open (was: Patch Available)
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch,
> HADOOP-16686.3.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969423#comment-16969423
]
David Mollitor commented on HADOOP-16686:
-
[~elgoiri] Maybe the 'HiddenField' check should be modified to allow the
current syntax. It is very common in setters to use the current format.
[https://stackoverflow.com/questions/7776046/checkstyle-how-to-resolve-hidden-field-error]
I'll address the other ones.
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968481#comment-16968481
]
David Mollitor commented on HADOOP-16686:
-
[~elgoiri] On this one, let's go with the patch attached to this JIRA. I will
do PRs moving forward.
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968375#comment-16968375
]
David Mollitor commented on HADOOP-16686:
-
[~elgoiri] Thanks for the review! I am very hesitant to and got back and fix
these "issues." This patch results in 41 fewer checkstyle violations. I
think that's win enough. Alas, you are the committer, so let me know if you
absolutely need these changes and I will comply.
{code:none}
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/BooleanWritable.java:48:
/**: First sentence should end with a period. [JavadocStyle]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ByteWritable.java:42:
public void set(byte value) {:24: 'value' hides a field. [HiddenField]
{code}
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967997#comment-16967997
]
David Mollitor edited comment on HADOOP-16686 at 11/6/19 1:18 AM:
--
Including new patch that fixes unit test.
Unit test was testing for specific values returned from the {{compare()}}
method, even though the only guarantees is that the method will return a
negative integer, zero, or a positive integer. One of the changes changed the
value returned, but it was still the correct sign.
https://docs.oracle.com/javase/8/docs/api/java/lang/Comparable.html#compareTo-T-
was (Author: belugabehr):
Including new patch that fixes unit test.
Unit test was testing for specific values returned from the {{compare()}}
method, even though the only guarantees is that the method will return a
negative integer, zero, or a positive integer.
https://docs.oracle.com/javase/8/docs/api/java/lang/Comparable.html#compareTo-T-
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-16686:
Status: Patch Available (was: Open)
Including new patch that fixes unit test.
Unit test was testing for specific values returned from the {{compare()}}
method, even though the only guarantees is that the method will return a
negative integer, zero, or a positive integer.
https://docs.oracle.com/javase/8/docs/api/java/lang/Comparable.html#compareTo-T-
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-16686:
Status: Open (was: Patch Available)
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16686) Review Writable Classes
[
https://issues.apache.org/jira/browse/HADOOP-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HADOOP-16686:
Attachment: HADOOP-16686.2.patch
> Review Writable Classes
> ---
>
> Key: HADOOP-16686
> URL: https://issues.apache.org/jira/browse/HADOOP-16686
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HADOOP-16686.1.patch, HADOOP-16686.2.patch
>
>
> Based on my work in HADOOP-16678, I figured I'd look at some of the other
> {{Writeable}} classes. I wanted these looking good because they are so
> core/base to the Hadoop framework.
> Formatted the classes to remove checkstyle violations, and spiffed things up
> where I saw fit. This includes improving some {{equals}} and {{hashcode}}
> implementation. For example:
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/LongWritable.java#L66]
>
> [https://stackoverflow.com/questions/4045063/how-should-i-map-long-to-int-in-hashcode]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
