[jira] [Updated] (HDFS-13566) Add configurable additional RPC listener to NameNode

2023-05-28 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-13566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDFS-13566:

Fix Version/s: 3.2.1

> Add configurable additional RPC listener to NameNode
> 
>
> Key: HDFS-13566
> URL: https://issues.apache.org/jira/browse/HDFS-13566
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ipc
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Fix For: 3.3.0, 3.2.1
>
> Attachments: HDFS-13566.001.patch, HDFS-13566.002.patch, 
> HDFS-13566.003.patch, HDFS-13566.004.patch, HDFS-13566.005.patch, 
> HDFS-13566.006.patch, HDFS-13566.007.patch, HDFS-13566.008.patch, 
> HDFS-13566.009.patch, HDFS-13566.010.patch, HDFS-13566.011.patch
>
>
> This Jira aims to add the capability to NameNode to run additional 
> listener(s). Such that NameNode can be accessed from multiple ports. 
> Fundamentally, this Jira tries to extend ipc.Server to allow configured with 
> more listeners, binding to different ports, but sharing the same call queue 
> and the handlers. Useful when different clients are only allowed to access 
> certain different ports. Combined with HDFS-13547, this also allows different 
> ports to have different SASL security levels. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2591) No tailMap needed for startIndex 0 in ContainerSet#listContainer

2019-11-21 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2591 started by Attila Doroszlai.
--
> No tailMap needed for startIndex 0 in ContainerSet#listContainer
> 
>
> Key: HDDS-2591
> URL: https://issues.apache.org/jira/browse/HDDS-2591
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: Ozone Datanode
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
>
> {{ContainerSet#listContainer}} has this code:
> {code:title=https://github.com/apache/hadoop-ozone/blob/3c334f6a7b344e0e5f52fec95071c369286cfdcb/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerSet.java#L198}
> map = containerMap.tailMap(containerMap.firstKey(), true);
> {code}
> This is equivalent to:
> {code}
> map = containerMap;
> {code}
> since {{tailMap}} is a sub-map with all keys larger than or equal to 
> ({{inclusive=true}}) {{firstKey}}, which is the lowest key in the map.  So it 
> is a sub-map with all keys, ie. the whole map.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2605) Use LongSupplier to avoid boxing

2019-11-21 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2605 started by Attila Doroszlai.
--
> Use LongSupplier to avoid boxing
> 
>
> Key: HDDS-2605
> URL: https://issues.apache.org/jira/browse/HDDS-2605
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: freon
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Trivial
>
> Freon's {{ProgressBar}} uses {{Supplier}}, which could be replaced with 
> {{LongSupplier}} to avoid boxing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2603) Avoid unnecessary boxing in XceiverClientReply

2019-11-21 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2603 started by Attila Doroszlai.
--
> Avoid unnecessary boxing in XceiverClientReply
> --
>
> Key: HDDS-2603
> URL: https://issues.apache.org/jira/browse/HDDS-2603
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
>
> {{XceiverClientReply#logIndex}} is unnecessarily boxed/unboxed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2605) Use LongSupplier to avoid boxing

2019-11-21 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2605:
--

 Summary: Use LongSupplier to avoid boxing
 Key: HDDS-2605
 URL: https://issues.apache.org/jira/browse/HDDS-2605
 Project: Hadoop Distributed Data Store
  Issue Type: Improvement
  Components: freon
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


Freon's {{ProgressBar}} uses {{Supplier}}, which could be replaced with 
{{LongSupplier}} to avoid boxing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2603) Avoid unnecessary boxing in XceiverClientReply

2019-11-21 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2603:
--

 Summary: Avoid unnecessary boxing in XceiverClientReply
 Key: HDDS-2603
 URL: https://issues.apache.org/jira/browse/HDDS-2603
 Project: Hadoop Distributed Data Store
  Issue Type: Improvement
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


{{XceiverClientReply#logIndex}} is unnecessarily boxed/unboxed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Commented] (HDDS-1812) Du while calculating used disk space reports that chunk files are file not found

2019-11-20 Thread Attila Doroszlai (Jira)


[ 
https://issues.apache.org/jira/browse/HDDS-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16978791#comment-16978791
 ] 

Attila Doroszlai commented on HDDS-1812:


Storage location report has 3 pieces of numeric information:
 # capacity
 # available space
 # space used by Datanode

* The first two pieces are cheap to obtain.
* Hadoop has two implementations for space usage: can be calculated expensively 
(using {{du}}), or approximated cheaply (using {{df}}, ie. {{capacity - 
available space}}).  The approximation is much better if the volume is 
dedicated to Datanode storage.  Is it fair to assume that if performance 
requires it, dedicated volumes will be used for Ozone?
* Do we really need the info on space used by Datanode?  It does not seem 
suitable for decisions regarding allocation, since the disk may be full with 
other data.

> Du while calculating used disk space reports that chunk files are file not 
> found
> 
>
> Key: HDDS-1812
> URL: https://issues.apache.org/jira/browse/HDDS-1812
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Ozone Datanode
>Affects Versions: 0.4.0
>Reporter: Mukul Kumar Singh
>Assignee: Attila Doroszlai
>Priority: Critical
>
> {code}
> 2019-07-16 08:16:49,787 WARN org.apache.hadoop.fs.CachingGetSpaceUsed: Could 
> not get disk usage information for path /data/3/ozone-0715
> ExitCodeException exitCode=1: du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/b113dd390e68e914d3ff405f3deec564_stream_60448f
> 77-6349-48fa-ae86-b2d311730569_chunk_1.tmp.1.14118085': No such file or 
> directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/37993af2849bdd0320d0f9d4a6ef4b92_stream_1f68be9f-e083-45e5-84a9-08809bc392ed
> _chunk_1.tmp.1.14118091': No such file or directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/a38677def61389ec0be9105b1b4fddff_stream_9c3c3741-f710-4482-8423-7ac6695be96b
> _chunk_1.tmp.1.14118102': No such file or directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/a689c89f71a75547471baf6182f3be01_stream_baf0f21d-2fb0-4cd8-84b0-eff1723019a0
> _chunk_1.tmp.1.14118105': No such file or directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/f58cf0fa5cb9360058ae25e8bc983e84_stream_d8d5ea61-995f-4ff5-88fb-4a9e97932f00
> _chunk_1.tmp.1.14118109': No such file or directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/a1d13ee6bbefd1f8156b1bd8db0d1b67_stream_db214bdd-a0c0-4f4a-8bc7-a3817e047e45_chunk_1.tmp.1.14118115':
>  No such file or directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/8f8a4bd3f6c31161a70f82cb5ab8ee60_stream_d532d657-3d87-4332-baf8-effad9b3db23_chunk_1.tmp.1.14118127':
>  No such file or directory
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008)
> at org.apache.hadoop.util.Shell.run(Shell.java:901)
> at org.apache.hadoop.fs.DU$DUShell.startRefresh(DU.java:62)
> at org.apache.hadoop.fs.DU.refresh(DU.java:53)
> at 
> org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:181)
> at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-1812) Du while calculating used disk space reports that chunk files are file not found

2019-11-20 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-1812 started by Attila Doroszlai.
--
> Du while calculating used disk space reports that chunk files are file not 
> found
> 
>
> Key: HDDS-1812
> URL: https://issues.apache.org/jira/browse/HDDS-1812
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Ozone Datanode
>Affects Versions: 0.4.0
>Reporter: Mukul Kumar Singh
>Assignee: Attila Doroszlai
>Priority: Critical
>
> {code}
> 2019-07-16 08:16:49,787 WARN org.apache.hadoop.fs.CachingGetSpaceUsed: Could 
> not get disk usage information for path /data/3/ozone-0715
> ExitCodeException exitCode=1: du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/b113dd390e68e914d3ff405f3deec564_stream_60448f
> 77-6349-48fa-ae86-b2d311730569_chunk_1.tmp.1.14118085': No such file or 
> directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/37993af2849bdd0320d0f9d4a6ef4b92_stream_1f68be9f-e083-45e5-84a9-08809bc392ed
> _chunk_1.tmp.1.14118091': No such file or directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/a38677def61389ec0be9105b1b4fddff_stream_9c3c3741-f710-4482-8423-7ac6695be96b
> _chunk_1.tmp.1.14118102': No such file or directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/a689c89f71a75547471baf6182f3be01_stream_baf0f21d-2fb0-4cd8-84b0-eff1723019a0
> _chunk_1.tmp.1.14118105': No such file or directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/f58cf0fa5cb9360058ae25e8bc983e84_stream_d8d5ea61-995f-4ff5-88fb-4a9e97932f00
> _chunk_1.tmp.1.14118109': No such file or directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/a1d13ee6bbefd1f8156b1bd8db0d1b67_stream_db214bdd-a0c0-4f4a-8bc7-a3817e047e45_chunk_1.tmp.1.14118115':
>  No such file or directory
> du: cannot access 
> '/data/3/ozone-0715/hdds/1b467d25-46cd-4de0-a4a1-e9405bde23ff/current/containerDir3/1724/chunks/8f8a4bd3f6c31161a70f82cb5ab8ee60_stream_d532d657-3d87-4332-baf8-effad9b3db23_chunk_1.tmp.1.14118127':
>  No such file or directory
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008)
> at org.apache.hadoop.util.Shell.run(Shell.java:901)
> at org.apache.hadoop.fs.DU$DUShell.startRefresh(DU.java:62)
> at org.apache.hadoop.fs.DU.refresh(DU.java:53)
> at 
> org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:181)
> at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2588) Consolidate compose environments

2019-11-20 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2588:
---
Status: Patch Available  (was: In Progress)

> Consolidate compose environments
> 
>
> Key: HDDS-2588
> URL: https://issues.apache.org/jira/browse/HDDS-2588
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: docker
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> There are a few slightly different sample docker compose environments: ozone, 
> ozoneperf, ozones3, ozone-recon. This issue proposes to merge these 4 by 
> minor additions to ozoneperf:
>  # add {{recon}} service from {{ozone-recon}}
>  # run GDPR and S3 tests
>  # expose datanode web port (eg. for profiling)
> Plus: also run ozone-shell test (from basic suite), which is currently run 
> only in ozonesecure
> I also propose to rename {{ozoneperf}} to {{ozone}} for simplicity.
> Consolidating these 4 environments would slightly reduce both code 
> duplication and the time needed for acceptance tests.
> CC [~elek]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Commented] (HDDS-2591) No tailMap needed for startIndex 0 in ContainerSet#listContainer

2019-11-20 Thread Attila Doroszlai (Jira)


[ 
https://issues.apache.org/jira/browse/HDDS-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16978713#comment-16978713
 ] 

Attila Doroszlai commented on HDDS-2591:


Actually, the method is only used by tests.  [~bharat], do you think it can be 
removed, or is there a plan to use it later?

> No tailMap needed for startIndex 0 in ContainerSet#listContainer
> 
>
> Key: HDDS-2591
> URL: https://issues.apache.org/jira/browse/HDDS-2591
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: Ozone Datanode
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
>
> {{ContainerSet#listContainer}} has this code:
> {code:title=https://github.com/apache/hadoop-ozone/blob/3c334f6a7b344e0e5f52fec95071c369286cfdcb/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerSet.java#L198}
> map = containerMap.tailMap(containerMap.firstKey(), true);
> {code}
> This is equivalent to:
> {code}
> map = containerMap;
> {code}
> since {{tailMap}} is a sub-map with all keys larger than or equal to 
> ({{inclusive=true}}) {{firstKey}}, which is the lowest key in the map.  So it 
> is a sub-map with all keys, ie. the whole map.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2591) No tailMap needed for startIndex 0 in ContainerSet#listContainer

2019-11-20 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2591:
--

 Summary: No tailMap needed for startIndex 0 in 
ContainerSet#listContainer
 Key: HDDS-2591
 URL: https://issues.apache.org/jira/browse/HDDS-2591
 Project: Hadoop Distributed Data Store
  Issue Type: Improvement
  Components: Ozone Datanode
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


{{ContainerSet#listContainer}} has this code:

{code:title=https://github.com/apache/hadoop-ozone/blob/3c334f6a7b344e0e5f52fec95071c369286cfdcb/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerSet.java#L198}
map = containerMap.tailMap(containerMap.firstKey(), true);
{code}

This is equivalent to:

{code}
map = containerMap;
{code}

since {{tailMap}} is a sub-map with all keys larger than or equal to 
({{inclusive=true}}) {{firstKey}}, which is the lowest key in the map.  So it 
is a sub-map with all keys, ie. the whole map.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2588) Consolidate compose environments

2019-11-20 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2588 started by Attila Doroszlai.
--
> Consolidate compose environments
> 
>
> Key: HDDS-2588
> URL: https://issues.apache.org/jira/browse/HDDS-2588
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: docker
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>
> There are a few slightly different sample docker compose environments: ozone, 
> ozoneperf, ozones3, ozone-recon. This issue proposes to merge these 4 by 
> minor additions to ozoneperf:
>  # add {{recon}} service from {{ozone-recon}}
>  # run GDPR and S3 tests
>  # expose datanode web port (eg. for profiling)
> Plus: also run ozone-shell test (from basic suite), which is currently run 
> only in ozonesecure
> I also propose to rename {{ozoneperf}} to {{ozone}} for simplicity.
> Consolidating these 4 environments would slightly reduce both code 
> duplication and the time needed for acceptance tests.
> CC [~elek]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2589) Ozone Shell PutKeyHandler inconsistent config override

2019-11-20 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2589:
--

 Summary: Ozone Shell PutKeyHandler inconsistent config override
 Key: HDDS-2589
 URL: https://issues.apache.org/jira/browse/HDDS-2589
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: Ozone CLI
Reporter: Attila Doroszlai


Ozone Shell allows overriding config file via {{\-conf }} option, and 
also specific settings via {{\-\-set key=value}}. This is partly ignored by 
{{PutKeyHandler}} (which implements {{ozone sh key put}}). By [creating a new 
config 
object|https://github.com/apache/hadoop-ozone/blob/b1ac520ed95fe35b454e55695a05ed71299d8797/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/PutKeyHandler.java#L101]
 instead of using the one from the [factory method 
{{createOzoneConfiguration()}}|https://github.com/apache/hadoop-ozone/blob/b1ac520ed95fe35b454e55695a05ed71299d8797/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/PutKeyHandler.java#L80],
 it bypasses the config set via command-line. This affects the following 
settings: replication factor and type, chunk size (used only for client-side 
buffer sizing).

The goal of this task is to first clarify if this is intentional. Then, if it 
is not, the fix is simple:
 * save the object from {{createOzoneConfiguration()}} call to the local 
variable {{conf}}
 * remove {{new OzoneConfiguration()}} call

To confirm the difference in config override handling:
 * {{ozone sh \-\-set ozone.om.address=no-such-host key put vol1/bucket1/asdf 
/etc/passwd}} fails due to unknown host for OM, so it respects the setting
 * {{ozone sh \-\-set ozone.replication.type=STAND_ALONE key put 
vol1/bucket1/asdf /etc/passwd}} creates a key with RATIS replication despite 
the setting to use STAND_ALONE replication

CC [~elek] for input on whether this is intentional or can be fixed



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2580) Ensure resources are closed in Get/PutKeyHandler

2019-11-20 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2580:
---
Status: Patch Available  (was: In Progress)

> Ensure resources are closed in Get/PutKeyHandler
> 
>
> Key: HDDS-2580
> URL: https://issues.apache.org/jira/browse/HDDS-2580
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Dinesh Chitlangia
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: newbie, pull-request-available, sonar
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Use try-with-resources or close this "FileOutputStream" in a "finally" clause.
> GetKeyHandler: 
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW6HHKTfdBVcJdcVFsvC&open=AW6HHKTfdBVcJdcVFsvC]
>  
> Use try-with-resources or close this "OzoneOutputStream" in a "finally" 
> clause.
> PutKeyHandler: 
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW6HHKRodBVcJdcVFsvB&open=AW6HHKRodBVcJdcVFsvB]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2588) Consolidate compose environments

2019-11-20 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2588:
--

 Summary: Consolidate compose environments
 Key: HDDS-2588
 URL: https://issues.apache.org/jira/browse/HDDS-2588
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: docker
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


There are a few slightly different sample docker compose environments: ozone, 
ozoneperf, ozones3, ozone-recon. This issue proposes to merge these 4 by minor 
additions to ozoneperf:

 # add {{recon}} service from {{ozone-recon}}
 # run GDPR and S3 tests
 # expose datanode web port (eg. for profiling)

Plus: also run ozone-shell test (from basic suite), which is currently run only 
in ozonesecure

I also propose to rename {{ozoneperf}} to {{ozone}} for simplicity.

Consolidating these 4 environments would slightly reduce both code duplication 
and the time needed for acceptance tests.

CC [~elek]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2580) Ensure resources are closed in Get/PutKeyHandler

2019-11-20 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2580:
---
Summary: Ensure resources are closed in Get/PutKeyHandler  (was: Sonar: 
Close resources in xxxKeyHandler)

> Ensure resources are closed in Get/PutKeyHandler
> 
>
> Key: HDDS-2580
> URL: https://issues.apache.org/jira/browse/HDDS-2580
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Dinesh Chitlangia
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: newbie, sonar
>
> Use try-with-resources or close this "FileOutputStream" in a "finally" clause.
> GetKeyHandler: 
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW6HHKTfdBVcJdcVFsvC&open=AW6HHKTfdBVcJdcVFsvC]
>  
> Use try-with-resources or close this "OzoneOutputStream" in a "finally" 
> clause.
> PutKeyHandler: 
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW6HHKRodBVcJdcVFsvB&open=AW6HHKRodBVcJdcVFsvB]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Assigned] (HDDS-2580) Sonar: Close resources in xxxKeyHandler

2019-11-20 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai reassigned HDDS-2580:
--

Assignee: Attila Doroszlai

> Sonar: Close resources in xxxKeyHandler
> ---
>
> Key: HDDS-2580
> URL: https://issues.apache.org/jira/browse/HDDS-2580
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Dinesh Chitlangia
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: newbie, sonar
>
> Use try-with-resources or close this "FileOutputStream" in a "finally" clause.
> GetKeyHandler: 
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW6HHKTfdBVcJdcVFsvC&open=AW6HHKTfdBVcJdcVFsvC]
>  
> Use try-with-resources or close this "OzoneOutputStream" in a "finally" 
> clause.
> PutKeyHandler: 
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW6HHKRodBVcJdcVFsvB&open=AW6HHKRodBVcJdcVFsvB]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2580) Sonar: Close resources in xxxKeyHandler

2019-11-20 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2580 started by Attila Doroszlai.
--
> Sonar: Close resources in xxxKeyHandler
> ---
>
> Key: HDDS-2580
> URL: https://issues.apache.org/jira/browse/HDDS-2580
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Dinesh Chitlangia
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: newbie, sonar
>
> Use try-with-resources or close this "FileOutputStream" in a "finally" clause.
> GetKeyHandler: 
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW6HHKTfdBVcJdcVFsvC&open=AW6HHKTfdBVcJdcVFsvC]
>  
> Use try-with-resources or close this "OzoneOutputStream" in a "finally" 
> clause.
> PutKeyHandler: 
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW6HHKRodBVcJdcVFsvB&open=AW6HHKRodBVcJdcVFsvB]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-19 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2523:
---
Status: Patch Available  (was: In Progress)

> BufferPool.releaseBuffer may release a buffer different than the head of the 
> list
> -
>
> Key: HDDS-2523
> URL: https://issues.apache.org/jira/browse/HDDS-2523
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Ozone Client
>Reporter: Tsz-wo Sze
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: pull-request-available
> Attachments: a.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {code}
> //BufferPool
>   public void releaseBuffer(ByteBuffer byteBuffer) {
> // always remove from head of the list and append at last
> ByteBuffer buffer = bufferList.remove(0);
> // Ensure the buffer to be removed is always at the head of the list.
> Preconditions.checkArgument(buffer.equals(byteBuffer));
> buffer.clear();
> bufferList.add(buffer);
> Preconditions.checkArgument(currentBufferIndex >= 0);
> currentBufferIndex--;
>   }
> {code}
> In the code above, it expects buffer and byteBuffer are the same object, i.e. 
>  buffer == byteBuffer.  However the precondition is checking 
> buffer.equals(byteBuffer). Unfortunately, both buffer and byteBuffer have 
> remaining() == 0 so that equals(..) returns true and the precondition does 
> not catch the bug.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Resolved] (HDDS-2540) Fix accepetance test failure introduced by wait_for_safemode_exit

2019-11-19 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai resolved HDDS-2540.

Resolution: Fixed

Fixed by revert of HDDS-2034.

> Fix accepetance test failure introduced by wait_for_safemode_exit
> -
>
> Key: HDDS-2540
> URL: https://issues.apache.org/jira/browse/HDDS-2540
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Sammi Chen
>Assignee: Sammi Chen
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> https://github.com/apache/hadoop-ozone/blob/1b72718dcab7f83ebdac67b6242c729f03a8f103/hadoop-ozone/dist/src/main/compose/testlib.sh#L97
> - status=`docker-compose -f "${compose_file}" exec -T scm bash -c 
> "kinit -k HTTP/s...@example.com -t /etc/security/keytabs/HTTP.keytab && 
> $command'"`
> + status=`docker-compose -f "${compose_file}" exec -T scm bash -c 
> "kinit -k HTTP/s...@example.com -t /etc/security/keytabs/HTTP.keytab && 
> $command"`



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Resolved] (HDDS-2508) Fix TestDeadNodeHandler

2019-11-19 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai resolved HDDS-2508.

Resolution: Fixed

Fixed by reverting HDDS-2034.  Unit tests are green now:

https://github.com/apache/hadoop-ozone/runs/309695469

> Fix TestDeadNodeHandler
> ---
>
> Key: HDDS-2508
> URL: https://issues.apache.org/jira/browse/HDDS-2508
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: test
>Reporter: Attila Doroszlai
>Priority: Major
>
> {code}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 
> 63.647 s <<< FAILURE! - in org.apache.hadoop.hdds.scm.node.TestDeadNodeHandler
> [ERROR] testOnMessage(org.apache.hadoop.hdds.scm.node.TestDeadNodeHandler)  
> Time elapsed: 63.562 s  <<< ERROR!
> java.io.IOException: Could not allocate container. Cannot get any matching 
> pipeline for Type:RATIS, Factor:THREE, State:PipelineState.OPEN
>   at 
> org.apache.hadoop.hdds.scm.container.ContainerStateManager.allocateContainer(ContainerStateManager.java:261)
>   at 
> org.apache.hadoop.hdds.scm.container.SCMContainerManager.allocateContainer(SCMContainerManager.java:255)
>   at 
> org.apache.hadoop.hdds.scm.TestUtils.allocateContainer(TestUtils.java:488)
>   at 
> org.apache.hadoop.hdds.scm.node.TestDeadNodeHandler.testOnMessage(TestDeadNodeHandler.java:154)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Reopened] (HDDS-2034) Async RATIS pipeline creation and destroy through heartbeat commands

2019-11-19 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai reopened HDDS-2034:


> Async RATIS pipeline creation and destroy through heartbeat commands
> 
>
> Key: HDDS-2034
> URL: https://issues.apache.org/jira/browse/HDDS-2034
> Project: Hadoop Distributed Data Store
>  Issue Type: Sub-task
>Reporter: Sammi Chen
>Assignee: Sammi Chen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.0
>
>  Time Spent: 13.5h
>  Remaining Estimate: 0h
>
> Currently, pipeline creation and destroy are synchronous operations. SCM 
> directly connect to each datanode of the pipeline through gRPC channel to 
> create the pipeline to destroy the pipeline.  
> This task is to remove the gRPC channel, send pipeline creation and destroy 
> action through heartbeat command to each datanode.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2525) Sonar : replace lambda with method reference in SCM BufferPool

2019-11-19 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2525:
---
Labels: pull-request-available sonar  (was: pull-request-available)

> Sonar : replace lambda with method reference in SCM BufferPool
> --
>
> Key: HDDS-2525
> URL: https://issues.apache.org/jira/browse/HDDS-2525
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Supratim Deka
>Assignee: Supratim Deka
>Priority: Minor
>  Labels: pull-request-available, sonar
> Fix For: 0.5.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> As per Sonar, method references are more compact than lambda - this applies 
> to java 8, not older versions.
> Sonar report:
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_5KcVY8lQ4ZsVn&open=AW5md-_5KcVY8lQ4ZsVn



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2519) Sonar: Double Brace Initialization should not be used

2019-11-19 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2519:
---
Labels: pull-request-available sonar  (was: pull-request-available)

> Sonar: Double Brace Initialization should not be used
> -
>
> Key: HDDS-2519
> URL: https://issues.apache.org/jira/browse/HDDS-2519
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Dinesh Chitlangia
>Assignee: Dinesh Chitlangia
>Priority: Major
>  Labels: pull-request-available, sonar
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md_APKcVY8lQ4ZsWN&open=AW5md_APKcVY8lQ4ZsWN



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2520) Sonar: Avoid temporary variable scmSecurityClient

2019-11-19 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2520:
---
Labels: pull-request-available sonar  (was: pull-request-available)

> Sonar: Avoid temporary variable scmSecurityClient
> -
>
> Key: HDDS-2520
> URL: https://issues.apache.org/jira/browse/HDDS-2520
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Dinesh Chitlangia
>Assignee: Dinesh Chitlangia
>Priority: Major
>  Labels: pull-request-available, sonar
> Fix For: 0.5.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md_APKcVY8lQ4ZsWL&open=AW5md_APKcVY8lQ4ZsWL



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Commented] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-18 Thread Attila Doroszlai (Jira)


[ 
https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16976761#comment-16976761
 ] 

Attila Doroszlai commented on HDDS-2523:


Thanks [~szetszwo], that would be great.  I also debugged it and found so far 
that this happens with standalone replication type (which TestContainerMapper 
is using).  I have a simple fix idea, but need to test it with other cases, too.

> BufferPool.releaseBuffer may release a buffer different than the head of the 
> list
> -
>
> Key: HDDS-2523
> URL: https://issues.apache.org/jira/browse/HDDS-2523
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Ozone Client
>Reporter: Tsz-wo Sze
>Assignee: Attila Doroszlai
>Priority: Major
>
> {code}
> //BufferPool
>   public void releaseBuffer(ByteBuffer byteBuffer) {
> // always remove from head of the list and append at last
> ByteBuffer buffer = bufferList.remove(0);
> // Ensure the buffer to be removed is always at the head of the list.
> Preconditions.checkArgument(buffer.equals(byteBuffer));
> buffer.clear();
> bufferList.add(buffer);
> Preconditions.checkArgument(currentBufferIndex >= 0);
> currentBufferIndex--;
>   }
> {code}
> In the code above, it expects buffer and byteBuffer are the same object, i.e. 
>  buffer == byteBuffer.  However the precondition is checking 
> buffer.equals(byteBuffer). Unfortunately, both buffer and byteBuffer have 
> remaining() == 0 so that equals(..) returns true and the precondition does 
> not catch the bug.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-18 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2523 started by Attila Doroszlai.
--
> BufferPool.releaseBuffer may release a buffer different than the head of the 
> list
> -
>
> Key: HDDS-2523
> URL: https://issues.apache.org/jira/browse/HDDS-2523
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Tsz-wo Sze
>Assignee: Attila Doroszlai
>Priority: Major
>
> {code}
> //BufferPool
>   public void releaseBuffer(ByteBuffer byteBuffer) {
> // always remove from head of the list and append at last
> ByteBuffer buffer = bufferList.remove(0);
> // Ensure the buffer to be removed is always at the head of the list.
> Preconditions.checkArgument(buffer.equals(byteBuffer));
> buffer.clear();
> bufferList.add(buffer);
> Preconditions.checkArgument(currentBufferIndex >= 0);
> currentBufferIndex--;
>   }
> {code}
> In the code above, it expects buffer and byteBuffer are the same object, i.e. 
>  buffer == byteBuffer.  However the precondition is checking 
> buffer.equals(byteBuffer). Unfortunately, both buffer and byteBuffer have 
> remaining() == 0 so that equals(..) returns true and the precondition does 
> not catch the bug.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Assigned] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-18 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai reassigned HDDS-2523:
--

Assignee: Attila Doroszlai

> BufferPool.releaseBuffer may release a buffer different than the head of the 
> list
> -
>
> Key: HDDS-2523
> URL: https://issues.apache.org/jira/browse/HDDS-2523
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Tsz-wo Sze
>Assignee: Attila Doroszlai
>Priority: Major
>
> {code}
> //BufferPool
>   public void releaseBuffer(ByteBuffer byteBuffer) {
> // always remove from head of the list and append at last
> ByteBuffer buffer = bufferList.remove(0);
> // Ensure the buffer to be removed is always at the head of the list.
> Preconditions.checkArgument(buffer.equals(byteBuffer));
> buffer.clear();
> bufferList.add(buffer);
> Preconditions.checkArgument(currentBufferIndex >= 0);
> currentBufferIndex--;
>   }
> {code}
> In the code above, it expects buffer and byteBuffer are the same object, i.e. 
>  buffer == byteBuffer.  However the precondition is checking 
> buffer.equals(byteBuffer). Unfortunately, both buffer and byteBuffer have 
> remaining() == 0 so that equals(..) returns true and the precondition does 
> not catch the bug.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2272) Avoid buffer copying in GrpcReplicationClient

2019-11-18 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2272:
---
Labels: performance pull-request-available  (was: pull-request-available)

> Avoid buffer copying in GrpcReplicationClient
> -
>
> Key: HDDS-2272
> URL: https://issues.apache.org/jira/browse/HDDS-2272
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Tsz-wo Sze
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: performance, pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In StreamDownloader.onNext, CopyContainerResponseProto is copied to a byte[] 
> and then it is written out to the stream.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2273) Avoid buffer copying in GrpcReplicationService

2019-11-18 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2273:
---
Labels: performance pull-request-available  (was: pull-request-available)

> Avoid buffer copying in GrpcReplicationService
> --
>
> Key: HDDS-2273
> URL: https://issues.apache.org/jira/browse/HDDS-2273
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Tsz-wo Sze
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: performance, pull-request-available
> Fix For: 0.5.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In GrpcOutputStream, it writes data to a ByteArrayOutputStream and copies 
> them to a ByteString.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2270) Avoid buffer copying in ContainerStateMachine.loadSnapshot/persistContainerSet

2019-11-18 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2270:
---
Labels: performance pull-request-available  (was: pull-request-available)

> Avoid buffer copying in ContainerStateMachine.loadSnapshot/persistContainerSet
> --
>
> Key: HDDS-2270
> URL: https://issues.apache.org/jira/browse/HDDS-2270
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: Ozone Datanode
>Reporter: Tsz-wo Sze
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: performance, pull-request-available
> Fix For: 0.5.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> ContainerStateMachine:
> - In loadSnapshot(..), it first reads the snapshotFile to a  byte[] and then 
> parses it to ContainerProtos.Container2BCSIDMapProto.  The buffer copying can 
> be avoided.
> {code}
> try (FileInputStream fin = new FileInputStream(snapshotFile)) {
>   byte[] container2BCSIDData = IOUtils.toByteArray(fin);
>   ContainerProtos.Container2BCSIDMapProto proto =
>   ContainerProtos.Container2BCSIDMapProto
>   .parseFrom(container2BCSIDData);
>   ...
> }
> {code}
> - persistContainerSet(..) has similar problem.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2464) Avoid unnecessary allocations for FileChannel.open call

2019-11-18 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2464:
---
Labels: performance pull-request-available  (was: pull-request-available)

> Avoid unnecessary allocations for FileChannel.open call
> ---
>
> Key: HDDS-2464
> URL: https://issues.apache.org/jira/browse/HDDS-2464
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: Ozone Datanode
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
>  Labels: performance, pull-request-available
> Fix For: 0.5.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> {{ChunkUtils}} calls {{FileChannel#open(Path, OpenOption...)}}.  Vararg array 
> elements are then added to a new {{HashSet}} to call {{FileChannel#open(Path, 
> Set, FileAttribute...)}}.  We can call the latter 
> directly instead.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2415) Completely disable tracer if hdds.tracing.enabled=false

2019-11-18 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2415:
---
Labels: performance pull-request-available  (was: perfomance 
pull-request-available)

> Completely disable tracer if hdds.tracing.enabled=false
> ---
>
> Key: HDDS-2415
> URL: https://issues.apache.org/jira/browse/HDDS-2415
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
>  Labels: performance, pull-request-available
> Fix For: 0.5.0
>
> Attachments: allocations.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> There is a config setting to enable/disable OpenTracing-based distributed 
> tracing in Ozone ({{hdds.tracing.enabled}}).  However, setting it to false 
> does not prevent tracer initialization, which causes unnecessary object 
> allocations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2458) Avoid list copy in ChecksumData

2019-11-18 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2458:
---
Labels: performance pull-request-available  (was: pull-request-available)

> Avoid list copy in ChecksumData
> ---
>
> Key: HDDS-2458
> URL: https://issues.apache.org/jira/browse/HDDS-2458
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: Ozone Datanode
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
>  Labels: performance, pull-request-available
> Fix For: 0.5.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> {{ChecksumData}} is initially created with empty list of checksums, then it 
> is updated with computed checksums, copying the list.  The computed list can 
> be set directly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2445) Replace ToStringBuilder in BlockData

2019-11-18 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2445:
---
Labels: performance pull-request-available  (was: perfomance 
pull-request-available)

> Replace ToStringBuilder in BlockData
> 
>
> Key: HDDS-2445
> URL: https://issues.apache.org/jira/browse/HDDS-2445
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
>  Labels: performance, pull-request-available
> Fix For: 0.5.0
>
> Attachments: blockdata.png, setchunks.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> {{BlockData#toString}} uses {{ToStringBuilder}} for ease of implementation.  
> This has a few problems:
> # {{ToStringBuilder}} uses {{StringBuffer}}, which is synchronized
> # the default buffer is 512 bytes, more than needed here
> # {{BlockID}} and {{ContainerBlockID}} both use another {{StringBuilder}} or 
> {{StringBuffer}} for their {{toString}} implementation, leading to several 
> allocations and copies
> The flame graph shows that {{BlockData#toString}} may be responsible for 1.5% 
> of total allocations while putting keys.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2405) int2ByteString unnecessary byte array allocation

2019-11-18 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2405:
---
Labels: performance pull-request-available  (was: pull-request-available)

> int2ByteString unnecessary byte array allocation
> 
>
> Key: HDDS-2405
> URL: https://issues.apache.org/jira/browse/HDDS-2405
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
>  Labels: performance, pull-request-available
> Fix For: 0.5.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> {{int2ByteString}} implementations (currently duplicated in 
> [RatisHelper|https://github.com/apache/hadoop-ozone/blob/6b2cda125b3647870ef5b01cf64e3b3e4cdc55db/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java#L280-L289]
>  and 
> [Checksum|https://github.com/apache/hadoop-ozone/blob/6b2cda125b3647870ef5b01cf64e3b3e4cdc55db/hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/Checksum.java#L64-L73],
>  but the first one is being removed in HDDS-2375) result in unnecessary byte 
> array allocations:
> # {{ByteString.Output}} creates 128-byte buffer by default, which is too 
> large for writing a single int
> # {{DataOutputStream}} allocates an [extra 8-byte 
> array|https://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/io/DataOutputStream.java#l204],
>  used only for writing longs
> # {{ByteString.Output}} also creates 10-element array for {{flushedBuffers}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2522) Fix TestSecureOzoneCluster

2019-11-17 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2522:
---
Status: Patch Available  (was: In Progress)

> Fix TestSecureOzoneCluster
> --
>
> Key: HDDS-2522
> URL: https://issues.apache.org/jira/browse/HDDS-2522
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: test
>Affects Versions: 0.5.0
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> TestSecureOzoneCluster is failing with {{failure to login}}.
> {code:title=https://github.com/elek/ozone-ci-03/blob/master/pr/pr-hdds-2291-5997d/integration/hadoop-ozone/integration-test/org.apache.hadoop.ozone.TestSecureOzoneCluster.txt}
> ---
> Test set: org.apache.hadoop.ozone.TestSecureOzoneCluster
> ---
> Tests run: 10, Failures: 0, Errors: 6, Skipped: 0, Time elapsed: 23.937 s <<< 
> FAILURE! - in org.apache.hadoop.ozone.TestSecureOzoneCluster
> testSCMSecurityProtocol(org.apache.hadoop.ozone.TestSecureOzoneCluster)  Time 
> elapsed: 2.474 s  <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: 
> failure to login: for principal: 
> scm/pr-hdds-2291-5997d-4279494...@example.com from keytab 
> /workdir/hadoop-ozone/integration-test/target/test-dir/TestSecureOzoneCluster/scm.keytab
>  javax.security.auth.login.LoginException: Unable to obtain password from user
>   at 
> org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1847)
>   at 
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1215)
>   at 
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1008)
>   at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:315)
>   at 
> org.apache.hadoop.hdds.scm.server.StorageContainerManager.loginAsSCMUser(StorageContainerManager.java:508)
>   at 
> org.apache.hadoop.hdds.scm.server.StorageContainerManager.(StorageContainerManager.java:254)
>   at 
> org.apache.hadoop.hdds.scm.server.StorageContainerManager.(StorageContainerManager.java:212)
>   at 
> org.apache.hadoop.hdds.scm.server.StorageContainerManager.createSCM(StorageContainerManager.java:600)
>   at 
> org.apache.hadoop.hdds.scm.HddsTestUtils.getScm(HddsTestUtils.java:91)
>   at 
> org.apache.hadoop.ozone.TestSecureOzoneCluster.testSCMSecurityProtocol(TestSecureOzoneCluster.java:299)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2522) Fix TestSecureOzoneCluster

2019-11-17 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2522:
--

 Summary: Fix TestSecureOzoneCluster
 Key: HDDS-2522
 URL: https://issues.apache.org/jira/browse/HDDS-2522
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: test
Affects Versions: 0.5.0
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


TestSecureOzoneCluster is failing with {{failure to login}}.

{code:title=https://github.com/elek/ozone-ci-03/blob/master/pr/pr-hdds-2291-5997d/integration/hadoop-ozone/integration-test/org.apache.hadoop.ozone.TestSecureOzoneCluster.txt}
---
Test set: org.apache.hadoop.ozone.TestSecureOzoneCluster
---
Tests run: 10, Failures: 0, Errors: 6, Skipped: 0, Time elapsed: 23.937 s <<< 
FAILURE! - in org.apache.hadoop.ozone.TestSecureOzoneCluster
testSCMSecurityProtocol(org.apache.hadoop.ozone.TestSecureOzoneCluster)  Time 
elapsed: 2.474 s  <<< ERROR!
org.apache.hadoop.security.KerberosAuthException: 
failure to login: for principal: scm/pr-hdds-2291-5997d-4279494...@example.com 
from keytab 
/workdir/hadoop-ozone/integration-test/target/test-dir/TestSecureOzoneCluster/scm.keytab
 javax.security.auth.login.LoginException: Unable to obtain password from user

at 
org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1847)
at 
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1215)
at 
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1008)
at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:315)
at 
org.apache.hadoop.hdds.scm.server.StorageContainerManager.loginAsSCMUser(StorageContainerManager.java:508)
at 
org.apache.hadoop.hdds.scm.server.StorageContainerManager.(StorageContainerManager.java:254)
at 
org.apache.hadoop.hdds.scm.server.StorageContainerManager.(StorageContainerManager.java:212)
at 
org.apache.hadoop.hdds.scm.server.StorageContainerManager.createSCM(StorageContainerManager.java:600)
at 
org.apache.hadoop.hdds.scm.HddsTestUtils.getScm(HddsTestUtils.java:91)
at 
org.apache.hadoop.ozone.TestSecureOzoneCluster.testSCMSecurityProtocol(TestSecureOzoneCluster.java:299)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2522) Fix TestSecureOzoneCluster

2019-11-17 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2522 started by Attila Doroszlai.
--
> Fix TestSecureOzoneCluster
> --
>
> Key: HDDS-2522
> URL: https://issues.apache.org/jira/browse/HDDS-2522
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: test
>Affects Versions: 0.5.0
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>
> TestSecureOzoneCluster is failing with {{failure to login}}.
> {code:title=https://github.com/elek/ozone-ci-03/blob/master/pr/pr-hdds-2291-5997d/integration/hadoop-ozone/integration-test/org.apache.hadoop.ozone.TestSecureOzoneCluster.txt}
> ---
> Test set: org.apache.hadoop.ozone.TestSecureOzoneCluster
> ---
> Tests run: 10, Failures: 0, Errors: 6, Skipped: 0, Time elapsed: 23.937 s <<< 
> FAILURE! - in org.apache.hadoop.ozone.TestSecureOzoneCluster
> testSCMSecurityProtocol(org.apache.hadoop.ozone.TestSecureOzoneCluster)  Time 
> elapsed: 2.474 s  <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: 
> failure to login: for principal: 
> scm/pr-hdds-2291-5997d-4279494...@example.com from keytab 
> /workdir/hadoop-ozone/integration-test/target/test-dir/TestSecureOzoneCluster/scm.keytab
>  javax.security.auth.login.LoginException: Unable to obtain password from user
>   at 
> org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1847)
>   at 
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1215)
>   at 
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1008)
>   at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:315)
>   at 
> org.apache.hadoop.hdds.scm.server.StorageContainerManager.loginAsSCMUser(StorageContainerManager.java:508)
>   at 
> org.apache.hadoop.hdds.scm.server.StorageContainerManager.(StorageContainerManager.java:254)
>   at 
> org.apache.hadoop.hdds.scm.server.StorageContainerManager.(StorageContainerManager.java:212)
>   at 
> org.apache.hadoop.hdds.scm.server.StorageContainerManager.createSCM(StorageContainerManager.java:600)
>   at 
> org.apache.hadoop.hdds.scm.HddsTestUtils.getScm(HddsTestUtils.java:91)
>   at 
> org.apache.hadoop.ozone.TestSecureOzoneCluster.testSCMSecurityProtocol(TestSecureOzoneCluster.java:299)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2521) Multipart upload failing with NPE

2019-11-17 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2521:
---
Status: Patch Available  (was: In Progress)

> Multipart upload failing with NPE
> -
>
> Key: HDDS-2521
> URL: https://issues.apache.org/jira/browse/HDDS-2521
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: S3
>Affects Versions: 0.5.0
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Critical
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> S3 multipart upload is 
> [failing|https://elek.github.io/ozone-ci-03/pr/pr-hdds-2501-b5dhd/acceptance/summary.html#s1-s11-s5]
>  with 
> [NPE|https://github.com/elek/ozone-ci-03/blob/ddbaf4dd92ee5f855fea3e84c59b702fb2dda663/pr/pr-hdds-2501-b5dhd/acceptance/docker-ozones3-ozones3-s3-scm.log#L740-L747].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Commented] (HDDS-2274) Avoid buffer copying in Codec

2019-11-17 Thread Attila Doroszlai (Jira)


[ 
https://issues.apache.org/jira/browse/HDDS-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16975978#comment-16975978
 ] 

Attila Doroszlai commented on HDDS-2274:


Thanks [~szetszwo] for confirmation.  Let me reassign this back to you for 
planning.

> Avoid buffer copying in Codec
> -
>
> Key: HDDS-2274
> URL: https://issues.apache.org/jira/browse/HDDS-2274
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Tsz-wo Sze
>Assignee: Attila Doroszlai
>Priority: Major
>
> Codec declares byte[] as a parameter in fromPersistedFormat(..) and a return 
> type in toPersistedFormat(..).  It leads to buffer copying when using it with 
> ByteString.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Assigned] (HDDS-2274) Avoid buffer copying in Codec

2019-11-17 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai reassigned HDDS-2274:
--

Assignee: Tsz-wo Sze  (was: Attila Doroszlai)

> Avoid buffer copying in Codec
> -
>
> Key: HDDS-2274
> URL: https://issues.apache.org/jira/browse/HDDS-2274
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Tsz-wo Sze
>Assignee: Tsz-wo Sze
>Priority: Major
>
> Codec declares byte[] as a parameter in fromPersistedFormat(..) and a return 
> type in toPersistedFormat(..).  It leads to buffer copying when using it with 
> ByteString.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2521) Multipart upload failing with NPE

2019-11-17 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2521 started by Attila Doroszlai.
--
> Multipart upload failing with NPE
> -
>
> Key: HDDS-2521
> URL: https://issues.apache.org/jira/browse/HDDS-2521
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: S3
>Affects Versions: 0.5.0
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Critical
>
> S3 multipart upload is 
> [failing|https://elek.github.io/ozone-ci-03/pr/pr-hdds-2501-b5dhd/acceptance/summary.html#s1-s11-s5]
>  with 
> [NPE|https://github.com/elek/ozone-ci-03/blob/ddbaf4dd92ee5f855fea3e84c59b702fb2dda663/pr/pr-hdds-2501-b5dhd/acceptance/docker-ozones3-ozones3-s3-scm.log#L740-L747].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2521) Multipart upload failing with NPE

2019-11-17 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2521:
--

 Summary: Multipart upload failing with NPE
 Key: HDDS-2521
 URL: https://issues.apache.org/jira/browse/HDDS-2521
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: S3
Affects Versions: 0.5.0
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


S3 multipart upload is 
[failing|https://elek.github.io/ozone-ci-03/pr/pr-hdds-2501-b5dhd/acceptance/summary.html#s1-s11-s5]
 with 
[NPE|https://github.com/elek/ozone-ci-03/blob/ddbaf4dd92ee5f855fea3e84c59b702fb2dda663/pr/pr-hdds-2501-b5dhd/acceptance/docker-ozones3-ozones3-s3-scm.log#L740-L747].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2485) Disable XML external entity processing

2019-11-17 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2485:
---
Status: Patch Available  (was: In Progress)

> Disable XML external entity processing
> --
>
> Key: HDDS-2485
> URL: https://issues.apache.org/jira/browse/HDDS-2485
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Security
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: pull-request-available, sonar
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Disable XML external entity processing in
> * NodeSchemaLoader: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-2nKcVY8lQ4ZsNm&open=AW5md-2nKcVY8lQ4ZsNm
> * ConfigFileAppender:
> ** 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVY&open=AW5md-_uKcVY8lQ4ZsVY
> ** 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVZ&open=AW5md-_uKcVY8lQ4ZsVZ
> * MultiDeleteRequestUnmarshaller: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-kDKcVY8lQ4Zr-N&open=AW5md-kDKcVY8lQ4Zr-N



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Assigned] (HDDS-2485) Disable XML external entity processing

2019-11-16 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai reassigned HDDS-2485:
--

Assignee: Attila Doroszlai

> Disable XML external entity processing
> --
>
> Key: HDDS-2485
> URL: https://issues.apache.org/jira/browse/HDDS-2485
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Security
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: sonar
>
> Disable XML external entity processing in
> * NodeSchemaLoader: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-2nKcVY8lQ4ZsNm&open=AW5md-2nKcVY8lQ4ZsNm
> * ConfigFileAppender:
> ** 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVY&open=AW5md-_uKcVY8lQ4ZsVY
> ** 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVZ&open=AW5md-_uKcVY8lQ4ZsVZ
> * MultiDeleteRequestUnmarshaller: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-kDKcVY8lQ4Zr-N&open=AW5md-kDKcVY8lQ4Zr-N



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2485) Disable XML external entity processing

2019-11-16 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2485 started by Attila Doroszlai.
--
> Disable XML external entity processing
> --
>
> Key: HDDS-2485
> URL: https://issues.apache.org/jira/browse/HDDS-2485
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Security
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: sonar
>
> Disable XML external entity processing in
> * NodeSchemaLoader: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-2nKcVY8lQ4ZsNm&open=AW5md-2nKcVY8lQ4ZsNm
> * ConfigFileAppender:
> ** 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVY&open=AW5md-_uKcVY8lQ4ZsVY
> ** 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVZ&open=AW5md-_uKcVY8lQ4ZsVZ
> * MultiDeleteRequestUnmarshaller: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-kDKcVY8lQ4Zr-N&open=AW5md-kDKcVY8lQ4Zr-N



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2485) Disable XML external entity processing

2019-11-16 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2485:
---
Description: 
Disable XML external entity processing in

* NodeSchemaLoader: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-2nKcVY8lQ4ZsNm&open=AW5md-2nKcVY8lQ4ZsNm
* ConfigFileAppender:
** 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVY&open=AW5md-_uKcVY8lQ4ZsVY
** 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVZ&open=AW5md-_uKcVY8lQ4ZsVZ
* MultiDeleteRequestUnmarshaller: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-kDKcVY8lQ4Zr-N&open=AW5md-kDKcVY8lQ4Zr-N

  was:
Disable XML external entity processing in

* NodeSchemaLoader: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-2nKcVY8lQ4ZsNm&open=AW5md-2nKcVY8lQ4ZsNm
* ConfigFileAppender: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVY&open=AW5md-_uKcVY8lQ4ZsVY
* MultiDeleteRequestUnmarshaller: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-kDKcVY8lQ4Zr-N&open=AW5md-kDKcVY8lQ4Zr-N


> Disable XML external entity processing
> --
>
> Key: HDDS-2485
> URL: https://issues.apache.org/jira/browse/HDDS-2485
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Security
>Reporter: Attila Doroszlai
>Priority: Major
>  Labels: sonar
>
> Disable XML external entity processing in
> * NodeSchemaLoader: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-2nKcVY8lQ4ZsNm&open=AW5md-2nKcVY8lQ4ZsNm
> * ConfigFileAppender:
> ** 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVY&open=AW5md-_uKcVY8lQ4ZsVY
> ** 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVZ&open=AW5md-_uKcVY8lQ4ZsVZ
> * MultiDeleteRequestUnmarshaller: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-kDKcVY8lQ4Zr-N&open=AW5md-kDKcVY8lQ4Zr-N



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2405) int2ByteString unnecessary byte array allocation

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2405:
---
Status: Patch Available  (was: Open)

> int2ByteString unnecessary byte array allocation
> 
>
> Key: HDDS-2405
> URL: https://issues.apache.org/jira/browse/HDDS-2405
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{int2ByteString}} implementations (currently duplicated in 
> [RatisHelper|https://github.com/apache/hadoop-ozone/blob/6b2cda125b3647870ef5b01cf64e3b3e4cdc55db/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java#L280-L289]
>  and 
> [Checksum|https://github.com/apache/hadoop-ozone/blob/6b2cda125b3647870ef5b01cf64e3b3e4cdc55db/hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/Checksum.java#L64-L73],
>  but the first one is being removed in HDDS-2375) result in unnecessary byte 
> array allocations:
> # {{ByteString.Output}} creates 128-byte buffer by default, which is too 
> large for writing a single int
> # {{DataOutputStream}} allocates an [extra 8-byte 
> array|https://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/io/DataOutputStream.java#l204],
>  used only for writing longs
> # {{ByteString.Output}} also creates 10-element array for {{flushedBuffers}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2516) Code cleanup in EventQueue

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2516:
---
Status: Patch Available  (was: In Progress)

> Code cleanup in EventQueue
> --
>
> Key: HDDS-2516
> URL: https://issues.apache.org/jira/browse/HDDS-2516
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: pull-request-available, sonar
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> https://sonarcloud.io/project/issues?fileUuids=AW5md-HgKcVY8lQ4ZrfB&id=hadoop-ozone&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Commented] (HDDS-2508) Fix TestDeadNodeHandler

2019-11-15 Thread Attila Doroszlai (Jira)


[ 
https://issues.apache.org/jira/browse/HDDS-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16975397#comment-16975397
 ] 

Attila Doroszlai commented on HDDS-2508:


I haven't investigated it, but it passes locally, both from IDE and Maven.

> Fix TestDeadNodeHandler
> ---
>
> Key: HDDS-2508
> URL: https://issues.apache.org/jira/browse/HDDS-2508
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: test
>Reporter: Attila Doroszlai
>Priority: Major
>
> {code}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 
> 63.647 s <<< FAILURE! - in org.apache.hadoop.hdds.scm.node.TestDeadNodeHandler
> [ERROR] testOnMessage(org.apache.hadoop.hdds.scm.node.TestDeadNodeHandler)  
> Time elapsed: 63.562 s  <<< ERROR!
> java.io.IOException: Could not allocate container. Cannot get any matching 
> pipeline for Type:RATIS, Factor:THREE, State:PipelineState.OPEN
>   at 
> org.apache.hadoop.hdds.scm.container.ContainerStateManager.allocateContainer(ContainerStateManager.java:261)
>   at 
> org.apache.hadoop.hdds.scm.container.SCMContainerManager.allocateContainer(SCMContainerManager.java:255)
>   at 
> org.apache.hadoop.hdds.scm.TestUtils.allocateContainer(TestUtils.java:488)
>   at 
> org.apache.hadoop.hdds.scm.node.TestDeadNodeHandler.testOnMessage(TestDeadNodeHandler.java:154)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2517) Immediately return this expression instead of assigning it to the temporary variable

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2517:
---
Labels: pull-request-available sonar  (was: pull-request-available)

> Immediately return this expression instead of assigning it to the temporary 
> variable
> 
>
> Key: HDDS-2517
> URL: https://issues.apache.org/jira/browse/HDDS-2517
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Abhishek Purohit
>Assignee: Abhishek Purohit
>Priority: Major
>  Labels: pull-request-available, sonar
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Related to : 
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&open=AW5md_AGKcVY8lQ4ZsV1&resolved=false]
> Immediately return this expression instead of assigning it to the temporary 
> variable



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2516) Code cleanup in EventQueue

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2516 started by Attila Doroszlai.
--
> Code cleanup in EventQueue
> --
>
> Key: HDDS-2516
> URL: https://issues.apache.org/jira/browse/HDDS-2516
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: sonar
>
> https://sonarcloud.io/project/issues?fileUuids=AW5md-HgKcVY8lQ4ZrfB&id=hadoop-ozone&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2516) Code cleanup in EventQueue

2019-11-15 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2516:
--

 Summary: Code cleanup in EventQueue
 Key: HDDS-2516
 URL: https://issues.apache.org/jira/browse/HDDS-2516
 Project: Hadoop Distributed Data Store
  Issue Type: Improvement
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai




https://sonarcloud.io/project/issues?fileUuids=AW5md-HgKcVY8lQ4ZrfB&id=hadoop-ozone&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2514) Remove this unused method parameter "encodedToken"

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2514:
---
Labels: sonar  (was: )

> Remove this unused method parameter "encodedToken"
> --
>
> Key: HDDS-2514
> URL: https://issues.apache.org/jira/browse/HDDS-2514
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Abhishek Purohit
>Assignee: Abhishek Purohit
>Priority: Major
>  Labels: sonar
>
> Remove this unused method parameter "encodedToken"
> method: connectToDatanode
> Class: XceiverClientGrpc
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2515) No need to call "toString()" method as formatting and string conversion is done by the Formatter

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2515:
---
Labels: sonar  (was: )

> No need to call "toString()" method as formatting and string conversion is 
> done by the Formatter
> 
>
> Key: HDDS-2515
> URL: https://issues.apache.org/jira/browse/HDDS-2515
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Abhishek Purohit
>Assignee: Abhishek Purohit
>Priority: Major
>  Labels: sonar
>
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&open=AW5md_AGKcVY8lQ4ZsV4&resolved=false]
> Class:  XceiverClientGrpc
> {code:java}
>  if (LOG.isDebugEnabled()) {  LOG.debug("Nodes in pipeline : {}", 
> pipeline.getNodes().toString());
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2513) Remove this unused "COMPONENT" private field.

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2513:
---
Labels: sonar  (was: )

> Remove this unused "COMPONENT" private field.
> -
>
> Key: HDDS-2513
> URL: https://issues.apache.org/jira/browse/HDDS-2513
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Abhishek Purohit
>Assignee: Abhishek Purohit
>Priority: Minor
>  Labels: sonar
>
> Remove this unused "COMPONENT" private field in class 
> XceiverClientGrpc
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&open=AW5md_AGKcVY8lQ4ZsWG&resolved=false]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2509) Code cleanup in replication package

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2509:
---
Status: Patch Available  (was: In Progress)

> Code cleanup in replication package
> ---
>
> Key: HDDS-2509
> URL: https://issues.apache.org/jira/browse/HDDS-2509
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: pull-request-available, sonar
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Fix couple of [issues 
> reported|https://sonarcloud.io/project/issues?directories=hadoop-hdds%2Fcontainer-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fozone%2Fcontainer%2Freplication%2Chadoop-hdds%2Fcontainer-service%2Fsrc%2Ftest%2Fjava%2Forg%2Fapache%2Fhadoop%2Fozone%2Fcontainer%2Freplication&id=hadoop-ozone&resolved=false]
>  in {{org.apache.hadoop.ozone.container.replication}} package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2512) Sonar TraceAllMethod NPE Could be Thrown

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2512:
---
Labels: sonar  (was: )

> Sonar TraceAllMethod NPE Could be Thrown
> 
>
> Key: HDDS-2512
> URL: https://issues.apache.org/jira/browse/HDDS-2512
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Matthew Sharp
>Assignee: Matthew Sharp
>Priority: Minor
>  Labels: sonar
>
> Sonar cleanup: 
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-2WKcVY8lQ4ZsNQ&open=AW5md-2WKcVY8lQ4ZsNQ]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2509) Code cleanup in replication package

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2509 started by Attila Doroszlai.
--
> Code cleanup in replication package
> ---
>
> Key: HDDS-2509
> URL: https://issues.apache.org/jira/browse/HDDS-2509
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: sonar
>
> Fix couple of [issues 
> reported|https://sonarcloud.io/project/issues?directories=hadoop-hdds%2Fcontainer-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fozone%2Fcontainer%2Freplication%2Chadoop-hdds%2Fcontainer-service%2Fsrc%2Ftest%2Fjava%2Forg%2Fapache%2Fhadoop%2Fozone%2Fcontainer%2Freplication&id=hadoop-ozone&resolved=false]
>  in {{org.apache.hadoop.ozone.container.replication}} package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2509) Code cleanup in replication package

2019-11-15 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2509:
--

 Summary: Code cleanup in replication package
 Key: HDDS-2509
 URL: https://issues.apache.org/jira/browse/HDDS-2509
 Project: Hadoop Distributed Data Store
  Issue Type: Improvement
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


Fix couple of [issues 
reported|https://sonarcloud.io/project/issues?directories=hadoop-hdds%2Fcontainer-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fozone%2Fcontainer%2Freplication%2Chadoop-hdds%2Fcontainer-service%2Fsrc%2Ftest%2Fjava%2Forg%2Fapache%2Fhadoop%2Fozone%2Fcontainer%2Freplication&id=hadoop-ozone&resolved=false]
 in {{org.apache.hadoop.ozone.container.replication}} package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2505) Fix logic related to SCM address calculation in HddsUtils

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2505:
---
Status: Patch Available  (was: Open)

> Fix logic related to SCM address calculation in HddsUtils
> -
>
> Key: HDDS-2505
> URL: https://issues.apache.org/jira/browse/HDDS-2505
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: SCM
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: pull-request-available, sonar
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{HddsUtils}} has 3 methods to calculate SCM address for various client 
> types.  All have an unreachable {{if}} branch, because:
> # {{iterator().next()}} throws exception for empty list
> # {{getSCMAddresses}} never returns empty list anyway, it throws exception
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPX&open=AW5md-4qKcVY8lQ4ZsPX
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPY&open=AW5md-4qKcVY8lQ4ZsPY
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPW&open=AW5md-4qKcVY8lQ4ZsPW
> Ideally code duplication among these methods should be reduced, too.
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPU&open=AW5md-4qKcVY8lQ4ZsPU
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPT&open=AW5md-4qKcVY8lQ4ZsPT
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPV&open=AW5md-4qKcVY8lQ4ZsPV
> Complete list of issues in the same file:
> https://sonarcloud.io/project/issues?fileUuids=AW5md-HhKcVY8lQ4Zrjn&id=hadoop-ozone&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2508) Fix TestDeadNodeHandler

2019-11-15 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2508:
--

 Summary: Fix TestDeadNodeHandler
 Key: HDDS-2508
 URL: https://issues.apache.org/jira/browse/HDDS-2508
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: test
Reporter: Attila Doroszlai


{code}
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 63.647 
s <<< FAILURE! - in org.apache.hadoop.hdds.scm.node.TestDeadNodeHandler
[ERROR] testOnMessage(org.apache.hadoop.hdds.scm.node.TestDeadNodeHandler)  
Time elapsed: 63.562 s  <<< ERROR!
java.io.IOException: Could not allocate container. Cannot get any matching 
pipeline for Type:RATIS, Factor:THREE, State:PipelineState.OPEN
at 
org.apache.hadoop.hdds.scm.container.ContainerStateManager.allocateContainer(ContainerStateManager.java:261)
at 
org.apache.hadoop.hdds.scm.container.SCMContainerManager.allocateContainer(SCMContainerManager.java:255)
at 
org.apache.hadoop.hdds.scm.TestUtils.allocateContainer(TestUtils.java:488)
at 
org.apache.hadoop.hdds.scm.node.TestDeadNodeHandler.testOnMessage(TestDeadNodeHandler.java:154)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2505) Fix logic related to SCM address calculation in HddsUtils

2019-11-15 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2505:
--

 Summary: Fix logic related to SCM address calculation in HddsUtils
 Key: HDDS-2505
 URL: https://issues.apache.org/jira/browse/HDDS-2505
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: SCM
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


{{HddsUtils}} has 3 methods to calculate SCM address for various client types.  
All have an unreachable {{if}} branch, because:

# {{iterator().next()}} throws exception for empty list
# {{getSCMAddresses}} never returns empty list anyway, it throws exception

* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPX&open=AW5md-4qKcVY8lQ4ZsPX
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPY&open=AW5md-4qKcVY8lQ4ZsPY
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPW&open=AW5md-4qKcVY8lQ4ZsPW

Ideally code duplication among these methods should be reduced, too.

* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPU&open=AW5md-4qKcVY8lQ4ZsPU
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPT&open=AW5md-4qKcVY8lQ4ZsPT
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPV&open=AW5md-4qKcVY8lQ4ZsPV

Complete list of issues in the same file:

https://sonarcloud.io/project/issues?fileUuids=AW5md-HhKcVY8lQ4Zrjn&id=hadoop-ozone&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2504) Handle InterruptedException properly

2019-11-15 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2504:
--

 Summary: Handle InterruptedException properly
 Key: HDDS-2504
 URL: https://issues.apache.org/jira/browse/HDDS-2504
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
Reporter: Attila Doroszlai


bq. Either re-interrupt or rethrow the {{InterruptedException}}

in several files (39 issues)

https://sonarcloud.io/project/issues?id=hadoop-ozone&resolved=false&rules=squid%3AS2142&statuses=OPEN&types=BUG

Feel free to create sub-tasks if needed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2503) Close FlushOptions in RDBStore

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2503:
---
Status: Patch Available  (was: In Progress)

> Close FlushOptions in RDBStore
> --
>
> Key: HDDS-2503
> URL: https://issues.apache.org/jira/browse/HDDS-2503
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: pull-request-available, sonar
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{FlushOptions}} should be closed after use.
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-zwKcVY8lQ4ZsJ4&open=AW5md-zwKcVY8lQ4ZsJ4
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-zwKcVY8lQ4ZsJ5&open=AW5md-zwKcVY8lQ4ZsJ5
> Sonar also reported 15 further issues in the same file:
> https://sonarcloud.io/project/issues?fileUuids=AW5md-HgKcVY8lQ4Zrga&id=hadoop-ozone&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Assigned] (HDDS-2503) Close FlushOptions in RDBStore

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai reassigned HDDS-2503:
--

Assignee: Attila Doroszlai

> Close FlushOptions in RDBStore
> --
>
> Key: HDDS-2503
> URL: https://issues.apache.org/jira/browse/HDDS-2503
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: sonar
>
> {{FlushOptions}} should be closed after use.
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-zwKcVY8lQ4ZsJ4&open=AW5md-zwKcVY8lQ4ZsJ4
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-zwKcVY8lQ4ZsJ5&open=AW5md-zwKcVY8lQ4ZsJ5
> Sonar also reported 15 further issues in the same file:
> https://sonarcloud.io/project/issues?fileUuids=AW5md-HgKcVY8lQ4Zrga&id=hadoop-ozone&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2503) Close FlushOptions in RDBStore

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2503 started by Attila Doroszlai.
--
> Close FlushOptions in RDBStore
> --
>
> Key: HDDS-2503
> URL: https://issues.apache.org/jira/browse/HDDS-2503
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: sonar
>
> {{FlushOptions}} should be closed after use.
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-zwKcVY8lQ4ZsJ4&open=AW5md-zwKcVY8lQ4ZsJ4
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-zwKcVY8lQ4ZsJ5&open=AW5md-zwKcVY8lQ4ZsJ5
> Sonar also reported 15 further issues in the same file:
> https://sonarcloud.io/project/issues?fileUuids=AW5md-HgKcVY8lQ4Zrga&id=hadoop-ozone&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2503) Close FlushOptions in RDBStore

2019-11-15 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2503:
--

 Summary: Close FlushOptions in RDBStore
 Key: HDDS-2503
 URL: https://issues.apache.org/jira/browse/HDDS-2503
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
Reporter: Attila Doroszlai


{{FlushOptions}} should be closed after use.

* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-zwKcVY8lQ4ZsJ4&open=AW5md-zwKcVY8lQ4ZsJ4
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-zwKcVY8lQ4ZsJ5&open=AW5md-zwKcVY8lQ4ZsJ5

Sonar also reported 15 further issues in the same file:

https://sonarcloud.io/project/issues?fileUuids=AW5md-HgKcVY8lQ4Zrga&id=hadoop-ozone&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2502) Close ScmClient in RatisInsight

2019-11-15 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2502:
--

 Summary: Close ScmClient in RatisInsight
 Key: HDDS-2502
 URL: https://issues.apache.org/jira/browse/HDDS-2502
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
Reporter: Attila Doroszlai


{{ScmClient}} in {{RatisInsight}} should be closed after use.

https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-mYKcVY8lQ4Zr_s&open=AW5md-mYKcVY8lQ4Zr_s

Also two other minor issues reported in the same file:

https://sonarcloud.io/project/issues?fileUuids=AW5md-HeKcVY8lQ4ZrXL&id=hadoop-ozone&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2485) Disable XML external entity processing

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2485:
---
Description: 
Disable XML external entity processing in

* NodeSchemaLoader: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-2nKcVY8lQ4ZsNm&open=AW5md-2nKcVY8lQ4ZsNm
* ConfigFileAppender: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVY&open=AW5md-_uKcVY8lQ4ZsVY
* MultiDeleteRequestUnmarshaller: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-kDKcVY8lQ4Zr-N&open=AW5md-kDKcVY8lQ4Zr-N

  was:
Disable XML external entity processing in

* NodeSchemaLoader: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-2nKcVY8lQ4ZsNm&open=AW5md-2nKcVY8lQ4ZsNm
* ConfigFileAppender: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVY&open=AW5md-_uKcVY8lQ4ZsVY


> Disable XML external entity processing
> --
>
> Key: HDDS-2485
> URL: https://issues.apache.org/jira/browse/HDDS-2485
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Attila Doroszlai
>Priority: Major
>  Labels: sonar
>
> Disable XML external entity processing in
> * NodeSchemaLoader: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-2nKcVY8lQ4ZsNm&open=AW5md-2nKcVY8lQ4ZsNm
> * ConfigFileAppender: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVY&open=AW5md-_uKcVY8lQ4ZsVY
> * MultiDeleteRequestUnmarshaller: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-kDKcVY8lQ4Zr-N&open=AW5md-kDKcVY8lQ4Zr-N



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2501) Ensure stream is closed in ObjectEndpoint

2019-11-15 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2501:
--

 Summary: Ensure stream is closed in ObjectEndpoint
 Key: HDDS-2501
 URL: https://issues.apache.org/jira/browse/HDDS-2501
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: S3
Reporter: Attila Doroszlai


Ensure {{ObjectOutputStream}} is closed in {{ObjectEndpoint}}:

https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-j-KcVY8lQ4Zr96&open=AW5md-j-KcVY8lQ4Zr96

And fix other issues in the same file:

https://sonarcloud.io/project/issues?fileUuids=AW5md-HdKcVY8lQ4ZrVc&id=hadoop-ozone&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2500) Avoid fall-through in CloseContainerCommandHandler

2019-11-15 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2500:
--

 Summary: Avoid fall-through in CloseContainerCommandHandler
 Key: HDDS-2500
 URL: https://issues.apache.org/jira/browse/HDDS-2500
 Project: Hadoop Distributed Data Store
  Issue Type: Improvement
  Components: Ozone Datanode
Reporter: Attila Doroszlai


Two instances of fall-through:

* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-7UKcVY8lQ4ZsRk&open=AW5md-7UKcVY8lQ4ZsRk
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-7UKcVY8lQ4ZsRj&open=AW5md-7UKcVY8lQ4ZsRj

Both seem OK, but unnecessary (the next branch is {{break}}-only).  Could be 
made more explicit by moving/adding {{break}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2500) Avoid fall-through in CloseContainerCommandHandler

2019-11-15 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2500:
---
Labels: sonar  (was: )

> Avoid fall-through in CloseContainerCommandHandler
> --
>
> Key: HDDS-2500
> URL: https://issues.apache.org/jira/browse/HDDS-2500
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: Ozone Datanode
>Reporter: Attila Doroszlai
>Priority: Minor
>  Labels: sonar
>
> Two instances of fall-through:
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-7UKcVY8lQ4ZsRk&open=AW5md-7UKcVY8lQ4ZsRk
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-7UKcVY8lQ4ZsRj&open=AW5md-7UKcVY8lQ4ZsRj
> Both seem OK, but unnecessary (the next branch is {{break}}-only).  Could be 
> made more explicit by moving/adding {{break}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2498) Sonar: Fix issues found in StorageContainerManager class

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2498:
---
Labels: pull-request-available sonar  (was: pull-request-available)

> Sonar: Fix issues found in StorageContainerManager class
> 
>
> Key: HDDS-2498
> URL: https://issues.apache.org/jira/browse/HDDS-2498
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: SCM
>Affects Versions: 0.5.0
>Reporter: Siddharth Wagle
>Assignee: Siddharth Wagle
>Priority: Major
>  Labels: pull-request-available, sonar
> Fix For: 0.5.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> https://sonarcloud.io/project/issues?fileUuids=AW5md-HfKcVY8lQ4ZrcG&id=hadoop-ozone&open=AW5md-tIKcVY8lQ4ZsEr&resolved=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2493) Sonar: Locking on a parameter in NetUtils.removeOutscope

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2493:
---
Labels: pull-request-available sonar  (was: pull-request-available)

> Sonar: Locking on a parameter in NetUtils.removeOutscope
> 
>
> Key: HDDS-2493
> URL: https://issues.apache.org/jira/browse/HDDS-2493
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: SCM
>Affects Versions: 0.5.0
>Reporter: Siddharth Wagle
>Assignee: Siddharth Wagle
>Priority: Major
>  Labels: pull-request-available, sonar
> Fix For: 0.5.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> https://sonarcloud.io/project/issues?id=hadoop-ozone&open=AW5md-2hKcVY8lQ4ZsNd&resolved=false&types=BUG



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2495) Sonar - "notify" may not wake up the appropriate thread

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2495:
---
Labels: sonar  (was: )

> Sonar - "notify" may not wake up the appropriate thread
> ---
>
> Key: HDDS-2495
> URL: https://issues.apache.org/jira/browse/HDDS-2495
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Matthew Sharp
>Assignee: Matthew Sharp
>Priority: Minor
>  Labels: sonar
>
> Addresses same issue within ReplicationManager:
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-sVKcVY8lQ4ZsDi&open=AW5md-sVKcVY8lQ4ZsDi]
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-sVKcVY8lQ4ZsDh&open=AW5md-sVKcVY8lQ4ZsDh]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2494) Sonar - BigDecimal(double) should not be used

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2494:
---
Labels: pull-request-available sonar  (was: pull-request-available)

> Sonar - BigDecimal(double) should not be used
> -
>
> Key: HDDS-2494
> URL: https://issues.apache.org/jira/browse/HDDS-2494
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Matthew Sharp
>Assignee: Matthew Sharp
>Priority: Minor
>  Labels: pull-request-available, sonar
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Sonar Issue:  
> [https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-0AKcVY8lQ4ZsKR&open=AW5md-0AKcVY8lQ4ZsKR]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2487) Ensure streams are closed

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2487:
---
Status: Patch Available  (was: In Progress)

> Ensure streams are closed
> -
>
> Key: HDDS-2487
> URL: https://issues.apache.org/jira/browse/HDDS-2487
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: pull-request-available, sonar
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> * ContainerDataYaml: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-6IKcVY8lQ4ZsQU&open=AW5md-6IKcVY8lQ4ZsQU
> * OmUtils: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-hdKcVY8lQ4Zr76&open=AW5md-hdKcVY8lQ4Zr76



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work started] (HDDS-2487) Ensure streams are closed

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HDDS-2487 started by Attila Doroszlai.
--
> Ensure streams are closed
> -
>
> Key: HDDS-2487
> URL: https://issues.apache.org/jira/browse/HDDS-2487
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>  Labels: sonar
>
> * ContainerDataYaml: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-6IKcVY8lQ4ZsQU&open=AW5md-6IKcVY8lQ4ZsQU
> * OmUtils: 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-hdKcVY8lQ4Zr76&open=AW5md-hdKcVY8lQ4Zr76



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2442) Add ServiceName support for for getting Signed Cert.

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2442:
---
Summary: Add ServiceName support for for getting Signed Cert.  (was: Add 
ServiceName support for for getting Singed Cert.)

> Add ServiceName support for for getting Signed Cert.
> 
>
> Key: HDDS-2442
> URL: https://issues.apache.org/jira/browse/HDDS-2442
> Project: Hadoop Distributed Data Store
>  Issue Type: Sub-task
>  Components: SCM
>Reporter: Anu Engineer
>Assignee: Abhishek Purohit
>Priority: Major
>
> We need to add support for adding Service name into the Certificate Signing 
> Request.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2442) Add ServiceName support for getting Signed Cert.

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2442:
---
Summary: Add ServiceName support for getting Signed Cert.  (was: Add 
ServiceName support for for getting Signed Cert.)

> Add ServiceName support for getting Signed Cert.
> 
>
> Key: HDDS-2442
> URL: https://issues.apache.org/jira/browse/HDDS-2442
> Project: Hadoop Distributed Data Store
>  Issue Type: Sub-task
>  Components: SCM
>Reporter: Anu Engineer
>Assignee: Abhishek Purohit
>Priority: Major
>
> We need to add support for adding Service name into the Certificate Signing 
> Request.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2489) Sonar: Anonymous class based initialization in HddsClientUtils

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2489:
---
Labels: sonar  (was: )

> Sonar: Anonymous class based initialization in HddsClientUtils
> --
>
> Key: HDDS-2489
> URL: https://issues.apache.org/jira/browse/HDDS-2489
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: SCM Client
>Affects Versions: 0.5.0
>Reporter: Siddharth Wagle
>Assignee: Siddharth Wagle
>Priority: Major
>  Labels: sonar
> Fix For: 0.5.0
>
>
> https://sonarcloud.io/project/issues?id=hadoop-ozone&open=AW5md_APKcVY8lQ4ZsWN&resolved=false&types=BUG



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2490) Ensure OzoneClient is closed in Ozone Shell handlers

2019-11-14 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2490:
--

 Summary: Ensure OzoneClient is closed in Ozone Shell handlers
 Key: HDDS-2490
 URL: https://issues.apache.org/jira/browse/HDDS-2490
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: Ozone CLI
Reporter: Attila Doroszlai


OzoneClient should be closed in all command handlers ({{Handler}} subclasses).

* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-Y0KcVY8lQ4Zrz6&open=AW5md-Y0KcVY8lQ4Zrz6
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-ZGKcVY8lQ4Zr0b&open=AW5md-ZGKcVY8lQ4Zr0b
etc.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2488) Not enough arguments for log messages in GrpcXceiverService

2019-11-14 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2488:
--

 Summary: Not enough arguments for log messages in 
GrpcXceiverService
 Key: HDDS-2488
 URL: https://issues.apache.org/jira/browse/HDDS-2488
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
Reporter: Attila Doroszlai


GrpcXceiverService has log messages with too few arguments for placeholders.  
Only one of them is flagged by Sonar, but all seem to have the same problem.

https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-69KcVY8lQ4ZsRZ&open=AW5md-69KcVY8lQ4ZsRZ



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2487) Ensure streams are closed

2019-11-14 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2487:
--

 Summary: Ensure streams are closed
 Key: HDDS-2487
 URL: https://issues.apache.org/jira/browse/HDDS-2487
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


* ContainerDataYaml: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-6IKcVY8lQ4ZsQU&open=AW5md-6IKcVY8lQ4ZsQU
* OmUtils: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-hdKcVY8lQ4Zr76&open=AW5md-hdKcVY8lQ4Zr76



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2486) Empty test method in TestRDBTableStore

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2486:
---
Description: 
{{TestRDBTableStore#toIOException}} is empty.

https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-5kKcVY8lQ4ZsQH&open=AW5md-5kKcVY8lQ4ZsQH

Also {{TestTypedRDBTableStore#toIOException}}:

https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-5qKcVY8lQ4ZsQJ&open=AW5md-5qKcVY8lQ4ZsQJ

  was:
{{TestRDBTableStore#toIOException}} is empty.

https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-5kKcVY8lQ4ZsQH&open=AW5md-5kKcVY8lQ4ZsQH


> Empty test method in TestRDBTableStore
> --
>
> Key: HDDS-2486
> URL: https://issues.apache.org/jira/browse/HDDS-2486
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: test
>Reporter: Attila Doroszlai
>Priority: Minor
>  Labels: sonar
>
> {{TestRDBTableStore#toIOException}} is empty.
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-5kKcVY8lQ4ZsQH&open=AW5md-5kKcVY8lQ4ZsQH
> Also {{TestTypedRDBTableStore#toIOException}}:
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-5qKcVY8lQ4ZsQJ&open=AW5md-5qKcVY8lQ4ZsQJ



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2486) Empty test method in TestRDBTableStore

2019-11-14 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2486:
--

 Summary: Empty test method in TestRDBTableStore
 Key: HDDS-2486
 URL: https://issues.apache.org/jira/browse/HDDS-2486
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: test
Reporter: Attila Doroszlai


{{TestRDBTableStore#toIOException}} is empty.

https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-5kKcVY8lQ4ZsQH&open=AW5md-5kKcVY8lQ4ZsQH



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2485) Disable XML external entity processing

2019-11-14 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2485:
--

 Summary: Disable XML external entity processing
 Key: HDDS-2485
 URL: https://issues.apache.org/jira/browse/HDDS-2485
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
Reporter: Attila Doroszlai


Disable XML external entity processing in

* NodeSchemaLoader: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-2nKcVY8lQ4ZsNm&open=AW5md-2nKcVY8lQ4ZsNm
* ConfigFileAppender: 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-_uKcVY8lQ4ZsVY&open=AW5md-_uKcVY8lQ4ZsVY



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2483) Avoid fall-through in HddsUtils#getBlockID

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2483:
---
Description: 
{{switch}} in {{HddsUtils#getBlockID}} has potential fall-through.  It should 
be handled explicitly (eg. throw exception or {{return null}}).

* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPf&open=AW5md-4qKcVY8lQ4ZsPf
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPg&open=AW5md-4qKcVY8lQ4ZsPg
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPh&open=AW5md-4qKcVY8lQ4ZsPh
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPi&open=AW5md-4qKcVY8lQ4ZsPi
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPj&open=AW5md-4qKcVY8lQ4ZsPj

  was:
{{switch}} in {{HddsUtils#getBlockID}} has potential fall-through.  It should 
be handled explicitly (eg. throw exception or {{return null}}).

* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPf&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPg&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPh&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPi&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPj&open=AW5md-4qKcVY8lQ4ZsPe


> Avoid fall-through in HddsUtils#getBlockID
> --
>
> Key: HDDS-2483
> URL: https://issues.apache.org/jira/browse/HDDS-2483
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Attila Doroszlai
>Priority: Minor
>  Labels: sonar
>
> {{switch}} in {{HddsUtils#getBlockID}} has potential fall-through.  It should 
> be handled explicitly (eg. throw exception or {{return null}}).
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPe
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPf&open=AW5md-4qKcVY8lQ4ZsPf
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPg&open=AW5md-4qKcVY8lQ4ZsPg
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPh&open=AW5md-4qKcVY8lQ4ZsPh
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPi&open=AW5md-4qKcVY8lQ4ZsPi
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPj&open=AW5md-4qKcVY8lQ4ZsPj



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2483) Avoid fall-through in HddsUtils#getBlockID

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2483:
---
Description: 
{{switch}} in {{HddsUtils#getBlockID}} has potential fall-through.  It should 
be handled explicitly (eg. throw exception or {{return null}}).

* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPf&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPg&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPh&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPi&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPj&open=AW5md-4qKcVY8lQ4ZsPe

  was:
{{switch}} in {{HddsUtils#getBlockID}} has potential fall-through.  It should 
be handled explicitly (eg. throw exception or {{return null}}).

* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPf
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPg
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPh
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPi
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPj&open=AW5md-4qKcVY8lQ4ZsPj


> Avoid fall-through in HddsUtils#getBlockID
> --
>
> Key: HDDS-2483
> URL: https://issues.apache.org/jira/browse/HDDS-2483
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Attila Doroszlai
>Priority: Minor
>  Labels: sonar
>
> {{switch}} in {{HddsUtils#getBlockID}} has potential fall-through.  It should 
> be handled explicitly (eg. throw exception or {{return null}}).
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPe
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPf&open=AW5md-4qKcVY8lQ4ZsPe
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPg&open=AW5md-4qKcVY8lQ4ZsPe
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPh&open=AW5md-4qKcVY8lQ4ZsPe
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPi&open=AW5md-4qKcVY8lQ4ZsPe
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPj&open=AW5md-4qKcVY8lQ4ZsPe



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2483) Avoid fall-through in HddsUtils#getBlockID

2019-11-14 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2483:
--

 Summary: Avoid fall-through in HddsUtils#getBlockID
 Key: HDDS-2483
 URL: https://issues.apache.org/jira/browse/HDDS-2483
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
Reporter: Attila Doroszlai


{{switch}} in {{HddsUtils#getBlockID}} has potential fall-through.  It should 
be handled explicitly (eg. throw exception or {{return null}}).

* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPe
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPf
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPg
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPh
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPe&open=AW5md-4qKcVY8lQ4ZsPi
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-4qKcVY8lQ4ZsPj&open=AW5md-4qKcVY8lQ4ZsPj



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2481) Close streams in TarContainerPacker

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2481:
---
Status: Patch Available  (was: Open)

> Close streams in TarContainerPacker
> ---
>
> Key: HDDS-2481
> URL: https://issues.apache.org/jira/browse/HDDS-2481
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Ozone Datanode
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
>  Labels: pull-request-available, sonar
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ensure various streams are closed in {{TarContainerPacker}}:
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-9bKcVY8lQ4ZsUH&open=AW5md-9bKcVY8lQ4ZsUH
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-9bKcVY8lQ4ZsUL&open=AW5md-9bKcVY8lQ4ZsUL
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-9bKcVY8lQ4ZsUK&open=AW5md-9bKcVY8lQ4ZsUK
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-9bKcVY8lQ4ZsUJ&open=AW5md-9bKcVY8lQ4ZsUJ
> * 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-9bKcVY8lQ4ZsUI&open=AW5md-9bKcVY8lQ4ZsUI



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2473) Fix code reliability issues found by Sonar in Ozone Recon module.

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2473:
---
Labels: pull-request-available sonar  (was: pull-request-available)

> Fix code reliability issues found by Sonar in Ozone Recon module.
> -
>
> Key: HDDS-2473
> URL: https://issues.apache.org/jira/browse/HDDS-2473
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Ozone Recon
>Affects Versions: 0.5.0
>Reporter: Aravindan Vijayan
>Assignee: Aravindan Vijayan
>Priority: Major
>  Labels: pull-request-available, sonar
> Fix For: 0.5.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> sonarcloud.io has flagged a number of code reliability issues in Ozone recon 
> (https://sonarcloud.io/code?id=hadoop-ozone&selected=hadoop-ozone%3Ahadoop-ozone%2Frecon%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fozone%2Frecon).
> Following issues will be triaged / fixed.
> * Double Brace Initialization should not be used
> * Resources should be closed
> * InterruptedException should not be ignored



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2479) Sonar : replace instanceof with catch block in XceiverClientGrpc.sendCommandWithRetry

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2479:
---
Labels: sonar  (was: )

> Sonar : replace instanceof with catch block in 
> XceiverClientGrpc.sendCommandWithRetry
> -
>
> Key: HDDS-2479
> URL: https://issues.apache.org/jira/browse/HDDS-2479
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Supratim Deka
>Assignee: Supratim Deka
>Priority: Minor
>  Labels: sonar
>
> Sonar issue:
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md_AGKcVY8lQ4ZsV_&open=AW5md_AGKcVY8lQ4ZsV_



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2480) Sonar : remove log spam for exceptions inside XceiverClientGrpc.reconnect

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2480:
---
Labels: sonar  (was: )

> Sonar : remove log spam for exceptions inside XceiverClientGrpc.reconnect
> -
>
> Key: HDDS-2480
> URL: https://issues.apache.org/jira/browse/HDDS-2480
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Supratim Deka
>Assignee: Supratim Deka
>Priority: Minor
>  Labels: sonar
>
> Sonar issue:
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md_AGKcVY8lQ4ZsWE&open=AW5md_AGKcVY8lQ4ZsWE



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2478) Sonar : remove temporary variable in XceiverClientGrpc.sendCommand

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2478:
---
Labels: pull-request-available sonar  (was: pull-request-available)

> Sonar : remove temporary variable in XceiverClientGrpc.sendCommand
> --
>
> Key: HDDS-2478
> URL: https://issues.apache.org/jira/browse/HDDS-2478
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Supratim Deka
>Assignee: Supratim Deka
>Priority: Minor
>  Labels: pull-request-available, sonar
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Sonar issues :
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md_AGKcVY8lQ4ZsV1&open=AW5md_AGKcVY8lQ4ZsV1
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md_AGKcVY8lQ4ZsV2&open=AW5md_AGKcVY8lQ4ZsV2



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDDS-2472) Use try-with-resources while creating FlushOptions in RDBStore.

2019-11-14 Thread Attila Doroszlai (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Doroszlai updated HDDS-2472:
---
Labels: pull-request-available sonar  (was: pull-request-available)

> Use try-with-resources while creating FlushOptions in RDBStore.
> ---
>
> Key: HDDS-2472
> URL: https://issues.apache.org/jira/browse/HDDS-2472
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Ozone Manager
>Affects Versions: 0.5.0
>Reporter: Aravindan Vijayan
>Assignee: Aravindan Vijayan
>Priority: Major
>  Labels: pull-request-available, sonar
> Fix For: 0.5.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Link to the sonar issue flag - 
> https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-zwKcVY8lQ4ZsJ4&open=AW5md-zwKcVY8lQ4ZsJ4.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2481) Close streams in TarContainerPacker

2019-11-14 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2481:
--

 Summary: Close streams in TarContainerPacker
 Key: HDDS-2481
 URL: https://issues.apache.org/jira/browse/HDDS-2481
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: Ozone Datanode
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


Ensure various streams are closed in {{TarContainerPacker}}:

* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-9bKcVY8lQ4ZsUH&open=AW5md-9bKcVY8lQ4ZsUH
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-9bKcVY8lQ4ZsUL&open=AW5md-9bKcVY8lQ4ZsUL
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-9bKcVY8lQ4ZsUK&open=AW5md-9bKcVY8lQ4ZsUK
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-9bKcVY8lQ4ZsUJ&open=AW5md-9bKcVY8lQ4ZsUJ
* 
https://sonarcloud.io/project/issues?id=hadoop-ozone&issues=AW5md-9bKcVY8lQ4ZsUI&open=AW5md-9bKcVY8lQ4ZsUI



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Created] (HDDS-2476) Share more code between metadata and data scanners

2019-11-13 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-2476:
--

 Summary: Share more code between metadata and data scanners
 Key: HDDS-2476
 URL: https://issues.apache.org/jira/browse/HDDS-2476
 Project: Hadoop Distributed Data Store
  Issue Type: Sub-task
  Components: Ozone Datanode
Reporter: Attila Doroszlai


There are several duplicated / similar pieces of code in metadata and data 
scanners.  More code should be reused.

Examples:

# ContainerDataScrubberMetrics and ContainerMetadataScrubberMetrics have 3 
common metrics
# lifecycle of ContainerMetadataScanner and ContainerDataScanner (main loop, 
iteration, metrics processing, shutdown)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



  1   2   3   >