[
https://issues.apache.org/jira/browse/HDDS-2405?focusedWorklogId=344625&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-344625
]
ASF GitHub Bot logged work on HDDS-2405:
----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Nov/19 22:30
Start Date: 15/Nov/19 22:30
Worklog Time Spent: 10m
Work Description: adoroszlai commented on pull request #197: HDDS-2405.
int2ByteString unnecessary byte array allocation
URL: https://github.com/apache/hadoop-ozone/pull/197
## What changes were proposed in this pull request?
Simplify `int` to `ByteString` conversion. Avoid allocating ~7x memory
overhead (vs. the size of the `ByteString` created).
https://issues.apache.org/jira/browse/HDDS-2405
## How was this patch tested?
Tested memory usage of various methods:
1. previous implementation
2. using `ByteString.Output` with restricted initial capacity (4 bytes
instead of default 128)
3. eliminating `DataOutputStream` by writing directly to `ByteString.Output`
4. proposed implementation (simple `copyFrom`)
The last one uses the least amount of memory.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 344625)
Remaining Estimate: 0h
Time Spent: 10m
> int2ByteString unnecessary byte array allocation
> ------------------------------------------------
>
> Key: HDDS-2405
> URL: https://issues.apache.org/jira/browse/HDDS-2405
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Attila Doroszlai
> Assignee: Attila Doroszlai
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{int2ByteString}} implementations (currently duplicated in
> [RatisHelper|https://github.com/apache/hadoop-ozone/blob/6b2cda125b3647870ef5b01cf64e3b3e4cdc55db/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java#L280-L289]
> and
> [Checksum|https://github.com/apache/hadoop-ozone/blob/6b2cda125b3647870ef5b01cf64e3b3e4cdc55db/hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/Checksum.java#L64-L73],
> but the first one is being removed in HDDS-2375) result in unnecessary byte
> array allocations:
> # {{ByteString.Output}} creates 128-byte buffer by default, which is too
> large for writing a single int
> # {{DataOutputStream}} allocates an [extra 8-byte
> array|https://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/io/DataOutputStream.java#l204],
> used only for writing longs
> # {{ByteString.Output}} also creates 10-element array for {{flushedBuffers}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
