[jira] [Commented] (HDDS-12998) Bring real container size in pb message when exporting/importing containers
[ https://issues.apache.org/jira/browse/HDDS-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18010864#comment-18010864 ] Sammi Chen commented on HDDS-12998: --- [~siddhant], do you have plan to work on this one? > Bring real container size in pb message when exporting/importing containers > --- > > Key: HDDS-12998 > URL: https://issues.apache.org/jira/browse/HDDS-12998 > Project: Apache Ozone > Issue Type: Sub-task > Components: Ozone Datanode >Reporter: Chu Cheng Li >Assignee: Chu Cheng Li >Priority: Major > > Generally, the tar zip file of container will have smaller size than it's > real size, so reserve a 2* max container size is conservative enough in most > cases. But there are container over-allocated case, where container size can > be double or triple the max container size as > [@siddhantsangwan|https://github.com/siddhantsangwan] saw in user's > environment, have a accurate container size can handle this case welly, and > that info could be brought in [proto > message|https://github.com/apache/ozone/blob/f52069b7aa757d39e4819bae7d06b12955660576/hadoop-hdds/interface-client/src/main/proto/DatanodeClientProtocol.proto#L529-L545]. > > For backward compatibility, if that field unset, fallback to get the > container size from config. > > comments: > * [https://github.com/apache/ozone/pull/8360#issuecomment-2850023149] > * [https://github.com/apache/ozone/pull/8360#issuecomment-2857154196] > * [https://github.com/apache/ozone/pull/8360#issuecomment-2857213750] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (HDDS-12998) Bring real container size in pb message when exporting/importing containers
[ https://issues.apache.org/jira/browse/HDDS-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17986634#comment-17986634 ] Chu Cheng Li commented on HDDS-12998: - Oh sure, please~ Please also request my review whenever the open the PR, thanks! > Bring real container size in pb message when exporting/importing containers > --- > > Key: HDDS-12998 > URL: https://issues.apache.org/jira/browse/HDDS-12998 > Project: Apache Ozone > Issue Type: Sub-task > Components: Ozone Datanode >Reporter: Chu Cheng Li >Assignee: Chu Cheng Li >Priority: Major > > Generally, the tar zip file of container will have smaller size than it's > real size, so reserve a 2* max container size is conservative enough in most > cases. But there are container over-allocated case, where container size can > be double or triple the max container size as > [@siddhantsangwan|https://github.com/siddhantsangwan] saw in user's > environment, have a accurate container size can handle this case welly, and > that info could be brought in [proto > message|https://github.com/apache/ozone/blob/f52069b7aa757d39e4819bae7d06b12955660576/hadoop-hdds/interface-client/src/main/proto/DatanodeClientProtocol.proto#L529-L545]. > > For backward compatibility, if that field unset, fallback to get the > container size from config. > > comments: > * [https://github.com/apache/ozone/pull/8360#issuecomment-2850023149] > * [https://github.com/apache/ozone/pull/8360#issuecomment-2857154196] > * [https://github.com/apache/ozone/pull/8360#issuecomment-2857213750] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (HDDS-12998) Bring real container size in pb message when exporting/importing containers
[ https://issues.apache.org/jira/browse/HDDS-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17986500#comment-17986500 ] Siddhant Sangwan commented on HDDS-12998: - If not, I can take it up. > Bring real container size in pb message when exporting/importing containers > --- > > Key: HDDS-12998 > URL: https://issues.apache.org/jira/browse/HDDS-12998 > Project: Apache Ozone > Issue Type: Sub-task > Components: Ozone Datanode >Reporter: Chu Cheng Li >Assignee: Chu Cheng Li >Priority: Major > > Generally, the tar zip file of container will have smaller size than it's > real size, so reserve a 2* max container size is conservative enough in most > cases. But there are container over-allocated case, where container size can > be double or triple the max container size as > [@siddhantsangwan|https://github.com/siddhantsangwan] saw in user's > environment, have a accurate container size can handle this case welly, and > that info could be brought in [proto > message|https://github.com/apache/ozone/blob/f52069b7aa757d39e4819bae7d06b12955660576/hadoop-hdds/interface-client/src/main/proto/DatanodeClientProtocol.proto#L529-L545]. > > For backward compatibility, if that field unset, fallback to get the > container size from config. > > comments: > * [https://github.com/apache/ozone/pull/8360#issuecomment-2850023149] > * [https://github.com/apache/ozone/pull/8360#issuecomment-2857154196] > * [https://github.com/apache/ozone/pull/8360#issuecomment-2857213750] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (HDDS-12998) Bring real container size in pb message when exporting/importing containers
[ https://issues.apache.org/jira/browse/HDDS-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17986493#comment-17986493 ] Siddhant Sangwan commented on HDDS-12998: - Hi [~peterxcli], are you planning to work on this? > Bring real container size in pb message when exporting/importing containers > --- > > Key: HDDS-12998 > URL: https://issues.apache.org/jira/browse/HDDS-12998 > Project: Apache Ozone > Issue Type: Sub-task > Components: Ozone Datanode >Reporter: Chu Cheng Li >Assignee: Chu Cheng Li >Priority: Major > > Generally, the tar zip file of container will have smaller size than it's > real size, so reserve a 2* max container size is conservative enough in most > cases. But there are container over-allocated case, where container size can > be double or triple the max container size as > [@siddhantsangwan|https://github.com/siddhantsangwan] saw in user's > environment, have a accurate container size can handle this case welly, and > that info could be brought in [proto > message|https://github.com/apache/ozone/blob/f52069b7aa757d39e4819bae7d06b12955660576/hadoop-hdds/interface-client/src/main/proto/DatanodeClientProtocol.proto#L529-L545]. > > For backward compatibility, if that field unset, fallback to get the > container size from config. > > comments: > * [https://github.com/apache/ozone/pull/8360#issuecomment-2850023149] > * [https://github.com/apache/ozone/pull/8360#issuecomment-2857154196] > * [https://github.com/apache/ozone/pull/8360#issuecomment-2857213750] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
