[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[ https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455066#comment-16455066 ] Hudson commented on HDFS-11769: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14070 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14070/]) HDFS-11769. Ozone: KSM: Add createVolume API. Contributed by Mukul Kumar (omalley: rev c9e6d4378decfdde9bb8978ca9efc8e8003b43e2) * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/helpers/KsmVolumeArgs.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/VolumeManager.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/exceptions/KSMException.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/VolumeManagerImpl.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KSMConfigKeys.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/web/TestLocalOzoneVolumes.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KeySpaceManager.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/exceptions/package-info.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/KeySpaceManagerProtocol.proto * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/storage/DistributedStorageHandler.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocol/KeyspaceManagerProtocol.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/ObjectStoreHandler.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/utils/LevelDBStore.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocolPB/KeySpaceManagerProtocolClientSideTranslatorPB.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/protocolPB/KeyspaceManagerProtocolServerSideTranslatorPB.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/web/TestOzoneHelper.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/MiniOzoneCluster.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/web/TestDistributedOzoneVolumes.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/utils/OzoneUtils.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/web/TestOzoneVolumes.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KSMMetrics.java > Ozone: KSM: Add createVolume API > > > Key: HDFS-11769 > URL: https://issues.apache.org/jira/browse/HDFS-11769 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Mukul Kumar Singh >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-11769-HDFS-7240.001.patch, > HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch, > HDFS-11769-HDFS-7240.004.patch, HDFS-11769-HDFS-7240.005.patch, > HDFS-11769-HDFS-7240.006.patch, HDFS-11769-HDFS-7240.007.patch, > HDFS-11769-HDFS-7240.008.patch > > > Add createVolume API in KSM. > createVolume API allows administrators to create a volume. The arguments to > the API are: > * Admin Name - The name of the administrator who is creating this volume. > Volumes can be created only by administrators. > * User Name - The name of the owner of this volume. > * Quota - Quota information for this volume. > This JIRA proposes to add the protobuf layer for createVolume and handling in > KSM. We will file a followup JIRA for the KSM client. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[ https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450657#comment-16450657 ] Hudson commented on HDFS-11769: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14057 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14057/]) HDFS-11769. Ozone: KSM: Add createVolume API. Contributed by Mukul Kumar (aengineer: rev a8393b4b875f8e69d5244ac1ce1f1a5abd474525) * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/web/TestDistributedOzoneVolumes.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/utils/OzoneUtils.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocolPB/KeySpaceManagerProtocolClientSideTranslatorPB.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KSMMetrics.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/utils/LevelDBStore.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KeySpaceManager.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/VolumeManagerImpl.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/KeySpaceManagerProtocol.proto * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/exceptions/package-info.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/web/TestOzoneHelper.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/protocolPB/KeyspaceManagerProtocolServerSideTranslatorPB.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/ObjectStoreHandler.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/helpers/KsmVolumeArgs.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/web/TestOzoneVolumes.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KSMConfigKeys.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/exceptions/KSMException.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/MiniOzoneCluster.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/web/TestLocalOzoneVolumes.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/VolumeManager.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/storage/DistributedStorageHandler.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocol/KeyspaceManagerProtocol.java > Ozone: KSM: Add createVolume API > > > Key: HDFS-11769 > URL: https://issues.apache.org/jira/browse/HDFS-11769 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Mukul Kumar Singh >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-11769-HDFS-7240.001.patch, > HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch, > HDFS-11769-HDFS-7240.004.patch, HDFS-11769-HDFS-7240.005.patch, > HDFS-11769-HDFS-7240.006.patch, HDFS-11769-HDFS-7240.007.patch, > HDFS-11769-HDFS-7240.008.patch > > > Add createVolume API in KSM. > createVolume API allows administrators to create a volume. The arguments to > the API are: > * Admin Name - The name of the administrator who is creating this volume. > Volumes can be created only by administrators. > * User Name - The name of the owner of this volume. > * Quota - Quota information for this volume. > This JIRA proposes to add the protobuf layer for createVolume and handling in > KSM. We will file a followup JIRA for the KSM client. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16014518#comment-16014518
]
Xiaoyu Yao commented on HDFS-11769:
---
Just to clarify the ksm.db schema based on discussion on HDFS-11769 and
HDFS-11775.
{code}
/{VolumeName} -> {VolumeInfo}
/{VolumeName}/{BucketName} -> {BucketInfo}
/{VolumeName}/{BucketName}/{KeyName} -> {KeyInfo}
${UserName} ->
{code}
The ${UserName} already exists with HDFS-11769. "/" prefix will be added in
HDFS-11770. The pro of having a "/" is that the Volume name does not have to
check for userName prefix $. The cons of having "/" prefix is the additional
storage cost, which is relative low.
> Ozone: KSM: Add createVolume API
>
>
> Key: HDFS-11769
> URL: https://issues.apache.org/jira/browse/HDFS-11769
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Mukul Kumar Singh
> Fix For: HDFS-7240
>
> Attachments: HDFS-11769-HDFS-7240.001.patch,
> HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch,
> HDFS-11769-HDFS-7240.004.patch, HDFS-11769-HDFS-7240.005.patch,
> HDFS-11769-HDFS-7240.006.patch, HDFS-11769-HDFS-7240.007.patch,
> HDFS-11769-HDFS-7240.008.patch
>
>
> Add createVolume API in KSM.
> createVolume API allows administrators to create a volume. The arguments to
> the API are:
> * Admin Name - The name of the administrator who is creating this volume.
> Volumes can be created only by administrators.
> * User Name - The name of the owner of this volume.
> * Quota - Quota information for this volume.
> This JIRA proposes to add the protobuf layer for createVolume and handling in
> KSM. We will file a followup JIRA for the KSM client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16013385#comment-16013385
]
Xiaoyu Yao commented on HDFS-11769:
---
Good question, [~vagarychen]. I agree that we will need prefix to differentiate
the entry types. We currently have the following schema:
{code}
Volume -> VolumeInfo
Volume/Bucket -> BucketInfo
Volume/Bucket/Key -> KeyInfo
User->VolumeList
{code}
How about changing to the following with Volumes/Buckets/Keys/Users as prefix
for different types?
{code}
/Volumes/{VolumeName} -> {VolumeInfo}
/Buckets/{VolumeName}/{BucketName} -> {BucketInfo}
/Keys/{VolumeName}/{BucketName}/{KeyName} -> {KeyInfo}
/Users/{UserName} ->
{code}
> Ozone: KSM: Add createVolume API
>
>
> Key: HDFS-11769
> URL: https://issues.apache.org/jira/browse/HDFS-11769
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Mukul Kumar Singh
> Fix For: HDFS-7240
>
> Attachments: HDFS-11769-HDFS-7240.001.patch,
> HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch,
> HDFS-11769-HDFS-7240.004.patch, HDFS-11769-HDFS-7240.005.patch,
> HDFS-11769-HDFS-7240.006.patch, HDFS-11769-HDFS-7240.007.patch,
> HDFS-11769-HDFS-7240.008.patch
>
>
> Add createVolume API in KSM.
> createVolume API allows administrators to create a volume. The arguments to
> the API are:
> * Admin Name - The name of the administrator who is creating this volume.
> Volumes can be created only by administrators.
> * User Name - The name of the owner of this volume.
> * Quota - Quota information for this volume.
> This JIRA proposes to add the protobuf layer for createVolume and handling in
> KSM. We will file a followup JIRA for the KSM client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16013279#comment-16013279
]
Chen Liang commented on HDFS-11769:
---
I just got a question about the usage of ksm.db, did not realize this earlier,
also please correct me if I'm wrong on any statement below.
when a volume gets created, there is
{code}
batch.put(DFSUtil.string2Bytes(args.getVolume()), volumeInfo.toByteArray());
{code}
and
{code}
batch.put(DFSUtil.string2Bytes(dbUserName), newVolList.toByteArray());
{code}
so there are two types of key-value information pushed/updated in ksm.db on
each volume creation:
{{volumeName -> volumeInfo}}
{{userName -> volumeListOfThisUser}}
So my concern about this is that, if we try to iterate over ksm.db (say, for
debug purpose), we may have a hard time figuring out what type of information
the current entry is...e.g., say we get an entry from iterator, is this entry a
"volumeName -> volumeInfo" entry? or is it "userName -> volumeListOfThisUser"?
Seems no way to tell, except for something like blindly trying to parse it and
catch the error...I ran into this when try to get the DEBUG CLI work for
ksm.db. In addition, if we allow the odd cases where a user name can be the
same as an existing volume name (or vice versa), either we will overwrite
things that we should not, or we will have to reject the insertion.
I checked the design doc in HDFS-11768, seems to me that maintaining this one
single ksm.db instance is actually by design. In which case this is not just
about this JIRA but about everything in KSM, and we will have more than two
types of information in ksm.db in the future. So I wonder, should we have more
db instance based on type of entries? or is it better to have some way to be
able to distinguish the entries in the DB?
I think one easy way to be able distinguish the entry types, is to add a prefix
to key for each type of entries, assuming keys are all strings...e.g. for
"volumeName - volumeInfo" type entry, we actually insert the entry
"VOLUMENAME_volumeName - volumeInfo" i.e. adding prefix "VOLUMENAME_" to volume
name. (It doesn't have to be as long as "VOLUMENAME", can be as short as one
single character). This way when iterating the db, we will know this is a
volume info entry.
Any comments? cc. [~xyao], [~msingh], [~anu], [~cheersyang]
> Ozone: KSM: Add createVolume API
>
>
> Key: HDFS-11769
> URL: https://issues.apache.org/jira/browse/HDFS-11769
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Mukul Kumar Singh
> Fix For: HDFS-7240
>
> Attachments: HDFS-11769-HDFS-7240.001.patch,
> HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch,
> HDFS-11769-HDFS-7240.004.patch, HDFS-11769-HDFS-7240.005.patch,
> HDFS-11769-HDFS-7240.006.patch, HDFS-11769-HDFS-7240.007.patch,
> HDFS-11769-HDFS-7240.008.patch
>
>
> Add createVolume API in KSM.
> createVolume API allows administrators to create a volume. The arguments to
> the API are:
> * Admin Name - The name of the administrator who is creating this volume.
> Volumes can be created only by administrators.
> * User Name - The name of the owner of this volume.
> * Quota - Quota information for this volume.
> This JIRA proposes to add the protobuf layer for createVolume and handling in
> KSM. We will file a followup JIRA for the KSM client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011471#comment-16011471
]
Hadoop QA commented on HDFS-11769:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
0s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
37s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s{color} | {color:green} HDFS-7240 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
39s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in HDFS-7240
has 2 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
55s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in HDFS-7240 has 10
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
46s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 43s{color}
| {color:red} hadoop-hdfs-project generated 5 new + 109 unchanged - 25 fixed =
114 total (was 134) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 44s{color} | {color:orange} hadoop-hdfs-project: The patch generated 4 new +
0 unchanged - 5 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
14s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 75m 5s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
23s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}118m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSUpgradeFromImage |
| | hadoop.hdfs.server.balancer.TestBalancer |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090 |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11769 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868154/HDFS-11769-HDFS-7240.008.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux a8ffb216ab41 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UT
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[ https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011395#comment-16011395 ] Anu Engineer commented on HDFS-11769: - +1, pending jenkins. I will commit this soon. There is a small typo that I will fix while committing. > Ozone: KSM: Add createVolume API > > > Key: HDFS-11769 > URL: https://issues.apache.org/jira/browse/HDFS-11769 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Mukul Kumar Singh > Attachments: HDFS-11769-HDFS-7240.001.patch, > HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch, > HDFS-11769-HDFS-7240.004.patch, HDFS-11769-HDFS-7240.005.patch, > HDFS-11769-HDFS-7240.006.patch, HDFS-11769-HDFS-7240.007.patch, > HDFS-11769-HDFS-7240.008.patch > > > Add createVolume API in KSM. > createVolume API allows administrators to create a volume. The arguments to > the API are: > * Admin Name - The name of the administrator who is creating this volume. > Volumes can be created only by administrators. > * User Name - The name of the owner of this volume. > * Quota - Quota information for this volume. > This JIRA proposes to add the protobuf layer for createVolume and handling in > KSM. We will file a followup JIRA for the KSM client. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011020#comment-16011020
]
Anu Engineer commented on HDFS-11769:
-
Excellent work, [~msingh] It looks very good. Thank you very much for getting
this done.
I am also a +1 after the following nits are fixed.
# DistributedStorageHandler.java: replace :
{{.setOwnerName(args.getAdminName())}} ==>
{{.setOwnerName(args.getOwnerName())}}
# KeySpaceManager.java#getClientRpcAddress -- Please remove this function if it
is not used. If we are using it or planning to use it, please fix the comments
and the name of this function.
# KeySpaceManager.java#stop --
{{LOG.info("Volume Manager stop failed.", e);}} ==> {{LOG.info("Key Space
Manager stop failed.", e);}}
# KeySpaceManager.proto: nit: typo:
{{INTERAL_ERROR}} ==> {{INTERNAL_ERROR}}
# KeySpaceManagerProtocolClientSideTranslatorPB.java: nit : typo :
{{implement}} --> {{implementation}}
# MiniOzoneCluster.java#shutdown :
if(ksm != null) check before we call ksm.stop();
# MiniOzoneCluster.java#createKeySpaceManagerClient -- Looks like we are not
using this. if not needed you can remove or if this is needed in future
patches, you might want to consider fixing the comments. The current comments
refer to SCM instead of KSM.
> Ozone: KSM: Add createVolume API
>
>
> Key: HDFS-11769
> URL: https://issues.apache.org/jira/browse/HDFS-11769
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11769-HDFS-7240.001.patch,
> HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch,
> HDFS-11769-HDFS-7240.004.patch, HDFS-11769-HDFS-7240.005.patch,
> HDFS-11769-HDFS-7240.006.patch, HDFS-11769-HDFS-7240.007.patch
>
>
> Add createVolume API in KSM.
> createVolume API allows administrators to create a volume. The arguments to
> the API are:
> * Admin Name - The name of the administrator who is creating this volume.
> Volumes can be created only by administrators.
> * User Name - The name of the owner of this volume.
> * Quota - Quota information for this volume.
> This JIRA proposes to add the protobuf layer for createVolume and handling in
> KSM. We will file a followup JIRA for the KSM client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010997#comment-16010997
]
Xiaoyu Yao commented on HDFS-11769:
---
Thanks [~msingh] for the update. The patch looks excellent to me. Just a few
more NITs, +1 after that being fixed.
KeySpaceManager.java
Line 63: this should be {{ private final VolumeManager volumeManager;}}
MiniOzoneCluster.java
Line 141-143 should be moved before line 137. We also need to check (ksm !=
null) just in case.
A few more checkstyle issues that seems fixable.
> Ozone: KSM: Add createVolume API
>
>
> Key: HDFS-11769
> URL: https://issues.apache.org/jira/browse/HDFS-11769
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11769-HDFS-7240.001.patch,
> HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch,
> HDFS-11769-HDFS-7240.004.patch, HDFS-11769-HDFS-7240.005.patch,
> HDFS-11769-HDFS-7240.006.patch, HDFS-11769-HDFS-7240.007.patch
>
>
> Add createVolume API in KSM.
> createVolume API allows administrators to create a volume. The arguments to
> the API are:
> * Admin Name - The name of the administrator who is creating this volume.
> Volumes can be created only by administrators.
> * User Name - The name of the owner of this volume.
> * Quota - Quota information for this volume.
> This JIRA proposes to add the protobuf layer for createVolume and handling in
> KSM. We will file a followup JIRA for the KSM client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010373#comment-16010373
]
Hadoop QA commented on HDFS-11769:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
47s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
35s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
39s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s{color} | {color:green} HDFS-7240 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
41s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in HDFS-7240
has 2 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
59s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in HDFS-7240 has 10
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 30s{color}
| {color:red} hadoop-hdfs-project generated 5 new + 109 unchanged - 25 fixed =
114 total (was 134) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 44s{color} | {color:orange} hadoop-hdfs-project: The patch generated 6 new +
0 unchanged - 5 fixed = 6 total (was 5) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
18s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 10s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}129m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSUpgradeFromImage |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.cblock.TestCBlockServerPersistence |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.TestFileCorruption |
| | hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11769 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868038/HDFS-11769-HDFS-7240.007.patch
|
| Optional Te
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009819#comment-16009819
]
Hadoop QA commented on HDFS-11769:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
10s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
54s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s{color} | {color:green} HDFS-7240 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
43s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in HDFS-7240
has 2 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
4s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in HDFS-7240 has 10
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 53s{color}
| {color:red} hadoop-hdfs-project generated 5 new + 109 unchanged - 25 fixed =
114 total (was 134) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 43s{color} | {color:orange} hadoop-hdfs-project: The patch generated 6 new +
0 unchanged - 5 fixed = 6 total (was 5) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
23s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 7s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}111m 30s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 |
| | hadoop.hdfs.TestDFSUpgradeFromImage |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11769 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867991/HDFS-11769-HDFS-7240.006.patch
|
| Optional Tests | asflicense compile
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[ https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009780#comment-16009780 ] Mukul Kumar Singh commented on HDFS-11769: -- TestQuotaByStorageType.java This change is not related. We can fix the usage of this deprecated APIs outside ozone in a separate Apache JIRA on trunk. ---> done, reverted LevelDBStore.java Line 174: can we use the flyweight pattern for the sync option used by put/commit to avoid creating small objects per write? This can be moved to a class member that instantiate only once and used for all the writes. this.syncOption = new WriteOptions(); syncOption.sync(true); ---> done VolumeManager.java VolumeManager interface is to improve the testability of individual components (Volume/Bucket/Key) without depending on the actual implementation, e.g., test mocked KeyManager with a KeyManagerImpl using mocked VolumeManager/BuckManager that simply assumes the Volume/Bucket exists. > done, added a new interface for VolumeManager MiniOzoneCluster.java We need to ensure KSM is shutdown in MiniOzoneCluster#shutdown() to avoid leaking. ---> done TestDistributedOzoneVolumes.java Line 57: I see we are using "distributed" for ozone handler type in line 74. Should we update the comment to differentiate TestLocalOzoneVolumes.java? -> done Line 120: assert the getNumVolumeCreateFails=1 > this test hasn't been enabled, I will enable these tests while implementing other APIs are done,. TestLocalOzoneVolumes.java Line 52-54: Are these the default configurations of MiniOzone cluster. If yes, we can remove this comments as they are not being changed by the tests. ---> no these are not the default configurations, local option is set using "OZONE_HANDLER_LOCAL" in line 72 > Ozone: KSM: Add createVolume API > > > Key: HDFS-11769 > URL: https://issues.apache.org/jira/browse/HDFS-11769 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Mukul Kumar Singh > Attachments: HDFS-11769-HDFS-7240.001.patch, > HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch, > HDFS-11769-HDFS-7240.004.patch, HDFS-11769-HDFS-7240.005.patch > > > Add createVolume API in KSM. > createVolume API allows administrators to create a volume. The arguments to > the API are: > * Admin Name - The name of the administrator who is creating this volume. > Volumes can be created only by administrators. > * User Name - The name of the owner of this volume. > * Quota - Quota information for this volume. > This JIRA proposes to add the protobuf layer for createVolume and handling in > KSM. We will file a followup JIRA for the KSM client. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009756#comment-16009756
]
Xiaoyu Yao commented on HDFS-11769:
---
Thanks [~msingh] for updating the patch. The latest patch looks pretty good to
me. Just a few more minor issues:
TestQuotaByStorageType.java
This change is not related. We can fix the usage of this deprecated APIs
outside ozone in a separate Apache JIRA on trunk.
LevelDBStore.java
Line 174: can we use the flyweight pattern for the sync option used by
put/commit to avoid creating small objects per write?
This can be moved to a class member that instantiate only once and used for all
the writes.
{code}
this.syncOption = new WriteOptions();
syncOption.sync(true);
{code}
VolumeManager.java
VolumeManager interface is to improve the testability of individual components
(Volume/Bucket/Key) without
depending on the actual implementation, e.g., test mocked KeyManager with a
KeyManagerImpl using mocked VolumeManager/BuckManager that simply assumes the
Volume/Bucket exists.
MiniOzoneCluster.java
We need to ensure KSM is shutdown in MiniOzoneCluster#shutdown() to avoid
leaking.
TestDistributedOzoneVolumes.java
Line 57: I see we are using "distributed" for ozone handler type in line 74.
Should we update the comment to differentiate
TestLocalOzoneVolumes.java?
Line 120: assert the getNumVolumeCreateFails=1
TestLocalOzoneVolumes.java
Line 52-54: Are these the default configurations of MiniOzone cluster. If yes,
we can remove this comments as they are not being
changed by the tests.
> Ozone: KSM: Add createVolume API
>
>
> Key: HDFS-11769
> URL: https://issues.apache.org/jira/browse/HDFS-11769
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11769-HDFS-7240.001.patch,
> HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch,
> HDFS-11769-HDFS-7240.004.patch, HDFS-11769-HDFS-7240.005.patch
>
>
> Add createVolume API in KSM.
> createVolume API allows administrators to create a volume. The arguments to
> the API are:
> * Admin Name - The name of the administrator who is creating this volume.
> Volumes can be created only by administrators.
> * User Name - The name of the owner of this volume.
> * Quota - Quota information for this volume.
> This JIRA proposes to add the protobuf layer for createVolume and handling in
> KSM. We will file a followup JIRA for the KSM client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009714#comment-16009714
]
Hadoop QA commented on HDFS-11769:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
45s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
33s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s{color} | {color:green} HDFS-7240 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
33s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in HDFS-7240
has 2 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
53s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in HDFS-7240 has 10
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 29s{color}
| {color:red} hadoop-hdfs-project generated 3 new + 109 unchanged - 25 fixed =
112 total (was 134) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 40s{color} | {color:orange} hadoop-hdfs-project: The patch generated 7 new +
47 unchanged - 5 fixed = 54 total (was 52) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
14s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 44s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
21s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}106m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSUpgradeFromImage |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.server.balancer.TestBalancer |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11769 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867983/HDFS-11769-HDFS-7240.005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux 1d8271
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009657#comment-16009657
]
Hadoop QA commented on HDFS-11769:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
36s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
41s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
45s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s{color} | {color:green} HDFS-7240 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
48s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in HDFS-7240
has 2 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
3s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in HDFS-7240 has 10
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
37s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 51s{color}
| {color:red} hadoop-hdfs-project generated 3 new + 109 unchanged - 25 fixed =
112 total (was 134) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 44s{color} | {color:orange} hadoop-hdfs-project: The patch generated 7 new +
47 unchanged - 5 fixed = 54 total (was 52) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
20s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}111m 3s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}153m 56s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ozone.scm.TestSCMCli |
| | hadoop.cblock.TestCBlockCLI |
| | hadoop.hdfs.TestDFSUpgradeFromImage |
| | hadoop.ozone.TestContainerOperations |
| | hadoop.ozone.scm.TestAllocateContainer |
| | hadoop.ozone.scm.TestContainerSmallFile |
| | hadoop.cblock.TestCBlockServerPersistence |
| | hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure |
| | hadoop.hdfs.server.datanode.TestDataNodeMetrics |
| | hadoop.cblock.TestLocalBlockCache |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11769 |
| JIRA Patch URL |
https://issues.apache.or
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009607#comment-16009607
]
Hadoop QA commented on HDFS-11769:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
0s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
45s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s{color} | {color:green} HDFS-7240 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
47s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in HDFS-7240
has 2 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
2s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in HDFS-7240 has 10
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 53s{color}
| {color:red} hadoop-hdfs-project generated 3 new + 109 unchanged - 25 fixed =
112 total (was 134) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 43s{color} | {color:orange} hadoop-hdfs-project: The patch generated 6 new +
47 unchanged - 5 fixed = 53 total (was 52) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
21s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 52s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}109m 12s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ozone.web.TestOzoneRestWithMiniCluster |
| | hadoop.ozone.web.TestOzoneWebAccess |
| | hadoop.ozone.TestMiniOzoneCluster |
| | hadoop.ozone.container.ozoneimpl.TestOzoneContainer |
| | hadoop.ozone.web.client.TestVolume |
| | hadoop.hdfs.TestDFSUpgradeFromImage |
| | hadoop.ozone.scm.TestSCMCli |
| | hadoop.ozone.TestContainerOperations |
| | hadoop.ozone.web.TestDistributedOzoneVolumes |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 |
| | hadoop.ozone.scm.TestSCMMXBean |
| | hadoop.ozone.container.common.impl.TestContainerPersistence |
| | hadoop.ozone.scm.TestContainerS
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009604#comment-16009604
]
Anu Engineer commented on HDFS-11769:
-
[~msingh] Thanks for posting the patch. I have a minor comment about
{{KeySpaceManager.java}}. It looks like it has a misplaced line.
{code}
public void join() {
try {
volumeManager.stop();
ksmRpcServer.join();
{code}
Did you intend to write
{code}
public void join() {
try {
ksmRpcServer.join();
volumeManager.stop();
{code}
> Ozone: KSM: Add createVolume API
>
>
> Key: HDFS-11769
> URL: https://issues.apache.org/jira/browse/HDFS-11769
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11769-HDFS-7240.001.patch,
> HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch
>
>
> Add createVolume API in KSM.
> createVolume API allows administrators to create a volume. The arguments to
> the API are:
> * Admin Name - The name of the administrator who is creating this volume.
> Volumes can be created only by administrators.
> * User Name - The name of the owner of this volume.
> * Quota - Quota information for this volume.
> This JIRA proposes to add the protobuf layer for createVolume and handling in
> KSM. We will file a followup JIRA for the KSM client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[ https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009582#comment-16009582 ] Mukul Kumar Singh commented on HDFS-11769: -- Thanks for the review [~nandakumar131], I have fixed this in the v3 patch. > Ozone: KSM: Add createVolume API > > > Key: HDFS-11769 > URL: https://issues.apache.org/jira/browse/HDFS-11769 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Mukul Kumar Singh > Attachments: HDFS-11769-HDFS-7240.001.patch, > HDFS-11769-HDFS-7240.002.patch, HDFS-11769-HDFS-7240.003.patch > > > Add createVolume API in KSM. > createVolume API allows administrators to create a volume. The arguments to > the API are: > * Admin Name - The name of the administrator who is creating this volume. > Volumes can be created only by administrators. > * User Name - The name of the owner of this volume. > * Quota - Quota information for this volume. > This JIRA proposes to add the protobuf layer for createVolume and handling in > KSM. We will file a followup JIRA for the KSM client. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009532#comment-16009532
]
Nandakumar commented on HDFS-11769:
---
{{VolumeManager#createVolume}} throws
{{KSMException(FAILED_VOLUME_ALREADY_EXISTS)}} when volume already exists,
since the throw statement is within try cache block and KSMException extends
IOException, {{catch (IOException | DBException ex)}} will catch the exception
again and throw a new {{KSMException(FAILED_INTERNAL_ERROR)}}. So even in case
of VOLUME_ALREADY_EXISTS we will get INTERNAL_ERROR.
> Ozone: KSM: Add createVolume API
>
>
> Key: HDFS-11769
> URL: https://issues.apache.org/jira/browse/HDFS-11769
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11769-HDFS-7240.001.patch,
> HDFS-11769-HDFS-7240.002.patch
>
>
> Add createVolume API in KSM.
> createVolume API allows administrators to create a volume. The arguments to
> the API are:
> * Admin Name - The name of the administrator who is creating this volume.
> Volumes can be created only by administrators.
> * User Name - The name of the owner of this volume.
> * Quota - Quota information for this volume.
> This JIRA proposes to add the protobuf layer for createVolume and handling in
> KSM. We will file a followup JIRA for the KSM client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009526#comment-16009526
]
Hadoop QA commented on HDFS-11769:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
34s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
33s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s{color} | {color:green} HDFS-7240 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
34s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in HDFS-7240
has 2 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
51s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in HDFS-7240 has 10
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 30s{color}
| {color:red} hadoop-hdfs-project generated 3 new + 109 unchanged - 25 fixed =
112 total (was 134) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 40s{color} | {color:orange} hadoop-hdfs-project: The patch generated 6 new +
47 unchanged - 5 fixed = 53 total (was 52) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
14s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 49s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
21s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}105m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ozone.web.TestOzoneRestWithMiniCluster |
| | hadoop.ozone.web.TestOzoneWebAccess |
| | hadoop.ozone.TestMiniOzoneCluster |
| | hadoop.ozone.container.ozoneimpl.TestOzoneContainer |
| | hadoop.ozone.web.client.TestVolume |
| | hadoop.hdfs.TestDFSUpgradeFromImage |
| | hadoop.ozone.scm.TestSCMCli |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.ozone.TestContainerOperations |
| | hadoop.hdfs.server.balancer.TestBalancer |
| | hadoop.ozone.web.TestDistributedOzoneVolumes |
| | hadoop.ozone.scm.TestSCMMXBean |
| | hadoop.ozone.container.common.impl.TestContainerPersistence |
|
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009463#comment-16009463
]
Mukul Kumar Singh commented on HDFS-11769:
--
Thanks [~anu] and [~xyao] for the review.
Can we abstract VolumeManager as an interface and put the implementation into
VolumeManagerImpl for better testability?
This way, KeySpaceManager, BucketManager and KeyManager,etc. can be easily
developed/tested with a mocked VolumeManager?
---> This is already done using KeyspaceManagerProtocol
Line 51:
can you add some comments on KSM DB and the schema for volumes. Note the same
DB will be used for bucket/key.
---> Will address this as part of another bug
Line 52: can you add some comments on the expected usage of the lock? Note it
will
be shared among VolumeManager/BucketManager/KeyManager for KSM DB read/write.
{{Will address this as part of another bug}}
We need to have a shutdown method to ensure the db is closed properly upon
shutdown of KSM.
---> done
Line 82/92/111/117:
Can we use DFSUtil.bytes2String/DFSUtil.string2Bytes to take care of string
encoding/decoding?
---> done
Line 102: can we define the magic number 1024 in OzoneConsts?
---> done
Also, we should add a KSM Exception as you commented in TODO.
---> done
Line 118: we need to close the batch to avoid resource leak.
Can you add wrapper for createWriteBatch()/writeBatch()/closeBatch() for
LevelDBStore and
modify the code to atomic update DB for volume creation using the wrappers like
below:
---> done
TestDistributedOzoneVolumes.java
The tests added look good to me. One suggestion: is it possible to abstract
some common helpers to dedup.
---> done
KeySpaceManagerProtocolClientSideTranslatorPB.java
resp = rpcProxy.createVolume(NULL_RPC_CONTROLLER,
req.build());
resp has a status field and if it is not OK, we should throw.
---> done
ObjectStoreHandler.java
Not really part of this change, but in one of our earlier patches
we stopped using this string – distributed.
if ("distributed".equalsIgnoreCase(shType))
---> this was solved when the workspace updated
VolumeManager.java#VolumeManager
File ksmDBFile = new File(ksmDBPath, KSM_DB_NAME);
Should create this path if this does not exist ?
---> done
options.createIfMissing(true); – we don't need this since this is the default
option for levelDB.
---> done
VolumeManager.java#createVolume
throw new IOException("Too many volumes for user:" + dbUserName);
dbUserName --> args.getOwnerName()
---> done
DistributedStorageHandler.java
Thanks for fixing the comment in line 78. I think we should add a new param
instead of modifying the StorageContainerLocationProtocol param.
---> done
> Ozone: KSM: Add createVolume API
>
>
> Key: HDFS-11769
> URL: https://issues.apache.org/jira/browse/HDFS-11769
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11769-HDFS-7240.001.patch,
> HDFS-11769-HDFS-7240.002.patch
>
>
> Add createVolume API in KSM.
> createVolume API allows administrators to create a volume. The arguments to
> the API are:
> * Admin Name - The name of the administrator who is creating this volume.
> Volumes can be created only by administrators.
> * User Name - The name of the owner of this volume.
> * Quota - Quota information for this volume.
> This JIRA proposes to add the protobuf layer for createVolume and handling in
> KSM. We will file a followup JIRA for the KSM client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16007342#comment-16007342
]
Anu Engineer commented on HDFS-11769:
-
[~msingh] Thanks for uploading the patch. Overall it looks excellent. I have
some very minor comments.
_KeySpaceManagerProtocolClientSideTranslatorPB.java_
# resp = rpcProxy.createVolume(NULL_RPC_CONTROLLER,
req.build());
-- resp has a status field and if it is not OK, we should throw.
_ObjectStoreHandler.java_
# Not really part of this change, but in one of our earlier patches
we stopped using this string -- distributed.
{{if ("distributed".equalsIgnoreCase(shType))}}
_VolumeManager.java#VolumeManager_
# File ksmDBFile = new File(ksmDBPath, KSM_DB_NAME);
-- Should create this path if this does not exist ?
# {{options.createIfMissing(true);}} -- we don't need this since this is the
default option for levelDB.
_VolumeManager.java#createVolume_
# {{throw new IOException("Too many volumes for user:" + dbUserName);}}
dbUserName --> args.getOwnerName()
_DistributedStorageHandler.java_
# Thanks for fixing the comment in line 78. I think we should add a new param
instead of modifying the StorageContainerLocationProtocol param.
> Ozone: KSM: Add createVolume API
>
>
> Key: HDFS-11769
> URL: https://issues.apache.org/jira/browse/HDFS-11769
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11769-HDFS-7240.001.patch
>
>
> Add createVolume API in KSM.
> createVolume API allows administrators to create a volume. The arguments to
> the API are:
> * Admin Name - The name of the administrator who is creating this volume.
> Volumes can be created only by administrators.
> * User Name - The name of the owner of this volume.
> * Quota - Quota information for this volume.
> This JIRA proposes to add the protobuf layer for createVolume and handling in
> KSM. We will file a followup JIRA for the KSM client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11769) Ozone: KSM: Add createVolume API
[
https://issues.apache.org/jira/browse/HDFS-11769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005582#comment-16005582
]
Xiaoyu Yao commented on HDFS-11769:
---
Thanks [~msingh] for working on this. The patch looks good to me overall. It
has some overlap with HDFS-11768 due dependency. That should be cleared out
when HDFS-11768 is in.
Below are some earlier feedback:
*VolumeManager.java*
Can we abstract VolumeManager as an interface and put the implementation into
VolumeManagerImpl for better testability?
This way, KeySpaceManager, BucketManager and KeyManager,etc. can be easily
developed/tested with a mocked VolumeManager?
Line 51:
can you add some comments on KSM DB and the schema for volumes. Note the same
DB will be used for bucket/key.
Line 52: can you add some comments on the expected usage of the lock? Note it
will
be shared among VolumeManager/BucketManager/KeyManager for KSM DB read/write.
We need to have a shutdown method to ensure the db is closed properly upon
shutdown of KSM.
Line 82/92/111/117:
Can we use DFSUtil.bytes2String/DFSUtil.string2Bytes to take care of string
encoding/decoding?
Line 102: can we define the magic number 1024 in OzoneConsts?
Also, we should add a KSM Exception as you commented in TODO.
Line 118: we need to close the batch to avoid resource leak.
Can you add wrapper for createWriteBatch()/writeBatch()/closeBatch() for
LevelDBStore and
modify the code to atomic update DB for volume creation using the wrappers like
below:
{code}
WriteBatch batch = store.createWriteBatch();
try {
batch.put(...);
batch.put(...);
store.writeBatch(batch);
} finally {
store.closeBatch(batch);
}
{code}
*TestDistributedOzoneVolumes.java*
The tests added look good to me. One suggestion: is it possible to abstract
some common helpers to dedup.
> Ozone: KSM: Add createVolume API
>
>
> Key: HDFS-11769
> URL: https://issues.apache.org/jira/browse/HDFS-11769
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11769-HDFS-7240.001.patch
>
>
> Add createVolume API in KSM.
> createVolume API allows administrators to create a volume. The arguments to
> the API are:
> * Admin Name - The name of the administrator who is creating this volume.
> Volumes can be created only by administrators.
> * User Name - The name of the owner of this volume.
> * Quota - Quota information for this volume.
> This JIRA proposes to add the protobuf layer for createVolume and handling in
> KSM. We will file a followup JIRA for the KSM client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
