[jira] [Comment Edited] (HDDS-1745) Add integration test for createDirectory for OM HA
[ https://issues.apache.org/jira/browse/HDDS-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169706#comment-17169706 ] Rui Wang edited comment on HDDS-1745 at 8/3/20, 4:20 AM: - [~bharat] Can I work on this JIRA? Can you share some context? Right now I want to know: 1. For OM HA model, which JIRA/paper describes it? (thus I can understand the correct behavior for createDirectory in HA mode) 2. what kind of integration test infrastructure/tool existing so I can utilize to create the test? Any existing integration test that I can use as a reference? was (Author: amaliujia): [~bharat] Can I work on this JIRA? Can you share some context? Right now I want to know: 1. For OM HA model, which JIRA/paper describes it? (thus I can understand the correct behavior for createDirectory in HA mode) 2. what kind of integration test infrastructure/tool existing so I can utilize to create the test? > Add integration test for createDirectory for OM HA > -- > > Key: HDDS-1745 > URL: https://issues.apache.org/jira/browse/HDDS-1745 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task > Components: OM HA, Ozone Manager >Reporter: Bharat Viswanadham >Priority: Major > Labels: newbie > > Add an integration test for createDirectory which is implemented as part of > HDDS-1730 for OM HA. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1745) Add integration test for createDirectory for OM HA
[ https://issues.apache.org/jira/browse/HDDS-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169706#comment-17169706 ] Rui Wang commented on HDDS-1745: [~bharat] Can I work on this JIRA? Can you share some context? Right now I want to know: 1. For OM HA model, which JIRA/paper describes it? (thus I can understand the correct behavior for createDirectory in HA mode) 2. what kind of integration test infrastructure/tool existing so I can utilize to create the test? > Add integration test for createDirectory for OM HA > -- > > Key: HDDS-1745 > URL: https://issues.apache.org/jira/browse/HDDS-1745 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task > Components: OM HA, Ozone Manager >Reporter: Bharat Viswanadham >Priority: Major > Labels: newbie > > Add an integration test for createDirectory which is implemented as part of > HDDS-1730 for OM HA. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] xiaoyuyao commented on pull request #1258: HDDS-4029. Recon unable to add a new container which is in CLOSED state.
xiaoyuyao commented on pull request #1258: URL: https://github.com/apache/hadoop-ozone/pull/1258#issuecomment-667749339 +1. Thanks @avijayanhwx for the patience. The latest LGTM, +1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1258: HDDS-4029. Recon unable to add a new container which is in CLOSED state.
xiaoyuyao commented on a change in pull request #1258:
URL: https://github.com/apache/hadoop-ozone/pull/1258#discussion_r464145469
##
File path:
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/scm/TestReconContainerManager.java
##
@@ -86,12 +104,39 @@ public void testCheckAndAddNewContainer() throws
IOException {
ReconContainerManager containerManager = getContainerManager();
assertFalse(containerManager.exists(containerID));
DatanodeDetails datanodeDetails = randomDatanodeDetails();
-containerManager.checkAndAddNewContainer(containerID, datanodeDetails);
+containerManager.checkAndAddNewContainer(containerID,
+OPEN, datanodeDetails);
assertTrue(containerManager.exists(containerID));
// Doing it one more time should not change any state.
-containerManager.checkAndAddNewContainer(containerID, datanodeDetails);
+containerManager.checkAndAddNewContainer(containerID, OPEN,
+datanodeDetails);
assertTrue(containerManager.exists(containerID));
+assertEquals(LifeCycleState.OPEN,
+getContainerManager().getContainer(containerID).getState());
}
+ @Test
+ public void testUpdateContainerStateFromOpen() throws IOException {
+ContainerWithPipeline containerWithPipeline =
+getTestContainer(LifeCycleState.OPEN);
+
+long id = containerWithPipeline.getContainerInfo().getContainerID();
+ContainerID containerID =
+containerWithPipeline.getContainerInfo().containerID();
+
+// Adding container #100.
+getContainerManager().addNewContainer(id, containerWithPipeline);
+assertEquals(LifeCycleState.OPEN,
+getContainerManager().getContainer(containerID).getState());
+
+DatanodeDetails datanodeDetails = randomDatanodeDetails();
+
+// First report with "CLOSED" replica state moves container state to
+// "CLOSING".
+getContainerManager().checkAndAddNewContainer(containerID, State.CLOSED,
+datanodeDetails);
+assertEquals(CLOSING,
+getContainerManager().getContainer(containerID).getState());
Review comment:
LGTM.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4052) Remove master/slave terminology from Ozone
[ https://issues.apache.org/jira/browse/HDDS-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4052: --- Fix Version/s: 0.7.0 > Remove master/slave terminology from Ozone > -- > > Key: HDDS-4052 > URL: https://issues.apache.org/jira/browse/HDDS-4052 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Eric Badger >Assignee: Attila Doroszlai >Priority: Major > Labels: pull-request-available > Fix For: 0.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4052) Remove master/slave terminology from Ozone
[ https://issues.apache.org/jira/browse/HDDS-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4052: --- Labels: (was: pull-request-available) > Remove master/slave terminology from Ozone > -- > > Key: HDDS-4052 > URL: https://issues.apache.org/jira/browse/HDDS-4052 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Eric Badger >Assignee: Attila Doroszlai >Priority: Major > Fix For: 0.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on pull request #1270: HDDS-4044. Deprecate ozone.s3g.volume.name.
elek commented on pull request #1270: URL: https://github.com/apache/hadoop-ozone/pull/1270#issuecomment-667714888 Thinking a little more: if I understoof well, you are worried about admins who will start multiple s3g servers with different configs. On the other hand I would prefer to suport customizable volume name (but practically the same for all s3g instances). The problem is that we couldn't check if different services are started with different settings or not. We had earlier a plan to do some kind of configuration download during the service startup to simplify the configuration of the services. With such approach, you can be sure that all the services use the same configs (but power users can do any evil actions, anyway...) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on pull request #1270: HDDS-4044. Deprecate ozone.s3g.volume.name.
elek commented on pull request #1270: URL: https://github.com/apache/hadoop-ozone/pull/1270#issuecomment-667714388 > Let's say when s3G started all buckets are created under volume s3v. > > Now different S3G's started using different volumes, now when the user wants to use the bucket in o3fs, now he is not sure which volume to use. I might have a different view this, because I think here are multiple, different roles, which are mixed here. * The **administrators** can modify the volume mounting. * **Users** will use s3g I think it's acceptable, if the administrator modifies some fundamental part of the environment setup, the user should be notified. It's very similar to a DNS name refactor. There are cases where the knowledge of the users should be updated. I think it's acceptable, and this is the responsibility of the administrator to judge if it's ok or not in a specific environment / company culture. There are also cases when the administrator would like to start with `/home` instead of `s3v` volume names from the day 1. The volume name won't be changed anymore, but in this setup, a different > And also one more point, using a single volume for S3G buckets came to eliminate a command which was there before ozone s3 path, ... now with this feature, we will be back to the same issue. Similar, but not the same. There are significant differences: 1. volume names are not cryptic anymore 2. **all** buckets use the same volume name (don't need to check for every bucket, enough to remember to the configuration) > Okay coming back to the comment, to get this in are you proposing we need to remove the warning which we have added in the code? Actually it was suggested by @arp7, but yes. I am fine with that. I think we shold keep it in `ozone-default.xml`, too. We can assume that if somebody checks the `ozone-default.xml`, they are power users or developers. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #1283: HDDS-4057. Failed acceptance test missing from bundle
adoroszlai opened a new pull request #1283: URL: https://github.com/apache/hadoop-ozone/pull/1283 ## What changes were proposed in this pull request? The problem is caused by this line in `test-all.sh`: https://github.com/apache/hadoop-ozone/blob/e219aae7c7bcd8eacd592ca2ccfcf58626477f37/hadoop-ozone/dist/src/main/compose/test-all.sh#L28 because `testlib.sh` turns on "exit on error": https://github.com/apache/hadoop-ozone/blob/e219aae7c7bcd8eacd592ca2ccfcf58626477f37/hadoop-ozone/dist/src/main/compose/testlib.sh#L17 So `test-all.sh` exits after the first failed `test.sh` execution, `... is FAILED` is not present in output. The fix is simple: temporarily turn off "exit on error" before running `test.sh`, then turn it back on. We also need to make `rebot` return zero code even if some tests failed. https://issues.apache.org/jira/browse/HDDS-4057 ## How was this patch tested? Added two always failing tests and two environments. These can be used to test failure handling in acceptance test runner. Without the fix, only the first test in the first environment is run, and its logs are not copied to `target/acceptance`: ``` $ OZONE_ACCEPTANCE_SUITE=failing ./hadoop-ozone/dev-support/checks/acceptance.sh ... Test1 :: This test always fails | FAIL | ... Output: /tmp/smoketest/failing1/result/robot-failing1-failing1-test1-scm.xml $ ls -1 target/acceptance ``` With the fix, both tests in both environments are run: ``` $ OZONE_ACCEPTANCE_SUITE=failing ./hadoop-ozone/dev-support/checks/acceptance.sh ... Test1 :: This test always fails | FAIL | ... Output: /tmp/smoketest/failing1/result/robot-failing1-failing1-test1-scm.xml ... ERROR: Test execution of ./failing1 is FAILED ... Test1 :: This test always fails | FAIL | ... Output: /tmp/smoketest/failing2/result/robot-failing2-failing2-test1-scm.xml ... ERROR: Test execution of ./failing2 is FAILED $ ls -1 target/acceptance docker-failing1-failing1-test1-scm.log docker-failing2-failing2-test1-scm.log failing1.xml failing2.xml log.html report.html summary.html ``` Regular CI: https://github.com/adoroszlai/hadoop-ozone/runs/937941098 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4057) Failed acceptance test missing from bundle
[
https://issues.apache.org/jira/browse/HDDS-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-4057:
-
Labels: pull-request-available (was: )
> Failed acceptance test missing from bundle
> --
>
> Key: HDDS-4057
> URL: https://issues.apache.org/jira/browse/HDDS-4057
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: build, test
>Affects Versions: 0.7.0
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
>
> Acceptance test run is stopped after any failed test, and its logs are
> missing from the bundle.
> {code:title=misc suite}
> hadoop27
> hadoop31
> hadoop32
> ozone-csi
> ozone-mr
> ozone-om-ha-s3
> ozone-topology
> ozones3-haproxy
> ozonesecure-mr
> ozonesecure-om-ha
> upgrade
> {code}
> * {{ozone-topology}} failed in [this
> run|https://github.com/apache/hadoop-ozone/runs/927545620],
> {{ozones3-haproxy}} and subsequent tests were skipped, {{ozone-topology}}
> robot log is missing from
> [bundle|https://github.com/apache/hadoop-ozone/suites/984510888/artifacts/12629978]
> * {{ozone-secure}} failed in [this
> run|https://github.com/adoroszlai/hadoop-ozone/runs/933098576], it is the
> only test in {{secure}} suite, so {{acceptance-secure.zip}} was not even
> created
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4057) Failed acceptance test missing from bundle
Attila Doroszlai created HDDS-4057:
--
Summary: Failed acceptance test missing from bundle
Key: HDDS-4057
URL: https://issues.apache.org/jira/browse/HDDS-4057
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: build, test
Affects Versions: 0.7.0
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai
Acceptance test run is stopped after any failed test, and its logs are missing
from the bundle.
{code:title=misc suite}
hadoop27
hadoop31
hadoop32
ozone-csi
ozone-mr
ozone-om-ha-s3
ozone-topology
ozones3-haproxy
ozonesecure-mr
ozonesecure-om-ha
upgrade
{code}
* {{ozone-topology}} failed in [this
run|https://github.com/apache/hadoop-ozone/runs/927545620], {{ozones3-haproxy}}
and subsequent tests were skipped, {{ozone-topology}} robot log is missing from
[bundle|https://github.com/apache/hadoop-ozone/suites/984510888/artifacts/12629978]
* {{ozone-secure}} failed in [this
run|https://github.com/adoroszlai/hadoop-ozone/runs/933098576], it is the only
test in {{secure}} suite, so {{acceptance-secure.zip}} was not even created
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
