[jira] [Resolved] (HDDS-4214) Fix failed UT: TestContainerStateMachineFailures#testApplyTransactionFailure
[ https://issues.apache.org/jira/browse/HDDS-4214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang resolved HDDS-4214. -- Resolution: Duplicate > Fix failed UT: TestContainerStateMachineFailures#testApplyTransactionFailure > > > Key: HDDS-4214 > URL: https://issues.apache.org/jira/browse/HDDS-4214 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: runzhiwang >Assignee: runzhiwang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4214) Fix failed UT: TestContainerStateMachineFailures#testApplyTransactionFailure
runzhiwang created HDDS-4214: Summary: Fix failed UT: TestContainerStateMachineFailures#testApplyTransactionFailure Key: HDDS-4214 URL: https://issues.apache.org/jira/browse/HDDS-4214 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: runzhiwang Assignee: runzhiwang -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-4212) TestSCMStateMachine
[ https://issues.apache.org/jira/browse/HDDS-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191358#comment-17191358 ] Rui Wang commented on HDDS-4212: To clarify, this JIRA should work on: org.apache.hadoop.hdds.scm.ha.SCMStateMachine > TestSCMStateMachine > --- > > Key: HDDS-4212 > URL: https://issues.apache.org/jira/browse/HDDS-4212 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > I cannot find a TestSCMStateMachine in HDDS-2323 feature branch. So in this > JIRA I propose add tests for SCMStateMachine -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1393: HDDS-4143. Implement a factory for OM Requests that returns an instance based on layout version.
linyiqun commented on a change in pull request #1393:
URL: https://github.com/apache/hadoop-ozone/pull/1393#discussion_r484082776
##
File path:
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOmRequestFactory.java
##
@@ -0,0 +1,116 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.upgrade;
+
+import static org.apache.hadoop.ozone.om.upgrade.OMLayoutFeature.CREATE_EC;
+import static
org.apache.hadoop.ozone.om.upgrade.OMLayoutFeature.INITIAL_VERSION;
+import static
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.Type.CreateKey;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.lang.reflect.Modifier;
+import java.util.Set;
+
+import org.apache.hadoop.ozone.om.OMStorage;
+import org.apache.hadoop.ozone.om.exceptions.OMException;
+import org.apache.hadoop.ozone.om.request.OMClientRequest;
+import org.apache.hadoop.ozone.om.request.key.OMECKeyCreateRequest;
+import org.apache.hadoop.ozone.om.request.key.OMKeyCreateRequest;
+import
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.LayoutVersion;
+import
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMRequest;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.reflections.Reflections;
+
+/**
+ * Test OmVersionFactory.
+ */
+public class TestOmRequestFactory {
+
+ private static OmRequestFactory omRequestFactory;
+ private static OMLayoutVersionManager omVersionManager;
+
+ @BeforeClass
+ public static void setup() throws OMException {
+OMStorage omStorage = mock(OMStorage.class);
+when(omStorage.getLayoutVersion()).thenReturn(0);
+omVersionManager = OMLayoutVersionManager.initialize(omStorage);
+omRequestFactory = omVersionManager.getVersionFactory();
+ }
+
+ @Test
+ public void testKeyCreateRequest() {
+
+// Try getting v1 of 'CreateKey'.
+Class requestType =
+omRequestFactory.getRequestType(OMRequest.newBuilder()
+.setCmdType(CreateKey)
+.setClientId("c1")
+.setLayoutVersion(LayoutVersion
+.newBuilder()
+.setVersion(INITIAL_VERSION.layoutVersion())
+.build())
+.build());
+Assert.assertEquals(requestType, OMKeyCreateRequest.class);
+
+// Try getting 'CreateECKey' (V2). Should fail.
+try {
+ omRequestFactory.getRequestType(OMRequest.newBuilder()
+ .setCmdType(CreateKey)
+ .setClientId("c1")
+ .setLayoutVersion(LayoutVersion
+ .newBuilder()
+ .setVersion(CREATE_EC.layoutVersion())
+ .build())
+ .build());
+ Assert.fail();
+} catch (IllegalArgumentException ex) {
Review comment:
Can we do the verify for error message that contained in
IllegalArgumentException? This will let us know why failure was happened.
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OzoneManagerRequestHandler.java
##
@@ -290,6 +292,11 @@ public void validateRequest(OMRequest omRequest) throws
OMException {
throw new OMException("ClientId is null",
OMException.ResultCodes.INVALID_REQUEST);
}
+
+if (omRequest.getLayoutVersion() == null) {
Review comment:
Would be better to add a comment here to explain why we do layout
version empty check on handling write request.
##
File path:
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMVersionManager.java
##
@@ -54,11 +61,39 @@ public void testOMLayoutVersionManager() throws IOException
{
assertEquals(2, omVersionManager.getMetadataLayoutVersion());
}
+ @Test
+ public void testOMLayoutVersionManagerInitError() {
+OMStorage omStorage = mock(OMStorage.class);
+when(omStorage.getLayoutVersion()).thenReturn(
+OMLayoutFeature.values()[OMLayoutFeature.values().length - 1]
+.layoutVersion() + 1);
+try {
+ OMLayoutVersionManager.initialize(omStorage);
+ Assert.fail();
+} catch (OMException ex) {
+
[jira] [Commented] (HDDS-4155) Directory and filename can end up with same name in a path
[ https://issues.apache.org/jira/browse/HDDS-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191280#comment-17191280 ] Marton Elek commented on HDDS-4155: --- Does it mean that we are planning to support files and directories with the same name in ofs/o3fs (!!!) when strict s3 compatibility is configured? > Directory and filename can end up with same name in a path > -- > > Key: HDDS-4155 > URL: https://issues.apache.org/jira/browse/HDDS-4155 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > > Scenario: > Create Key via S3, and Create Directory through Fs. > # open key -> /a/b/c > # CreateDirectory -> /a/b/c > # CommitKey -> /a/b/c > So, now in Ozone we will have directory and file with name "c" > When created through Fs interface. > # create file -> /a/b/c > # CreateDirectory -> /a/b/c > # CommitKey -> /a/b/c > So, now in Ozone we will have directory and file with name "c" > > # InitiateMPU /a/b/c > # Create Part1 /a/b/c > # Commit Part1 /a/b/c > # Create Directory /a/b/c > # Complete MPU /a/b/c > So, now in Ozone, we will have directory and file with name "c". In MPU this > is one example scenario. > > Few proposals/ideas to solve this: > # Check during commit whether a directory already exists with same name. But > disadvantage is after user uploads the entire data during last stage we fail. > (File system with create in progress acts similarly. Scenario: 1. vi t1 2. > mkdir t1 3. Save t1: (Fail:"t1" is a directory) > # During create directory check are there any open key creation with same > name and fail. > > Any of the above approaches are not final, this Jira is opened to discuss > this issue and come up with solution. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] maobaolong opened a new pull request #1402: HDDS-4213. Print the dead datanode when DEAD message comming
maobaolong opened a new pull request #1402: URL: https://github.com/apache/hadoop-ozone/pull/1402 ## What changes were proposed in this pull request? Print the dead datanode when DEAD message comming ## What is the link to the Apache JIRA HDDS-4213 ## How was this patch tested? Kill a datanode, after x minute past, watching the log of SCM, you can see the log `A dead datanode detected.`, and you can get the dead datanode information clearly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4213) Print the dead datanode when DEAD message comming
[ https://issues.apache.org/jira/browse/HDDS-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4213: - Labels: pull-request-available (was: ) > Print the dead datanode when DEAD message comming > - > > Key: HDDS-4213 > URL: https://issues.apache.org/jira/browse/HDDS-4213 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: maobaolong >Assignee: maobaolong >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4213) Print the dead datanode when DEAD message comming
maobaolong created HDDS-4213: Summary: Print the dead datanode when DEAD message comming Key: HDDS-4213 URL: https://issues.apache.org/jira/browse/HDDS-4213 Project: Hadoop Distributed Data Store Issue Type: New Feature Reporter: maobaolong Assignee: maobaolong -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] runitao commented on pull request #1289: HDDS-4039. Reduce the number of fields in hdds.proto to improve performance
runitao commented on pull request #1289: URL: https://github.com/apache/hadoop-ozone/pull/1289#issuecomment-687712159 @elek Would you help me review this PR if you have time? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
