[jira] [Updated] (HDDS-3959) Avoid HddsProtos.PipelineID#toString
[
https://issues.apache.org/jira/browse/HDDS-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-3959:
---
Status: Patch Available (was: In Progress)
> Avoid HddsProtos.PipelineID#toString
>
>
> Key: HDDS-3959
> URL: https://issues.apache.org/jira/browse/HDDS-3959
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Datanode
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
>
> {{PipelineID}} was recently changed to have integer-based ID in addition to
> the string ID. Now log messages including {{PipelineID}} span multiple lines:
> {code:title=https://github.com/elek/ozone-build-results/blob/92d31c9b58065b37a371c71c97b346f99163318d/2020/07/11/1626/acceptance/docker-ozone-ozone-freon-scm.log#L218-L223}
> datanode_1 | 2020-07-11 13:07:00,540 [Command processor thread] INFO
> commandhandler.CreatePipelineCommandHandler: Created Pipeline RATIS ONE #id:
> "8101dcbf-1a28-4f20-863a-0616b4e4bc4b"
> datanode_1 | uuid128 {
> datanode_1 | mostSigBits: -9150790254504423648
> datanode_1 | leastSigBits: -8774694229384053685
> datanode_1 | }
> datanode_1 | .
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4320) Let Ozone input streams implement CanUnbuffer
[
https://issues.apache.org/jira/browse/HDDS-4320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4320:
---
Status: Patch Available (was: In Progress)
> Let Ozone input streams implement CanUnbuffer
> -
>
> Key: HDDS-4320
> URL: https://issues.apache.org/jira/browse/HDDS-4320
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
>
> Implement Hadoop's {{CanUnbuffer}} interface in {{OzoneFSInputStream}} and
> the underlying other input streams. Note: {{CanUnbuffer}} is available in
> 2.7 (HDFS-7694), but {{StreamCapabilities#UNBUFFER}} is new to 2.9.1
> (HADOOP-15012).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4362) Change hadoop32 test to use 3.2 image
[
https://issues.apache.org/jira/browse/HDDS-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4362:
---
Fix Version/s: 1.1.0
Resolution: Implemented
Status: Resolved (was: Patch Available)
> Change hadoop32 test to use 3.2 image
> -
>
> Key: HDDS-4362
> URL: https://issues.apache.org/jira/browse/HDDS-4362
> Project: Hadoop Distributed Data Store
> Issue Type: Task
> Components: test
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> {{ozone-mr/hadoop32}} acceptance test currently uses "latest" {{hadoop:3}}
> docker image, which is currently Hadoop 3.2. If it gets updated to Hadoop
> 3.3, Ozone acceptance test will be broken. We should explicitly use some 3.2
> release-based image.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4362) Change hadoop32 test to use 3.2 image
[
https://issues.apache.org/jira/browse/HDDS-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4362:
---
Labels: (was: pull-request-available)
> Change hadoop32 test to use 3.2 image
> -
>
> Key: HDDS-4362
> URL: https://issues.apache.org/jira/browse/HDDS-4362
> Project: Hadoop Distributed Data Store
> Issue Type: Task
> Components: test
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Fix For: 1.1.0
>
>
> {{ozone-mr/hadoop32}} acceptance test currently uses "latest" {{hadoop:3}}
> docker image, which is currently Hadoop 3.2. If it gets updated to Hadoop
> 3.3, Ozone acceptance test will be broken. We should explicitly use some 3.2
> release-based image.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4362) Change hadoop32 test to use 3.2 image
[
https://issues.apache.org/jira/browse/HDDS-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4362:
---
Status: Patch Available (was: In Progress)
> Change hadoop32 test to use 3.2 image
> -
>
> Key: HDDS-4362
> URL: https://issues.apache.org/jira/browse/HDDS-4362
> Project: Hadoop Distributed Data Store
> Issue Type: Task
> Components: test
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
>
> {{ozone-mr/hadoop32}} acceptance test currently uses "latest" {{hadoop:3}}
> docker image, which is currently Hadoop 3.2. If it gets updated to Hadoop
> 3.3, Ozone acceptance test will be broken. We should explicitly use some 3.2
> release-based image.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4362) Change hadoop32 test to use 3.2 image
Attila Doroszlai created HDDS-4362:
--
Summary: Change hadoop32 test to use 3.2 image
Key: HDDS-4362
URL: https://issues.apache.org/jira/browse/HDDS-4362
Project: Hadoop Distributed Data Store
Issue Type: Task
Components: test
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai
{{ozone-mr/hadoop32}} acceptance test currently uses "latest" {{hadoop:3}}
docker image, which is currently Hadoop 3.2. If it gets updated to Hadoop 3.3,
Ozone acceptance test will be broken. We should explicitly use some 3.2
release-based image.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4306) Ozone checkstyle rule can't be imported to IntelliJ
[ https://issues.apache.org/jira/browse/HDDS-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4306. Fix Version/s: 1.1.0 Resolution: Fixed > Ozone checkstyle rule can't be imported to IntelliJ > --- > > Key: HDDS-4306 > URL: https://issues.apache.org/jira/browse/HDDS-4306 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > CheckStyle: Move LineLength Check parent from TreeWalker to Checker, > otherwise fail to import to latest IntelliJ > Similar issue has been reported here and I've verified the fix locally that > the IntelliJ can import checkstyle rule after the fix. > https://github.com/checkstyle/checkstyle/issues/2116 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4328) Provide fallback cache restore key
[
https://issues.apache.org/jira/browse/HDDS-4328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4328:
---
Fix Version/s: 1.1.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Provide fallback cache restore key
> --
>
> Key: HDDS-4328
> URL: https://issues.apache.org/jira/browse/HDDS-4328
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: build
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> Maven dependency cache hit or miss in GitHub Actions workflow is based on the
> hash of all POM files. If any POM is changed, all dependencies need to be
> downloaded from scratch. Providing {{restore-keys}} would allow it to fall
> back to one of the previous caches, potentially avoiding most of the
> downloads.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4328) Provide fallback cache restore key
[
https://issues.apache.org/jira/browse/HDDS-4328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4328:
---
Labels: (was: pull-request-available)
> Provide fallback cache restore key
> --
>
> Key: HDDS-4328
> URL: https://issues.apache.org/jira/browse/HDDS-4328
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: build
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Fix For: 1.1.0
>
>
> Maven dependency cache hit or miss in GitHub Actions workflow is based on the
> hash of all POM files. If any POM is changed, all dependencies need to be
> downloaded from scratch. Providing {{restore-keys}} would allow it to fall
> back to one of the previous caches, potentially avoiding most of the
> downloads.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDDS-2413) Set configuration variables from annotated java objects
[
https://issues.apache.org/jira/browse/HDDS-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai reassigned HDDS-2413:
--
Assignee: Attila Doroszlai (was: Marton Elek)
> Set configuration variables from annotated java objects
> ---
>
> Key: HDDS-2413
> URL: https://issues.apache.org/jira/browse/HDDS-2413
> Project: Hadoop Distributed Data Store
> Issue Type: Task
>Reporter: Marton Elek
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
>
> HDDS-1469 introduced a new method to handle configuration. Configuration can
> be injected directly to java objects which makes all the java constants
> unnecessary.
>
> Almost.
>
> To read the configuration it's enough to have an annotated java object. For
> example:
>
> {code:java}
> @ConfigGroup(prefix = "hdds.scm")
> public class ScmConfig {
> private String principal;
> private String keytab; @Config(key = "kerberos.principal",
> type = ConfigType.STRING,
> defaultValue = "",
> tags = { ConfigTag.SECURITY, ConfigTag.OZONE },
> description = "This Kerberos principal is used by the SCM service."
> )
> public void setKerberosPrincipal(String kerberosPrincipal) {
> this.principal = kerberosPrincipal; {code}
> And the configuration can be set in ozone-site.xml
> Unfortunately during the unit testing we need to inject the configuration
> variables programmatically which requires a String constant:
> {code:java}
> configuration.set(ScmConfig.ConfigStrings.HDDS_SCM_KERBEROS_PRINCIPAL_KEY,
>
> "scm/" + host + "@" + realm); {code}
> I propose to implement a simple setter in the OzoneConfiguration which may
> help to set configuration based on an annotated configuration object instance:
> {code:java}
> OzoneConfiguration conf = new OzoneConfiguration();
> SCMHTTPServerConfig httpConfig = SCMHTTPServerConfig(principal1,...);
> conf.setFromObject(httpConfig){code}
> This is the opposite direction of the existing OzoneConfiguration.getObject()
> and can be implemented with a similar approach.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4158) Provide a class type for Java based configuration
[ https://issues.apache.org/jira/browse/HDDS-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4158: --- Description: Provide a class type for Java based configuration. > Provide a class type for Java based configuration > - > > Key: HDDS-4158 > URL: https://issues.apache.org/jira/browse/HDDS-4158 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Affects Versions: 1.0.0 >Reporter: maobaolong >Assignee: maobaolong >Priority: Major > Fix For: 1.1.0 > > > Provide a class type for Java based configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4158) Provide a class type for Java based configuration
[ https://issues.apache.org/jira/browse/HDDS-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4158. Fix Version/s: 1.1.0 Assignee: maobaolong Resolution: Implemented > Provide a class type for Java based configuration > - > > Key: HDDS-4158 > URL: https://issues.apache.org/jira/browse/HDDS-4158 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Affects Versions: 1.0.0 >Reporter: maobaolong >Assignee: maobaolong >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4158) Provide a class type for Java based configuration
[ https://issues.apache.org/jira/browse/HDDS-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4158: --- Labels: (was: pull-request-available) > Provide a class type for Java based configuration > - > > Key: HDDS-4158 > URL: https://issues.apache.org/jira/browse/HDDS-4158 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Affects Versions: 1.0.0 >Reporter: maobaolong >Assignee: maobaolong >Priority: Major > Fix For: 1.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4328) Provide fallback cache restore key
Attila Doroszlai created HDDS-4328:
--
Summary: Provide fallback cache restore key
Key: HDDS-4328
URL: https://issues.apache.org/jira/browse/HDDS-4328
Project: Hadoop Distributed Data Store
Issue Type: Improvement
Components: build
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai
Maven dependency cache hit or miss in GitHub Actions workflow is based on the
hash of all POM files. If any POM is changed, all dependencies need to be
downloaded from scratch. Providing {{restore-keys}} would allow it to fall
back to one of the previous caches, potentially avoiding most of the downloads.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4285) Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()
[
https://issues.apache.org/jira/browse/HDDS-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4285:
---
Status: Patch Available (was: In Progress)
> Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()
> --
>
> Key: HDDS-4285
> URL: https://issues.apache.org/jira/browse/HDDS-4285
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Marton Elek
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2020-09-28-16-19-17-581.png,
> profile-20200928-161631-180518.svg
>
>
> Ozone read operation turned out to be slow mainly because we do a new
> UGI.getCurrentUser for block token for each of the calls.
> We need to cache the block token / UGI.getCurrentUserCall() to make it faster.
> !image-2020-09-28-16-19-17-581.png!
> To reproduce:
> Checkout: https://github.com/elek/hadoop-ozone/tree/mocked-read
> {code}
> cd hadoop-ozone/client
> export
> MAVEN_OPTS=-agentpath:/home/elek/prog/async-profiler/build/libasyncProfiler.so=start,file=/tmp/profile-%t-%p.svg
> mvn compile exec:java
> -Dexec.mainClass=org.apache.hadoop.ozone.client.io.TestKeyOutputStreamUnit
> -Dexec.classpathScope=test
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4316) Upgrade to angular 1.8.0 due to CVE-2020-7676
[ https://issues.apache.org/jira/browse/HDDS-4316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4316: --- Fix Version/s: 1.1.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade to angular 1.8.0 due to CVE-2020-7676 > - > > Key: HDDS-4316 > URL: https://issues.apache.org/jira/browse/HDDS-4316 > Project: Hadoop Distributed Data Store > Issue Type: Task >Affects Versions: 1.0.0 >Reporter: Vivek Ratnavel Subramanian >Assignee: Vivek Ratnavel Subramanian >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > Angular versions < 1.8.0 are vulnerable to cross-site scripting > [https://nvd.nist.gov/vuln/detail/CVE-2020-7676] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4325) Incompatible return codes from Ozone getconf -confKey
[
https://issues.apache.org/jira/browse/HDDS-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4325:
---
Status: Patch Available (was: In Progress)
> Incompatible return codes from Ozone getconf -confKey
> -
>
> Key: HDDS-4325
> URL: https://issues.apache.org/jira/browse/HDDS-4325
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone CLI
>Affects Versions: 1.0.0
>Reporter: Uma Maheswara Rao G
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
>
> It seems that the return codes of ozone getconf -confKey are different in
> prior 1.0 and after 1.0.
> Looking at the code:
> in old code:
> /** Method to be overridden by sub classes for specific behavior. */
> int doWorkInternal(OzoneGetConf tool, String[] args) throws Exception {
> {code:java}
> String value = tool.getConf().getTrimmed(key);
> if (value != null) {
> tool.printOut(value);
> return 0;
> }
> tool.printError("Configuration " + key + " is missing.");
> return -1;
> }
> {code}
> with 1.0 code:
> @Override
> public Void call() throws Exception {
> String value = tool.getConf().getTrimmed(confKey);
> if (value != null) {
> tool.printOut(value);
> } else {
> tool.printError("Configuration " + confKey + " is missing.");
> }
> return null;
> }
> We are returning null irrespective of the cases.
> Some applications/tests depending on this codes.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDDS-4325) Incompatible return codes from Ozone getconf -confKey
[
https://issues.apache.org/jira/browse/HDDS-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai reassigned HDDS-4325:
--
Assignee: Attila Doroszlai
> Incompatible return codes from Ozone getconf -confKey
> -
>
> Key: HDDS-4325
> URL: https://issues.apache.org/jira/browse/HDDS-4325
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone CLI
>Affects Versions: 1.0.0
>Reporter: Uma Maheswara Rao G
>Assignee: Attila Doroszlai
>Priority: Major
>
> It seems that the return codes of ozone getconf -confKey are different in
> prior 1.0 and after 1.0.
> Looking at the code:
> in old code:
> /** Method to be overridden by sub classes for specific behavior. */
> int doWorkInternal(OzoneGetConf tool, String[] args) throws Exception {
> {code:java}
> String value = tool.getConf().getTrimmed(key);
> if (value != null) {
> tool.printOut(value);
> return 0;
> }
> tool.printError("Configuration " + key + " is missing.");
> return -1;
> }
> {code}
> with 1.0 code:
> @Override
> public Void call() throws Exception {
> String value = tool.getConf().getTrimmed(confKey);
> if (value != null) {
> tool.printOut(value);
> } else {
> tool.printError("Configuration " + confKey + " is missing.");
> }
> return null;
> }
> We are returning null irrespective of the cases.
> Some applications/tests depending on this codes.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4319) Compile error with Java 11
[
https://issues.apache.org/jira/browse/HDDS-4319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4319:
---
Status: Patch Available (was: Open)
> Compile error with Java 11
> --
>
> Key: HDDS-4319
> URL: https://issues.apache.org/jira/browse/HDDS-4319
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: build
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
>
> {code:title=https://github.com/apache/hadoop-ozone/runs/1217093596#step:6:5632}
> Error: Failed to execute goal
> org.codehaus.mojo:aspectj-maven-plugin:1.10:compile (default) on project
> hadoop-ozone-ozone-manager: Execution default of goal
> org.codehaus.mojo:aspectj-maven-plugin:1.10:compile failed: Plugin
> org.codehaus.mojo:aspectj-maven-plugin:1.10 or one of its dependencies could
> not be resolved: Could not find artifact com.sun:tools:jar:11.0.8 at
> specified path /opt/hostedtoolcache/jdk/11.0.8/x64/../lib/tools.jar -> [Help
> 1]
> {code}
> https://github.com/mojohaus/aspectj-maven-plugin/issues/24#issuecomment-419077658
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4320) Let Ozone input streams implement CanUnbuffer
[
https://issues.apache.org/jira/browse/HDDS-4320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4320:
---
Description: Implement Hadoop's {{CanUnbuffer}} interface in
{{OzoneFSInputStream}} and the underlying other input streams. Note:
{{CanUnbuffer}} is available in 2.7 (HDFS-7694), but
{{StreamCapabilities#UNBUFFER}} is new to 2.9.1 (HADOOP-15012). (was:
Implement Hadoop's {{CanUnbuffer}} interface in {{OzoneFSInputStream}} and the
underlying other input streams.)
> Let Ozone input streams implement CanUnbuffer
> -
>
> Key: HDDS-4320
> URL: https://issues.apache.org/jira/browse/HDDS-4320
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>
> Implement Hadoop's {{CanUnbuffer}} interface in {{OzoneFSInputStream}} and
> the underlying other input streams. Note: {{CanUnbuffer}} is available in
> 2.7 (HDFS-7694), but {{StreamCapabilities#UNBUFFER}} is new to 2.9.1
> (HADOOP-15012).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4320) Let Ozone input streams implement CanUnbuffer
Attila Doroszlai created HDDS-4320:
--
Summary: Let Ozone input streams implement CanUnbuffer
Key: HDDS-4320
URL: https://issues.apache.org/jira/browse/HDDS-4320
Project: Hadoop Distributed Data Store
Issue Type: Improvement
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai
Implement Hadoop's {{CanUnbuffer}} interface in {{OzoneFSInputStream}} and the
underlying other input streams.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4319) Compile error with Java 11
[
https://issues.apache.org/jira/browse/HDDS-4319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4319:
---
Description:
{code:title=https://github.com/apache/hadoop-ozone/runs/1217093596#step:6:5632}
Error: Failed to execute goal
org.codehaus.mojo:aspectj-maven-plugin:1.10:compile (default) on project
hadoop-ozone-ozone-manager: Execution default of goal
org.codehaus.mojo:aspectj-maven-plugin:1.10:compile failed: Plugin
org.codehaus.mojo:aspectj-maven-plugin:1.10 or one of its dependencies could
not be resolved: Could not find artifact com.sun:tools:jar:11.0.8 at specified
path /opt/hostedtoolcache/jdk/11.0.8/x64/../lib/tools.jar -> [Help 1]
{code}
https://github.com/mojohaus/aspectj-maven-plugin/issues/24#issuecomment-419077658
was:
{code:title=https://github.com/apache/hadoop-ozone/runs/1217093596#step:6:5632}
Error: Failed to execute goal
org.codehaus.mojo:aspectj-maven-plugin:1.10:compile (default) on project
hadoop-ozone-ozone-manager: Execution default of goal
org.codehaus.mojo:aspectj-maven-plugin:1.10:compile failed: Plugin
org.codehaus.mojo:aspectj-maven-plugin:1.10 or one of its dependencies could
not be resolved: Could not find artifact com.sun:tools:jar:11.0.8 at specified
path /opt/hostedtoolcache/jdk/11.0.8/x64/../lib/tools.jar -> [Help 1]
{code}
> Compile error with Java 11
> --
>
> Key: HDDS-4319
> URL: https://issues.apache.org/jira/browse/HDDS-4319
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: build
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>
> {code:title=https://github.com/apache/hadoop-ozone/runs/1217093596#step:6:5632}
> Error: Failed to execute goal
> org.codehaus.mojo:aspectj-maven-plugin:1.10:compile (default) on project
> hadoop-ozone-ozone-manager: Execution default of goal
> org.codehaus.mojo:aspectj-maven-plugin:1.10:compile failed: Plugin
> org.codehaus.mojo:aspectj-maven-plugin:1.10 or one of its dependencies could
> not be resolved: Could not find artifact com.sun:tools:jar:11.0.8 at
> specified path /opt/hostedtoolcache/jdk/11.0.8/x64/../lib/tools.jar -> [Help
> 1]
> {code}
> https://github.com/mojohaus/aspectj-maven-plugin/issues/24#issuecomment-419077658
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4319) Compile error with Java 11
Attila Doroszlai created HDDS-4319:
--
Summary: Compile error with Java 11
Key: HDDS-4319
URL: https://issues.apache.org/jira/browse/HDDS-4319
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: build
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai
{code:title=https://github.com/apache/hadoop-ozone/runs/1217093596#step:6:5632}
Error: Failed to execute goal
org.codehaus.mojo:aspectj-maven-plugin:1.10:compile (default) on project
hadoop-ozone-ozone-manager: Execution default of goal
org.codehaus.mojo:aspectj-maven-plugin:1.10:compile failed: Plugin
org.codehaus.mojo:aspectj-maven-plugin:1.10 or one of its dependencies could
not be resolved: Could not find artifact com.sun:tools:jar:11.0.8 at specified
path /opt/hostedtoolcache/jdk/11.0.8/x64/../lib/tools.jar -> [Help 1]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4313) Create thread-local instance of FileSystem in HadoopFsGenerator
[
https://issues.apache.org/jira/browse/HDDS-4313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4313:
---
Status: Patch Available (was: In Progress)
> Create thread-local instance of FileSystem in HadoopFsGenerator
> ---
>
> Key: HDDS-4313
> URL: https://issues.apache.org/jira/browse/HDDS-4313
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: freon
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
>
> [~elek]'s most recent performance test uncovered a client-side bottleneck in
> Freon's Hadoop FS generator: a global {{FileSystem}} instance causes lock
> contention among test threads.
> https://github.com/elek/ozone-notes/blob/master/static/results/23_hcfs_write/profile.svg
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4313) Create thread-local instance of FileSystem in HadoopFsGenerator
Attila Doroszlai created HDDS-4313:
--
Summary: Create thread-local instance of FileSystem in
HadoopFsGenerator
Key: HDDS-4313
URL: https://issues.apache.org/jira/browse/HDDS-4313
Project: Hadoop Distributed Data Store
Issue Type: Improvement
Components: freon
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai
[~elek]'s most recent performance test uncovered a client-side bottleneck in
Freon's Hadoop FS generator: a global {{FileSystem}} instance causes lock
contention among test threads.
https://github.com/elek/ozone-notes/blob/master/static/results/23_hcfs_write/profile.svg
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4311) Type-safe config design doc points to OM HA
[ https://issues.apache.org/jira/browse/HDDS-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4311: --- Status: Patch Available (was: In Progress) > Type-safe config design doc points to OM HA > --- > > Key: HDDS-4311 > URL: https://issues.apache.org/jira/browse/HDDS-4311 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: documentation >Reporter: Attila Doroszlai >Assignee: Attila Doroszlai >Priority: Minor > > Abstract and links for > http://hadoop.apache.org/ozone/docs/1.0.0/design/typesafeconfig.html are > wrong, reference OM HA design doc. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4312) findbugs check succeeds despite compile error
[
https://issues.apache.org/jira/browse/HDDS-4312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4312:
---
Status: Patch Available (was: Open)
> findbugs check succeeds despite compile error
> -
>
> Key: HDDS-4312
> URL: https://issues.apache.org/jira/browse/HDDS-4312
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: build
>Affects Versions: 1.1.0
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
>
> Findbugs check has been silently failing but reporting success for some time
> now. The problem is that {{findbugs.sh}} determines exit code based on the
> number of findbugs failures. If {{compile}} step fails, exit code is 0, ie.
> success.
> {code:title=https://github.com/apache/hadoop-ozone/runs/1210535433#step:3:866}
> 2020-10-02T18:37:57.0699502Z [ERROR] Failed to execute goal on project
> hadoop-hdds-client: Could not resolve dependencies for project
> org.apache.hadoop:hadoop-hdds-client:jar:1.1.0-SNAPSHOT: Could not find
> artifact org.apache.hadoop:hadoop-hdds-common:jar:tests:1.1.0-SNAPSHOT in
> apache.snapshots.https
> (https://repository.apache.org/content/repositories/snapshots) -> [Help 1]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4312) findbugs check succeeds despite compile error
[
https://issues.apache.org/jira/browse/HDDS-4312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4312:
---
Summary: findbugs check succeeds despite compile error (was: findbugs
check succeeds despite failure)
> findbugs check succeeds despite compile error
> -
>
> Key: HDDS-4312
> URL: https://issues.apache.org/jira/browse/HDDS-4312
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: build
>Affects Versions: 1.1.0
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
>
> Findbugs check has been silently failing but reporting success for some time
> now. The problem is that {{findbugs.sh}} determines exit code based on the
> number of findbugs failures. If {{compile}} step fails, exit code is 0, ie.
> success.
> {code:title=https://github.com/apache/hadoop-ozone/runs/1210535433#step:3:866}
> 2020-10-02T18:37:57.0699502Z [ERROR] Failed to execute goal on project
> hadoop-hdds-client: Could not resolve dependencies for project
> org.apache.hadoop:hadoop-hdds-client:jar:1.1.0-SNAPSHOT: Could not find
> artifact org.apache.hadoop:hadoop-hdds-common:jar:tests:1.1.0-SNAPSHOT in
> apache.snapshots.https
> (https://repository.apache.org/content/repositories/snapshots) -> [Help 1]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4311) Type-safe config design doc points to OM HA
Attila Doroszlai created HDDS-4311: -- Summary: Type-safe config design doc points to OM HA Key: HDDS-4311 URL: https://issues.apache.org/jira/browse/HDDS-4311 Project: Hadoop Distributed Data Store Issue Type: Bug Components: documentation Reporter: Attila Doroszlai Assignee: Attila Doroszlai Abstract and links for http://hadoop.apache.org/ozone/docs/1.0.0/design/typesafeconfig.html are wrong, reference OM HA design doc. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4312) findbugs check succeeds despite failure
Attila Doroszlai created HDDS-4312:
--
Summary: findbugs check succeeds despite failure
Key: HDDS-4312
URL: https://issues.apache.org/jira/browse/HDDS-4312
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: build
Affects Versions: 1.1.0
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai
Findbugs check has been silently failing but reporting success for some time
now. The problem is that {{findbugs.sh}} determines exit code based on the
number of findbugs failures. If {{compile}} step fails, exit code is 0, ie.
success.
{code:title=https://github.com/apache/hadoop-ozone/runs/1210535433#step:3:866}
2020-10-02T18:37:57.0699502Z [ERROR] Failed to execute goal on project
hadoop-hdds-client: Could not resolve dependencies for project
org.apache.hadoop:hadoop-hdds-client:jar:1.1.0-SNAPSHOT: Could not find
artifact org.apache.hadoop:hadoop-hdds-common:jar:tests:1.1.0-SNAPSHOT in
apache.snapshots.https
(https://repository.apache.org/content/repositories/snapshots) -> [Help 1]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4310) Ozone getconf broke the compatibility
[
https://issues.apache.org/jira/browse/HDDS-4310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai resolved HDDS-4310.
Fix Version/s: 1.1.0
Resolution: Fixed
> Ozone getconf broke the compatibility
> -
>
> Key: HDDS-4310
> URL: https://issues.apache.org/jira/browse/HDDS-4310
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone CLI
>Affects Versions: 1.0.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> Currently ozone getconf '-confKey' does not work as 'HDDS-3102' removed the
> need of prepending - with options.
> {code:java}
> RUNNING: /opt/cloudera/parcels/CDH/bin/ozone getconf -confKey
> ozone.om.service.ids 2020-10-05 19:10:09,110|INFO|MainThread|machine.py:180 -
> run()||GUID=8644ce5b-cfe9-4e6b-9b3f-55c29c950489|Unknown options: '-confKey',
> 'ozone.om.service.ids' 2020-10-05 19:10:09,111|INFO|MainThread|machine.py:180
> - run()||GUID=8644ce5b-cfe9-4e6b-9b3f-55c29c950489|Possible solutions: -conf
> {code}
> There are some users which did the automation with the commands and this
> change broke them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4310) Ozone getconf broke the compatibility
[
https://issues.apache.org/jira/browse/HDDS-4310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4310:
---
Labels: (was: pull-request-available)
> Ozone getconf broke the compatibility
> -
>
> Key: HDDS-4310
> URL: https://issues.apache.org/jira/browse/HDDS-4310
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone CLI
>Affects Versions: 1.0.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
>Priority: Major
> Fix For: 1.1.0
>
>
> Currently ozone getconf '-confKey' does not work as 'HDDS-3102' removed the
> need of prepending - with options.
> {code:java}
> RUNNING: /opt/cloudera/parcels/CDH/bin/ozone getconf -confKey
> ozone.om.service.ids 2020-10-05 19:10:09,110|INFO|MainThread|machine.py:180 -
> run()||GUID=8644ce5b-cfe9-4e6b-9b3f-55c29c950489|Unknown options: '-confKey',
> 'ozone.om.service.ids' 2020-10-05 19:10:09,111|INFO|MainThread|machine.py:180
> - run()||GUID=8644ce5b-cfe9-4e6b-9b3f-55c29c950489|Possible solutions: -conf
> {code}
> There are some users which did the automation with the commands and this
> change broke them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4302) Shade the org.apache.common.lang3 package as this is coming from other hadoop packages as well.
[ https://issues.apache.org/jira/browse/HDDS-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4302. Fix Version/s: 1.1.0 Resolution: Fixed > Shade the org.apache.common.lang3 package as this is coming from other hadoop > packages as well. > --- > > Key: HDDS-4302 > URL: https://issues.apache.org/jira/browse/HDDS-4302 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > In one of our duplicate classes tests, we noticed the duplicate classes > because of commons-lang3. To avoid class collisions, it's good to shade the > common-lang3 package as well. > java.lang.Exception: Duplicate class > 'org.apache.commons.lang3.arch.Processor$Arch.class' detected in > '/Users/umagangumalla/Work/repos/Gerrit/xxx/xxx/target/xxx-client-x2.3-dependencies/hadoop-ozone-filesystem-hadoop3-.jar', > class is already present in > '/Users/umagangumalla/Work/repos/Gerrit/xxx/xxx/target/xxx-client-xxx-dependencies/commons-lang3-3.9.jar' -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4299) Display Ratis version with ozone version
[ https://issues.apache.org/jira/browse/HDDS-4299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4299: --- Labels: (was: pull-request-available) > Display Ratis version with ozone version > > > Key: HDDS-4299 > URL: https://issues.apache.org/jira/browse/HDDS-4299 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Major > Fix For: 1.1.0 > > > During the development Ozone uses snapshot releases from Ratis. It can be > useful to print out the exact version of the used Ratis as part of the output > of "ozone version". > Ratis versions are part of the jar files since RATIS-1050 > It can make the testing easier, as it's easier to check which Ratis version > is used. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4299) Display Ratis version with ozone version
[ https://issues.apache.org/jira/browse/HDDS-4299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4299. Fix Version/s: 1.1.0 Resolution: Implemented > Display Ratis version with ozone version > > > Key: HDDS-4299 > URL: https://issues.apache.org/jira/browse/HDDS-4299 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > During the development Ozone uses snapshot releases from Ratis. It can be > useful to print out the exact version of the used Ratis as part of the output > of "ozone version". > Ratis versions are part of the jar files since RATIS-1050 > It can make the testing easier, as it's easier to check which Ratis version > is used. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4290) Enable insight point for SCM heartbeat protocol
[ https://issues.apache.org/jira/browse/HDDS-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4290: --- Labels: (was: pull-request-available) > Enable insight point for SCM heartbeat protocol > --- > > Key: HDDS-4290 > URL: https://issues.apache.org/jira/browse/HDDS-4290 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Major > Fix For: 1.1.0 > > > The registration of the already implemented insigh-tpoint seems to be missing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4290) Enable insight point for SCM heartbeat protocol
[ https://issues.apache.org/jira/browse/HDDS-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4290. Fix Version/s: 1.1.0 Resolution: Fixed > Enable insight point for SCM heartbeat protocol > --- > > Key: HDDS-4290 > URL: https://issues.apache.org/jira/browse/HDDS-4290 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > The registration of the already implemented insigh-tpoint seems to be missing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDDS-4285) Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()
[
https://issues.apache.org/jira/browse/HDDS-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai reassigned HDDS-4285:
--
Assignee: Attila Doroszlai (was: Marton Elek)
> Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()
> --
>
> Key: HDDS-4285
> URL: https://issues.apache.org/jira/browse/HDDS-4285
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Marton Elek
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2020-09-28-16-19-17-581.png,
> profile-20200928-161631-180518.svg
>
>
> Ozone read operation turned out to be slow mainly because we do a new
> UGI.getCurrentUser for block token for each of the calls.
> We need to cache the block token / UGI.getCurrentUserCall() to make it faster.
> !image-2020-09-28-16-19-17-581.png!
> To reproduce:
> Checkout: https://github.com/elek/hadoop-ozone/tree/mocked-read
> {code}
> cd hadoop-ozone/client
> export
> MAVEN_OPTS=-agentpath:/home/elek/prog/async-profiler/build/libasyncProfiler.so=start,file=/tmp/profile-%t-%p.svg
> mvn compile exec:java
> -Dexec.mainClass=org.apache.hadoop.ozone.client.io.TestKeyOutputStreamUnit
> -Dexec.classpathScope=test
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4285) Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()
[
https://issues.apache.org/jira/browse/HDDS-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4285:
---
Summary: Read is slow due to frequent calls to UGI.getCurrentUser() and
getTokens() (was: Read is slow due to the frequent usage of
UGI.getCurrentUserCall())
> Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()
> --
>
> Key: HDDS-4285
> URL: https://issues.apache.org/jira/browse/HDDS-4285
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Marton Elek
>Assignee: Marton Elek
>Priority: Major
> Attachments: image-2020-09-28-16-19-17-581.png,
> profile-20200928-161631-180518.svg
>
>
> Ozone read operation turned out to be slow mainly because we do a new
> UGI.getCurrentUser for block token for each of the calls.
> We need to cache the block token / UGI.getCurrentUserCall() to make it faster.
> !image-2020-09-28-16-19-17-581.png!
> To reproduce:
> Checkout: https://github.com/elek/hadoop-ozone/tree/mocked-read
> {code}
> cd hadoop-ozone/client
> export
> MAVEN_OPTS=-agentpath:/home/elek/prog/async-profiler/build/libasyncProfiler.so=start,file=/tmp/profile-%t-%p.svg
> mvn compile exec:java
> -Dexec.mainClass=org.apache.hadoop.ozone.client.io.TestKeyOutputStreamUnit
> -Dexec.classpathScope=test
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4232) Use single thread for KeyDeletingService
[ https://issues.apache.org/jira/browse/HDDS-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4232: --- Labels: (was: pull-request-available) > Use single thread for KeyDeletingService > > > Key: HDDS-4232 > URL: https://issues.apache.org/jira/browse/HDDS-4232 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Fix For: 1.1.0 > > > KeyDeletingService scan the keys from a particular rocksdb table and sends > deletion request to SCM. Every thread would scan the table and send deletion > requests. This can lead to multiple deletion request for a particular block. > There is currently no way to distribute the keys to be deleted amongst > multiple threads. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4232) Use single thread for KeyDeletingService
[ https://issues.apache.org/jira/browse/HDDS-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4232. Fix Version/s: 1.1.0 Resolution: Fixed > Use single thread for KeyDeletingService > > > Key: HDDS-4232 > URL: https://issues.apache.org/jira/browse/HDDS-4232 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > KeyDeletingService scan the keys from a particular rocksdb table and sends > deletion request to SCM. Every thread would scan the table and send deletion > requests. This can lead to multiple deletion request for a particular block. > There is currently no way to distribute the keys to be deleted amongst > multiple threads. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4231) Background Service blocks on task results
[ https://issues.apache.org/jira/browse/HDDS-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4231: --- Labels: (was: pull-request-available) > Background Service blocks on task results > - > > Key: HDDS-4231 > URL: https://issues.apache.org/jira/browse/HDDS-4231 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Fix For: 1.1.0 > > > Background service currently waits on the results of the tasks. The idea is > to track the time it took for the task to execute and log if task takes more > than configured timeout. > This does not require waiting on the task results and can be achieved by just > comparing the execution time of a task with the timeout value. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4231) Background Service blocks on task results
[ https://issues.apache.org/jira/browse/HDDS-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4231: --- Fix Version/s: 1.1.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Background Service blocks on task results > - > > Key: HDDS-4231 > URL: https://issues.apache.org/jira/browse/HDDS-4231 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > Background service currently waits on the results of the tasks. The idea is > to track the time it took for the task to execute and log if task takes more > than configured timeout. > This does not require waiting on the task results and can be achieved by just > comparing the execution time of a task with the timeout value. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-4283) Remove unsupported upgrade command in ozone cli
[
https://issues.apache.org/jira/browse/HDDS-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203077#comment-17203077
]
Attila Doroszlai commented on HDDS-4283:
I think this was already removed in HDDS-3992.
> Remove unsupported upgrade command in ozone cli
> ---
>
> Key: HDDS-4283
> URL: https://issues.apache.org/jira/browse/HDDS-4283
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
>Priority: Minor
>
> In HDDS-1383, we introduce a new upgrade command for supporting to in-place
> upgrade from HDFS to Ozone.
> {noformat}
> upgrade HDFS to Ozone in-place upgrade tool
>
> Usage: ozone upgrade [-hV] [--verbose] [-conf=]
> [-D=]... [COMMAND]
> Convert raw HDFS data to Ozone data without data movement.
> --verbose More verbose output. Show the stack trace of the errors.
> -conf=
> -D, --set=
> -h, --help Show this help message and exit.
> -V, --version Print version information and exit.
> Commands:
> plan Plan existing HDFS block distribution and give.estimation.
> balance Move the HDFS blocks for a better distribution usage.
> execute Start/restart upgrade from HDFS to Ozone cluster.
> [hdfs@lyq ~]$ ~/ozone/bin/ozone upgrade plan
> [In-Place upgrade : plan] is not yet supported.
> [hdfs@lyq ~]$ ~/ozone/bin/ozone upgrade balance
> [In-Place upgrade : balance] is not yet supported.
> [hdfs@lyq ~]$ ~/ozone/bin/ozone upgrade execute
> In-Place upgrade : execute] is not yet supported
> {noformat}
> But this feature has not been implemented yet and is a very big feature.
> I don't think it's good to expose a cli command that is not supported and
> meanwhile that cannot be quickly implemented in the short term.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4194) Create a script to check AWS S3 compatibility
[ https://issues.apache.org/jira/browse/HDDS-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4194: --- Labels: (was: pull-request-available) > Create a script to check AWS S3 compatibility > - > > Key: HDDS-4194 > URL: https://issues.apache.org/jira/browse/HDDS-4194 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Major > Fix For: 1.1.0 > > > Ozone S3G implements the REST interface of AWS S3 protocol. Our robot test > based scripts check if it's possible to use Ozone S3 with the AWS client tool. > But occasionally we should check if our robot test definitions are valid: > robot tests should be executed with using real AWS endpoint and bucket(s) and > all the test cases should be passed. > This patch provides a simple shell script to make this cross-check easier. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4194) Create a script to check AWS S3 compatibility
[ https://issues.apache.org/jira/browse/HDDS-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4194. Fix Version/s: 1.1.0 Resolution: Implemented > Create a script to check AWS S3 compatibility > - > > Key: HDDS-4194 > URL: https://issues.apache.org/jira/browse/HDDS-4194 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > Ozone S3G implements the REST interface of AWS S3 protocol. Our robot test > based scripts check if it's possible to use Ozone S3 with the AWS client tool. > But occasionally we should check if our robot test definitions are valid: > robot tests should be executed with using real AWS endpoint and bucket(s) and > all the test cases should be passed. > This patch provides a simple shell script to make this cross-check easier. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4270) Add more reusable byteman scripts to debug ofs/o3fs performance
[ https://issues.apache.org/jira/browse/HDDS-4270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4270: --- Labels: (was: pull-request-available) > Add more reusable byteman scripts to debug ofs/o3fs performance > --- > > Key: HDDS-4270 > URL: https://issues.apache.org/jira/browse/HDDS-4270 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Major > Fix For: 1.1.0 > > > I am using https://byteman.jboss.org to debug the performance of spark + > terage with different scripts. Some byteman scripts are already shared by > HDDS-4095 or HDDS-342 but it seems to be a good practice to share the newer > scripts to make it possible to reproduce performance problems. > For using byteman with Ozone, see this video: > https://www.youtube.com/watch?v=_4eYsH8F50E=PLCaV-jpCBO8U_WqyySszmbmnL-dhlzF6o=5 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4270) Add more reusable byteman scripts to debug ofs/o3fs performance
[ https://issues.apache.org/jira/browse/HDDS-4270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4270. Fix Version/s: 1.1.0 Resolution: Implemented > Add more reusable byteman scripts to debug ofs/o3fs performance > --- > > Key: HDDS-4270 > URL: https://issues.apache.org/jira/browse/HDDS-4270 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > I am using https://byteman.jboss.org to debug the performance of spark + > terage with different scripts. Some byteman scripts are already shared by > HDDS-4095 or HDDS-342 but it seems to be a good practice to share the newer > scripts to make it possible to reproduce performance problems. > For using byteman with Ozone, see this video: > https://www.youtube.com/watch?v=_4eYsH8F50E=PLCaV-jpCBO8U_WqyySszmbmnL-dhlzF6o=5 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-2660) Create insight point for datanode container protocol
[
https://issues.apache.org/jira/browse/HDDS-2660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-2660:
---
Labels: (was: pull-request-available)
> Create insight point for datanode container protocol
>
>
> Key: HDDS-2660
> URL: https://issues.apache.org/jira/browse/HDDS-2660
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: Ozone Datanode
>Reporter: Attila Doroszlai
>Assignee: Marton Elek
>Priority: Major
> Fix For: 1.1.0
>
>
> The goal of this task is to create a new insight point for the datanode
> container protocol ({{HddsDispatcher}}) to be able to debug
> {{client<->datanode}} communication.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-2660) Create insight point for datanode container protocol
[
https://issues.apache.org/jira/browse/HDDS-2660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai resolved HDDS-2660.
Fix Version/s: 1.1.0
Resolution: Implemented
> Create insight point for datanode container protocol
>
>
> Key: HDDS-2660
> URL: https://issues.apache.org/jira/browse/HDDS-2660
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: Ozone Datanode
>Reporter: Attila Doroszlai
>Assignee: Marton Elek
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> The goal of this task is to create a new insight point for the datanode
> container protocol ({{HddsDispatcher}}) to be able to debug
> {{client<->datanode}} communication.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4271) Avoid logging chunk content in Ozone Insight
Attila Doroszlai created HDDS-4271: -- Summary: Avoid logging chunk content in Ozone Insight Key: HDDS-4271 URL: https://issues.apache.org/jira/browse/HDDS-4271 Project: Hadoop Distributed Data Store Issue Type: Improvement Components: Tools Reporter: Attila Doroszlai Assignee: Attila Doroszlai HDDS-2660 added an insight point for the datanode dispatcher. At trace level it logs all chunk content, which can be huge and contain control characters, so I think we should avoid it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4197) Failed to load existing service definition files: ...SubcommandWithParent
[
https://issues.apache.org/jira/browse/HDDS-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4197:
---
Labels: jdk11 (was: jdk11 pull-request-available)
> Failed to load existing service definition files: ...SubcommandWithParent
> -
>
> Key: HDDS-4197
> URL: https://issues.apache.org/jira/browse/HDDS-4197
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: build
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: jdk11
> Fix For: 1.1.0
>
>
> {code}
> [INFO] Apache Hadoop HDDS Tools ... FAILURE
> ...
> [ERROR] Failed to load existing service definition files:
> java.nio.file.NoSuchFileException:
> hadoop-hdds/tools/target/classes/META-INF/services/org.apache.hadoop.hdds.cli.SubcommandWithParent
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4206) Attempt pipeline creation more frequently in acceptance tests
[ https://issues.apache.org/jira/browse/HDDS-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4206: --- Fix Version/s: 1.1.0 Resolution: Implemented Status: Resolved (was: Patch Available) > Attempt pipeline creation more frequently in acceptance tests > - > > Key: HDDS-4206 > URL: https://issues.apache.org/jira/browse/HDDS-4206 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: test >Reporter: Attila Doroszlai >Assignee: Attila Doroszlai >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > Acceptance tests are frequently failing recently due to SCM not being able to > exit safe mode for 3 minutes. Background pipeline creator attempts pipeline > creation every 2 minutes. It is safe to decrease this period in tests for > faster startup and less idle wait time. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4206) Attempt pipeline creation more frequently in acceptance tests
[ https://issues.apache.org/jira/browse/HDDS-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4206: --- Labels: (was: pull-request-available) > Attempt pipeline creation more frequently in acceptance tests > - > > Key: HDDS-4206 > URL: https://issues.apache.org/jira/browse/HDDS-4206 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: test >Reporter: Attila Doroszlai >Assignee: Attila Doroszlai >Priority: Major > Fix For: 1.1.0 > > > Acceptance tests are frequently failing recently due to SCM not being able to > exit safe mode for 3 minutes. Background pipeline creator attempts pipeline > creation every 2 minutes. It is safe to decrease this period in tests for > faster startup and less idle wait time. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4194) Create a script to check AWS S3 compatibility
[ https://issues.apache.org/jira/browse/HDDS-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4194: --- Summary: Create a script to check AWS S3 compatibility (was: Create a script to check AWS S3 compatiblity) > Create a script to check AWS S3 compatibility > - > > Key: HDDS-4194 > URL: https://issues.apache.org/jira/browse/HDDS-4194 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Major > Labels: pull-request-available > > Ozone S3G implements the REST interface of AWS S3 protocol. Our robot test > based scripts check if it's possible to use Ozone S3 with the AWS client tool. > But occasionally we should check if our robot test definitions are valid: > robot tests should be executed with using real AWS endpoint and bucket(s) and > all the test cases should be passed. > This patch provides a simple shell script to make this cross-check easier. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4233) Interrupted exeception printed out from DatanodeStateMachine
[
https://issues.apache.org/jira/browse/HDDS-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4233:
---
Labels: (was: pull-request-available)
> Interrupted exeception printed out from DatanodeStateMachine
>
>
> Key: HDDS-4233
> URL: https://issues.apache.org/jira/browse/HDDS-4233
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Marton Elek
>Assignee: Marton Elek
>Priority: Major
> Fix For: 1.1.0
>
>
> A strange exception is visible in the log during normal run:
> {code}
> 2020-09-10 11:31:41 WARN DatanodeStateMachine:245 - Interrupt the execution.
> java.lang.InterruptedException: sleep interrupted
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.start(DatanodeStateMachine.java:243)
> at
> org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$startDaemon$0(DatanodeStateMachine.java:405)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The most common reason to this is triggering a new HB request.
> As this is a normal behavior we shouldn't log exception on WARN level.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4233) Interrupted exeception printed out from DatanodeStateMachine
[
https://issues.apache.org/jira/browse/HDDS-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai resolved HDDS-4233.
Fix Version/s: 1.1.0
Resolution: Done
> Interrupted exeception printed out from DatanodeStateMachine
>
>
> Key: HDDS-4233
> URL: https://issues.apache.org/jira/browse/HDDS-4233
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Marton Elek
>Assignee: Marton Elek
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> A strange exception is visible in the log during normal run:
> {code}
> 2020-09-10 11:31:41 WARN DatanodeStateMachine:245 - Interrupt the execution.
> java.lang.InterruptedException: sleep interrupted
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.start(DatanodeStateMachine.java:243)
> at
> org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$startDaemon$0(DatanodeStateMachine.java:405)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The most common reason to this is triggering a new HB request.
> As this is a normal behavior we shouldn't log exception on WARN level.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3947) Sort DNs for client when the key is a file for #getFileStatus #listStatus APIs
[
https://issues.apache.org/jira/browse/HDDS-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-3947:
---
Labels: (was: pull-request-available)
> Sort DNs for client when the key is a file for #getFileStatus #listStatus APIs
> --
>
> Key: HDDS-3947
> URL: https://issues.apache.org/jira/browse/HDDS-3947
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Manager
>Reporter: Rakesh Radhakrishnan
>Assignee: Rakesh Radhakrishnan
>Priority: Major
> Fix For: 1.1.0
>
>
> Similar to {{OzoneManagerFS#lookupFile(OmKeyArgs args, String
> clientAddress)}}, it would be good to sort the DNs for the given client
> address when the key is a file.
> Thanks [~xyao] for your comments in
> [HDDS-3824-PR|https://github.com/apache/hadoop-ozone/pull/1164].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-3947) Sort DNs for client when the key is a file for #getFileStatus #listStatus APIs
[
https://issues.apache.org/jira/browse/HDDS-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai resolved HDDS-3947.
Fix Version/s: 1.1.0
Resolution: Implemented
> Sort DNs for client when the key is a file for #getFileStatus #listStatus APIs
> --
>
> Key: HDDS-3947
> URL: https://issues.apache.org/jira/browse/HDDS-3947
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Manager
>Reporter: Rakesh Radhakrishnan
>Assignee: Rakesh Radhakrishnan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> Similar to {{OzoneManagerFS#lookupFile(OmKeyArgs args, String
> clientAddress)}}, it would be good to sort the DNs for the given client
> address when the key is a file.
> Thanks [~xyao] for your comments in
> [HDDS-3824-PR|https://github.com/apache/hadoop-ozone/pull/1164].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4255) Remove unused Ant and Jdiff dependency versions
[ https://issues.apache.org/jira/browse/HDDS-4255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4255. Fix Version/s: 1.1.0 Resolution: Implemented > Remove unused Ant and Jdiff dependency versions > --- > > Key: HDDS-4255 > URL: https://issues.apache.org/jira/browse/HDDS-4255 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > Versions of Ant and JDiff are not used in ozone project, but we have some > version declaration (inherited from the Hadoo parent pom which was used as a > base for the main pom.xml). > As the (unused) ANT version has security issues, I would remove them to avoid > any confusion -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4255) Remove unused Ant and Jdiff dependency versions
[ https://issues.apache.org/jira/browse/HDDS-4255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4255: --- Labels: (was: pull-request-available) > Remove unused Ant and Jdiff dependency versions > --- > > Key: HDDS-4255 > URL: https://issues.apache.org/jira/browse/HDDS-4255 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Major > Fix For: 1.1.0 > > > Versions of Ant and JDiff are not used in ozone project, but we have some > version declaration (inherited from the Hadoo parent pom which was used as a > base for the main pom.xml). > As the (unused) ANT version has security issues, I would remove them to avoid > any confusion -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4247) Fixed log4j usage in some places
[
https://issues.apache.org/jira/browse/HDDS-4247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4247:
---
Labels: (was: pull-request-available)
> Fixed log4j usage in some places
>
>
> Key: HDDS-4247
> URL: https://issues.apache.org/jira/browse/HDDS-4247
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: build
>Affects Versions: 1.0.0
>Reporter: Xie Lei
>Assignee: Xie Lei
>Priority: Minor
> Fix For: 1.1.0
>
>
> Fixed log4j usage in some places.
> examples, {} is redundant.
> {code:java}
> LOG.error("Error while scrubbing pipelines {}", e);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4247) Fixed log4j usage in some places
[
https://issues.apache.org/jira/browse/HDDS-4247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai resolved HDDS-4247.
Resolution: Fixed
> Fixed log4j usage in some places
>
>
> Key: HDDS-4247
> URL: https://issues.apache.org/jira/browse/HDDS-4247
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: build
>Affects Versions: 1.0.0
>Reporter: Xie Lei
>Assignee: Xie Lei
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> Fixed log4j usage in some places.
> examples, {} is redundant.
> {code:java}
> LOG.error("Error while scrubbing pipelines {}", e);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDDS-4250) Fix wrong logger name
[
https://issues.apache.org/jira/browse/HDDS-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai reassigned HDDS-4250:
--
Assignee: Xie Lei
> Fix wrong logger name
> -
>
> Key: HDDS-4250
> URL: https://issues.apache.org/jira/browse/HDDS-4250
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: build
>Affects Versions: 1.1.0
>Reporter: Xie Lei
>Assignee: Xie Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> Fix wrong logger name, the logger name doesn't match the class name.
> example
> {code:java}
> public class OMBucketSetAclRequest extends OMBucketAclRequest {
> private static final Logger LOG =
> LoggerFactory.getLogger(OMBucketAddAclRequest.class);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDDS-4247) Fixed log4j usage in some places
[
https://issues.apache.org/jira/browse/HDDS-4247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai reassigned HDDS-4247:
--
Assignee: Xie Lei
> Fixed log4j usage in some places
>
>
> Key: HDDS-4247
> URL: https://issues.apache.org/jira/browse/HDDS-4247
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: build
>Affects Versions: 1.0.0
>Reporter: Xie Lei
>Assignee: Xie Lei
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> Fixed log4j usage in some places.
> examples, {} is redundant.
> {code:java}
> LOG.error("Error while scrubbing pipelines {}", e);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4104) Provide a way to get the default value and key of java-based-configuration easily
[ https://issues.apache.org/jira/browse/HDDS-4104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4104: --- Labels: (was: pull-request-available) > Provide a way to get the default value and key of java-based-configuration > easily > - > > Key: HDDS-4104 > URL: https://issues.apache.org/jira/browse/HDDS-4104 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Affects Versions: 1.0.0 >Reporter: maobaolong >Assignee: maobaolong >Priority: Minor > Fix For: 1.1.0 > > > - getDefaultValue > - getKeyName -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4104) Provide a way to get the default value and key of java-based-configuration easily
[ https://issues.apache.org/jira/browse/HDDS-4104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4104. Fix Version/s: 1.1.0 Resolution: Implemented > Provide a way to get the default value and key of java-based-configuration > easily > - > > Key: HDDS-4104 > URL: https://issues.apache.org/jira/browse/HDDS-4104 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Affects Versions: 1.0.0 >Reporter: maobaolong >Assignee: maobaolong >Priority: Minor > Labels: pull-request-available > Fix For: 1.1.0 > > > - getDefaultValue > - getKeyName -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-4166) Documentation index page redirects to the wrong address
[ https://issues.apache.org/jira/browse/HDDS-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195990#comment-17195990 ] Attila Doroszlai commented on HDDS-4166: Thanks [~linyiqun] for reporting this issue, [~cku328] for fixing it, and [~cxorm] for the review. > Documentation index page redirects to the wrong address > --- > > Key: HDDS-4166 > URL: https://issues.apache.org/jira/browse/HDDS-4166 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: documentation >Reporter: Yiqun Lin >Assignee: Neo Yang >Priority: Minor > Fix For: 1.1.0 > > Attachments: image-2020-08-29-10-35-34-633.png > > > Reading Chinese doc of Ozone that introduced in HDDS-2708. I find a page > error that index page redirects a wrong page. > The steps: > 1. Jump into > [https://ci-hadoop.apache.org/view/Hadoop%20Ozone/job/ozone-doc-master/lastSuccessfulBuild/artifact/hadoop-hdds/docs/public/index.html] > 2. Click the page switch button. > 3. The wrong page we will jumped into: > > [https://ci-hadoop.apache.org/view/Hadoop%20Ozone/job/ozone-doc-master/lastSuccessfulBuild/artifact/hadoop-hdds/docs/public/zh/] > !image-2020-08-29-10-35-34-633.png! > It missed the 'index.html' at the end of the address, actually this address > is expected to > [https://ci-hadoop.apache.org/view/Hadoop%20Ozone/job/ozone-doc-master/lastSuccessfulBuild/artifact/hadoop-hdds/docs/public/zh/index.html] > The same error happened when I switch the index page from Chinese doc to > English doc. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4166) Documentation index page redirects to the wrong address
[ https://issues.apache.org/jira/browse/HDDS-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4166. Fix Version/s: 1.1.0 Resolution: Fixed > Documentation index page redirects to the wrong address > --- > > Key: HDDS-4166 > URL: https://issues.apache.org/jira/browse/HDDS-4166 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: documentation >Reporter: Yiqun Lin >Assignee: Neo Yang >Priority: Minor > Labels: pull-request-available > Fix For: 1.1.0 > > Attachments: image-2020-08-29-10-35-34-633.png > > > Reading Chinese doc of Ozone that introduced in HDDS-2708. I find a page > error that index page redirects a wrong page. > The steps: > 1. Jump into > [https://ci-hadoop.apache.org/view/Hadoop%20Ozone/job/ozone-doc-master/lastSuccessfulBuild/artifact/hadoop-hdds/docs/public/index.html] > 2. Click the page switch button. > 3. The wrong page we will jumped into: > > [https://ci-hadoop.apache.org/view/Hadoop%20Ozone/job/ozone-doc-master/lastSuccessfulBuild/artifact/hadoop-hdds/docs/public/zh/] > !image-2020-08-29-10-35-34-633.png! > It missed the 'index.html' at the end of the address, actually this address > is expected to > [https://ci-hadoop.apache.org/view/Hadoop%20Ozone/job/ozone-doc-master/lastSuccessfulBuild/artifact/hadoop-hdds/docs/public/zh/index.html] > The same error happened when I switch the index page from Chinese doc to > English doc. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4166) Documentation index page redirects to the wrong address
[ https://issues.apache.org/jira/browse/HDDS-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4166: --- Labels: (was: pull-request-available) > Documentation index page redirects to the wrong address > --- > > Key: HDDS-4166 > URL: https://issues.apache.org/jira/browse/HDDS-4166 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: documentation >Reporter: Yiqun Lin >Assignee: Neo Yang >Priority: Minor > Fix For: 1.1.0 > > Attachments: image-2020-08-29-10-35-34-633.png > > > Reading Chinese doc of Ozone that introduced in HDDS-2708. I find a page > error that index page redirects a wrong page. > The steps: > 1. Jump into > [https://ci-hadoop.apache.org/view/Hadoop%20Ozone/job/ozone-doc-master/lastSuccessfulBuild/artifact/hadoop-hdds/docs/public/index.html] > 2. Click the page switch button. > 3. The wrong page we will jumped into: > > [https://ci-hadoop.apache.org/view/Hadoop%20Ozone/job/ozone-doc-master/lastSuccessfulBuild/artifact/hadoop-hdds/docs/public/zh/] > !image-2020-08-29-10-35-34-633.png! > It missed the 'index.html' at the end of the address, actually this address > is expected to > [https://ci-hadoop.apache.org/view/Hadoop%20Ozone/job/ozone-doc-master/lastSuccessfulBuild/artifact/hadoop-hdds/docs/public/zh/index.html] > The same error happened when I switch the index page from Chinese doc to > English doc. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4110) Intermittent failure in TestEndpoints#testGetDatanodes
[
https://issues.apache.org/jira/browse/HDDS-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai resolved HDDS-4110.
Assignee: Vivek Ratnavel Subramanian
Resolution: Duplicate
> Intermittent failure in TestEndpoints#testGetDatanodes
> --
>
> Key: HDDS-4110
> URL: https://issues.apache.org/jira/browse/HDDS-4110
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Recon, test
>Reporter: Attila Doroszlai
>Assignee: Vivek Ratnavel Subramanian
>Priority: Major
>
> {code}
> [ERROR] Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 10.693 s <<< FAILURE! - in org.apache.hadoop.ozone.recon.api.TestEndpoints
> [ERROR] testGetDatanodes(org.apache.hadoop.ozone.recon.api.TestEndpoints)
> Time elapsed: 0.877 s <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<0>
> ...
> at
> org.apache.hadoop.ozone.recon.api.TestEndpoints.testGetDatanodes(TestEndpoints.java:377)
> {code}
> https://github.com/elek/ozone-build-results/tree/master/2020/08/11/2337/unit/hadoop-ozone/recon/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4206) Attempt pipeline creation more frequently in acceptance tests
[ https://issues.apache.org/jira/browse/HDDS-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4206: --- Status: Patch Available (was: In Progress) > Attempt pipeline creation more frequently in acceptance tests > - > > Key: HDDS-4206 > URL: https://issues.apache.org/jira/browse/HDDS-4206 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: test >Reporter: Attila Doroszlai >Assignee: Attila Doroszlai >Priority: Major > Labels: pull-request-available > > Acceptance tests are frequently failing recently due to SCM not being able to > exit safe mode for 3 minutes. Background pipeline creator attempts pipeline > creation every 2 minutes. It is safe to decrease this period in tests for > faster startup and less idle wait time. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4204) upgrade docker environment does not work with KEEP_RUNNING=true
[
https://issues.apache.org/jira/browse/HDDS-4204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4204:
---
Status: Patch Available (was: In Progress)
> upgrade docker environment does not work with KEEP_RUNNING=true
> ---
>
> Key: HDDS-4204
> URL: https://issues.apache.org/jira/browse/HDDS-4204
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: docker, test
>Affects Versions: 1.0.0
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
> Labels: pull-request-available
>
> Ozone {{upgrade}} Docker Compose environment fails if run with
> {{KEEP_RUNNING=true}}. The variable is applied to both runs (pre- and
> post-upgrade), but pre-upgrade containers should be stopped anyway, since
> they will be replaced by the new ones.
> {code}
> $ cd hadoop-ozone/dist/target/ozone-1.1.0-SNAPSHOT/compose/upgrade
> $ KEEP_RUNNING=true ./test.sh
> ...
> Failed: IO error: While lock file: scm.db/LOCK: Resource temporarily
> unavailable
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4198) Compile Ozone with multiple Java versions
[ https://issues.apache.org/jira/browse/HDDS-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4198: --- Status: Patch Available (was: In Progress) > Compile Ozone with multiple Java versions > - > > Key: HDDS-4198 > URL: https://issues.apache.org/jira/browse/HDDS-4198 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: build >Reporter: Attila Doroszlai >Assignee: Attila Doroszlai >Priority: Major > Labels: pull-request-available > > Add matrix build in GitHub Actions for compiling Ozone with both Java 8 and > 11. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4205) Fix or disable coverage upload to codecov (pull requests)
[
https://issues.apache.org/jira/browse/HDDS-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4205:
---
Labels: (was: pull-request-available)
> Fix or disable coverage upload to codecov (pull requests)
> -
>
> Key: HDDS-4205
> URL: https://issues.apache.org/jira/browse/HDDS-4205
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Marton Elek
>Assignee: Marton Elek
>Priority: Major
> Fix For: 1.1.0
>
>
> I would like to start a conversation about disabling the coverage data
> uploading to codecov
> 1. It seems to be unreliable, HTTP 400 is a very common answer. I checked
> and the @v1 is the latest version from the action
> {code}
> -> Pinging Codecov
> https://codecov.io/upload/v4?package=bash-20200825-997b141=secret=HDDS-4193=361ddccd0666a8bd24240cda2652a903d4b69013=237476928_url=http%3A%2F%2Fgithub.com%2Fapache%2Fhadoop-ozone%2Factions%2Fruns%2F237476928=codecov-umbrella==apache%2Fhadoop-ozone=github-actions==1384=_args=f,n,F,Z
> HTTP 400
> {code}
> 2. The reported coverage data is meaningless as there is no good mechanism
> to use the latest stable build as baseline.
> I think we should disable codecov (sonar is still available) until we will
> have a solution for these issue.
> [~adoroszlai], [~vivekratnavel]: What do you think?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4205) Fix or disable coverage upload to codecov (pull requests)
[
https://issues.apache.org/jira/browse/HDDS-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai resolved HDDS-4205.
Fix Version/s: 1.1.0
Resolution: Done
> Fix or disable coverage upload to codecov (pull requests)
> -
>
> Key: HDDS-4205
> URL: https://issues.apache.org/jira/browse/HDDS-4205
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Marton Elek
>Assignee: Marton Elek
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> I would like to start a conversation about disabling the coverage data
> uploading to codecov
> 1. It seems to be unreliable, HTTP 400 is a very common answer. I checked
> and the @v1 is the latest version from the action
> {code}
> -> Pinging Codecov
> https://codecov.io/upload/v4?package=bash-20200825-997b141=secret=HDDS-4193=361ddccd0666a8bd24240cda2652a903d4b69013=237476928_url=http%3A%2F%2Fgithub.com%2Fapache%2Fhadoop-ozone%2Factions%2Fruns%2F237476928=codecov-umbrella==apache%2Fhadoop-ozone=github-actions==1384=_args=f,n,F,Z
> HTTP 400
> {code}
> 2. The reported coverage data is meaningless as there is no good mechanism
> to use the latest stable build as baseline.
> I think we should disable codecov (sonar is still available) until we will
> have a solution for these issue.
> [~adoroszlai], [~vivekratnavel]: What do you think?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4201) Improve the performance of OmKeyLocationInfoGroup
[ https://issues.apache.org/jira/browse/HDDS-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4201. Fix Version/s: 1.1.0 Resolution: Implemented > Improve the performance of OmKeyLocationInfoGroup > - > > Key: HDDS-4201 > URL: https://issues.apache.org/jira/browse/HDDS-4201 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: runzhiwang >Assignee: runzhiwang >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4201) Improve the performance of OmKeyLocationInfoGroup
[ https://issues.apache.org/jira/browse/HDDS-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4201: --- Labels: (was: pull-request-available) > Improve the performance of OmKeyLocationInfoGroup > - > > Key: HDDS-4201 > URL: https://issues.apache.org/jira/browse/HDDS-4201 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: runzhiwang >Assignee: runzhiwang >Priority: Major > Fix For: 1.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4207) Include version information in docs
Attila Doroszlai created HDDS-4207: -- Summary: Include version information in docs Key: HDDS-4207 URL: https://issues.apache.org/jira/browse/HDDS-4207 Project: Hadoop Distributed Data Store Issue Type: Improvement Components: documentation Reporter: Attila Doroszlai Assignee: Attila Doroszlai Ozone docs should include version information from POM (eg. "1.0.0") and from version control if available (short hash, last change date, etc.). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4206) Attempt pipeline creation more frequently in acceptance tests
Attila Doroszlai created HDDS-4206: -- Summary: Attempt pipeline creation more frequently in acceptance tests Key: HDDS-4206 URL: https://issues.apache.org/jira/browse/HDDS-4206 Project: Hadoop Distributed Data Store Issue Type: Improvement Components: test Reporter: Attila Doroszlai Assignee: Attila Doroszlai Acceptance tests are frequently failing recently due to SCM not being able to exit safe mode for 3 minutes. Background pipeline creator attempts pipeline creation every 2 minutes. It is safe to decrease this period in tests for faster startup and less idle wait time. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3840) [OFS] Use OFS in MapReduce acceptance test
[ https://issues.apache.org/jira/browse/HDDS-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3840: --- Labels: (was: pull-request-available) > [OFS] Use OFS in MapReduce acceptance test > -- > > Key: HDDS-3840 > URL: https://issues.apache.org/jira/browse/HDDS-3840 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Attila Doroszlai >Priority: Major > Fix For: 1.1.0 > > > [mapreduce.robot|https://github.com/apache/hadoop-ozone/blob/072370b947416d89fae11d00a84a1d9a6b31beaa/hadoop-ozone/dist/src/main/smoketest/mapreduce.robot#L42] > currently uses o3fs. > We should run MR acceptance with OFS as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3840) [OFS] Use OFS in MapReduce acceptance test
[ https://issues.apache.org/jira/browse/HDDS-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3840: --- Fix Version/s: 1.1.0 Resolution: Implemented Status: Resolved (was: Patch Available) > [OFS] Use OFS in MapReduce acceptance test > -- > > Key: HDDS-3840 > URL: https://issues.apache.org/jira/browse/HDDS-3840 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Attila Doroszlai >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > [mapreduce.robot|https://github.com/apache/hadoop-ozone/blob/072370b947416d89fae11d00a84a1d9a6b31beaa/hadoop-ozone/dist/src/main/smoketest/mapreduce.robot#L42] > currently uses o3fs. > We should run MR acceptance with OFS as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-4205) Fix or disable coverage upload to codecov (pull requests)
[
https://issues.apache.org/jira/browse/HDDS-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190190#comment-17190190
]
Attila Doroszlai commented on HDDS-4205:
I agree that the action is unreliable and its comments on PRs are mostly
useless. I think we should still try to upload to Codecov, but avoid failing
the build on error. Its visualization and/or browser might be a useful
alternative to Sonar.
> Fix or disable coverage upload to codecov (pull requests)
> -
>
> Key: HDDS-4205
> URL: https://issues.apache.org/jira/browse/HDDS-4205
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Marton Elek
>Assignee: Marton Elek
>Priority: Major
>
> I would like to start a conversation about disabling the coverage data
> uploading to codecov
> 1. It seems to be unreliable, HTTP 400 is a very common answer. I checked
> and the @v1 is the latest version from the action
> {code}
> -> Pinging Codecov
> https://codecov.io/upload/v4?package=bash-20200825-997b141=secret=HDDS-4193=361ddccd0666a8bd24240cda2652a903d4b69013=237476928_url=http%3A%2F%2Fgithub.com%2Fapache%2Fhadoop-ozone%2Factions%2Fruns%2F237476928=codecov-umbrella==apache%2Fhadoop-ozone=github-actions==1384=_args=f,n,F,Z
> HTTP 400
> {code}
> 2. The reported coverage data is meaningless as there is no good mechanism
> to use the latest stable build as baseline.
> I think we should disable codecov (sonar is still available) until we will
> have a solution for these issue.
> [~adoroszlai], [~vivekratnavel]: What do you think?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4204) upgrade docker environment does not work with KEEP_RUNNING=true
Attila Doroszlai created HDDS-4204:
--
Summary: upgrade docker environment does not work with
KEEP_RUNNING=true
Key: HDDS-4204
URL: https://issues.apache.org/jira/browse/HDDS-4204
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: docker, test
Affects Versions: 1.0.0
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai
Ozone {{upgrade}} Docker Compose environment fails if run with
{{KEEP_RUNNING=true}}. The variable is applied to both runs (pre- and
post-upgrade), but pre-upgrade containers should be stopped anyway, since they
will be replaced by the new ones.
{code}
$ cd hadoop-ozone/dist/target/ozone-1.1.0-SNAPSHOT/compose/upgrade
$ KEEP_RUNNING=true ./test.sh
...
Failed: IO error: While lock file: scm.db/LOCK: Resource temporarily unavailable
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4190) Intermittent failure in TestOMVolumeSetOwnerRequest and TestOMVolumeSetQuotaRequest
[
https://issues.apache.org/jira/browse/HDDS-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4190:
---
Description:
Original issue:
{code:title=https://github.com/elek/ozone-build-results/blob/master/2020/09/01/2686/unit/hadoop-ozone/ozone-manager/org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.txt}
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.214 s <<<
FAILURE! - in org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest
testValidateAndUpdateCache(org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest)
Time elapsed: 0.129 s <<< FAILURE!
java.lang.AssertionError: Values should be different. Actual: 1598964934681
...
at org.junit.Assert.assertNotEquals(Assert.java:209)
at
org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.testValidateAndUpdateCache(TestOMAllocateBlockRequest.java:100)
{code}
Remaining issue:
Similar assertions failing locally in:
*
TestOMVolumeSetQuotaRequest.testValidateAndUpdateCacheSuccess(TestOMVolumeSetQuotaRequest.java:101)
*
TestOMVolumeSetOwnerRequest.testValidateAndUpdateCacheSuccess(TestOMVolumeSetOwnerRequest.java:100)
was:
Original issue:
{code:title=https://github.com/elek/ozone-build-results/blob/master/2020/09/01/2686/unit/hadoop-ozone/ozone-manager/org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.txt}
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.214 s <<<
FAILURE! - in org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest
testValidateAndUpdateCache(org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest)
Time elapsed: 0.129 s <<< FAILURE!
java.lang.AssertionError: Values should be different. Actual: 1598964934681
...
at org.junit.Assert.assertNotEquals(Assert.java:209)
at
org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.testValidateAndUpdateCache(TestOMAllocateBlockRequest.java:100)
{code}
> Intermittent failure in TestOMVolumeSetOwnerRequest and
> TestOMVolumeSetQuotaRequest
> ---
>
> Key: HDDS-4190
> URL: https://issues.apache.org/jira/browse/HDDS-4190
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: test
>Reporter: Attila Doroszlai
>Priority: Critical
>
> Original issue:
> {code:title=https://github.com/elek/ozone-build-results/blob/master/2020/09/01/2686/unit/hadoop-ozone/ozone-manager/org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.txt}
> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.214 s <<<
> FAILURE! - in
> org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest
> testValidateAndUpdateCache(org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest)
> Time elapsed: 0.129 s <<< FAILURE!
> java.lang.AssertionError: Values should be different. Actual: 1598964934681
> ...
> at org.junit.Assert.assertNotEquals(Assert.java:209)
> at
> org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.testValidateAndUpdateCache(TestOMAllocateBlockRequest.java:100)
> {code}
> Remaining issue:
> Similar assertions failing locally in:
> *
> TestOMVolumeSetQuotaRequest.testValidateAndUpdateCacheSuccess(TestOMVolumeSetQuotaRequest.java:101)
> *
> TestOMVolumeSetOwnerRequest.testValidateAndUpdateCacheSuccess(TestOMVolumeSetOwnerRequest.java:100)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4190) Intermittent failure in TestOMVolumeSetOwnerRequest and TestOMVolumeSetQuotaRequest
[
https://issues.apache.org/jira/browse/HDDS-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4190:
---
Summary: Intermittent failure in TestOMVolumeSetOwnerRequest and
TestOMVolumeSetQuotaRequest (was: Intermittent failure in
TestOMVolumeSetOwnerRequest and )
> Intermittent failure in TestOMVolumeSetOwnerRequest and
> TestOMVolumeSetQuotaRequest
> ---
>
> Key: HDDS-4190
> URL: https://issues.apache.org/jira/browse/HDDS-4190
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: test
>Reporter: Attila Doroszlai
>Priority: Critical
>
> {code:title=https://github.com/elek/ozone-build-results/blob/master/2020/09/01/2686/unit/hadoop-ozone/ozone-manager/org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.txt}
> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.214 s <<<
> FAILURE! - in
> org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest
> testValidateAndUpdateCache(org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest)
> Time elapsed: 0.129 s <<< FAILURE!
> java.lang.AssertionError: Values should be different. Actual: 1598964934681
> ...
> at org.junit.Assert.assertNotEquals(Assert.java:209)
> at
> org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.testValidateAndUpdateCache(TestOMAllocateBlockRequest.java:100)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4190) Intermittent failure in TestOMVolumeSetOwnerRequest and
[
https://issues.apache.org/jira/browse/HDDS-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4190:
---
Summary: Intermittent failure in TestOMVolumeSetOwnerRequest and (was:
Intermittent failure in TestOMAllocateBlockRequest#testValidateAndUpdateCache)
> Intermittent failure in TestOMVolumeSetOwnerRequest and
>
>
> Key: HDDS-4190
> URL: https://issues.apache.org/jira/browse/HDDS-4190
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: test
>Reporter: Attila Doroszlai
>Priority: Critical
>
> {code:title=https://github.com/elek/ozone-build-results/blob/master/2020/09/01/2686/unit/hadoop-ozone/ozone-manager/org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.txt}
> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.214 s <<<
> FAILURE! - in
> org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest
> testValidateAndUpdateCache(org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest)
> Time elapsed: 0.129 s <<< FAILURE!
> java.lang.AssertionError: Values should be different. Actual: 1598964934681
> ...
> at org.junit.Assert.assertNotEquals(Assert.java:209)
> at
> org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.testValidateAndUpdateCache(TestOMAllocateBlockRequest.java:100)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4190) Intermittent failure in TestOMVolumeSetOwnerRequest and TestOMVolumeSetQuotaRequest
[
https://issues.apache.org/jira/browse/HDDS-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4190:
---
Description:
Original issue:
{code:title=https://github.com/elek/ozone-build-results/blob/master/2020/09/01/2686/unit/hadoop-ozone/ozone-manager/org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.txt}
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.214 s <<<
FAILURE! - in org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest
testValidateAndUpdateCache(org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest)
Time elapsed: 0.129 s <<< FAILURE!
java.lang.AssertionError: Values should be different. Actual: 1598964934681
...
at org.junit.Assert.assertNotEquals(Assert.java:209)
at
org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.testValidateAndUpdateCache(TestOMAllocateBlockRequest.java:100)
{code}
was:
{code:title=https://github.com/elek/ozone-build-results/blob/master/2020/09/01/2686/unit/hadoop-ozone/ozone-manager/org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.txt}
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.214 s <<<
FAILURE! - in org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest
testValidateAndUpdateCache(org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest)
Time elapsed: 0.129 s <<< FAILURE!
java.lang.AssertionError: Values should be different. Actual: 1598964934681
...
at org.junit.Assert.assertNotEquals(Assert.java:209)
at
org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.testValidateAndUpdateCache(TestOMAllocateBlockRequest.java:100)
{code}
> Intermittent failure in TestOMVolumeSetOwnerRequest and
> TestOMVolumeSetQuotaRequest
> ---
>
> Key: HDDS-4190
> URL: https://issues.apache.org/jira/browse/HDDS-4190
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: test
>Reporter: Attila Doroszlai
>Priority: Critical
>
> Original issue:
> {code:title=https://github.com/elek/ozone-build-results/blob/master/2020/09/01/2686/unit/hadoop-ozone/ozone-manager/org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.txt}
> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.214 s <<<
> FAILURE! - in
> org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest
> testValidateAndUpdateCache(org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest)
> Time elapsed: 0.129 s <<< FAILURE!
> java.lang.AssertionError: Values should be different. Actual: 1598964934681
> ...
> at org.junit.Assert.assertNotEquals(Assert.java:209)
> at
> org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.testValidateAndUpdateCache(TestOMAllocateBlockRequest.java:100)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-4190) Intermittent failure in TestOMAllocateBlockRequest#testValidateAndUpdateCache
[
https://issues.apache.org/jira/browse/HDDS-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190087#comment-17190087
]
Attila Doroszlai commented on HDDS-4190:
TestOMAllocateBlockRequest was fixed in HDDS-4199 since then.
TestOMVolumeSetQuotaRequest and TestOMVolumeSetOwnerRequest are still to be
fixed here.
> Intermittent failure in TestOMAllocateBlockRequest#testValidateAndUpdateCache
> -
>
> Key: HDDS-4190
> URL: https://issues.apache.org/jira/browse/HDDS-4190
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: test
>Reporter: Attila Doroszlai
>Priority: Critical
>
> {code:title=https://github.com/elek/ozone-build-results/blob/master/2020/09/01/2686/unit/hadoop-ozone/ozone-manager/org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.txt}
> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.214 s <<<
> FAILURE! - in
> org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest
> testValidateAndUpdateCache(org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest)
> Time elapsed: 0.129 s <<< FAILURE!
> java.lang.AssertionError: Values should be different. Actual: 1598964934681
> ...
> at org.junit.Assert.assertNotEquals(Assert.java:209)
> at
> org.apache.hadoop.ozone.om.request.key.TestOMAllocateBlockRequest.testValidateAndUpdateCache(TestOMAllocateBlockRequest.java:100)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4199) Fix failed UT: TestOMAllocateBlockRequest#testValidateAndUpdateCache
[ https://issues.apache.org/jira/browse/HDDS-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4199. Fix Version/s: 1.1.0 Resolution: Fixed > Fix failed UT: TestOMAllocateBlockRequest#testValidateAndUpdateCache > > > Key: HDDS-4199 > URL: https://issues.apache.org/jira/browse/HDDS-4199 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: runzhiwang >Assignee: runzhiwang >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > Attachments: screenshot-1.png > > > !screenshot-1.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-4189) Use a unified Cli syntax for both getting OM and SCM status
[ https://issues.apache.org/jira/browse/HDDS-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4189. Fix Version/s: 1.1.0 Resolution: Fixed > Use a unified Cli syntax for both getting OM and SCM status > --- > > Key: HDDS-4189 > URL: https://issues.apache.org/jira/browse/HDDS-4189 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Labels: pull-request-available > Fix For: 1.1.0 > > > https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481380452 > suggests a unification for OM and SCM for getting status by Cli. > https://github.com/apache/hadoop-ozone/pull/1346 updated for SCM case. > This JIRA proposes to change > ozone admin om getserviceroles > to > ozone admin om roles -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4189) Use a unified Cli syntax for both getting OM and SCM status
[ https://issues.apache.org/jira/browse/HDDS-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4189: --- Issue Type: Improvement (was: Task) > Use a unified Cli syntax for both getting OM and SCM status > --- > > Key: HDDS-4189 > URL: https://issues.apache.org/jira/browse/HDDS-4189 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Labels: pull-request-available > > https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481380452 > suggests a unification for OM and SCM for getting status by Cli. > https://github.com/apache/hadoop-ozone/pull/1346 updated for SCM case. > This JIRA proposes to change > ozone admin om getserviceroles > to > ozone admin om roles -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4189) Use a unified Cli syntax for both getting OM and SCM status
[ https://issues.apache.org/jira/browse/HDDS-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4189: --- Labels: (was: pull-request-available) > Use a unified Cli syntax for both getting OM and SCM status > --- > > Key: HDDS-4189 > URL: https://issues.apache.org/jira/browse/HDDS-4189 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Fix For: 1.1.0 > > > https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481380452 > suggests a unification for OM and SCM for getting status by Cli. > https://github.com/apache/hadoop-ozone/pull/1346 updated for SCM case. > This JIRA proposes to change > ozone admin om getserviceroles > to > ozone admin om roles -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3762) Intermittent failure in TestDeleteWithSlowFollower
[
https://issues.apache.org/jira/browse/HDDS-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-3762:
---
Labels: (was: pull-request-available)
> Intermittent failure in TestDeleteWithSlowFollower
> --
>
> Key: HDDS-3762
> URL: https://issues.apache.org/jira/browse/HDDS-3762
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: test
>Affects Versions: 1.0.0
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Fix For: 1.1.0
>
>
> TestDeleteWithSlowFollower failed soon after it was re-enabled in HDDS-3330.
> {code:title=https://github.com/apache/hadoop-ozone/runs/753363338}
> [INFO] Running org.apache.hadoop.ozone.client.rpc.TestDeleteWithSlowFollower
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 28.647 s <<< FAILURE! - in
> org.apache.hadoop.ozone.client.rpc.TestDeleteWithSlowFollower
> [ERROR]
> testDeleteKeyWithSlowFollower(org.apache.hadoop.ozone.client.rpc.TestDeleteWithSlowFollower)
> Time elapsed: 0.163 s <<< FAILURE!
> java.lang.AssertionError
> ...
> at org.junit.Assert.assertNotNull(Assert.java:631)
> at
> org.apache.hadoop.ozone.client.rpc.TestDeleteWithSlowFollower.testDeleteKeyWithSlowFollower(TestDeleteWithSlowFollower.java:225)
> {code}
> CC [~shashikant] [~elek]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3840) [OFS] Use OFS in MapReduce acceptance test
[ https://issues.apache.org/jira/browse/HDDS-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3840: --- Status: Patch Available (was: In Progress) > [OFS] Use OFS in MapReduce acceptance test > -- > > Key: HDDS-3840 > URL: https://issues.apache.org/jira/browse/HDDS-3840 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Attila Doroszlai >Priority: Major > Labels: pull-request-available > > [mapreduce.robot|https://github.com/apache/hadoop-ozone/blob/072370b947416d89fae11d00a84a1d9a6b31beaa/hadoop-ozone/dist/src/main/smoketest/mapreduce.robot#L42] > currently uses o3fs. > We should run MR acceptance with OFS as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4056) Convert OzoneAdmin to pluggable model
[
https://issues.apache.org/jira/browse/HDDS-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4056:
---
Labels: (was: pull-request-available)
> Convert OzoneAdmin to pluggable model
> -
>
> Key: HDDS-4056
> URL: https://issues.apache.org/jira/browse/HDDS-4056
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone CLI
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Fix For: 1.1.0
>
>
> Ozone Shell's {{OzoneAdmin}} implements {{WithScmClient}} interface to be
> able to provide SCM client to sub-commands. We can convert it to a
> {{Mixin}}, which would allow converting {{OzoneAdmin}} to the pluggable model
> introduced by HDDS-4046.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4198) Compile Ozone with multiple Java versions
Attila Doroszlai created HDDS-4198: -- Summary: Compile Ozone with multiple Java versions Key: HDDS-4198 URL: https://issues.apache.org/jira/browse/HDDS-4198 Project: Hadoop Distributed Data Store Issue Type: Improvement Components: build Reporter: Attila Doroszlai Assignee: Attila Doroszlai Add matrix build in GitHub Actions for compiling Ozone with both Java 8 and 11. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4197) Failed to load existing service definition files: ...SubcommandWithParent
[
https://issues.apache.org/jira/browse/HDDS-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-4197:
---
Description:
{code}
[INFO] Apache Hadoop HDDS Tools ... FAILURE
...
[ERROR] Failed to load existing service definition files:
java.nio.file.NoSuchFileException:
hadoop-hdds/tools/target/classes/META-INF/services/org.apache.hadoop.hdds.cli.SubcommandWithParent
{code}
> Failed to load existing service definition files: ...SubcommandWithParent
> -
>
> Key: HDDS-4197
> URL: https://issues.apache.org/jira/browse/HDDS-4197
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: build
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: jdk11
>
> {code}
> [INFO] Apache Hadoop HDDS Tools ... FAILURE
> ...
> [ERROR] Failed to load existing service definition files:
> java.nio.file.NoSuchFileException:
> hadoop-hdds/tools/target/classes/META-INF/services/org.apache.hadoop.hdds.cli.SubcommandWithParent
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4197) Failed to load existing service definition files: ...SubcommandWithParent
Attila Doroszlai created HDDS-4197: -- Summary: Failed to load existing service definition files: ...SubcommandWithParent Key: HDDS-4197 URL: https://issues.apache.org/jira/browse/HDDS-4197 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Attila Doroszlai Assignee: Attila Doroszlai -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4197) Failed to load existing service definition files: ...SubcommandWithParent
[ https://issues.apache.org/jira/browse/HDDS-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4197: --- Component/s: build > Failed to load existing service definition files: ...SubcommandWithParent > - > > Key: HDDS-4197 > URL: https://issues.apache.org/jira/browse/HDDS-4197 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: build >Reporter: Attila Doroszlai >Assignee: Attila Doroszlai >Priority: Major > Labels: jdk11 > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
