[jira] [Created] (HADOOP-17245) Add RootedOzFS AbstractFileSystem to core-default.xml
Bharat Viswanadham created HADOOP-17245: --- Summary: Add RootedOzFS AbstractFileSystem to core-default.xml Key: HADOOP-17245 URL: https://issues.apache.org/jira/browse/HADOOP-17245 Project: Hadoop Common Issue Type: Bug Reporter: Bharat Viswanadham Assignee: Bharat Viswanadham When "ofs" is default, when running mapreduce job, YarnClient fails with below exception. Caused by: org.apache.hadoop.fs.UnsupportedFileSystemException: fs.AbstractFileSystem.ofs.impl=null: No AbstractFileSystem configured for scheme: ofs at org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:176) at org.apache.hadoop.fs.AbstractFileSystem.get(AbstractFileSystem.java:265) at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:341) at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:338) at java.security.AccessController.doPrivileged(Native Method) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17244) ITestS3AFileContextMainOperations#testRenameDirectoryAsNonExistentDirectory test failure on -Dauth
Steve Loughran created HADOOP-17244:
---
Summary:
ITestS3AFileContextMainOperations#testRenameDirectoryAsNonExistentDirectory
test failure on -Dauth
Key: HADOOP-17244
URL: https://issues.apache.org/jira/browse/HADOOP-17244
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Affects Versions: 3.3.1
Reporter: Steve Loughran
Assignee: Steve Loughran
Test failure:
{{ITestS3AFileContextMainOperations#testRenameDirectoryAsNonExistentDirectory}}
This is repeatable on -Dauth runs (we haven't been running them, have we?)
Either its from the recent dir marker changes (initial hypothesis) or its been
lurking a while and not been picked up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/ [Sep 3, 2020 7:17:52 AM] (noreply) YARN-10419. Javadoc error in hadoop-yarn-server-common module. (#2271) [Sep 3, 2020 1:02:48 PM] (Peter Bacsko) YARN-10372. Create MappingRule class to represent each CS mapping rule. Contributed by Gergely Pollak [Sep 3, 2020 4:54:22 PM] (Peter Bacsko) YARN-10375. CS Mapping rule config parser should return MappingRule objects. Contributed by Gergely Pollak. [Sep 3, 2020 6:06:20 PM] (noreply) HDFS-15529: getChildFilesystems should include fallback fs as well (#2234). Contributed by Uma Maheswara Rao G. [Sep 3, 2020 6:35:24 PM] (noreply) HADOOP-17167 ITestS3AEncryptionWithDefaultS3Settings failing (#2187) -1 overall The following subsystems voted -1: asflicense pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml Failed junit tests : hadoop.hdfs.TestFileChecksum hadoop.hdfs.TestFileChecksumCompositeCrc hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting hadoop.hdfs.TestGetFileChecksum hadoop.hdfs.server.datanode.TestBPOfferService hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped hadoop.hdfs.server.federation.router.TestRouterRpc hadoop.yarn.applications.distributedshell.TestDistributedShell cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/diff-compile-cc-root.txt [48K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/diff-compile-javac-root.txt [568K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/diff-checkstyle-root.txt [16M] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/diff-patch-pylint.txt [60K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/diff-patch-shelldocs.txt [44K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/whitespace-eol.txt [13M] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/whitespace-tabs.txt [1.9M] xml: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/xml.txt [24K] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/diff-javadoc-javadoc-root.txt [1.3M] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [560K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt [452K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt [16K] asflicense: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/255/artifact/out/patch-asflicense-problems.txt [4.0K] Powered by Apache Yetus 0.12.0 https://yetus.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail:
[jira] [Resolved] (HADOOP-17167) ITestS3AEncryptionWithDefaultS3Settings fails if default bucket encryption != KMS
[ https://issues.apache.org/jira/browse/HADOOP-17167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17167. - Fix Version/s: 3.3.1 Resolution: Fixed in trunk / branch-3; not yet tested myself to be confident all is good. > ITestS3AEncryptionWithDefaultS3Settings fails if default bucket encryption != > KMS > - > > Key: HADOOP-17167 > URL: https://issues.apache.org/jira/browse/HADOOP-17167 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.0, 3.4.0 >Reporter: Steve Loughran >Assignee: Mukund Thakur >Priority: Minor > Labels: pull-request-available > Fix For: 3.3.1 > > Time Spent: 40m > Remaining Estimate: 0h > > The test ITestS3AEncryptionWithDefaultS3Settings fails if > * the test run is set up with a KMS key > * the test bucket has a default encryption of AES (maybe also unencrypted) > Proposed: downgrade the test to skip if the default encryption is determined > to be something other than S3-KMS -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17227) improve s3guard markers command line tool
[ https://issues.apache.org/jira/browse/HADOOP-17227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17227. - Fix Version/s: 3.3.1 Resolution: Fixed Fixed in branch-3.3 > improve s3guard markers command line tool > - > > Key: HADOOP-17227 > URL: https://issues.apache.org/jira/browse/HADOOP-17227 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Fix For: 3.3.1 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > The s3guard markers audit -expected N command is meant to verify that the > marker count is N, but > * it isn't verified if the #of markers is 0, so you can't use it to assert > that markers have been created > * it doesn't work for tests where you expect a minimum number of markers. > It's essentially setting a max #of markers > Proposed: explicit -min, -max args to declare a specific range of values -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/ [Sep 3, 2020 5:29:30 PM] (Steve Loughran) HADOOP-17199. S3A Directory Marker HADOOP-13230 backport -1 overall The following subsystems voted -1: asflicense hadolint jshint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-build-tools/src/main/resources/checkstyle/checkstyle.xml hadoop-build-tools/src/main/resources/checkstyle/suppressions.xml hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml hadoop-tools/hadoop-azure/src/config/checkstyle.xml hadoop-tools/hadoop-resourceestimator/src/config/checkstyle.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml Failed junit tests : hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain hadoop.hdfs.TestDecommission hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.server.federation.router.TestRouterQuota hadoop.hdfs.server.federation.router.TestRouterNamenodeHeartbeat hadoop.hdfs.server.federation.resolver.order.TestLocalResolver hadoop.hdfs.server.federation.resolver.TestMultipleDestinationResolver hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.mapreduce.jobhistory.TestHistoryViewerPrinter hadoop.resourceestimator.solver.impl.TestLpSolver hadoop.resourceestimator.service.TestResourceEstimatorService hadoop.yarn.server.resourcemanager.TestClientRMService jshint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/diff-patch-jshint.txt [208K] cc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/diff-compile-javac-root.txt [408K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/diff-checkstyle-root.txt [16M] hadolint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/diff-patch-pylint.txt [60K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/diff-patch-shellcheck.txt [56K] shelldocs: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/diff-patch-shelldocs.txt [8.0K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/whitespace-eol.txt [12M] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/whitespace-tabs.txt [1.3M] xml: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/xml.txt [4.0K] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/diff-javadoc-javadoc-root.txt [20K] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [264K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt [36K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs_src_contrib_bkjournal.txt [12K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt [96K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/46/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt [104K]
