Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo
Thanks everyone for voting. We got 12 binding +1s from committers. The vote passed. We will go ahead to create a repo "submarine", and will allow existing Hadoop committers to commit code to submarine. Best, Wangda On Thu, Aug 29, 2019 at 10:50 PM Arpit Agarwal wrote: > +1 > > > On Aug 23, 2019, at 7:05 PM, Wangda Tan wrote: > > > > Hi devs, > > > > This is a voting thread to move Submarine source code, documentation from > > Hadoop repo to a separate Apache Git repo. Which is based on discussions > of > > > https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E > > > > Contributors who have permissions to push to Hadoop Git repository will > > have permissions to push to the new Submarine repository. > > > > This voting thread will run for 7 days and will end at Aug 30th. > > > > Please let me know if you have any questions. > > > > Thanks, > > Wangda Tan > >
[jira] [Created] (HADOOP-16540) Pluggable Filesystem Caching Support in FileSystem Class
Arun Ravi M V created HADOOP-16540: -- Summary: Pluggable Filesystem Caching Support in FileSystem Class Key: HADOOP-16540 URL: https://issues.apache.org/jira/browse/HADOOP-16540 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Arun Ravi M V Provide an option to use Custom Cache Class in FileSystem Class. Currently, the caching is enabled by default and uses the URI schema and authority value to determine whether to create a new FS instance for the given URI or to fetch an already existing one from the cache. In case of AWS S3 FS Impl, for an S3 path, the authority name will be bucket name, ie Filesystem object will be cached at the bucket level, but providing a custom caching logic can empower the user to cache it at some prefix level and provide more flexibility. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16539) ABFS: Add missing query parameter for getFileStatus
Da Zhou created HADOOP-16539: Summary: ABFS: Add missing query parameter for getFileStatus Key: HADOOP-16539 URL: https://issues.apache.org/jira/browse/HADOOP-16539 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.2.0 Reporter: Da Zhou When calling [getPathStatus|https://github.com/apache/hadoop/blob/e220dac15cc9972ebdd54ea9c82f288f234fca51/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java#L356], query parameter "action=getStatus" is missing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Merge YARN-8200 to branch-2 and branch-3.0
Thanks all, +1 from me too. There's three binding +1, two non-binding +1, and no -1 so I'll merge YARN-8200 to branch-2 shortly. I'll skip branch-3.0 since it's EOL as others have mentioned. Jonathan Hung On Tue, Aug 27, 2019 at 11:49 AM Konstantin Shvachko wrote: > +1 for the merge. > > We probably should not bother with branch-3.0 merge since it's been voted > EOL. > > Thanks, > --Konstantin > > On Thu, Aug 22, 2019 at 4:43 PM Jonathan Hung > wrote: > >> Hi folks, >> >> As per [1], starting a vote to merge YARN-8200 (and YARN-8200.branch3) >> feature branch to branch-2 (and branch-3.0). >> >> Vote runs for 7 days, to Thursday, Aug 29 5:00PM PDT. >> >> Thanks. >> >> [1] >> >> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201908.mbox/%3cCAHzWLgcX7f5Tr3q=csrqgysvpdf7mh-iu17femgx89dhr+1...@mail.gmail.com%3e >> >> Jonathan Hung >> >
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1243/ [Aug 28, 2019 4:13:46 PM] (aengineer) HDDS-1596. Create service endpoint to download configuration from SCM. [Aug 28, 2019 4:53:31 PM] (aengineer) HDDS-738. Removing REST protocol support from OzoneClient. Contributed [Aug 28, 2019 5:05:20 PM] (arp7) HDDS-1094. Performance test infrastructure : skip writing user data on [Aug 28, 2019 5:19:03 PM] (aengineer) HDDS-1881. Design doc: decommissioning in Ozone (#1196) [Aug 28, 2019 5:48:00 PM] (inigoiri) HDFS-14710. RBF: Improve some RPC performance by using previous block. [Aug 28, 2019 6:28:23 PM] (surendralilhore) HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun. [Aug 28, 2019 6:40:37 PM] (aengineer) HDDS-1937. Acceptance tests fail if scm webui shows invalid json [Aug 28, 2019 6:56:50 PM] (aengineer) HDDS-1942. Support copy during S3 multipart upload part creation [Aug 28, 2019 7:08:16 PM] (aengineer) HDDS-1950. S3 MPU part-list call fails if there are no parts [Aug 28, 2019 8:28:07 PM] (aengineer) HDDS-2051. Rat check failure in decommissioning.md (#1372) [Aug 28, 2019 8:40:34 PM] (ericp) YARN-9756: Create metric that sums total memory/vcores preempted per [Aug 28, 2019 9:59:49 PM] (bharat) HDDS-1941. Unused executor in SimpleContainerDownloader (#1367) -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core Class org.apache.hadoop.applications.mawo.server.common.TaskStatus implements Cloneable but does not define or use clone method At TaskStatus.java:does not define or use clone method At TaskStatus.java:[lines 39-346] Equals method for org.apache.hadoop.applications.mawo.server.worker.WorkerId assumes the argument is of type WorkerId At WorkerId.java:the argument is of type WorkerId At WorkerId.java:[line 114] org.apache.hadoop.applications.mawo.server.worker.WorkerId.equals(Object) does not check for null argument At WorkerId.java:null argument At WorkerId.java:[lines 114-115] Failed CTEST tests : test_test_libhdfs_ops_hdfs_static test_test_libhdfs_threaded_hdfs_static test_test_libhdfs_zerocopy_hdfs_static test_test_native_mini_dfs test_libhdfs_threaded_hdfspp_test_shim_static test_hdfspp_mini_dfs_smoke_hdfspp_test_shim_static libhdfs_mini_stress_valgrind_hdfspp_test_static memcheck_libhdfs_mini_stress_valgrind_hdfspp_test_static test_libhdfs_mini_stress_hdfspp_test_shim_static test_hdfs_ext_hdfspp_test_shim_static Failed junit tests : hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy hadoop.hdfs.TestErasureCodingPolicies hadoop.hdfs.server.federation.router.TestRouterWithSecureStartup hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun hadoop.mapreduce.v2.app.TestRecovery cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1243/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1243/artifact/out/diff-compile-javac-root.txt [332K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1243/artifact/out/diff-checkstyle-root.txt [17M] hadolint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1243/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1243/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1243/artifact/out/diff-patch-pylint.txt [220K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1243/artifact/out/diff-patch-shellcheck.txt [20K] shel
Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo
+1 > On Aug 23, 2019, at 7:05 PM, Wangda Tan wrote: > > Hi devs, > > This is a voting thread to move Submarine source code, documentation from > Hadoop repo to a separate Apache Git repo. Which is based on discussions of > https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E > > Contributors who have permissions to push to Hadoop Git repository will > have permissions to push to the new Submarine repository. > > This voting thread will run for 7 days and will end at Aug 30th. > > Please let me know if you have any questions. > > Thanks, > Wangda Tan - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo
+1 (non-binding) On Thu, Aug 29, 2019 at 3:43 PM Jonathan Eagles wrote: > +1. Will new mailing list aliases be created? hdfs-* is getting pretty > verbose. > > On Mon, Aug 26, 2019 at 11:29 PM Sunil Govindan wrote: > > > +1 > > > > - Sunil > > > > On Sat, Aug 24, 2019 at 7:36 AM Wangda Tan wrote: > > > > > Hi devs, > > > > > > This is a voting thread to move Submarine source code, documentation > from > > > Hadoop repo to a separate Apache Git repo. Which is based on > discussions > > of > > > > > > > > > https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E > > > > > > Contributors who have permissions to push to Hadoop Git repository will > > > have permissions to push to the new Submarine repository. > > > > > > This voting thread will run for 7 days and will end at Aug 30th. > > > > > > Please let me know if you have any questions. > > > > > > Thanks, > > > Wangda Tan > > > > > >
Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/ [Aug 28, 2019 9:26:03 PM] (ericp) YARN-9756: Create metric that sums total memory/vcores preempted per [Aug 28, 2019 10:42:28 PM] (tmarq) HADOOP-16242. ABFS: add bufferpool to AbfsOutputStream. [Aug 28, 2019 10:42:28 PM] (tmarq) HADOOP-16269. ABFS: add listFileStatus with StartFrom. [Aug 28, 2019 10:42:28 PM] (tmarq) HADOOP-16251. ABFS: add FSMainOperationsBaseTest [Aug 28, 2019 10:42:28 PM] (tmarq) Revert "HADOOP-16251. ABFS: add FSMainOperationsBaseTest" [Aug 28, 2019 10:42:28 PM] (tmarq) HADOOP-16376. ABFS: Override access() to no-op. [Aug 28, 2019 10:42:28 PM] (tmarq) HADOOP-16404. ABFS default blocksize change(256MB from 512MB) [Aug 28, 2019 10:42:28 PM] (tmarq) HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time [Aug 28, 2019 10:42:28 PM] (tmarq) HADOOP-16315. ABFS: transform full UPN for named user in AclStatus [Aug 28, 2019 10:42:28 PM] (tmarq) HADOOP-16460: ABFS: fix for Sever Name Indication (SNI) -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client Boxed value is unboxed and then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:[line 335] Failed junit tests : hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.server.datanode.TestDirectoryScanner hadoop.yarn.client.cli.TestRMAdminCLI hadoop.registry.secure.TestSecureLogins hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2 cc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/diff-compile-cc-root-jdk1.7.0_95.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/diff-compile-javac-root-jdk1.7.0_95.txt [328K] cc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/diff-compile-cc-root-jdk1.8.0_222.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/diff-compile-javac-root-jdk1.8.0_222.txt [308K] checkstyle: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/diff-checkstyle-root.txt [16M] hadolint: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/diff-patch-shellcheck.txt [72K] shelldocs: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/diff-patch-shelldocs.txt [8.0K] whitespace: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/whitespace-eol.txt [12M] https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/whitespace-tabs.txt [1.2M] xml: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/xml.txt [12K] findbugs: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-client-warnings.html [8.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/428/artifact/out/diff-javadoc-javadoc-root-jdk1
Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo
+1. Will new mailing list aliases be created? hdfs-* is getting pretty verbose. On Mon, Aug 26, 2019 at 11:29 PM Sunil Govindan wrote: > +1 > > - Sunil > > On Sat, Aug 24, 2019 at 7:36 AM Wangda Tan wrote: > > > Hi devs, > > > > This is a voting thread to move Submarine source code, documentation from > > Hadoop repo to a separate Apache Git repo. Which is based on discussions > of > > > > > https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E > > > > Contributors who have permissions to push to Hadoop Git repository will > > have permissions to push to the new Submarine repository. > > > > This voting thread will run for 7 days and will end at Aug 30th. > > > > Please let me know if you have any questions. > > > > Thanks, > > Wangda Tan > > >
