[jira] [Created] (HADOOP-14907) Memory leak in FileSystem cache
cen yuhai created HADOOP-14907: -- Summary: Memory leak in FileSystem cache Key: HADOOP-14907 URL: https://issues.apache.org/jira/browse/HADOOP-14907 Project: Hadoop Common Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.4 Reporter: cen yuhai There is a memory leak in FileSystem cache. It will take a lot of memory. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/ [Sep 25, 2017 12:12:25 PM] (templedf) HADOOP-14897. Loosen compatibility guidelines for native dependencies [Sep 25, 2017 4:25:07 PM] (aajisaka) HDFS-12304. Remove unused parameter from FsDatasetImpl#addVolume. [Sep 25, 2017 4:47:02 PM] (aajisaka) HADOOP-14892. MetricsSystemImpl should use Time.monotonicNow for [Sep 25, 2017 5:03:30 PM] (aajisaka) YARN-7153. Remove duplicated code in AMRMClientAsyncImpl.java. [Sep 25, 2017 8:35:44 PM] (jlowe) HADOOP-14881. LoadGenerator should use Time.monotonicNow() to measure [Sep 25, 2017 8:55:09 PM] (aengineer) HDFS-12529. Get source for config tags from file name. Contributed by [Sep 25, 2017 10:30:51 PM] (rchiang) HADOOP-14903. Add json-smart explicitly to pom.xml. (rchiang) [Sep 25, 2017 10:59:38 PM] (fabbri) HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity [Sep 25, 2017 11:02:55 PM] (Arun Suresh) YARN-7240. Add more states and transitions to stabilize the NM Container [Sep 26, 2017 12:12:01 AM] (fabbri) HADOOP-14890 Move up to AWS SDK 1.11.199. Contributed by Steve Loughran. -1 overall The following subsystems voted -1: unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation hadoop.yarn.client.api.impl.TestAMRMClient hadoop.mapreduce.TestMapreduceConfigFields hadoop.mapreduce.v2.hs.webapp.TestHSWebApp hadoop.yarn.sls.TestSLSRunner hadoop.yarn.sls.TestReservationSystemInvariants Timed out junit tests : org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/diff-compile-javac-root.txt [292K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/diff-checkstyle-root.txt [17M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/whitespace-eol.txt [11M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/whitespace-tabs.txt [1.2M] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/diff-javadoc-javadoc-root.txt [1.9M] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [64K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt [28K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/539/artifact/out/patch-unit-hadoop-tools_hadoop-sls.txt [20K] Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-14890) Move up to AWS SDK 1.11.199
[
https://issues.apache.org/jira/browse/HADOOP-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran reopened HADOOP-14890:
-
reopening to backport to branch-2; about to submit patch
> Move up to AWS SDK 1.11.199
> ---
>
> Key: HADOOP-14890
> URL: https://issues.apache.org/jira/browse/HADOOP-14890
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build, fs/s3
>Affects Versions: 3.0.0-beta1
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HADOOP-14890-001.patch
>
>
> the AWS SDK in Hadoop 3.0.-beta-1 prints a warning whenever you call abort()
> on a stream, which is what we need to do whenever doing long-distance seeks
> in a large file opened with fadvise=normal
> {code}
> 2017-09-20 17:51:50,459 [ScalaTest-main-running-S3ASeekReadSuite] INFO
> s3.S3ASeekReadSuite (Logging.scala:logInfo(54)) -
> 2017-09-20 17:51:50,460 [ScalaTest-main-running-S3ASeekReadSuite] INFO
> s3.S3ASeekReadSuite (Logging.scala:logInfo(54)) - Starting read() [pos =
> 45603305]
> 2017-09-20 17:51:50,461 [ScalaTest-main-running-S3ASeekReadSuite] WARN
> internal.S3AbortableInputStream (S3AbortableInputStream.java:close(163)) -
> Not all bytes were read from the S3ObjectInputStream, aborting HTTP
> connection. This is likely an error and may result in sub-optimal behavior.
> Request only the bytes you need via a ranged GET or drain the input stream
> after use.
> 2017-09-20 17:51:51,263 [ScalaTest-main-running-S3ASeekReadSuite] INFO
> s3.S3ASeekReadSuite (Logging.scala:logInfo(54)) - Duration of read() [pos =
> 45603305] = 803,650,637 nS
> {code}
> This goes away if we upgrade to the latest SDK, at least for the
> non-localdynamo bits
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14908) CrossOriginFilter should trigger regex on more input
Allen Wittenauer created HADOOP-14908:
-
Summary: CrossOriginFilter should trigger regex on more input
Key: HADOOP-14908
URL: https://issues.apache.org/jira/browse/HADOOP-14908
Project: Hadoop Common
Issue Type: Bug
Components: common, security
Affects Versions: 3.0.0-beta1
Reporter: Allen Wittenauer
Currently, CrossOriginFilter.java limits regex matching only if there is an
asterisk (*) in the config.
{code}
if (allowedOrigin.contains("*")) {
{code}
This means that entries such as:
{code}
http?://foo.example.com
https://[a-z][0-9].example.com
{code}
... and other patterns that succinctly limit the input space need to either be
fully expanded or dramatically have their space increased by using an asterisk
in order to pass through the filter.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/ [Sep 26, 2017 7:46:03 AM] (aajisaka) HADOOP-14893. WritableRpcEngine should use Time.monotonicNow. [Sep 26, 2017 9:04:09 AM] (rohithsharmaks) YARN-65. Reduce RM app memory footprint once app has completed. [Sep 26, 2017 9:21:42 AM] (yqlin) HDFS-12495. TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks [Sep 26, 2017 3:38:03 PM] (jlowe) Revert "YARN-6570. No logs were found for running application, running" -1 overall The following subsystems voted -1: unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.crypto.key.TestValueQueue hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem hadoop.crypto.key.kms.server.TestKMS hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting hadoop.hdfs.TestSafeModeWithStripedFile hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer hadoop.yarn.server.router.webapp.TestRouterWebServicesREST hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.mapreduce.TestMapreduceConfigFields hadoop.mapreduce.v2.hs.webapp.TestHSWebApp hadoop.yarn.sls.TestReservationSystemInvariants hadoop.yarn.sls.TestSLSRunner Timed out junit tests : org.apache.hadoop.yarn.server.resourcemanager.TestRMStoreCommands org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA org.apache.hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/diff-compile-javac-root.txt [292K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/diff-checkstyle-root.txt [17M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/whitespace-eol.txt [11M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/whitespace-tabs.txt [1.2M] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/diff-javadoc-javadoc-root.txt [1.9M] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt [156K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/patch-unit-hadoop-common-project_hadoop-kms.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [352K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [64K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt [604K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt [28K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/540/artifact/out/patch-unit-hadoop-tools_hadoop-sls.txt [24K] Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14909) Fix the word of "erasure encoding" in the top page
Takanobu Asanuma created HADOOP-14909: - Summary: Fix the word of "erasure encoding" in the top page Key: HADOOP-14909 URL: https://issues.apache.org/jira/browse/HADOOP-14909 Project: Hadoop Common Issue Type: Improvement Components: documentation Reporter: Takanobu Asanuma Assignee: Takanobu Asanuma Priority: Trivial Since "erasure coding" is a more general word, we should use it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14910) Upgrade netty-all jar to 4.0.37.Final
Vinayakumar B created HADOOP-14910: -- Summary: Upgrade netty-all jar to 4.0.37.Final Key: HADOOP-14910 URL: https://issues.apache.org/jira/browse/HADOOP-14910 Project: Hadoop Common Issue Type: Bug Reporter: Vinayakumar B Priority: Critical Upgrade netty-all jar to 4.0.37.Final version to fix latest vulnerabilities reported. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
