[jira] [Created] (HADOOP-13764) WASB test runs leak storage containers.
Chris Nauroth created HADOOP-13764: -- Summary: WASB test runs leak storage containers. Key: HADOOP-13764 URL: https://issues.apache.org/jira/browse/HADOOP-13764 Project: Hadoop Common Issue Type: Bug Components: fs/azure Reporter: Steve Loughran It appears that WASB test runs dynamically allocate a container within the storage account, using a naming convention of "wasbtests--". These containers are not cleaned up automatically, so they remain in the storage account indefinitely. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13763) KMS REST API Documentation
Jeffrey E Rodriguez created HADOOP-13763: - Summary: KMS REST API Documentation Key: HADOOP-13763 URL: https://issues.apache.org/jira/browse/HADOOP-13763 Project: Hadoop Common Issue Type: Bug Components: kms Affects Versions: 3.0.0-alpha1, 2.6.5, 2.6.4, 2.7.3, 2.6.3, 2.7.2, 2.7.1, 2.7.0 Environment: All- This is a KMS REST API documentation typo Reporter: Jeffrey E Rodriguez Priority: Minor Fix For: 3.0.0-alpha1 Decrypt Encrypted Key URL REST definition has a typo: reads as: POST http://HOST:PORT/kms/v1/keyversion//_eek?ee_op=decrypt should be: POST http://HOST:PORT/kms/v1/keyversion//_eek?eek_op=decrypt -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/136/ [Oct 25, 2016 4:03:03 PM] (cnauroth) HADOOP-13309. Document S3A known limitations in file ownership and [Oct 25, 2016 4:59:12 PM] (xiao) HADOOP-13669. Addendum patch 2 for KMS Server should log exceptions [Oct 25, 2016 5:19:13 PM] (zhz) HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee. [Oct 25, 2016 6:20:23 PM] (junping_du) MAPREDUCE-6792. Allow user's full principal name as owner of MapReduce [Oct 25, 2016 6:24:03 PM] (xyao) HDFS-11011. Add unit tests for HDFS command 'dfsadmin [Oct 25, 2016 7:15:09 PM] (xiao) YARN-5777. TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails. [Oct 25, 2016 7:19:34 PM] (yzhang) Fix HDFS-11040 [Oct 25, 2016 7:25:02 PM] (yzhang) Revert "Fix HDFS-11040" [Oct 25, 2016 7:25:40 PM] (yzhang) HDFS-11040. Add documentation for HDFS-9820 distcp improvement. [Oct 25, 2016 8:35:47 PM] (kasha) YARN-5388. Deprecate and remove DockerContainerExecutor. (Daniel [Oct 25, 2016 8:45:52 PM] (wang) Revert "HADOOP-13696. change hadoop-common dependency scope of jsch to [Oct 25, 2016 10:35:39 PM] (wang) HDFS-5684. Annotate o.a.h.fs.viewfs.ViewFileSystem.MountPoint as [Oct 25, 2016 11:53:53 PM] (weichiu) HADOOP-11798. Native raw erasure coder in XOR codes. Contributed by [Oct 26, 2016 2:50:56 AM] (arp) HDFS-11054. Suppress verbose log message in BlockPlacementPolicyDefault. [Oct 26, 2016 3:40:31 AM] (kai.zheng) HDFS-10935. TestFileChecksum fails in some cases. Contributed by Sammi [Oct 26, 2016 6:32:07 AM] (aajisaka) YARN-5575. Many classes use bare yarn. properties instead of the defined [Oct 26, 2016 6:59:39 AM] (rkanter) YARN-5753. fix NPE in AMRMClientImpl.getMatchingRequests() (haibochen [Oct 26, 2016 11:39:09 AM] (weichiu) HADOOP-13659. Upgrade jaxb-api version. Contributed by Sean Mackrory. -1 overall The following subsystems voted -1: compile unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.yarn.server.nodemanager.recovery.TestNMLeveldbStateStoreService hadoop.yarn.server.nodemanager.TestNodeManagerShutdown hadoop.yarn.server.timeline.TestRollingLevelDB hadoop.yarn.server.timeline.TestTimelineDataManager hadoop.yarn.server.timeline.TestLeveldbTimelineStore hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer hadoop.yarn.server.timelineservice.storage.common.TestRowKeys hadoop.yarn.server.timelineservice.storage.common.TestKeyConverters hadoop.yarn.server.timelineservice.storage.common.TestSeparator hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore hadoop.yarn.server.resourcemanager.TestRMRestart hadoop.yarn.server.resourcemanager.TestResourceTrackerService hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.client.api.impl.TestNMClient hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorage hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun hadoop.yarn.server.timelineservice.storage.TestPhoenixOfflineAggregationWriterImpl hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowActivity hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.mapred.TestShuffleHandler hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService hadoop.fs.azure.TestNativeAzureFileSystemAtomicRenameDirList hadoop.fs.azure.TestFileSystemOperationsWithThreads hadoop.fs.azure.TestFileSystemOperationExceptionHandling hadoop.fs.azure.TestNativeAzureFileSystemContractEmulator hadoop.fs.azure.TestReadAndSeekPageBlobAfterWrite hadoop.fs.azure.TestNativeAzureFileSystemAppend hadoop.fs.azure.TestNativeAzureFileSystemBlockLocations hadoop.fs.azure.TestOutOfBandAzureBlobOperations hadoop.fs.azure.metrics.TestNativeAzureFileSystemMetricsSystem hadoop.fs.azure.TestBlobMetadata hadoop.fs.azure.TestNativeAzureFileSystemMocked
[jira] [Created] (HADOOP-13762) S3A: Set thread names with more specific information about the call.
Chris Nauroth created HADOOP-13762:
--
Summary: S3A: Set thread names with more specific information
about the call.
Key: HADOOP-13762
URL: https://issues.apache.org/jira/browse/HADOOP-13762
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Reporter: Chris Nauroth
Running {{jstack}} on a hung process and reading the stack traces is a helpful
way to determine exactly what code in the process is stuck. This would be even
more helpful if we included more descriptive information about the specific
file system method call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13614) Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
[
https://issues.apache.org/jira/browse/HADOOP-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth resolved HADOOP-13614.
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.0.0-alpha2
2.8.0
I have committed this to trunk, branch-2 and branch-2.8 after completing
pre-commit on HADOOP-13758. Steve, thank you for the contribution.
> Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
> ---
>
> Key: HADOOP-13614
> URL: https://issues.apache.org/jira/browse/HADOOP-13614
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: HADOOP-13614-branch-2-001.patch,
> HADOOP-13614-branch-2-002.patch, HADOOP-13614-branch-2-002.patch,
> HADOOP-13614-branch-2-004.patch, HADOOP-13614-branch-2-005.patch,
> HADOOP-13614-branch-2-006.patch, HADOOP-13614-branch-2-007.patch,
> HADOOP-13614-branch-2-008.patch, testrun.txt
>
>
> Some of the slow test cases contain tests that are now obsoleted by newer
> ones. For example, {{ITestS3ADeleteManyFiles}} has the test case
> {{testOpenCreate()}} which writes then reads files up 25 MB.
> Have a look at which of the s3a tests are taking time, review them to see if
> newer tests have superceded the slow ones; and cut them where appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-8134) DNS claims to return a hostname but returns a PTR record in some cases
[
https://issues.apache.org/jira/browse/HADOOP-8134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J resolved HADOOP-8134.
-
Resolution: Not A Problem
Assignee: (was: Harsh J)
This hasn't proven as a problem in late. Closing as stale.
> DNS claims to return a hostname but returns a PTR record in some cases
> --
>
> Key: HADOOP-8134
> URL: https://issues.apache.org/jira/browse/HADOOP-8134
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 0.23.0
>Reporter: Harsh J
>Priority: Minor
>
> Per Shrijeet on HBASE-4109:
> {quote}
> If you are using an interface anything other than 'default' (literally that
> keyword) DNS.java's getDefaultHost will return a string which will have a
> trailing period at the end. It seems javadoc of reverseDns in DNS.java (see
> below) is conflicting with what that function is actually doing.
> It is returning a PTR record while claims it returns a hostname. The PTR
> record always has period at the end , RFC:
> http://irbs.net/bog-4.9.5/bog47.html
> We make call to DNS.getDefaultHost at more than one places and treat that as
> actual hostname.
> Quoting HRegionServer for example
> String machineName = DNS.getDefaultHost(conf.get(
> "hbase.regionserver.dns.interface", "default"), conf.get(
> "hbase.regionserver.dns.nameserver", "default"));
> We may want to sanitize the string returned from DNS class. Or better we can
> take a path of overhauling the way we do DNS name matching all over.
> {quote}
> While HBase has worked around the issue, we should fix the methods that
> aren't doing what they've intended.
> 1. We fix the method. This may be an 'incompatible change'. But I do not know
> who outside of us uses DNS classes.
> 2. We fix HDFS's DN at the calling end, cause that is affected by the
> trailing period in its reporting back to the NN as well (Just affects NN->DN
> weblinks, non critical).
> For 2, we can close this and open a HDFS JIRA.
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/206/ [Oct 25, 2016 4:03:03 PM] (cnauroth) HADOOP-13309. Document S3A known limitations in file ownership and [Oct 25, 2016 4:59:12 PM] (xiao) HADOOP-13669. Addendum patch 2 for KMS Server should log exceptions [Oct 25, 2016 5:19:13 PM] (zhz) HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee. [Oct 25, 2016 6:20:23 PM] (junping_du) MAPREDUCE-6792. Allow user's full principal name as owner of MapReduce [Oct 25, 2016 6:24:03 PM] (xyao) HDFS-11011. Add unit tests for HDFS command 'dfsadmin [Oct 25, 2016 7:15:09 PM] (xiao) YARN-5777. TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails. [Oct 25, 2016 7:19:34 PM] (yzhang) Fix HDFS-11040 [Oct 25, 2016 7:25:02 PM] (yzhang) Revert "Fix HDFS-11040" [Oct 25, 2016 7:25:40 PM] (yzhang) HDFS-11040. Add documentation for HDFS-9820 distcp improvement. [Oct 25, 2016 8:35:47 PM] (kasha) YARN-5388. Deprecate and remove DockerContainerExecutor. (Daniel [Oct 25, 2016 8:45:52 PM] (wang) Revert "HADOOP-13696. change hadoop-common dependency scope of jsch to [Oct 25, 2016 10:35:39 PM] (wang) HDFS-5684. Annotate o.a.h.fs.viewfs.ViewFileSystem.MountPoint as [Oct 25, 2016 11:53:53 PM] (weichiu) HADOOP-11798. Native raw erasure coder in XOR codes. Contributed by [Oct 26, 2016 2:50:56 AM] (arp) HDFS-11054. Suppress verbose log message in BlockPlacementPolicyDefault. [Oct 26, 2016 3:40:31 AM] (kai.zheng) HDFS-10935. TestFileChecksum fails in some cases. Contributed by Sammi [Oct 26, 2016 6:32:07 AM] (aajisaka) YARN-5575. Many classes use bare yarn. properties instead of the defined -1 overall The following subsystems voted -1: asflicense unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.hdfs.TestMaintenanceState hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.server.TestContainerManagerSecurity hadoop.fs.azure.TestAzureFileSystemErrorConditions hadoop.fs.azure.TestNativeAzureFileSystemAppend hadoop.fs.azure.TestNativeAzureFileSystemMocked hadoop.fs.azure.TestWasbFsck hadoop.fs.azure.TestNativeAzureFSPageBlobLive hadoop.fs.azure.TestOutOfBandAzureBlobOperationsLive hadoop.fs.azure.TestNativeAzureFileSystemAtomicRenameDirList hadoop.fs.azure.TestOutOfBandAzureBlobOperations hadoop.fs.azure.TestNativeAzureFileSystemContractPageBlobLive hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked hadoop.fs.azure.metrics.TestAzureFileSystemInstrumentation hadoop.fs.azure.TestBlobTypeSpeedDifference hadoop.fs.azure.TestFileSystemOperationExceptionMessage hadoop.fs.azure.TestAzureConcurrentOutOfBandIo hadoop.fs.azure.TestBlobMetadata hadoop.fs.azure.TestContainerChecks hadoop.fs.azure.TestNativeAzureFileSystemContractLive hadoop.fs.azure.metrics.TestNativeAzureFileSystemMetricsSystem hadoop.fs.azure.TestBlobDataValidation hadoop.fs.azure.TestFileSystemOperationsWithThreads hadoop.fs.azure.TestFileSystemOperationsExceptionHandlingMultiThreaded hadoop.fs.azure.TestNativeAzureFileSystemBlockLocations hadoop.fs.azure.TestNativeAzureFileSystemConcurrency hadoop.fs.azure.TestNativeAzureFileSystemClientLogging hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck hadoop.fs.azure.TestWasbUriAndConfiguration hadoop.fs.azure.TestFileSystemOperationExceptionHandling hadoop.fs.azure.TestNativeAzureFileSystemLive hadoop.fs.azure.TestReadAndSeekPageBlobAfterWrite hadoop.fs.azure.TestNativeAzureFileSystemContractEmulator hadoop.fs.azure.TestNativeAzureFileSystemContractMocked Timed out junit tests : org.apache.hadoop.hdfs.server.namenode.TestEditLog org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork org.apache.hadoop.hdfs.server.blockmanagement.TestCachedBlocksList org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl org.apache.hadoop.hdfs.TestDFSShell cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/206/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/206/artifact/out/diff-compile-javac-root.txt [168K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/206/artifact/out/diff-checkstyle-root.txt [16M] pylint:
[jira] [Resolved] (HADOOP-7505) EOFException in RPC stack should have a nicer error message
[ https://issues.apache.org/jira/browse/HADOOP-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J resolved HADOOP-7505. - Resolution: Duplicate Assignee: (was: Harsh J) This seems to be taken care (in part) via HADOOP-7346 > EOFException in RPC stack should have a nicer error message > --- > > Key: HADOOP-7505 > URL: https://issues.apache.org/jira/browse/HADOOP-7505 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Affects Versions: 0.23.0 >Reporter: Eli Collins >Priority: Minor > > Lots of user logs involve a user running mismatched versions, and for some > reason or another, they get EOFException instead of a proper version mismatch > exception. We should be able to catch this at appropriate points, and have a > nicer exception message explaining that it's a possible version mismatch, or > that they're trying to connect to the incorrect port. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-8579) Websites for HDFS and MapReduce both send users to video training resource which is non-public
[ https://issues.apache.org/jira/browse/HADOOP-8579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J resolved HADOOP-8579. - Resolution: Not A Problem Assignee: (was: Harsh J) This does not appear to be a problem after the project re-merge. > Websites for HDFS and MapReduce both send users to video training resource > which is non-public > -- > > Key: HADOOP-8579 > URL: https://issues.apache.org/jira/browse/HADOOP-8579 > Project: Hadoop Common > Issue Type: Bug > Environment: website >Reporter: David L. Willson >Priority: Minor > Original Estimate: 2h > Remaining Estimate: 2h > > Main pages for HDFS and MapReduce send new user to unavailable training > resource. > These two pages: > http://hadoop.apache.org/mapreduce/ > http://hadoop.apache.org/hdfs/ > Link to this page: > http://vimeo.com/3584536 > That page is not public, and not shared to all registered Vimeo users, and I > see nothing indicating how to ask for access to the resource. > Please make the vids public, or remove the link of disappointment. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-8863) Eclipse plugin may not be working on Juno due to changes in it
[ https://issues.apache.org/jira/browse/HADOOP-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J resolved HADOOP-8863. - Resolution: Won't Fix Assignee: (was: Harsh J) The eclipse plugin is formally out. > Eclipse plugin may not be working on Juno due to changes in it > -- > > Key: HADOOP-8863 > URL: https://issues.apache.org/jira/browse/HADOOP-8863 > Project: Hadoop Common > Issue Type: Bug > Components: contrib/eclipse-plugin >Affects Versions: 1.2.0 >Reporter: Harsh J > > We need to debug/investigate why it is so. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
