[jira] [Created] (HADOOP-13819) Directory Scanner should log startup message time correctly
Xiao Chen created HADOOP-13819:
--
Summary: Directory Scanner should log startup message time
correctly
Key: HADOOP-13819
URL: https://issues.apache.org/jira/browse/HADOOP-13819
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.7.1
Reporter: Xiao Chen
Priority: Trivial
When DirectoryScanner is enabled, one can see the following log:
{noformat}
INFO org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: Periodic
Directory Tree Verification scan starting at 1479189895562ms with interval of
2160ms
{noformat}
The first is epoch time and should not have 'ms'. Or better yet, we can change
it to a human-readable format.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13818) TestLocalFileSystem#testSetTimes fails
Akira Ajisaka created HADOOP-13818:
--
Summary: TestLocalFileSystem#testSetTimes fails
Key: HADOOP-13818
URL: https://issues.apache.org/jira/browse/HADOOP-13818
Project: Hadoop Common
Issue Type: Bug
Components: test
Reporter: Akira Ajisaka
{noformat}
Running org.apache.hadoop.fs.TestLocalFileSystem
Tests run: 20, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 4.887 sec <<<
FAILURE! - in org.apache.hadoop.fs.TestLocalFileSystem
testSetTimes(org.apache.hadoop.fs.TestLocalFileSystem) Time elapsed: 0.084 sec
<<< FAILURE!
java.lang.AssertionError: expected:<23456000> but was:<1479176144000>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.fs.TestLocalFileSystem.checkTimesStatus(TestLocalFileSystem.java:391)
at
org.apache.hadoop.fs.TestLocalFileSystem.testSetTimes(TestLocalFileSystem.java:414)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/155/ [Nov 14, 2016 5:38:36 AM] (kasha) YARN-4033. In FairScheduler, parent queues should also display queue -1 overall The following subsystems voted -1: compile unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.crypto.key.kms.server.TestKMS hadoop.hdfs.server.datanode.TestDataNodeLifeline hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer hadoop.hdfs.tools.TestDebugAdmin hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.yarn.server.nodemanager.recovery.TestNMLeveldbStateStoreService hadoop.yarn.server.nodemanager.TestNodeManagerShutdown hadoop.yarn.server.timeline.TestRollingLevelDB hadoop.yarn.server.timeline.TestTimelineDataManager hadoop.yarn.server.timeline.TestLeveldbTimelineStore hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer hadoop.yarn.server.timelineservice.storage.common.TestRowKeys hadoop.yarn.server.timelineservice.storage.common.TestKeyConverters hadoop.yarn.server.timelineservice.storage.common.TestSeparator hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore hadoop.yarn.server.resourcemanager.TestTokenClientRMService hadoop.yarn.server.resourcemanager.TestRMRestart hadoop.yarn.server.resourcemanager.TestRMAdminService hadoop.yarn.server.resourcemanager.TestResourceTrackerService hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.client.api.impl.TestNMClient hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageApps hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageEntities hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun hadoop.yarn.server.timelineservice.storage.TestPhoenixOfflineAggregationWriterImpl hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowActivity hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.mapred.TestShuffleHandler hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService hadoop.mapred.pipes.TestPipeApplication hadoop.mapred.TestSpecialCharactersInOutputPath Timed out junit tests : org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache org.apache.hadoop.mapred.TestMRIntermediateDataEncryption compile: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/155/artifact/out/patch-compile-root.txt [168K] cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/155/artifact/out/patch-compile-root.txt [168K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/155/artifact/out/patch-compile-root.txt [168K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/155/artifact/out/patch-unit-hadoop-common-project_hadoop-kms.txt [24K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/155/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [196K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/155/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt [52K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/155/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt [52K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/155/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice.txt [20K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/155/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [88K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/155/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [316K]
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/ [Nov 14, 2016 5:38:36 AM] (kasha) YARN-4033. In FairScheduler, parent queues should also display queue -1 overall The following subsystems voted -1: asflicense findbugs unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.crypto.key.kms.server.TestKMS hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks hadoop.hdfs.TestRollingUpgrade hadoop.hdfs.server.namenode.ha.TestHAMetrics hadoop.fs.http.server.TestHttpFSServerNoACLs hadoop.yarn.server.resourcemanager.TestTokenClientRMService hadoop.yarn.server.TestDiskFailures hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.mapred.pipes.TestPipeApplication cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/diff-compile-javac-root.txt [164K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/diff-checkstyle-root.txt [16M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/diff-patch-shelldocs.txt [16K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/whitespace-eol.txt [11M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/whitespace-tabs.txt [1.3M] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [4.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/diff-javadoc-javadoc-root.txt [2.2M] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/patch-unit-hadoop-common-project_hadoop-kms.txt [24K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [256K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [72K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [316K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt [92K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt [124K] asflicense: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/225/artifact/out/patch-asflicense-problems.txt [4.0K] Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13817) Add a finite shell command timeout to ShellBasedUnixGroupsMapping
Harsh J created HADOOP-13817:
Summary: Add a finite shell command timeout to
ShellBasedUnixGroupsMapping
Key: HADOOP-13817
URL: https://issues.apache.org/jira/browse/HADOOP-13817
Project: Hadoop Common
Issue Type: Improvement
Components: security
Affects Versions: 2.6.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
The ShellBasedUnixGroupsMapping run various {{id}} commands via the
ShellCommandExecutor modules without a timeout set (its set to 0, which implies
infinite).
If this command hangs for a long time on the OS end due to an unresponsive
groups backend or other reasons, it also blocks the handlers that use it on the
NameNode (or other services that use this class). That inadvertently causes odd
timeout troubles on the client end where its forced to retry (only to likely
run into such hangs again with every attempt until at least one command
returns).
It would be helpful to have a finite command timeout after which we may give up
on the command and return the result equivalent of no groups found.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
