[jira] [Created] (YARN-6542) Fix the logger in TestAlignedPlanner and TestGreedyReservationAgent
Subru Krishnan created YARN-6542: Summary: Fix the logger in TestAlignedPlanner and TestGreedyReservationAgent Key: YARN-6542 URL: https://issues.apache.org/jira/browse/YARN-6542 Project: Hadoop YARN Issue Type: Sub-task Components: reservation system Reporter: Subru Krishnan Assignee: Subru Krishnan Jetty loggers are used in TestAlignedPlanner and TestGreedyReservationAgent while we actually need to use slf4j, this jira proposes switching the logger with slf4j. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: About 2.7.4 Release
On Mon, May 1, 2017 at 3:44 PM, Allen Wittenauer wrote: > > > On May 1, 2017, at 2:27 PM, Andrew Wang > wrote: > > I believe I asked about this on dev-yetus a while back. I'd prefer that > the presence of the fix version be sufficient to indicate whether a JIRA is > included in a release branch. Yetus requires that the JIRA be resolved as > "Fixed" to show up, which is why we are in our current situation. > > We can't do this because Hadoop is the only one that I've seen > that sets Fix version at close time. Everyone else is setting fix version > in place of target (which is a custom field, iirc). > > Let's see if I can revive the discussion over on a yetus list/jira. I think it's easier to add a new flag to Yetus than changing the Hadoop JIRA workflow, and it seems like this issue is becoming more acute.
Re: About 2.7.4 Release
> On May 1, 2017, at 2:27 PM, Andrew Wang wrote: > I believe I asked about this on dev-yetus a while back. I'd prefer that the > presence of the fix version be sufficient to indicate whether a JIRA is > included in a release branch. Yetus requires that the JIRA be resolved as > "Fixed" to show up, which is why we are in our current situation. We can't do this because Hadoop is the only one that I've seen that sets Fix version at close time. Everyone else is setting fix version in place of target (which is a custom field, iirc). - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6541) Optimize {{PeriodicRLESparseResourceAllocation}} by auto-expanding it's time period
Subru Krishnan created YARN-6541:
Summary: Optimize {{PeriodicRLESparseResourceAllocation}} by
auto-expanding it's time period
Key: YARN-6541
URL: https://issues.apache.org/jira/browse/YARN-6541
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Subru Krishnan
YARN-5531 adds a the specified {{PeriodicRLESparseResourceAllocation}} that
represents periodic allocation of resources. It seeds the period directly
using the max timePeriod which will result in storing multiple instances if the
user requested periods are small. For e.g: 24 instances of an hourly job if
timePeriod is one day. We need the max to prevent unbounded time period if user
decides to input prime numbers but we can auto-expand instead of statically
fixing it as the period
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/301/ [May 1, 2017 6:22:28 AM] (aajisaka) HADOOP-14359. Remove unnecessary shading of commons-httpclient. [May 1, 2017 6:32:20 AM] (aajisaka) HADOOP-13901. Fix ASF License warnings. [May 1, 2017 11:01:16 AM] (naganarasimha_gr) YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common. Contributed [May 1, 2017 11:19:30 AM] (naganarasimha_gr) YARN-6518. Fix warnings from Spotbugs in -1 overall The following subsystems voted -1: compile mvninstall unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120 hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140 hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean hadoop.hdfs.qjournal.server.TestJournalNode hadoop.hdfs.server.namenode.TestMetadataVersionOutput hadoop.hdfs.server.namenode.TestCheckpoint hadoop.hdfs.TestDFSUpgrade hadoop.hdfs.server.namenode.ha.TestBootstrapStandby hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110 hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130 hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.hdfs.server.datanode.TestDataNodeUUID hadoop.hdfs.server.namenode.TestStartup hadoop.hdfs.TestFileAppend hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService hadoop.mapred.TestShuffleHandler hadoop.tools.TestHadoopArchiveLogsRunner hadoop.mapred.gridmix.TestGridmixSubmission hadoop.metrics2.impl.TestKafkaMetrics hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.yarn.client.api.impl.TestAMRMClient hadoop.yarn.server.timeline.TestRollingLevelDB hadoop.yarn.server.timeline.TestTimelineDataManager hadoop.yarn.server.timeline.TestLeveldbTimelineStore hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore hadoop.yarn.server.resourcemanager.TestRMRestart hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore Timed out junit tests : org.apache.hadoop.hdfs.qjournal.client.TestQJMWithFaults org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStorePerf mvninstall: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/301/artifact/out/patch-mvninstall-root.txt [492K] compile: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/301/artifact/out/patch-compile-root.txt [20K] cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/301/artifact/out/patch-compile-root.txt [20K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/301/artifact/out/patch-compile-root.txt [20K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/301/artifact/out/patch-unit-hadoop-assemblies.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/301/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [2.1M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/301/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/301/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt [52K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/301/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapred
Re: About 2.7.4 Release
I didn't close JIRAs after the 3.0.0-alpha1 or alpha2 releases since closing makes the JIRAs read-only. This makes it more annoying to backport to older releases and for concurrent releases in general. I believe I asked about this on dev-yetus a while back. I'd prefer that the presence of the fix version be sufficient to indicate whether a JIRA is included in a release branch. Yetus requires that the JIRA be resolved as "Fixed" to show up, which is why we are in our current situation. On Thu, Apr 27, 2017 at 12:47 AM, Akira Ajisaka wrote: > Thanks Allen for the additional information. > > > At one point, JIRA was configured to refuse re-opening after a release > is cut. > > In the past, release manager closed the tickets and the process is written > in the wiki: https://wiki.apache.org/hadoop/HowToRelease > > > 10. In JIRA, close issues resolved in the release. Disable mail > notifications for this bulk change. > > Therefore, let's close them. > > On 2017/04/27 3:01, Allen Wittenauer wrote: > >> >> On Apr 25, 2017, at 12:35 AM, Akira Ajisaka wrote: >>> Maybe we should create a jira to track this? >>> >>> I think now either way (reopen or create) is fine. >>> >>> Release doc maker creates change logs by fetching information from JIRA, >>> so reopening the tickets should be avoided when a release process is in >>> progress. >>> >>> >> Keep in mind that the release documentation is part of the build >> process. Users who are doing their own builds will have incomplete >> documentation if we keep re-opening JIRAs after a release. At one point, >> JIRA was configured to refuse re-opening after a release is cut. I'm not >> sure why it stopped doing that, but it might be time to see if we can >> re-enable that functionality. >> >> >> - >> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org >> For additional commands, e-mail: common-dev-h...@hadoop.apache.org >> >> > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
[jira] [Created] (YARN-6540) Resource Manager is spelled "Resource Manger" in ResourceManagerRestart.md and ResourceManagerHA.md
Grant Sohn created YARN-6540: Summary: Resource Manager is spelled "Resource Manger" in ResourceManagerRestart.md and ResourceManagerHA.md Key: YARN-6540 URL: https://issues.apache.org/jira/browse/YARN-6540 Project: Hadoop YARN Issue Type: Bug Components: site Reporter: Grant Sohn Assignee: Grant Sohn Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6539) Create SecureLogin inside Router
Giovanni Matteo Fumarola created YARN-6539: -- Summary: Create SecureLogin inside Router Key: YARN-6539 URL: https://issues.apache.org/jira/browse/YARN-6539 Project: Hadoop YARN Issue Type: Sub-task Reporter: Giovanni Matteo Fumarola Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/390/ [Apr 30, 2017 6:49:36 AM] (liuml07) HADOOP-14363. Inconsistent default block location in FileSystem javadoc. [May 1, 2017 6:22:28 AM] (aajisaka) HADOOP-14359. Remove unnecessary shading of commons-httpclient. [May 1, 2017 6:32:20 AM] (aajisaka) HADOOP-13901. Fix ASF License warnings. -1 overall The following subsystems voted -1: findbugs unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-common-project/hadoop-minikdc Possible null pointer dereference in org.apache.hadoop.minikdc.MiniKdc.delete(File) due to return value of called method Dereferenced at MiniKdc.java:org.apache.hadoop.minikdc.MiniKdc.delete(File) due to return value of called method Dereferenced at MiniKdc.java:[line 368] FindBugs : module:hadoop-common-project/hadoop-auth org.apache.hadoop.security.authentication.server.MultiSchemeAuthenticationHandler.authenticate(HttpServletRequest, HttpServletResponse) makes inefficient use of keySet iterator instead of entrySet iterator At MultiSchemeAuthenticationHandler.java:of keySet iterator instead of entrySet iterator At MultiSchemeAuthenticationHandler.java:[line 192] FindBugs : module:hadoop-common-project/hadoop-common org.apache.hadoop.crypto.CipherSuite.setUnknownValue(int) unconditionally sets the field unknownValue At CipherSuite.java:unknownValue At CipherSuite.java:[line 44] org.apache.hadoop.crypto.CryptoProtocolVersion.setUnknownValue(int) unconditionally sets the field unknownValue At CryptoProtocolVersion.java:unknownValue At CryptoProtocolVersion.java:[line 67] Possible null pointer dereference in org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(File) due to return value of called method Dereferenced at FileUtil.java:org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(File) due to return value of called method Dereferenced at FileUtil.java:[line 118] Possible null pointer dereference in org.apache.hadoop.fs.RawLocalFileSystem.handleEmptyDstDirectoryOnWindows(Path, File, Path, File) due to return value of called method Dereferenced at RawLocalFileSystem.java:org.apache.hadoop.fs.RawLocalFileSystem.handleEmptyDstDirectoryOnWindows(Path, File, Path, File) due to return value of called method Dereferenced at RawLocalFileSystem.java:[line 387] Return value of org.apache.hadoop.fs.permission.FsAction.or(FsAction) ignored, but method has no side effect At FTPFileSystem.java:but method has no side effect At FTPFileSystem.java:[line 421] Useless condition:lazyPersist == true at this point At CommandWithDestination.java:[line 502] org.apache.hadoop.io.DoubleWritable.compareTo(DoubleWritable) incorrectly handles double value At DoubleWritable.java: At DoubleWritable.java:[line 78] org.apache.hadoop.io.DoubleWritable$Comparator.compare(byte[], int, int, byte[], int, int) incorrectly handles double value At DoubleWritable.java:int) incorrectly handles double value At DoubleWritable.java:[line 97] org.apache.hadoop.io.FloatWritable.compareTo(FloatWritable) incorrectly handles float value At FloatWritable.java: At FloatWritable.java:[line 71] org.apache.hadoop.io.FloatWritable$Comparator.compare(byte[], int, int, byte[], int, int) incorrectly handles float value At FloatWritable.java:int) incorrectly handles float value At FloatWritable.java:[line 89] Possible null pointer dereference in org.apache.hadoop.io.IOUtils.listDirectory(File, FilenameFilter) due to return value of called method Dereferenced at IOUtils.java:org.apache.hadoop.io.IOUtils.listDirectory(File, FilenameFilter) due to return value of called method Dereferenced at IOUtils.java:[line 350] org.apache.hadoop.io.erasurecode.ECSchema.toString() makes inefficient use of keySet iterator instead of entrySet iterator At ECSchema.java:keySet iterator instead of entrySet iterator At ECSchema.java:[line 193] Possible bad parsing of shift operation in org.apache.hadoop.io.file.tfile.Utils$Version.hashCode() At Utils.java:operation in org.apache.hadoop.io.file.tfile.Utils$Version.hashCode() At Utils.java:[line 398] org.apache.hadoop.metrics2.lib.DefaultMetricsFactory.setInstance(MutableMetricsFactory) unconditionally sets the field mmfImpl At DefaultMetricsFactory.java:mmfImpl At DefaultMetricsFactory.java:[line 49] org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.setMiniClusterMode(boolean) unconditionally sets the field miniClusterMode At DefaultMetricsSystem.java:miniClusterMode At DefaultMetricsSystem.java:[line 100] Useless object stored in variable seqOs of method org
