[jira] [Created] (YARN-9806) TestNMSimulator#testNMSimulator fails in branch-2
Jonathan Hung created YARN-9806:
---
Summary: TestNMSimulator#testNMSimulator fails in branch-2
Key: YARN-9806
URL: https://issues.apache.org/jira/browse/YARN-9806
Project: Hadoop YARN
Issue Type: Bug
Reporter: Jonathan Hung
{noformat}java.lang.AssertionError: expected:<10240> but was:<0>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.yarn.sls.nodemanager.TestNMSimulator.testNMSimulator(TestNMSimulator.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413){noformat}
This appears fixed in YARN-7929. We only need the bit in TestNMSimulator
though. This jira is to track getting this bit in branch-2.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-7585) NodeManager should go unhealthy when state store throws DBException
[
https://issues.apache.org/jira/browse/YARN-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hung resolved YARN-7585.
-
Fix Version/s: 2.10.0
Resolution: Fixed
Committed to branch-2.
> NodeManager should go unhealthy when state store throws DBException
>
>
> Key: YARN-7585
> URL: https://issues.apache.org/jira/browse/YARN-7585
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Labels: release-blocker
> Fix For: 2.10.0, 3.1.0
>
> Attachments: YARN-7585.001.patch, YARN-7585.002.patch,
> YARN-7585.003.patch
>
>
> If work preserving recover is enabled the NM will not start up if the state
> store does not initialise. However if the state store becomes unavailable
> after that for any reason the NM will not go unhealthy.
> Since the state store is not available new containers can not be started any
> more and the NM should become unhealthy:
> {code}
> AMLauncher: Error launching appattempt_1508806289867_268617_01. Got
> exception: org.apache.hadoop.yarn.exceptions.YarnException:
> java.io.IOException: org.iq80.leveldb.DBException: IO error:
> /dsk/app/var/lib/hadoop-yarn/yarn-nm-recovery/yarn-nm-state/028269.log:
> Read-only file system
> at o.a.h.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
> at
> o.a.h.y.s.n.cm.ContainerManagerImpl.startContainers(ContainerManagerImpl.java:721)
> ...
> Caused by: java.io.IOException: org.iq80.leveldb.DBException: IO error:
> /dsk/app/var/lib/hadoop-yarn/yarn-nm-recovery/yarn-nm-state/028269.log:
> Read-only file system
> at
> o.a.h.y.s.n.r.NMLeveldbStateStoreService.storeApplication(NMLeveldbStateStoreService.java:374)
> at
> o.a.h.y.s.n.cm.ContainerManagerImpl.startContainerInternal(ContainerManagerImpl.java:848)
> at
> o.a.h.y.s.n.cm.ContainerManagerImpl.startContainers(ContainerManagerImpl.java:712)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1244/ [Aug 29, 2019 6:13:40 AM] (abmodi) YARN-9754. Add support for arbitrary DAG AM Simulator. Contributed by [Aug 29, 2019 7:25:03 AM] (elek) Revert "HDDS-1596. Create service endpoint to download configuration [Aug 29, 2019 7:46:50 AM] (elek) HDDS-2045. Partially started compose cluster left running [Aug 29, 2019 10:14:39 AM] (wwei) YARN-9664. Improve response of scheduler/app activities for better [Aug 29, 2019 2:38:38 PM] (ayushsaxena) HDFS-14721. RBF: ProxyOpComplete is not accurate in [Aug 29, 2019 5:10:52 PM] (weichiu) HDFS-11246. FSNameSystem#logAuditEvent should be called outside the read [Aug 29, 2019 9:18:35 PM] (inigoiri) HDFS-14104. Review getImageTxIdToRetain. Contributed by David Mollitor. [Aug 29, 2019 9:21:09 PM] (inigoiri) HDFS-14256. Review Logging of NameNode Class. Contributed by David [Aug 29, 2019 11:56:14 PM] (weichiu) HDFS-8178. QJM doesn't move aside stale inprogress edits files. [Aug 29, 2019 11:57:02 PM] (aengineer) HDDS-2050. Error while compiling ozone-recon-web (#1374) [Aug 30, 2019 12:07:55 AM] (aengineer) HDDS-1935. Improve the visibility with Ozone Insight tool (#1255) [Aug 30, 2019 12:38:07 AM] (weichiu) HDFS-14706. Checksums are not checked if block meta file is less than 7 [Aug 30, 2019 1:13:10 AM] (aajisaka) YARN-9783. Remove low-level zookeeper test to be able to build Hadoop [Aug 30, 2019 1:35:26 AM] (weichiu) Make DataNodePeerMetrics#minOutlierDetectionSamples configurable [Aug 30, 2019 3:10:13 AM] (ayushsaxena) HDFS-12212. Options.Rename.To_TRASH is considered even when [Aug 30, 2019 4:09:04 AM] (busbey) HADOOP-15998. Ensure jar validation works on Windows. [Aug 30, 2019 5:05:09 AM] (rohithsharmaks) YARN-9714. ZooKeeper connection in ZKRMStateStore leaks after RM -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core Class org.apache.hadoop.applications.mawo.server.common.TaskStatus implements Cloneable but does not define or use clone method At TaskStatus.java:does not define or use clone method At TaskStatus.java:[lines 39-346] Equals method for org.apache.hadoop.applications.mawo.server.worker.WorkerId assumes the argument is of type WorkerId At WorkerId.java:the argument is of type WorkerId At WorkerId.java:[line 114] org.apache.hadoop.applications.mawo.server.worker.WorkerId.equals(Object) does not check for null argument At WorkerId.java:null argument At WorkerId.java:[lines 114-115] Failed CTEST tests : test_test_libhdfs_ops_hdfs_static test_test_libhdfs_threaded_hdfs_static test_test_libhdfs_zerocopy_hdfs_static test_test_native_mini_dfs test_libhdfs_threaded_hdfspp_test_shim_static test_hdfspp_mini_dfs_smoke_hdfspp_test_shim_static libhdfs_mini_stress_valgrind_hdfspp_test_static memcheck_libhdfs_mini_stress_valgrind_hdfspp_test_static test_libhdfs_mini_stress_hdfspp_test_shim_static test_hdfs_ext_hdfspp_test_shim_static Failed junit tests : hadoop.hdfs.server.federation.router.TestRouterWithSecureStartup hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken hadoop.yarn.client.api.impl.TestAMRMClient cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1244/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1244/artifact/out/diff-compile-javac-root.txt [332K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1244/artifact/out/diff-checkstyle-root.txt [17M] hadolint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1244/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen:
Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/429/ [Aug 30, 2019 12:02:46 AM] (jhung) YARN-4081. Add support for multiple resource types in the Resource [Aug 30, 2019 12:02:46 AM] (jhung) YARN-4172. Extend DominantResourceCalculator to account for all [Aug 30, 2019 12:02:46 AM] (jhung) YARN-4715. Add support to read resource types from a config file. [Aug 30, 2019 12:02:46 AM] (jhung) YARN-4829. Add support for binary units in Resource class.(vvasudev via [Aug 30, 2019 12:02:46 AM] (jhung) YARN-4830. Add support for resource types in the nodemanager. [Aug 30, 2019 12:02:46 AM] (jhung) YARN-5242. Update DominantResourceCalculator to consider all resource [Aug 30, 2019 12:02:46 AM] (jhung) YARN-5586. Update the Resources class to consider all resource types. [Aug 30, 2019 12:02:46 AM] (jhung) YARN-5587. [Partial backport] Add support for resource profiles. [Aug 30, 2019 12:02:46 AM] (jhung) YARN-5588. [Partial backport] [YARN-3926] Add support for resource [Aug 30, 2019 12:02:46 AM] (jhung) YARN-9181. Backport YARN-6232 for generic resource type usage to [Aug 30, 2019 12:02:46 AM] (jhung) YARN-9177. Use resource map for app metrics in [Aug 30, 2019 12:02:46 AM] (jhung) YARN-9182. Backport YARN-6445 resource profile performance improvements [Aug 30, 2019 12:02:46 AM] (jhung) YARN-6761. Fix build for YARN-3926 branch. Contributed by Varun Vasudev. [Aug 30, 2019 12:02:46 AM] (jhung) YARN-6786. [YARN-3926] ResourcePBImpl imports cleanup. Contributed by [Aug 30, 2019 12:02:46 AM] (jhung) YARN-6788. [YARN-3926] Improve performance of resource profile branch [Aug 30, 2019 12:02:46 AM] (jhung) YARN-6994. [YARN-3926] Remove last uses of Long from resource types [Aug 30, 2019 12:02:46 AM] (jhung) YARN-6892. [YARN-3926] Improve API implementation in Resources and [Aug 30, 2019 12:02:46 AM] (jhung) YARN-6610. [YARN-3926] DominantResourceCalculator#getResourceAsValue [Aug 30, 2019 12:02:46 AM] (jhung) YARN-7030. [YARN-3926] Performance optimizations in Resource and [Aug 30, 2019 12:02:46 AM] (jhung) YARN-7042. Clean up unit tests after YARN-6610. (Daniel Templeton via [Aug 30, 2019 12:02:46 AM] (jhung) YARN-6789. Add Client API to get all supported resource types from RM. [Aug 30, 2019 12:02:46 AM] (jhung) YARN-6781. [YARN-3926] ResourceUtils#initializeResourcesMap takes an [Aug 30, 2019 12:02:46 AM] (jhung) YARN-7043. [Partial backport] Cleanup ResourceProfileManager. (wangda) [Aug 30, 2019 12:02:46 AM] (jhung) YARN-7067. [YARN-3926] Optimize ResourceType information display in UI. [Aug 30, 2019 12:02:46 AM] (jhung) YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G [Aug 30, 2019 12:02:46 AM] (jhung) YARN-7093. Improve log message in ResourceUtils. (Sunil G via wangda) [Aug 30, 2019 12:02:47 AM] (jhung) YARN-6933. [YARN-3926] ResourceUtils.DISALLOWED_NAMES check is [Aug 30, 2019 12:02:47 AM] (jhung) YARN-7056. [Partial backport] Document Resource Profiles feature. (Sunil [Aug 30, 2019 12:02:47 AM] (jhung) YARN-9188. Port YARN-7136 to branch-2 [Aug 30, 2019 12:02:47 AM] (jhung) YARN-7270 addendum: Reapplied changes after YARN-3926 backports [Aug 30, 2019 12:02:47 AM] (jhung) YARN-7137. [YARN-3926] Move newly added APIs to unstable in YARN-3926 [Aug 30, 2019 12:02:47 AM] (jhung) YARN-9175. Null resources check in ResourceInfo for branch-3.0 [Aug 30, 2019 12:02:47 AM] (jhung) YARN-9187. Backport YARN-6852 for GPU-specific native changes to [Aug 30, 2019 12:02:47 AM] (jhung) YARN-9180. Port YARN-7033 NM recovery of assigned resources to branch-2 [Aug 30, 2019 12:02:47 AM] (jhung) YARN-9280. Backport YARN-6620 to YARN-8200/branch-2 for NodeManager-side [Aug 30, 2019 12:02:47 AM] (jhung) YARN-9174. Backport YARN-7224 for refactoring of GpuDevice class [Aug 30, 2019 12:02:47 AM] (jhung) YARN-7396. NPE when accessing container logs due to null dirsHandler. [Aug 30, 2019 12:02:47 AM] (jhung) YARN-9289. Backport YARN-7330 for GPU in UI to branch-2 [Aug 30, 2019 12:02:47 AM] (jhung) YARN-7573. Gpu Information page could be empty for nodes without GPU. [Aug 30, 2019 12:02:47 AM] (jhung) YARN-7594. TestNMWebServices#testGetNMResourceInfo fails on trunk. [Aug 30, 2019 12:02:47 AM] (jhung) YARN-7223. Document GPU isolation feature. Contributed by Wangda Tan. [Aug 30, 2019 12:02:47 AM] (jhung) YARN-9397. Fix empty NMResourceInfo object test failures in branch-2 [Aug 30, 2019 12:02:47 AM] (jhung) YARN-9291. Backport YARN-7637 to branch-2 [Aug 30, 2019 12:02:47 AM] (jhung) YARN-7345. GPU Isolation: Incorrect minor device numbers written to [Aug 30, 2019 12:02:47 AM] (jhung) YARN-7143. FileNotFound handling in ResourceUtils is inconsistent [Aug 30, 2019 12:02:47 AM] (jhung) YARN-7383. Node resource is not parsed correctly for resource names [Aug 30, 2019 12:02:47 AM] (jhung) YARN-8183. Fix ConcurrentModificationException inside [Aug 30, 2019 12:02:47 AM] (jhung) YARN-9271. Backport YARN-6927 for resource type support in MapReduce
[jira] [Created] (YARN-9805) Fine-grained SchedulerNode synchronization
Ahmed Hussein created YARN-9805: --- Summary: Fine-grained SchedulerNode synchronization Key: YARN-9805 URL: https://issues.apache.org/jira/browse/YARN-9805 Project: Hadoop YARN Issue Type: Improvement Reporter: Ahmed Hussein Assignee: Ahmed Hussein Yarn schedulerNode and RMNode are using synchronized methods on reading and updating the resources. Instead, use read-write reentrant locks to provide fine-grained locking and to avoid blocking concurrent reads. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo
Thanks everyone for voting. We got 12 binding +1s from committers. The vote passed. We will go ahead to create a repo "submarine", and will allow existing Hadoop committers to commit code to submarine. Best, Wangda On Thu, Aug 29, 2019 at 10:50 PM Arpit Agarwal wrote: > +1 > > > On Aug 23, 2019, at 7:05 PM, Wangda Tan wrote: > > > > Hi devs, > > > > This is a voting thread to move Submarine source code, documentation from > > Hadoop repo to a separate Apache Git repo. Which is based on discussions > of > > > https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E > > > > Contributors who have permissions to push to Hadoop Git repository will > > have permissions to push to the new Submarine repository. > > > > This voting thread will run for 7 days and will end at Aug 30th. > > > > Please let me know if you have any questions. > > > > Thanks, > > Wangda Tan > >
[jira] [Resolved] (YARN-4069) For long running apps (> 2 days), populate flow activity table
[ https://issues.apache.org/jira/browse/YARN-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S resolved YARN-4069. - Resolution: Won't Fix No activity for long time. In ATSv2 weekly call, discussed for cleaning up JIRA which are not going implement. Hence, closing the JIRA as Won't Fix. Feel free to reopen if still same design approach exists. > For long running apps (> 2 days), populate flow activity table > -- > > Key: YARN-4069 > URL: https://issues.apache.org/jira/browse/YARN-4069 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Vrushali C >Assignee: Varun Saxena >Priority: Major > Labels: YARN-5355 > > YARN-4063 / YARN-3901 contain the work being done to populate the flow > activity and flow run tables. > The flow activity table is updated each time a yarn application is created > and finishes. So if an application runs for more than 3 days, day1 has an > entry for the flow for start time, day3 has an entry for the flow for end > time but day2 has no entry for that flow. > Filing the jira to ensure that for long running apps, the flow activity table > does get a snapshot time entered for each day that an application is running > in that flow. > It may be the case that for ALL apps (long running or not) the same update > may be done in the flow activity table. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-3865) Backward compatibility of reader with ATSv1
[ https://issues.apache.org/jira/browse/YARN-3865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S resolved YARN-3865. - Resolution: Won't Fix No activity for long time. In ATSv2 weekly call, discussed for cleaning up JIRA which are not going implement. Hence, closing the JIRA as Won't Fix. Feel free to reopen if still same design approach exists. > Backward compatibility of reader with ATSv1 > --- > > Key: YARN-3865 > URL: https://issues.apache.org/jira/browse/YARN-3865 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena >Priority: Major > Labels: YARN-5355 > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-4343) Need to support Application History Server on ATSV2
[ https://issues.apache.org/jira/browse/YARN-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S resolved YARN-4343. - Resolution: Duplicate This functionality exist in latest trunk code. I don't remember which JIRA but [~abmodi] did it. If ATSv2 is enabled, then completed application which is not present in RM is fetched from ATSv2. > Need to support Application History Server on ATSV2 > --- > > Key: YARN-4343 > URL: https://issues.apache.org/jira/browse/YARN-4343 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R >Priority: Major > Labels: YARN-5355 > > AHS is used by the CLI and Webproxy(REST), if the application related > information is not found in RM then it tries to fetch from AHS and show -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S resolved YARN-5071. - Resolution: Won't Fix No activity for long time. In ATSv2 weekly call, discussed for cleaning up JIRA which are not going implement. Hence, closing the JIRA as Won't Fix. Feel free to reopen if still same design approach exists. Currently we support only hbase-1.2.4/5 hbase-2 versions. We have separate packages for both the versions. I guess this shouldn't be much issue as per current code in trunk. > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > Labels: YARN-5355 > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-4261) fix the order of timelinereader in yarn/yarn.cmd
[ https://issues.apache.org/jira/browse/YARN-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S resolved YARN-4261. - Resolution: Won't Fix No activity for long time. In ATSv2 weekly call, discussed for cleaning up JIRA which are not going implement. Hence, closing the JIRA as Won't Fix. Feel free to reopen if still same design approach exists. > fix the order of timelinereader in yarn/yarn.cmd > > > Key: YARN-4261 > URL: https://issues.apache.org/jira/browse/YARN-4261 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Varun Saxena >Priority: Trivial > Labels: YARN-5355 > > The order of the timelinereader command is not correct in yarn/yarn.cmd. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-4504) Retrospect on defaults for created time while querying
[ https://issues.apache.org/jira/browse/YARN-4504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S resolved YARN-4504. - Resolution: Won't Fix No activity for long time. In ATSv2 weekly call, discussed for cleaning up JIRA which are not going implement. Hence, closing the JIRA as Won't Fix. Feel free to reopen if still same design approach exists. > Retrospect on defaults for created time while querying > -- > > Key: YARN-4504 > URL: https://issues.apache.org/jira/browse/YARN-4504 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-5760) [ATSv2] Create HBase connection only if an app collector is publishing from NM
[ https://issues.apache.org/jira/browse/YARN-5760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S resolved YARN-5760. - Resolution: Won't Fix No activity for long time. In ATSv2 weekly call, discussed for cleaning up JIRA which are not going implement. Hence, closing the JIRA as Won't Fix. Feel free to reopen if still same design approach exists. > [ATSv2] Create HBase connection only if an app collector is publishing from NM > -- > > Key: YARN-5760 > URL: https://issues.apache.org/jira/browse/YARN-5760 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Varun Saxena >Assignee: Varun Saxena >Priority: Major > > Irrespective of NM handling an app or not, we initialize > HBaseTimelineWriterImpl in TimelineCollectorManager. > This in turn calls ConnectionFactory#createConnection to manage connections > with HBase. > But it seems this opens up a connection with Zookeeper (i.e. as soon as NM > starts up) instead of opening connection when atleast one app arrives for > publishing and closing it if no apps are being published from this NM. > This leads to unnecessary connections to Zookeeper. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-9804) Update ATSv2 document for latest feature supports
Rohith Sharma K S created YARN-9804: --- Summary: Update ATSv2 document for latest feature supports Key: YARN-9804 URL: https://issues.apache.org/jira/browse/YARN-9804 Project: Hadoop YARN Issue Type: Sub-task Reporter: Rohith Sharma K S Assignee: Rohith Sharma K S Revisit ATSv2 documents and update for GA features. And also for the road map. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
