[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369748#comment-15369748 ] Hudson commented on YARN-4700: -- SUCCESS: Integrated in Hadoop-trunk-Commit #10074 (See

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182247#comment-15182247 ] Naganarasimha G R commented on YARN-4700: - Thanks for the review and the commit [~varun_saxena],

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179926#comment-15179926 ] Varun Saxena commented on YARN-4700: I have committed this to branch YARN-2928. Thanks [~gtCarrera9]

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177826#comment-15177826 ] Varun Saxena commented on YARN-4700: Thanks [~Naganarasimha] for the latest patch. +1, looks good to

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177119#comment-15177119 ] Hadoop QA commented on YARN-4700: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176611#comment-15176611 ] Sangjin Lee commented on YARN-4700: --- It seems that the unit test failures are real. So is the javadoc

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176399#comment-15176399 ] Hadoop QA commented on YARN-4700: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176364#comment-15176364 ] Vrushali C commented on YARN-4700: -- Thanks [~Naganarasimha Garla] for the updated patch. Overall it looks

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175802#comment-15175802 ] Varun Saxena commented on YARN-4700: [~Naganarasimha], had a glance at the patch. It looks good to me

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175689#comment-15175689 ] Naganarasimha G R commented on YARN-4700: - As [~varun_saxena] pointed offline and the test results

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175020#comment-15175020 ] Hadoop QA commented on YARN-4700: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173510#comment-15173510 ] Varun Saxena commented on YARN-4700: Thanks [~Naganarasimha] for the patch. I have nothing further to

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172651#comment-15172651 ] Vrushali C commented on YARN-4700: -- Hi [~Naganarasimha] Thanks for the patch. I believe the constructor

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172617#comment-15172617 ] Hadoop QA commented on YARN-4700: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172453#comment-15172453 ] Naganarasimha G R commented on YARN-4700: - may be small correction in the patch, as

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172364#comment-15172364 ] Naganarasimha G R commented on YARN-4700: - Oops *TimelineStorageUtils.getTopOfTheDayTimestamp()* is

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172357#comment-15172357 ] Naganarasimha G R commented on YARN-4700: - Hi [~sjlee0], Based on the points from [~vrushalic] and

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172335#comment-15172335 ] Vrushali C commented on YARN-4700: -- Hi [~sjlee0] Yes, the flow activity table's row key always needs to

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172332#comment-15172332 ] Naganarasimha G R commented on YARN-4700: - It might be mostly the case of asynchronous or, its not

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172304#comment-15172304 ] Sangjin Lee commented on YARN-4700: --- I may have misread the comments in haste last Friday. If the

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170704#comment-15170704 ] Vrushali C commented on YARN-4700: -- Hi [~sjlee0] Let me add some more explanation. bq. Wait, I think

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170397#comment-15170397 ] Varun Saxena commented on YARN-4700: This event can be emitted on recovery for RUNNING apps as well. >

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170387#comment-15170387 ] Varun Saxena commented on YARN-4700: If we do not fix it in RM, as was the conclusion in those 2 JIRAs'

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170376#comment-15170376 ] Varun Saxena commented on YARN-4700: bq. I think we're using the day timestamp for a reason as this

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170102#comment-15170102 ] Li Lu commented on YARN-4700: - bq. In the code that writes to the flow activity table, can we check the

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170065#comment-15170065 ] Sangjin Lee commented on YARN-4700: --- Wait, I think we're using the day timestamp for a reason as this

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169939#comment-15169939 ] Vrushali C commented on YARN-4700: -- This was a good catch, thanks [~gtCarrera], [~varun_saxena] and

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169923#comment-15169923 ] Vrushali C commented on YARN-4700: -- I see. So when events are being replayed, we making new entries in the

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169764#comment-15169764 ] Naganarasimha G R commented on YARN-4700: - Yes these were the jira's where we discussed about it

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169756#comment-15169756 ] Naganarasimha G R commented on YARN-4700: - Thanks [~varun_saxena], I was also thinking in the same

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169734#comment-15169734 ] Varun Saxena commented on YARN-4700: When we recover an app we can also get start and finish time for

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169721#comment-15169721 ] Varun Saxena commented on YARN-4700: Hmm...I think the JIRAs' are YARN-3127 and YARN-4392. Let me see

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169531#comment-15169531 ] Li Lu commented on YARN-4700: - I think the redundant events are coming from the work preserving RM restart,

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168515#comment-15168515 ] Varun Saxena commented on YARN-4700: [~Naganarasimha], I remember there was a JIRA with you regarding

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168443#comment-15168443 ] Naganarasimha G R commented on YARN-4700: - Oops My mistake, i dint chk the default values set in

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168173#comment-15168173 ] Sangjin Lee commented on YARN-4700: --- I see. This is regarding the flow activity table, right? Writes to

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168104#comment-15168104 ] Li Lu commented on YARN-4700: - [~sjlee0] I checked the flowrun activity table and I can see the row keys for

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168080#comment-15168080 ] Sangjin Lee commented on YARN-4700: --- These are the current default values that are being set as the

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168063#comment-15168063 ] Sangjin Lee commented on YARN-4700: --- [~gtCarrera9], can you provide the exact row keys in this problem

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15167741#comment-15167741 ] Naganarasimha G R commented on YARN-4700: - Any suggestions for the above approach

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156208#comment-15156208 ] Li Lu commented on YARN-4700: - [~Naganarasimha] sure, please go ahead and take it. Thanks! > ATS storage has

[jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted

[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156126#comment-15156126 ] Naganarasimha G R commented on YARN-4700: - hi [~gtCarrera9], Thought of taking this issue up,