[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996675#comment-13996675
]
Bikas Saha commented on YARN-2052:
--
The RM identifier is effectively the epoch for the RM.
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13997781#comment-13997781
]
Tsuyoshi OZAWA commented on YARN-2052:
--
{quote}
e.g. container_XXX_1000 after epoch 1
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028707#comment-14028707
]
Jian He commented on YARN-2052:
---
bq. The monotonically increasing sequence could be a combina
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028717#comment-14028717
]
Tsuyoshi OZAWA commented on YARN-2052:
--
s/epoch number is/epoch number should be/
> C
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028716#comment-14028716
]
Tsuyoshi OZAWA commented on YARN-2052:
--
{quote}
we probably need to persist the epoch
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028728#comment-14028728
]
Jian He commented on YARN-2052:
---
On RM startup, RM may need to read and rewrite this epoch nu
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028734#comment-14028734
]
Bikas Saha commented on YARN-2052:
--
Dont we already read and write synchronously from the
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028744#comment-14028744
]
Jian He commented on YARN-2052:
---
bq. Dont we already read and write synchronously from the st
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029539#comment-14029539
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~jianhe], I think it's OK after fencing opera
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029578#comment-14029578
]
Jian He commented on YARN-2052:
---
bq. but one problem is recover() is invoked before the fenc
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029619#comment-14029619
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~jianhe], my bad, you're right. I misread tha
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14030936#comment-14030936
]
Jian He commented on YARN-2052:
---
I'm kind of leaning towards this approach even with extra pe
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14030966#comment-14030966
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
bq. e.g. container_XXX_1000
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14030971#comment-14030971
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
I forgot to add one more not
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031223#comment-14031223
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~jianhe] and [~vinodkv], thank you for the co
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031258#comment-14031258
]
Tsuyoshi OZAWA commented on YARN-2052:
--
{quote}
The only problem is users who are inte
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031997#comment-14031997
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
bq. BTW, I found that Conver
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032037#comment-14032037
]
Hadoop QA commented on YARN-2052:
-
{color:green}+1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032053#comment-14032053
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~vinodkv], thank you for the comment. I agree
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032061#comment-14032061
]
Hadoop QA commented on YARN-2052:
-
{color:green}+1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14033101#comment-14033101
]
Jian He commented on YARN-2052:
---
Application itself may possibly use Container.getId to diffe
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14033640#comment-14033640
]
Tsuyoshi OZAWA commented on YARN-2052:
--
{quote}
BTW, I think we should update Checkpoi
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14033662#comment-14033662
]
Hadoop QA commented on YARN-2052:
-
{color:green}+1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034448#comment-14034448
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
bq. BTW, I think we should u
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034452#comment-14034452
]
Jian He commented on YARN-2052:
---
Another question is how are we going to show the containerId
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034474#comment-14034474
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Vinod, OK. I'll create new JIRA to address it.
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034541#comment-14034541
]
Jian He commented on YARN-2052:
---
Seems more problem with the randomId approach if user wants
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034637#comment-14034637
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Basically, I agree with the approach. If we ta
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034691#comment-14034691
]
Bikas Saha commented on YARN-2052:
--
bq. Had an offline discussion with Vinod. Maybe it's s
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034702#comment-14034702
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~bikassaha], Yes, I think it's same.
> Conta
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034716#comment-14034716
]
Jian He commented on YARN-2052:
---
bq. One simple way is to fallback to RM-restart implemented
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034722#comment-14034722
]
Tsuyoshi OZAWA commented on YARN-2052:
--
I meant starting apps from a clean state after
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034731#comment-14034731
]
Bikas Saha commented on YARN-2052:
--
Why would ContainerId#compareTo fail? Existing contain
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034732#comment-14034732
]
Bikas Saha commented on YARN-2052:
--
Ah. I did not see the rest of the comment. Yes. Intege
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034746#comment-14034746
]
Tsuyoshi OZAWA commented on YARN-2052:
--
{quote}
We should make it a long in the same r
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034876#comment-14034876
]
Bikas Saha commented on YARN-2052:
--
With 32 bits for epoch number we have 4 billion restar
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14036488#comment-14036488
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~bikassaha], [~vinodkv], Sure, I'll update i
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037463#comment-14037463
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Updated patch to address the comments by Bikas
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037481#comment-14037481
]
Hadoop QA commented on YARN-2052:
-
{color:green}+1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037875#comment-14037875
]
Jian He commented on YARN-2052:
---
I think the conclusion was to not add any new fields into Co
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14041494#comment-14041494
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Jian, thank you for clarifying. I'm working to
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14041537#comment-14041537
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Brief design is as follows:
1. Adding getter
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14041548#comment-14041548
]
Jian He commented on YARN-2052:
---
sounds good
> ContainerId creation after work preserving re
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14041565#comment-14041565
]
Tsuyoshi OZAWA commented on YARN-2052:
--
OK!
> ContainerId creation after work preserv
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043264#comment-14043264
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043599#comment-14043599
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044059#comment-14044059
]
Jian He commented on YARN-2052:
---
- we can update the structure graph in ZKRMStateStore to ref
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045164#comment-14045164
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Thank you for the review, Jian! Updated a patc
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045240#comment-14045240
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045265#comment-14045265
]
Jian He commented on YARN-2052:
---
hmm, we need to think about toString compatibility too, let'
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045298#comment-14045298
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
I cursorily looked. Some sug
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045313#comment-14045313
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Thank you for suggestions, Vinod! I'll update
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045345#comment-14045345
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045373#comment-14045373
]
Tsuyoshi OZAWA commented on YARN-2052:
--
* Renamed RMEpochProto to EpochProto.
* Bump u
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045412#comment-14045412
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045681#comment-14045681
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045720#comment-14045720
]
Tsuyoshi OZAWA commented on YARN-2052:
--
The test failure of TestRMApplicationHistoryWr
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046419#comment-14046419
]
Jian He commented on YARN-2052:
---
Patch looks good overall, can you update MemoryStateStore al
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046504#comment-14046504
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
Not related to this patch, b
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046530#comment-14046530
]
Jian He commented on YARN-2052:
---
- Actually, FileSystem and ZK state store has separate versi
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046544#comment-14046544
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046557#comment-14046557
]
Jian He commented on YARN-2052:
---
can you rename RMEpoch.java to Epoch and similar RMEpochPBim
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046703#comment-14046703
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046707#comment-14046707
]
Tsuyoshi OZAWA commented on YARN-2052:
--
The test failure is not related.
> ContainerI
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047000#comment-14047000
]
Jian He commented on YARN-2052:
---
found that may be we can change epochProto to use int64 also
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047020#comment-14047020
]
Jian He commented on YARN-2052:
---
looks good, pending jenkins.
> ContainerId creation after w
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047026#comment-14047026
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047031#comment-14047031
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Thank you for the review, Jian. The test failu
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047202#comment-14047202
]
Hudson commented on YARN-2052:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #5799 (See
[https:
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047207#comment-14047207
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
Shouldn't epoch have a defau
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047210#comment-14047210
]
Jian He commented on YARN-2052:
---
bq. For numeric types, the default value is zero.
Copied fro
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047563#comment-14047563
]
Hudson commented on YARN-2052:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #599 (See
[https://b
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047665#comment-14047665
]
Hudson commented on YARN-2052:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1817 (See
[htt
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047686#comment-14047686
]
Hudson commented on YARN-2052:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1790 (See
[https://
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048216#comment-14048216
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Thank you for the review and comments, Jian, V
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048225#comment-14048225
]
Tsuyoshi OZAWA commented on YARN-2052:
--
I'm planning to define epoch format on YARN-22
76 matches
Mail list logo
