[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13808612#comment-13808612 ] Vinod Kumar Vavilapalli commented on YARN-947: -- Tx Mayank too for the reviews! Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch, YARN-947.6.patch, YARN-947.8.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803262#comment-13803262 ] Vinod Kumar Vavilapalli commented on YARN-947: -- Quick comments on the patch: - We don't need a new YarnContainerState, already have ContainerState. It can be argued that ContainerState should have more information. In that case, we should directly change ContainerState itself - but have to think about compatibility. Do we ever expose ContainerState in any existing APIs in 2.2.0? In any case I'd like to postpone that discussion into a separate JIRA and use ContainerState itself for now. - We in future may need a separate ApplicationAttemptStartDataProto and ApplicationAttemptRegisteredDataProto. For now, we can just store both of them after AM registration, but that increases the possibility of loss of information on RM restart. May be another JIRA. - We will need a APP_ATTEMPT_FINAL_SAVING state as being discussed on YARN-891. Please add it. - I think we should separate client facing records from the event records. For e.g. ContainerHistoryData.java shouldn't be in server package. Again another JIRA, but this one's more important. - SchedulerUtils.java changes are unnecessary. Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch, YARN-947.6.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803391#comment-13803391 ] Vinod Kumar Vavilapalli commented on YARN-947: -- Turns out ContainerHistoryData is a server side record that is used only by the reader. In that case, we can keep it in the server package but I don't think it should be a pb implementation. Thinking beyond, may be we should ditch AppilcationHistoryData etc completely and simply use ApplicationReport, ApplicationAttemptReport etc in the Reader. Thoughts? Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch, YARN-947.6.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803458#comment-13803458 ] Mayank Bansal commented on YARN-947: [~zjshen] I think we should have seprate store implemetation to clients facing that way we can change whenever we need to at store level. Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch, YARN-947.6.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803542#comment-13803542 ] Zhijie Shen commented on YARN-947: -- Thanks for the comments. Having discussion with [~vinodkv] and [~mayank_bansal]. Here's some update: 1. Remove PBImpl, and change HistoryData to a simple object that containing of the fields. Discuss the issue of changing readers to Report directly in a follow up ticket. 2. Now we can only modify the yarn_protos and YarnApplicationAttemptState to add the new state enum. Need to file a follow up ticket to update the conversion method in RMServerUtils whenever YARN-891 is done Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch, YARN-947.6.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803716#comment-13803716 ] Vinod Kumar Vavilapalli commented on YARN-947: -- Looks good, +1. Will check it in. Please file all the follow up tickets. Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch, YARN-947.6.patch, YARN-947.8.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13803743#comment-13803743 ] Zhijie Shen commented on YARN-947: -- Thanks for the review. Will file the follow up tickets. As it's ready to go. I'll update YARN-956 and YARN-975 accordingly Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch, YARN-947.6.patch, YARN-947.8.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801605#comment-13801605 ] Mayank Bansal commented on YARN-947: BTW yarnapplicationattemptstate is already part of YARN-978. I think we should use the same states In AHS as we have in RM and we don't think we should use others states then RM states as thats what we show users to RM UI/CLI to avoid confusion. Thoughts? Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802013#comment-13802013 ] Zhijie Shen commented on YARN-947: -- bq. BTW yarnapplicationattemptstate is already part of YARN-978. Yes, I know that. [~jeffgx619], are you fine if I takes YarnApplicationAttemptState part from your patch of YARN-978? This is because we want the branch is always compilable. bq. I think we should use the same states In AHS as we have in RM and we don't think we should use others states then RM states as thats what we show users to RM UI/CLI to avoid confusion. It's not the other states. Actually web UI and CLI is showing YarnApplicationState instead or RMAppState, as YarnApplicationState is a user oriented object (it is a proto). That's why we propose YarnApplicationAttemptState in YARN-978, and we also need YarnContainerState as well. Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[
https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802024#comment-13802024
]
Hadoop QA commented on YARN-947:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609679/YARN-947.6.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2257//console
This message is automatically generated.
Defining the history data classes for the implementation of the
reading/writing interface
-
Key: YARN-947
URL: https://issues.apache.org/jira/browse/YARN-947
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Fix For: YARN-321
Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch,
YARN-947.4.patch, YARN-947.5.patch, YARN-947.6.patch
We need to define the history data classes have the exact fields to be
stored. Therefore, all the implementations don't need to have the duplicate
logic to exact the required information from RMApp, RMAppAttempt and
RMContainer.
We use protobuf to define these classes, such that they can be ser/des
to/from bytes, which are easier for persistence.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802073#comment-13802073 ] Mayank Bansal commented on YARN-947: bq. Yes, I know that. Xuan Gong, are you fine if I takes YarnApplicationAttemptState part from your patch of YARN-978? This is because we want the branch is always compilable. Instead of taking it from different patch , can't we just depends on that JIRA and commit that JIRA first. That JIRA is anyway reviwed and we need that in any case. Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch, YARN-947.6.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802349#comment-13802349 ] Mayank Bansal commented on YARN-947: [~zjshen] and I had an offline discussion, and we decided to go ahed and add the yarn application attempt state to part of this JIRA] due to YARN-978 needs some more work and to unblock this JIRA. Looks goot to me +1 Thanks, Mayank Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch, YARN-947.6.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801038#comment-13801038 ] Mayank Bansal commented on YARN-947: [~zjshen] I think we still haven't decided on container status and container exit status to be part of container reports or not? This patch seems to remove container status Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801056#comment-13801056 ] Mayank Bansal commented on YARN-947: [~zjshen] Some comments: ApplicationAttemptFinishData should not have FinalApplicationStatus as we already removed them as part of attempt report. SImilarly from ApplicationAttemptHistoryData also should not contain FinalApplicationStatus Instead they should contain ApplicationAttemptState. Same comments for the PB classes Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801067#comment-13801067 ] Zhijie Shen commented on YARN-947: -- ApplicationAttemptState will make this jira depend on YARN-978 as well, which involves more work of review? Shall we simplify the work here? Anyway, it's a single field change. If we finally conclude that ApplicationAttemptState is necessary in YARN-978, it won't be big change for the history data records. Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801092#comment-13801092 ] Mayank Bansal commented on YARN-947: Yes it will make it depends on YARN-978 , however i think we decide as part of YARN-978 that we will not have final application status part of application attempt. And we should have application attempt state as part of application attempt history data as reports depends on that. YARN-978 is already done and reviewed. Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801470#comment-13801470 ] Zhijie Shen commented on YARN-947: -- I'm thinking that instead of putting rmappstate/rmappattemptstate/rmcontainerstate into the history data records, shall we put yarnapplicationstate/yarnapplicationattemptstate/yarncontainerstate instead (the latter two need to be defined)? This is because essentially we want AHS to be independent of RM (as is described in the proposal), rmstate is the internal values, and RM should expose the user friendly state to users. Here AHS is the user. Thoughts? Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch, YARN-947.4.patch, YARN-947.5.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[
https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798157#comment-13798157
]
Hadoop QA commented on YARN-947:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12608975/YARN-947.4.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2201//console
This message is automatically generated.
Defining the history data classes for the implementation of the
reading/writing interface
-
Key: YARN-947
URL: https://issues.apache.org/jira/browse/YARN-947
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Fix For: YARN-321
Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch,
YARN-947.4.patch
We need to define the history data classes have the exact fields to be
stored. Therefore, all the implementations don't need to have the duplicate
logic to exact the required information from RMApp, RMAppAttempt and
RMContainer.
We use protobuf to define these classes, such that they can be ser/des
to/from bytes, which are easier for persistence.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[ https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797452#comment-13797452 ] Mayank Bansal commented on YARN-947: Over all looks good, however for point 5 please go through this comment https://issues.apache.org/jira/browse/YARN-1123?focusedCommentId=13797450page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13797450 Defining the history data classes for the implementation of the reading/writing interface - Key: YARN-947 URL: https://issues.apache.org/jira/browse/YARN-947 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-321 Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch We need to define the history data classes have the exact fields to be stored. Therefore, all the implementations don't need to have the duplicate logic to exact the required information from RMApp, RMAppAttempt and RMContainer. We use protobuf to define these classes, such that they can be ser/des to/from bytes, which are easier for persistence. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[
https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13796399#comment-13796399
]
Hadoop QA commented on YARN-947:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12608648/YARN-947.3.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2184//console
This message is automatically generated.
Defining the history data classes for the implementation of the
reading/writing interface
-
Key: YARN-947
URL: https://issues.apache.org/jira/browse/YARN-947
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Fix For: YARN-321
Attachments: YARN-947.1.patch, YARN-947.2.patch, YARN-947.3.patch
We need to define the history data classes have the exact fields to be
stored. Therefore, all the implementations don't need to have the duplicate
logic to exact the required information from RMApp, RMAppAttempt and
RMContainer.
We use protobuf to define these classes, such that they can be ser/des
to/from bytes, which are easier for persistence.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-947) Defining the history data classes for the implementation of the reading/writing interface
[
https://issues.apache.org/jira/browse/YARN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13716035#comment-13716035
]
Hadoop QA commented on YARN-947:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593644/YARN-947.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1548//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1548//console
This message is automatically generated.
Defining the history data classes for the implementation of the
reading/writing interface
-
Key: YARN-947
URL: https://issues.apache.org/jira/browse/YARN-947
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Attachments: YARN-947.1.patch
We need to define the history data classes have the exact fields to be
stored. Therefore, all the implementations don't need to have the duplicate
logic to exact the required information from RMApp, RMAppAttempt and
RMContainer.
We use protobuf to define these classes, such that they can be ser/des
to/from bytes, which are easier for persistence.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
