[ https://issues.apache.org/jira/browse/TEZ-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314960#comment-14314960 ]
Siddharth Seth commented on TEZ-1233: ------------------------------------- bq. Eg. If I have to take a job thats having issues and want to rerun it once with a different config for a given vertex then can I do that after this change? No, but that can be a follow up jira, if required. Like I mentioned in my previous comment - I think defining something like this via configuration is best left to frameworks making use of Tez. Programmatic configuration is still helpful for case like #allowed attempts when a vertex may not be deterministic, or for setting speculation per vertex (e.g. Pig determines a vertex to be a Map / Reduce and sets up speculation parameters to be based on MR parameters, which from the looks of the PIG jira is what they'll do - at least initially). bq. Thinking about the comments here and in TEZ-2003, I would expect that a clear ExecutionContext like API would be created that would allow different execution contexts to be specified per vertex in an unambiguous and clearly defined manner. Instead it seems like the functionality depends on correctly setting a bunch of independent configuration properties via a configuration based API. Apart from being error prone, in general we have tried to stay clear of configuration based API's in Tez. So the change concerns me. For starters, this is going to be driven by programmatic configuration. Ideally, there'll be a central context which can provide entities for the various plugin points - TaskCommunication Plan, Scheduling, Launch. These will be central entities (specified at DAG level) which can be shared across multiple vertices. Individual vertices still need to specify what they intend to use - which can be via configuration or again via an explicit API call. IAC, that's a discussion for TEZ-2003, so I'll stop here. > Allow configuration of framework parameters per vertex > ------------------------------------------------------ > > Key: TEZ-1233 > URL: https://issues.apache.org/jira/browse/TEZ-1233 > Project: Apache Tez > Issue Type: Improvement > Reporter: Siddharth Seth > Assignee: Jeff Zhang > Attachments: TEZ-1233-1.patch, TEZ-1233-2.patch, TEZ-1233-3.patch, > TEZ-1233-4.patch > > > Currently, configuration properties specific in AMConfiguration are used to > configure tasks - e.g. memory configs, process tree, etc. These should be > configurable at a Vertex level. > Also, tasks end up reading these configs from the dist cache each time - > would be simpler to send over the wire as part of the TaskSpec. -- This message was sent by Atlassian JIRA (v6.3.4#6332)