[jira] [Updated] (TEZ-3866) add ability to pass information between plugins
[ https://issues.apache.org/jira/browse/TEZ-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated TEZ-3866: -- Description: Distinct from TEZ-3815, since even with a hack that makes plugins aware of each other, they cannot store task-specific info without calling into each others' per-task structures. In particular, I need a piece of info that is generated when the custom scheduler calls getContext().taskAllocated, to be propagated to communicator plugin registerRunningTaskAttempt, so that the custom communicator could include additional info in task submission. There doesn't seem to be any means to do it now. was: Distinct from TEZ-3815, since even with the hack that makes plugins aware of each other, they cannot store task-specific info without calling into each others' per-task structures. In particular, I need a piece of info that is generated when the custom scheduler calls getContext().taskAllocated, to be propagated to communicator plugin registerRunningTaskAttempt, so that the custom communicator could include additional info in task submission. There doesn't seem to be any means to do it now. > add ability to pass information between plugins > --- > > Key: TEZ-3866 > URL: https://issues.apache.org/jira/browse/TEZ-3866 > Project: Apache Tez > Issue Type: Bug >Reporter: Sergey Shelukhin > > Distinct from TEZ-3815, since even with a hack that makes plugins aware of > each other, they cannot store task-specific info without calling into each > others' per-task structures. > In particular, I need a piece of info that is generated when the custom > scheduler calls getContext().taskAllocated, to be propagated to communicator > plugin registerRunningTaskAttempt, so that the custom communicator could > include additional info in task submission. There doesn't seem to be any > means to do it now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TEZ-3866) add ability to pass information between plugins
[ https://issues.apache.org/jira/browse/TEZ-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254677#comment-16254677 ] Sergey Shelukhin commented on TEZ-3866: --- cc [~hagleitn] [~aplusplus] this is needed for Hive workload management to avoid some major ugly code that exists right now :) > add ability to pass information between plugins > --- > > Key: TEZ-3866 > URL: https://issues.apache.org/jira/browse/TEZ-3866 > Project: Apache Tez > Issue Type: Bug >Reporter: Sergey Shelukhin > > Distinct from TEZ-3815, since even with the hack that makes plugins aware of > each other, they cannot store task-specific info without calling into each > others' per-task structures. > In particular, I need a piece of info that is generated when the custom > scheduler calls getContext().taskAllocated, to be propagated to communicator > plugin registerRunningTaskAttempt, so that the custom communicator could > include additional info in task submission. There doesn't seem to be any > means to do it now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (TEZ-3866) add ability to pass information between plugins
Sergey Shelukhin created TEZ-3866: - Summary: add ability to pass information between plugins Key: TEZ-3866 URL: https://issues.apache.org/jira/browse/TEZ-3866 Project: Apache Tez Issue Type: Bug Reporter: Sergey Shelukhin Distinct from TEZ-3815, since even with the hack that makes plugins aware of each other, they cannot store task-specific info without calling into each others' per-task structures. In particular, I need a piece of info that is generated when the custom scheduler calls getContext().taskAllocated, to be propagated to communicator plugin registerRunningTaskAttempt, so that the custom communicator could include additional info in task submission. There doesn't seem to be any means to do it now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TEZ-3855) Allow vertex manager to send event to processor
[ https://issues.apache.org/jira/browse/TEZ-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254264#comment-16254264 ] Zhiyuan Yang commented on TEZ-3855: --- [~gopalv] It is sufficient. With addendum patch, buffer object is sealed intact within the event. It can be get multiple times, either for heartbeat or processor. Thanks for review! I'll commit this soon. > Allow vertex manager to send event to processor > --- > > Key: TEZ-3855 > URL: https://issues.apache.org/jira/browse/TEZ-3855 > Project: Apache Tez > Issue Type: Bug >Reporter: Zhiyuan Yang >Assignee: Zhiyuan Yang >Priority: Blocker > Fix For: 0.9.1 > > Attachments: TEZ-3855.1.patch, TEZ-3855.2.patch, TEZ-3855.3.patch, > TEZ-3855.addendum.patch, TEZ-3855.prototype.patch > > > Hive is trying to propagate some info from vertex manager to processor. The > task framework support processor event but there is no interface for VM to > send event out. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
