Sure, you are welcome to do so. You can also update the JIRA with your proposed change before sending the PR.
What is your Apache JIRA email? I have to add you to the twill project before I can assign the JIRA to you. Terence Sent from my iPhone > On Jul 3, 2017, at 12:57 AM, Сергей Филиппов <role...@gmail.com> wrote: > > Hi, Terence, > Thank you for description! Should I assign this issue to me? > > > Sergey > > сб, 1 июл. 2017 г. в 21:23, Terence Yim <cht...@gmail.com>: > >> Hi Sergey, >> >> I think you are talking about TWILL-147 ( >> https://issues.apache.org/jira/browse/TWILL-147), right? The idea for that >> is we don't need to start EmbeddedKafkaServer in AM at all, but rather it >> just take a configuration (via TwillPreparer, which can have a default >> value in the Configuration object passed to YarnTwillRunnerService), which >> the configuration specifies the Kafka broker list and topic that the AM >> will publish to. >> >> Since under this model, application logs from different application may >> send to the same Kafka topic (depends on the configuration), the LogEntry >> needs to be modified to carry the application and run id, so that the >> TwillController can filter based on it on the client side. >> >> Terence >> >> >> >>> On Sat, Jul 1, 2017 at 1:46 AM, Сергей Филиппов <role...@gmail.com> wrote: >>> >>> Hello, >>> I would like to implement possibility to use external kafka server for >> log >>> aggregation. >>> Now twill uses EmbededKafkaServer for that. I think implementation would >>> look like this: >>> 1. Add ZK path where kafka zk connection string will be stored. There >>> should should be only one such path per ApplicationMaster >>> 2. Use this path in ApplicationKafkaService while creating >>> EmbededKafkaService, if there on brokers right now >>> 3. For log aggregation there should be additional nodes in ZK for each >>> instance with kafka topic's name in it. Something like >>> "test-app-{UUID}-log". So publisher will send to this topic and consumer >>> will consume log messages on the job submission machine. >>> >>> What would you say? Is this sounds ok? >>> >>> Sergey >>> >>
