anyone attending Apache Big Data Miami?

Devs, I'm going to Apache Big Data in a couple weeks. Just wondering if anyone else on the list will be there. Perhaps we can get together, do a BOF or something. Cheers, Martin Serrano

[jira] [Comment Edited] (TWILL-217) AppMaster launcher should include eventHandler dependencies and nothing else from application

[ https://issues.apache.org/jira/browse/TWILL-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883217#comment-15883217 ] Martin Serrano edited comment on TWILL-217 at 2/24/17 6:09 PM: --- Yeah, I

[jira] [Commented] (TWILL-217) AppMaster launcher should include eventHandler dependencies and nothing else from application

[ https://issues.apache.org/jira/browse/TWILL-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883085#comment-15883085 ] Martin Serrano commented on TWILL-217: -- I know. I'm just suggesting the use of ClassPath to generate

[jira] [Commented] (TWILL-217) AppMaster launcher should include eventHandler dependencies and nothing else from application

[ https://issues.apache.org/jira/browse/TWILL-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882642#comment-15882642 ] Martin Serrano commented on TWILL-217: -- I was referring to the need to construct the jar

[jira] [Commented] (TWILL-217) AppMaster launcher should include eventHandler dependencies and nothing else from application

[ https://issues.apache.org/jira/browse/TWILL-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881207#comment-15881207 ] Martin Serrano commented on TWILL-217: -- Seems then that we will need some way to get the list of all

[jira] [Comment Edited] (TWILL-217) AppMaster launcher should include eventHandler dependencies and nothing else from application

[ https://issues.apache.org/jira/browse/TWILL-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881207#comment-15881207 ] Martin Serrano edited comment on TWILL-217 at 2/23/17 8:31 PM: --- Seems

Re: Release of Twill 0.10.0

Terence is correct. It is easily worked around by supplying the jar so shouldn't be consider a blocker. Martin Sent from my Verizon Wireless 4G LTE DROID On Feb 19, 2017 3:30 AM, Henry Saputra wrote: sounds good to me, thanks On Sat, Feb 18, 2017 at 11:40 PM, Terence

[jira] [Commented] (TWILL-217) AppMaster launcher should include eventHandler dependencies and nothing else from application

[ https://issues.apache.org/jira/browse/TWILL-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870007#comment-15870007 ] Martin Serrano commented on TWILL-217: -- That sounds reasonable. Thanks! > AppMaster launcher sho

[jira] [Created] (TWILL-218) The implicit jopt-simple dependency should be made explicit

Martin Serrano created TWILL-218: Summary: The implicit jopt-simple dependency should be made explicit Key: TWILL-218 URL: https://issues.apache.org/jira/browse/TWILL-218 Project: Apache Twill

classpath for the appmaster

Devs, In our full deployment environment I could not get the Kafka forwarding of logs to work. I kept getting Kafka errors on the AM trying to lookup the topic. Seeing as how I had been able to get this same runnable working in a unit test environment I figured it had to do with the

Re: twill dependency on logback

, Feb 10, 2017 at 9:24 AM, Martin Serrano <mar...@attivio.com> wrote: Terence, Correct, I don't want to use logback on the client side. slf4j is okay -- we ship with the log4j binding. In the twill containers, the AM doesn't matter for me, but the runnables does.So yes, flexi

Re: Gaining Control of Runnable after client terminates

Caleb, Sorry, made a mistake there. In the simple case the application has the simple classname, not the full one. -Martin On 02/10/2017 02:02 PM, Martin Serrano wrote: Caleb, The TwillRunnerService handles the ZK registration of applications for you. When TwillClient B starts

Re: twill dependency on logback

that you mentioned, I am just wondering what's your end goal looks like to shape a better solution for this. Terence Sent from my iPhone On Feb 9, 2017, at 11:33 AM, Martin Serrano <mar...@attivio.com> wrote: Terence, I'm familiar with the logback appender and Kafka code. My

Re: twill dependency on logback

seeing this behavior in deploying Twill app in latest YARN? - Henry On Wed, Feb 8, 2017 at 12:30 PM, Martin Serrano <mar...@attivio.com> wrote: > Hey Devs, > > It seems like the twill project goes through some pain to try to insulate > itself logging frameworks. I see use o

[jira] [Commented] (TWILL-215) Dependencies not on classpath lead to runtime startup error

[ https://issues.apache.org/jira/browse/TWILL-215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858607#comment-15858607 ] Martin Serrano commented on TWILL-215: -- The details got pretty messy. I still think a github diff/pr

twill dependency on logback

Hey Devs, It seems like the twill project goes through some pain to try to insulate itself logging frameworks. I see use of the slf4j API. However, the appmaster code has a dependency on logback via the org.apache.twill.internal.logging.Loggings class. The appmaster will not start up

[jira] [Commented] (TWILL-215) Dependencies not on classpath lead to runtime startup error

[ https://issues.apache.org/jira/browse/TWILL-215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858268#comment-15858268 ] Martin Serrano commented on TWILL-215: -- You get a {{ClassNotFoundException}} trying to start the app

[jira] [Commented] (TWILL-213) Increase of instances while starting up may lead to ignored retries and instance increases

[ https://issues.apache.org/jira/browse/TWILL-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856669#comment-15856669 ] Martin Serrano commented on TWILL-213: -- I've updated the description with what I think is happening

Re: interest in bare-bones cluster/agent solution?

be shared by different clustering control implementations. For instance, much of the code in YarnTwillPreparer, YarnContainerMain, YarnServiceMain, etc is not really yarn specific. Thoughts? Thanks, Martin On 02/01/2017 04:13 PM, Martin Serrano wrote: Hey Devs, I've been evangelizing Twill

[jira] [Updated] (TWILL-213) Increase of instances while starting up may lead to ignored retries and instance increases

[ https://issues.apache.org/jira/browse/TWILL-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Serrano updated TWILL-213: - Description: As seen in the test development for TWILL-181, if the number of instances

[jira] [Created] (TWILL-213) Increase of instances while starting up may lead to ignored retries and instance increases

Martin Serrano created TWILL-213: Summary: Increase of instances while starting up may lead to ignored retries and instance increases Key: TWILL-213 URL: https://issues.apache.org/jira/browse/TWILL-213

interest in bare-bones cluster/agent solution?

that Henry and I had a conversation at the last Apache Big Data that there had been talk of plugins for other clustering backends but I don't know if anything ever came of that. Cheers, Martin Serrano

Re: [DISCUSS] HasDependencies tagging interface

. On Tue, Jan 17, 2017 at 5:56 PM, Terence Yim <cht...@gmail.com> wrote: Hi Martin, Is it not doable via the TwillPreparer.withDependencies method? Terence On Tue, Jan 17, 2017 at 2:56 PM, Martin Serrano <mar...@attivio.com> wrote: Team, I have some untraceable dependencies for one of

[DISCUSS] HasDependencies tagging interface

Team, I have some untraceable dependencies for one of my runnables. It occurs to me that preparing and launching the runnable is not always the best place to define these dependencies (using withDependences method). The runnable itself will always have these deps (there is static xml

Re: Desired behavior at shutdown

Created https://issues.apache.org/jira/browse/TWILL-204 On 01/14/2017 11:47 AM, Martin Serrano wrote: Terence, I'm going to make a ticket and move discussion to that. Okay? -Martin

Re: Desired behavior at shutdown

a special log event emitted by AM after the AM shuts down all containers. The AM will wait for the message from controller before completely shutting down itself (with some reasonable timeout to avoid infinite wait). Terence On Fri, Jan 6, 2017 at 4:46 PM, Martin Serrano <mar...@attivio.com>

Re: Desired behavior at shutdown

twill apps put a heavy load on ZK). Do you have any suggestions how to do that? Thanks, Terence Sent from my iPhone On Jan 5, 2017, at 2:42 PM, Martin Serrano <mar...@attivio.com> wrote: Actually, after further investigation, I realize the server side has to be dealt with b

[jira] [Commented] (TWILL-181) Control the maximum number of retries for failed application starts

[ https://issues.apache.org/jira/browse/TWILL-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433574#comment-15433574 ] Martin Serrano commented on TWILL-181: -- I'm going to start taking a look at this. > Cont

Re: TwillRunnerService usage

Thanks! Same here. On 08/10/2016 05:48 PM, Henry Saputra wrote: > Hi Martin, > > Glad to see you in the dev@ list =) > > Looking forward to working with you and your team. > > - Henry > > > On Wed, Aug 10, 2016 at 1:39 PM, Martin Serrano <mar...@attivio.com>

[jira] [Commented] (TWILL-182) ApplicationBundler will overwrite dependencies with identical names

[ https://issues.apache.org/jira/browse/TWILL-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418006#comment-15418006 ] Martin Serrano commented on TWILL-182: -- will do. > ApplicationBundler will overwrite dependenc

Re: TwillRunnerService usage

acing more information > about individual app state/resource and cluster resources. Would you mind > file JIRA(s) for them as well? I will. I need to research a bit to write a good ticket. -Martin > Thanks, > Terence > > On Mon, Aug 8, 2016 at 1:07 PM, Martin Serrano <mar...@attiv

[jira] [Created] (TWILL-183) TwillRunnerService should provide a way to determine that TwillRunner background load is complete

Martin Serrano created TWILL-183: Summary: TwillRunnerService should provide a way to determine that TwillRunner background load is complete Key: TWILL-183 URL: https://issues.apache.org/jira/browse/TWILL-183