Re: forking off releases/branch-0.60 to work with hadoop-2.6

2014-11-05 Thread Sumit Mohanty
I am proposing that SLIDER-620 should be part of 0.60 release. While this is a debugging improvement, it seems to be a critical one especially when the error happens due to some bad configuration. I am working on the fix and testing it. ETA is Nov 6th morning. Let me know if there are any concern

Re: forking off releases/branch-0.60 to work with hadoop-2.6

2014-11-05 Thread Gour Saha
Steve, The changes for SLIDER-555 (AM log4j) has been merged into develop and releases/slider-0.60. -Gour On Wed, Nov 5, 2014 at 12:50 PM, Steve Loughran wrote: > FYI, there's going to be an RC of Hadoop 2.6 this weekend. To celebrate > this I'm creating a slider-0.60 release which will be in

[jira] [Commented] (SLIDER-555) AM needs to get log aggregation friendly log4j

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199930#comment-14199930 ] ASF subversion and git services commented on SLIDER-555: Commit f6

[jira] [Commented] (SLIDER-555) AM needs to get log aggregation friendly log4j

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199931#comment-14199931 ] ASF subversion and git services commented on SLIDER-555: Commit 83

[jira] [Created] (SLIDER-620) Execute command should allow polling for daemons that may stop within seconds

2014-11-05 Thread Sumit Mohanty (JIRA)
Sumit Mohanty created SLIDER-620: Summary: Execute command should allow polling for daemons that may stop within seconds Key: SLIDER-620 URL: https://issues.apache.org/jira/browse/SLIDER-620 Project:

[jira] [Updated] (SLIDER-620) Execute command should allow polling for daemons that may stop within seconds

2014-11-05 Thread Sumit Mohanty (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-620: - Sprint: Slider November #1 > Execute command should allow polling for daemons that may stop within

Re: Maven repository?

2014-11-05 Thread hsy...@gmail.com
This is git repository? What I mean is if I want to write some code depends on slider library. How would I include the dependency in pom.xml? Thanks! On Wed, Nov 5, 2014 at 7:23 PM, Ted Yu wrote: > Slider maven repo is here: > > https://git-wip-us.apache.org/repos/asf/incubator-slider.git > >

Re: Maven repository?

2014-11-05 Thread Ted Yu
Slider maven repo is here: https://git-wip-us.apache.org/repos/asf/incubator-slider.git You can checkout develop branch. Cheers On Wed, Nov 5, 2014 at 6:36 PM, hsy...@gmail.com wrote: > Is there a public maven repository that I can checkout the slider library? > > Best, > Siyuan >

Maven repository?

2014-11-05 Thread hsy...@gmail.com
Is there a public maven repository that I can checkout the slider library? Best, Siyuan

[jira] [Commented] (SLIDER-555) AM needs to get log aggregation friendly log4j

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199652#comment-14199652 ] ASF subversion and git services commented on SLIDER-555: Commit 81

[jira] [Resolved] (SLIDER-555) AM needs to get log aggregation friendly log4j

2014-11-05 Thread Gour Saha (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha resolved SLIDER-555. -- Resolution: Fixed > AM needs to get log aggregation friendly log4j > ---

[jira] [Commented] (SLIDER-555) AM needs to get log aggregation friendly log4j

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199603#comment-14199603 ] ASF subversion and git services commented on SLIDER-555: Commit a2

Re: Is it able to make AM to restart from previous state?

2014-11-05 Thread hsy...@gmail.com
Steve, I found out from the code that everything is kept in history folder in hdfs. You mentioned that if I add new component, the history layout would be discarded. What if I add more component instances in configuration? Do you try to launch instance from previous node and add new instance from

Re: Is it able to make AM to restart from previous state?

2014-11-05 Thread hsy...@gmail.com
Thanks Steve, Is No 1 a new feature in YARN (Not released yet)? And you mentioned slider saves the location in history files. What are the history files and where is it stored? Is it in HDFS? If the one of the previous machines is gone, will it try to get resource from new labeled machine? Tha

Re: slider report

2014-11-05 Thread Billie Rinaldi
Sure, you could add a couple of sentences to show what we've done. No need to go into much detail, as the goal is to illustrate project health rather than any particular technical milestone. On Wed, Nov 5, 2014 at 1:04 PM, Sumit Mohanty wrote: > Do we need more details on what we did in terms f

Re: slider report

2014-11-05 Thread Sumit Mohanty
Do we need more details on what we did in terms features developed etc? If yes, I can edit and add some details. On Wed, Nov 5, 2014 at 12:25 PM, Ted Yu wrote: > bq. We have added one committer / PMC member. > > I think we should mention Gour's name in the above sentence. > > lgtm > > On Wed, No

forking off releases/branch-0.60 to work with hadoop-2.6

2014-11-05 Thread Steve Loughran
FYI, there's going to be an RC of Hadoop 2.6 this weekend. To celebrate this I'm creating a slider-0.60 release which will be in sync. 1. The branch already exists: releases/branch-0.60 please try and stabilize this. new features into develop/ 2. I did one last-minute feature addition to slider

Re: Is it able to make AM to restart from previous state?

2014-11-05 Thread Steve Loughran
On 5 November 2014 20:21, hsy...@gmail.com wrote: > Hi guys, > > I noticed in the code when a container fails it will try to relaunch from > the same node. My question is if I restart whole application(Ex. AM got > killed, or manually restart the app). Does slider try to launch all > containers f

[jira] [Commented] (SLIDER-544) Release Slider 0.6.0

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199050#comment-14199050 ] ASF subversion and git services commented on SLIDER-544: Commit 4e

[jira] [Commented] (SLIDER-602) AgentRegistryIT-testAgentRegistry failing: ~ passdown

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199044#comment-14199044 ] ASF subversion and git services commented on SLIDER-602: Commit 7f

[jira] [Commented] (SLIDER-604) AppsThroughAgentIT brittle against slow startup

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199046#comment-14199046 ] ASF subversion and git services commented on SLIDER-604: Commit b6

[jira] [Commented] (SLIDER-544) Release Slider 0.6.0

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199049#comment-14199049 ] ASF subversion and git services commented on SLIDER-544: Commit 50

[jira] [Commented] (SLIDER-544) Release Slider 0.6.0

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199048#comment-14199048 ] ASF subversion and git services commented on SLIDER-544: Commit 50

[jira] [Commented] (SLIDER-614) Release Slider 0.52 against old registry code

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199047#comment-14199047 ] ASF subversion and git services commented on SLIDER-614: Commit 57

[jira] [Commented] (SLIDER-603) NPE in AppsThroughAgentIT

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199045#comment-14199045 ] ASF subversion and git services commented on SLIDER-603: Commit ec

[jira] [Resolved] (SLIDER-619) registry list and listconf commands to save output to --out file

2014-11-05 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SLIDER-619. --- Resolution: Fixed > registry list and listconf commands to save output to --out file > ---

[jira] [Commented] (SLIDER-619) registry list and listconf commands to save output to --out file

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199033#comment-14199033 ] ASF subversion and git services commented on SLIDER-619: Commit 16

[jira] [Commented] (SLIDER-616) re-enable AMFailuresIT

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199025#comment-14199025 ] ASF subversion and git services commented on SLIDER-616: Commit 89

[jira] [Commented] (SLIDER-544) Release Slider 0.6.0

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199030#comment-14199030 ] ASF subversion and git services commented on SLIDER-544: Commit 84

[jira] [Commented] (SLIDER-619) registry list and listconf commands to save output to --out file

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199024#comment-14199024 ] ASF subversion and git services commented on SLIDER-619: Commit 70

[jira] [Commented] (SLIDER-525) Storm app package for windows does not report component status correctly

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199022#comment-14199022 ] ASF subversion and git services commented on SLIDER-525: Commit 66

[jira] [Commented] (SLIDER-544) Release Slider 0.6.0

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199023#comment-14199023 ] ASF subversion and git services commented on SLIDER-544: Commit 84

[jira] [Commented] (SLIDER-616) re-enable AMFailuresIT

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199026#comment-14199026 ] ASF subversion and git services commented on SLIDER-616: Commit ad

[jira] [Commented] (SLIDER-619) registry list and listconf commands to save output to --out file

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199031#comment-14199031 ] ASF subversion and git services commented on SLIDER-619: Commit 70

[jira] [Commented] (SLIDER-616) re-enable AMFailuresIT

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199021#comment-14199021 ] ASF subversion and git services commented on SLIDER-616: Commit bb

[jira] [Commented] (SLIDER-616) re-enable AMFailuresIT

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199020#comment-14199020 ] ASF subversion and git services commented on SLIDER-616: Commit da

[jira] [Updated] (SLIDER-619) registry list and listconf commands to save output to --out file

2014-11-05 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SLIDER-619: -- Sprint: Slider November #1 > registry list and listconf commands to save output to --out file >

[jira] [Created] (SLIDER-619) registry list and listconf commands to save output to --out file

2014-11-05 Thread Steve Loughran (JIRA)
Steve Loughran created SLIDER-619: - Summary: registry list and listconf commands to save output to --out file Key: SLIDER-619 URL: https://issues.apache.org/jira/browse/SLIDER-619 Project: Slider

Re: slider report

2014-11-05 Thread Ted Yu
bq. We have added one committer / PMC member. I think we should mention Gour's name in the above sentence. lgtm On Wed, Nov 5, 2014 at 12:12 PM, Billie Rinaldi wrote: > I drafted the following Slider report. If anyone wants to edit or add > anything, please do so on the wiki: > https://wiki.a

Is it able to make AM to restart from previous state?

2014-11-05 Thread hsy...@gmail.com
Hi guys, I noticed in the code when a container fails it will try to relaunch from the same node. My question is if I restart whole application(Ex. AM got killed, or manually restart the app). Does slider try to launch all containers from the nodes where it was running? Thanks! Best, Siyuan

slider report

2014-11-05 Thread Billie Rinaldi
I drafted the following Slider report. If anyone wants to edit or add anything, please do so on the wiki: https://wiki.apache.org/incubator/November2014 Slider Slider is a collection of tools and technologies to package, deploy, and manage long running applications on Apache

Re: Memcahed doesn't start with latest slider code

2014-11-05 Thread Jon Maron
As Sumit indicated, the "yarn logs --applicationId ” command should dump those logs. — Jon On Nov 5, 2014, at 2:13 PM, Pushkar Raste wrote: > I am new to both slider and hadoop. Where can I find agent logs? > > On Wed, Nov 5, 2014 at 2:02 PM, Jon Maron wrote: > >> It may help to provide any

Re: Memcahed doesn't start with latest slider code

2014-11-05 Thread Pushkar Raste
I am new to both slider and hadoop. Where can I find agent logs? On Wed, Nov 5, 2014 at 2:02 PM, Jon Maron wrote: > It may help to provide any of the agent logs or memcached logs from the > node managers. This could occur for any number of reasons including wrong > java_home value. > > — Jon >

Re: Memcahed doesn't start with latest slider code

2014-11-05 Thread Sumit Mohanty
These three properties in appConfig.json will be my initial suspect (I am using the default values from the appConfig-default.json in the git repo): "application.def": ".slider/package/MEMCACHED/jmemcached-1.0.0.zip", Is the app pkg in the correct location in HDFS? The above path is relative t

Re: Memcahed doesn't start with latest slider code

2014-11-05 Thread Jon Maron
It may help to provide any of the agent logs or memcached logs from the node managers. This could occur for any number of reasons including wrong java_home value. — Jon On Nov 5, 2014, at 1:55 PM, Pushkar Raste wrote: > May I should provide entire log > > 2014-11-05 18:27:24,345 [main] INFO

Re: Memcahed doesn't start with latest slider code

2014-11-05 Thread Pushkar Raste
May I should provide entire log 2014-11-05 18:27:24,345 [main] INFO Configuration.deprecation - slider.registry.path is deprecated. Instead, use hadoop.registry.zk.root 2014-11-05 18:27:24,350 [main] INFO appmaster.SliderAppMaster - AM configuration: fs.defaultFS=hdfs://localhost:9000 hadoop.reg

Memcahed doesn't start with latest slider code

2014-11-05 Thread Pushkar Raste
I tried deploy jmemcached using latest slider built from dev branch. I see following error 2014-11-05 18:28:22,804 [AMRM Callback Handler Thread] ERROR appmaster.SliderAppMaster - Role instance RoleInstance{role='MEMCACHED', id='container_1415211406300_0001_01_02', container=ContainerID=contai

[jira] [Created] (SLIDER-618) Keep only one folder for conf

2014-11-05 Thread Sumit Mohanty (JIRA)
Sumit Mohanty created SLIDER-618: Summary: Keep only one folder for conf Key: SLIDER-618 URL: https://issues.apache.org/jira/browse/SLIDER-618 Project: Slider Issue Type: Bug Compon

[jira] [Created] (SLIDER-617) review conf/ directories in slider-assembly

2014-11-05 Thread Steve Loughran (JIRA)
Steve Loughran created SLIDER-617: - Summary: review conf/ directories in slider-assembly Key: SLIDER-617 URL: https://issues.apache.org/jira/browse/SLIDER-617 Project: Slider Issue Type: Test

[jira] [Commented] (SLIDER-585) Localize SSL certs for apps

2014-11-05 Thread Jonathan Maron (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198706#comment-14198706 ] Jonathan Maron commented on SLIDER-585: --- If the application is configured for two wa

[jira] [Resolved] (SLIDER-525) Storm app package for windows does not report component status correctly

2014-11-05 Thread Sumit Mohanty (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-525. -- Resolution: Fixed > Storm app package for windows does not report component status correctly > -

[jira] [Commented] (SLIDER-525) Storm app package for windows does not report component status correctly

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198646#comment-14198646 ] ASF subversion and git services commented on SLIDER-525: Commit 66

[jira] [Commented] (SLIDER-614) Release Slider 0.52 against old registry code

2014-11-05 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198525#comment-14198525 ] ASF subversion and git services commented on SLIDER-614: Commit 0f

[jira] [Created] (SLIDER-616) re-enable AMFailuresIT

2014-11-05 Thread Steve Loughran (JIRA)
Steve Loughran created SLIDER-616: - Summary: re-enable AMFailuresIT Key: SLIDER-616 URL: https://issues.apache.org/jira/browse/SLIDER-616 Project: Slider Issue Type: Test Components

[jira] [Updated] (SLIDER-616) re-enable AMFailuresIT

2014-11-05 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SLIDER-616: -- Sprint: Slider November #1 > re-enable AMFailuresIT > -- > >

[jira] [Commented] (SLIDER-585) Localize SSL certs for apps

2014-11-05 Thread Billie Rinaldi (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198507#comment-14198507 ] Billie Rinaldi commented on SLIDER-585: --- Sounds pretty comprehensive. It might be b

[jira] [Resolved] (SLIDER-614) Release Slider 0.52 against old registry code

2014-11-05 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SLIDER-614. --- Resolution: Fixed done; no major problems, except noticed that there were conflicting versions

[jira] [Commented] (SLIDER-585) Localize SSL certs for apps

2014-11-05 Thread Jonathan Maron (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198438#comment-14198438 ] Jonathan Maron commented on SLIDER-585: --- So I imagine the requirements here: - in

[jira] [Updated] (SLIDER-614) Release Slider 0.52 against old registry code

2014-11-05 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SLIDER-614: -- Summary: Release Slider 0.52 against old registry code (was: Release Slider 0.50.2 against old

[jira] [Resolved] (SLIDER-613) client registry retriever can be configured by ssl-client.xml for https connections

2014-11-05 Thread Gour Saha (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha resolved SLIDER-613. -- Resolution: Fixed > client registry retriever can be configured by ssl-client.xml for https > connectio