Re: [EXTERNAL] Re: slider 0.92 question

2018-04-22 Thread Manoj Samel
David, When local disks on the host running node manager are more than 90% full, nodemanager gives message like "10/12 local-dirs are bad:". In such cases, the node manager service keeps running but is not servicing any applications. Check if the host had multiple disk more than 90% full. Hope

Agent log shows connection refused error : hostname shows container name

2018-03-02 Thread Manoj Samel
Hello, Slider version 0.80 with CDH 5.5.1 Investigating a instance where Slider application errored out. slider.agent.log for many components show following trace - noticed that the "hostname" key is actually the container name e.g. "hostname": "container_e14_1513412386901_898934_03_03___a

Re: Slider upgrade command error : org.apache.slider.core.persist.LockAcquireFailedException: Failed to acquire lock

2018-01-17 Thread Manoj Samel
/.slider/cluster/spas/readlock > > > -Gour > > On 1/17/18, 1:05 PM, "Manoj Samel" wrote: > > >Hello, > > > >Slider version 0.80 on CDH 5.5.1 cluster with kerberos > > > >Slider upgrade --template /xxx/appConfig.json --resources > >/xxx/reso

Slider upgrade command error : org.apache.slider.core.persist.LockAcquireFailedException: Failed to acquire lock

2018-01-17 Thread Manoj Samel
Hello, Slider version 0.80 on CDH 5.5.1 cluster with kerberos Slider upgrade --template /xxx/appConfig.json --resources /xxx/resources.json --queue tenant --force failed with following trace 2018-01-17 20:31:23,030 [main] INFO tools.SliderUtils - JVM initialized into secure mode with kerberos

Re: Sometimes slider commands time out in a secured cluster

2017-09-22 Thread Manoj Samel
fails. Seems like your > issue is intermittent. RPC timeout for CLIs are set to 15 secs, so there > could be several reasons for which the timeout occurs. Do you see any > network/routing issue to connect to the host where the AM is running? > > -Gour > > On 9/21/17, 12:31 PM, &qu

Re: Sometimes slider commands time out in a secured cluster

2017-09-21 Thread Manoj Samel
the RM UI and load the ApplicationMaster web ui for > this app? > > -Gour > > On 9/21/17, 11:00 AM, "Manoj Samel" wrote: > > >Any thoughts ? > > > >On Mon, Sep 18, 2017 at 3:22 PM, Manoj Samel > >wrote: > > > >> > >>

Re: Sometimes slider commands time out in a secured cluster

2017-09-21 Thread Manoj Samel
Any thoughts ? On Mon, Sep 18, 2017 at 3:22 PM, Manoj Samel wrote: > > CDH 5.5.1 cluster with Kerberos, slider version 0.80 > > Sometimes Slider commands start hanging > > slider list --containers > > [r...@s-76zyl02.sys.az1.eng.pdx.wd ~]# slider list spas --conta

Sometimes slider commands time out in a secured cluster

2017-09-18 Thread Manoj Samel
CDH 5.5.1 cluster with Kerberos, slider version 0.80 Sometimes Slider commands start hanging slider list --containers [r...@s-76zyl02.sys.az1.eng.pdx.wd ~]# slider list spas --containers 2017-09-18 21:44:45,659 [main] INFO tools.SliderUtils - JVM initialized into secure mode with kerberos real

[jira] [Comment Edited] (SLIDER-1227) Component name with 3 "_" gives NPE in 0.92, is working in 0.80

2017-05-10 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005402#comment-16005402 ] Manoj Samel edited comment on SLIDER-1227 at 5/10/17 9:0

Re: Slider 0.92 on CDH 5.5.1 (Hadoop 2.6) - AM log shows NPE at component heardbeat URI

2017-05-10 Thread Manoj Samel
Filed https://issues.apache.org/jira/browse/SLIDER-1227. Thanks for everyones time ! On Wed, May 10, 2017 at 12:17 PM, Manoj Samel wrote: > Thanks Billie for role group explanation, seems like a good feature to > have ! > > Thinking a bit about the role group, following are

[jira] [Commented] (SLIDER-1227) Component name with 3 "_" gives NPE in 0.92, is working in 0.80

2017-05-10 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005402#comment-16005402 ] Manoj Samel commented on SLIDER-1227: - Billie Rinaldi explained the roleG

[jira] [Commented] (SLIDER-1227) Component name with 3 "_" gives NPE in 0.92, is working in 0.80

2017-05-10 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005387#comment-16005387 ] Manoj Samel commented on SLIDER-1227: - I think I found out what causes the NPE a

[jira] [Created] (SLIDER-1227) Component name with 3 "_" gives NPE in 0.92, is working in 0.80

2017-05-10 Thread Manoj Samel (JIRA)
Manoj Samel created SLIDER-1227: --- Summary: Component name with 3 "_" gives NPE in 0.92, is working in 0.80 Key: SLIDER-1227 URL: https://issues.apache.org/jira/browse/SLIDER-1227 Proje

Re: Slider 0.92 on CDH 5.5.1 (Hadoop 2.6) - AM log shows NPE at component heardbeat URI

2017-05-10 Thread Manoj Samel
e. > > Yeah, we could think about possible ways of solving this problem. Please > open a ticket along the lines of "allow LABEL_MAKER in component names OR > document that it should not be used." > > On Tue, May 9, 2017 at 7:33 PM, Manoj Samel > wrote: > > > I

Re: Slider 0.92 on CDH 5.5.1 (Hadoop 2.6) - AM log shows NPE at component heardbeat URI

2017-05-09 Thread Manoj Samel
not be used in any role names; then slider 0.92 should give error when creating cluster or accepting configs during any other operations etc. saying invalid role name etc. etc. Thanks in advance, On Tue, Apr 11, 2017 at 6:09 PM, Manoj Samel wrote: > Hi > > Running slider 0.92 on CDH 5.5.1

slider 0.92 - After upgrade, existing component keeps showing "awaiting heartbeat..."

2017-05-09 Thread Manoj Samel
Slider 0.92 on secured hadoop 2.6 cluster Have a app with component "tenant1" running. Add another component "tenant2" and do a upgrade. After that, the original component keeps showing "awaiting heartbeat..." in the output of "slider list --containers" ... The component and AM log does not seem

Re: Slider 0.92 on CDH 5.5.1 (Hadoop 2.6) - AM log shows NPE at component heardbeat URI

2017-05-03 Thread Manoj Samel
eve I've come across this error before. The problem was that the > metainfo.xml file in the application package I was trying to deploy was > malformed / missing required tags. > > > On 12 April 2017 at 02:09, Manoj Samel wrote: > > > Hi > > > > Runnin

Slider 0.92 on CDH 5.5.1 (Hadoop 2.6) - AM log shows NPE at component heardbeat URI

2017-04-11 Thread Manoj Samel
Hi Running slider 0.92 on CDH 5.5.1 (which is Hadoop 2.6), with Kerberos I am deploying a application with multiple components. The components start but fail to heart beat to slider AM. The slider AM log shows NPE at container heartbeat URLs as below. I have attached the complete slider AM log

Re: Slider 0.92 version download - no binaries available ?

2017-04-10 Thread Manoj Samel
t; > On 4/10/17, 3:00 PM, "Manoj Samel" wrote: > > >Hi, > > > >For slider version 0.92, on the download side, I could only find sources. > >I > >could not find full assemblies (like version 0.80 etc.). > > > >Thoughts? > > > >Thanks, > >

Re: Slider 0.92 version download - no binaries available ?

2017-04-10 Thread Manoj Samel
Thanks Gaur for prompt reply ! Manoj On Mon, Apr 10, 2017 at 5:32 PM, Gour Saha wrote: > https://repository.apache.org/content/groups/public/org/ > apache/slider/slide > r-assembly/0.92.0-incubating/ > > > On 4/10/17, 3:00 PM, "Manoj Samel" wrote: > > >H

Slider 0.92 version download - no binaries available ?

2017-04-10 Thread Manoj Samel
Hi, For slider version 0.92, on the download side, I could only find sources. I could not find full assemblies (like version 0.80 etc.). Thoughts? Thanks,

[jira] [Comment Edited] (SLIDER-1114) Provide option to run components as different user(s)

2017-03-09 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904142#comment-15904142 ] Manoj Samel edited comment on SLIDER-1114 at 3/10/17 12:3

[jira] [Commented] (SLIDER-1114) Provide option to run components as different user(s)

2017-03-09 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904142#comment-15904142 ] Manoj Samel commented on SLIDER-1114: - Last year I had reported that

[jira] [Updated] (SLIDER-1114) Provide option to run components as different user(s)

2017-03-09 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1114: Attachment: run_components_as_different_users.pdf > Provide option to run components as differ

[jira] [Updated] (SLIDER-1114) Provide option to run components as different user(s)

2017-03-09 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1114: Attachment: (was: run_component_as_different_users.pdf) > Provide option to run components

Re: First draft of Apache Slider November 2016 report

2016-11-02 Thread Manoj Samel
atures and bug fixes.² > > -Gour > > On 11/2/16, 10:29 AM, "Manoj Samel" wrote: > > >Hi Gour, > > > >As user of slider (in production), it would be great to have some idea on > >timelines so to plan to switch from "classic slider" to hadoop

Re: First draft of Apache Slider November 2016 report

2016-11-02 Thread Manoj Samel
Hi Gour, As user of slider (in production), it would be great to have some idea on timelines so to plan to switch from "classic slider" to hadoop branch. Given that hadoop version of branch itself will be not be a stable / battle tested and that "slider classic" development is slowing down (for r

Ballpark timeline for 1.0.0 release ?

2016-10-31 Thread Manoj Samel
Hello, I see many of the critical issues getting fixed in Slider 1.0.0 version. Can anyone comment if there is even a ballpark timeline for releasing 1.0.0 ? Thanks, Manoj

[jira] [Comment Edited] (SLIDER-1169) Slider not honoring zookeeper quorum values passed

2016-10-21 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595869#comment-15595869 ] Manoj Samel edited comment on SLIDER-1169 at 10/21/16 6:0

[jira] [Commented] (SLIDER-1169) Slider not honoring zookeeper quorum values passed

2016-10-21 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595869#comment-15595869 ] Manoj Samel commented on SLIDER-1169: - Hi [~gsaha], I applied the patch to ver

[jira] [Commented] (SLIDER-1169) Slider not honoring zookeeper quorum values passed

2016-09-30 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536520#comment-15536520 ] Manoj Samel commented on SLIDER-1169: - 1. Can this patch be applied to version

Re: Slider fails when first zookeeper in registry quorum is down

2016-09-30 Thread Manoj Samel
Thanks Gour ! Any idea when 1.0.0 will be available ? On Fri, Sep 30, 2016 at 7:23 AM, Gour Saha wrote: > I think you are hitting this - > https://issues.apache.org/jira/browse/SLIDER-1169 > > > On 9/29/16, 10:21 PM, "Manoj Samel" wrote: > > >Hi > >

Slider fails when first zookeeper in registry quorum is down

2016-09-29 Thread Manoj Samel
Hi Slider version .80 on secure cluster. In my xxx-site.xml files, the hadoop.registry.zk.quorum zk1_host:2181,zk2_host:2181,zk3_host:2181 However, it appears slider AM uses only the first ZK to connect for registry - and fails when the first ZK happens to be down. In the s

[jira] [Commented] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-08-04 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408171#comment-15408171 ] Manoj Samel commented on SLIDER-1158: - Hello, Can someone provide any fur

Re: Slider AM fails to run when RM in HA setup fails over

2016-08-01 Thread Manoj Samel
? Thanks in advance, Manoj On Thu, Jul 28, 2016 at 7:01 PM, Manoj Samel wrote: > Hi Gour, > > I added properties in /etc/hadoop/conf/yarn-site.xml and emptied the > /data/slider/conf/slider-client.xml and restarted both RMs. > >- hadoop.registry.zk.quorum >- ha

Re: Slider AM fails to run when RM in HA setup fails over

2016-07-29 Thread Manoj Samel
ng with status 0 foo RUNNING application_1469834604094_0001 http://xxx:23188/proxy/application_1469834604094_0001/ .. Thanks, On Thu, Jul 28, 2016 at 7:01 PM, Manoj Samel wrote: > Hi Gour, > > I added properties in /etc/hadoop/conf/yarn-site.xml and emptied the > /data/s

[jira] [Commented] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-29 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400111#comment-15400111 ] Manoj Samel commented on SLIDER-1158: - Hi [~jmaron] & [~gsaha], I have

[jira] [Updated] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-29 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1158: Attachment: slider-1158.hadoop_conf.tar.gz > Slider AM hits er

Re: Slider AM fails to run when RM in HA setup fails over

2016-07-28 Thread Manoj Samel
Thu, Jul 28, 2016 at 5:28 PM, Manoj Samel wrote: > Thanks. I will test with the updated config and then upload the latest > ones ... > > Thanks, > > Manoj > > On Thu, Jul 28, 2016 at 5:21 PM, Gour Saha wrote: > >> slider.zookeeper.quorum is

Re: Slider AM fails to run when RM in HA setup fails over

2016-07-28 Thread Manoj Samel
he command line using -D slider.yarn.queue=<> during the > create call. If indeed all slider apps should go to one and only one > queue, then this prop can be specified in any one of the existing site xml > files under /etc/hadoop/conf. > > -Gour > > On 7/28/16, 4:43 PM, &

Re: Slider AM fails to run when RM in HA setup fails over

2016-07-28 Thread Manoj Samel
; indication that there might be some issue with cluster configuration based > on files solely under HADOOP_CONF_DIR to begin with. > > Suggest you to upload all the config files to the jira to help debug this > further. > > -Gour > > On 7/28/16, 4:27 PM, "Manoj Samel&q

Re: Slider AM fails to run when RM in HA setup fails over

2016-07-28 Thread Manoj Samel
rties will be read from HADOOP_CONF_DIR files. Let me know if this could cause any issues. On Thu, Jul 28, 2016 at 3:36 PM, Gour Saha wrote: > No need to copy any files. Pointing HADOOP_CONF_DIR to /etc/hadoop/conf is > good. > > -Gour > > On 7/28/16, 3:24 PM, "Manoj Samel&

Re: Slider AM fails to run when RM in HA setup fails over

2016-07-28 Thread Manoj Samel
vance, Manoj On Tue, Jul 26, 2016 at 3:27 PM, Manoj Samel wrote: > Filed https://issues.apache.org/jira/browse/SLIDER-1158 with logs and my > analysis of logs. > > On Tue, Jul 26, 2016 at 10:36 AM, Gour Saha wrote: > >> Please file a JIRA and upload the logs to it. >> >&

[jira] [Updated] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-27 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1158: Attachment: SUCCESS_slider.log SUCCESS_rm1.log.gz > Slider AM hits er

[jira] [Commented] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-27 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396727#comment-15396727 ] Manoj Samel commented on SLIDER-1158: - Hi Jon, 1. Just to clarify, our ha

[jira] [Commented] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-27 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15395892#comment-15395892 ] Manoj Samel commented on SLIDER-1158: - Also, other applications on this cluster

[jira] [Commented] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-27 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15395889#comment-15395889 ] Manoj Samel commented on SLIDER-1158: - We are running CDH 5.5.1, which is Hadoop

[jira] [Updated] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-26 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1158: Description: In certain cases, when a RM fails over from RM1 to RM2, the Slider AM starts getting

Re: Slider AM fails to run when RM in HA setup fails over

2016-07-26 Thread Manoj Samel
Filed https://issues.apache.org/jira/browse/SLIDER-1158 with logs and my analysis of logs. On Tue, Jul 26, 2016 at 10:36 AM, Gour Saha wrote: > Please file a JIRA and upload the logs to it. > > On 7/26/16, 10:21 AM, "Manoj Samel" wrote: > > >Hi Gour, > > >

[jira] [Updated] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-26 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1158: Description: In certain cases, when a RM fails over from RM1 RM2, the Slider AM starts getting

[jira] [Updated] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-26 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1158: Attachment: slider.log rm2.log README_INFO_ANALYSIS > Slider

[jira] [Updated] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-26 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1158: Description: In certain cases, when a RM fails over from RM1 RM2, the Slider AM starts getting

[jira] [Updated] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-26 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1158: Description: In certain cases, when a RM fails over from RM1 RM2, the Slider AM starts getting

[jira] [Created] (SLIDER-1158) Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA

2016-07-26 Thread Manoj Samel (JIRA)
Manoj Samel created SLIDER-1158: --- Summary: Slider AM hits error org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN] when RM failover happens in RM HA Key: SLIDER-1158 URL

Re: Slider AM fails to run when RM in HA setup fails over

2016-07-26 Thread Manoj Samel
sues you are facing, you have to provide additional > logs for us to understand better. Let¹s start with - > 1. RM logs (specifically between the time when rm1->rm2 failover is > simulated) > 2. Slider App logs > > -Gour > > On 7/25/16, 5:16 PM, "Manoj Samel"

Re: Slider AM fails to run when RM in HA setup fails over

2016-07-25 Thread Manoj Samel
r everything out from > slider-client.xml. > > On 7/25/16, 4:12 PM, "Manoj Samel" wrote: > > >Hi Gour, > > > >Thanks for your prompt reply. > > > >FYI, issue happens when I create slider app when rm1 is active and when > >rm1 > >fails

Re: Slider AM fails to run when RM in HA setup fails over

2016-07-25 Thread Manoj Samel
ors? > > -Gour > > On 7/25/16, 2:28 PM, "Manoj Samel" wrote: > > >Another observation (whatever it is worth) > > > >If slider app is created and started when rm2 was active, then it seems to > >survive switches between rm2 and rm1 (and back). I.e >

What RM properties are must in slider-client.xml, if present in files in HADOOP_CONF_DIR ?

2016-07-25 Thread Manoj Samel
Hello, Slider version is 0.80, Hadoop is 2.6 with Kerberos Slider-client.xml allows specification of full path of hadoop conf using HADOOP_CONF_DIR. In our case, full hadoop configuration, including all HA configurations are available in the HADOOP_CONF_DIR for hdfs-site, core-site and yarn-site.

Re: Slider AM fails to run when RM in HA setup fails over

2016-07-25 Thread Manoj Samel
again. Slider AM still keeps running So, it seems if it starts with rm1 active, then the AM goes to "ACCEPTED" state when RM fails to rm2. If it starts with rm2 active, then it runs fine with any switches between rm1 and rm2. Any feedback ? Thanks, Manoj On Mon, Jul 25, 2016 at 12:25

Slider AM fails to run when RM in HA setup fails over

2016-07-25 Thread Manoj Samel
Setup - Hadoop 2.6 with RM HA, Kerberos enabled - Slider 0.80 - In my slider-client.xml, I have added all RM HA properties, including the ones mentioned in http://markmail.org/message/wnhpp2zn6ixo65e3. Following is the issue * rm1 is active, rm2 is standby * deploy and start slider application,

[jira] [Updated] (SLIDER-1151) Don't log Invalid port range values when there are no invalid ports specified

2016-07-07 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1151: Attachment: SLIDER-1151.1.patch > Don't log Invalid port range values when there are no

[jira] [Updated] (SLIDER-1151) Don't log Invalid port range values when there are no invalid ports specified

2016-07-07 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1151: Attachment: (was: check_empty.patch) > Don't log Invalid port range values when ther

[jira] [Commented] (SLIDER-1151) Don't log Invalid port range values when there are no invalid ports specified

2016-07-07 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366824#comment-15366824 ] Manoj Samel commented on SLIDER-1151: - [~gsaha], patch attached > Don't

[jira] [Updated] (SLIDER-1151) Don't log Invalid port range values when there are no invalid ports specified

2016-07-07 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1151: Attachment: check_empty.patch > Don't log Invalid port range values when there are no

[jira] [Created] (SLIDER-1151) Don't log Invalid port range values when there are no invalid ports specified

2016-07-06 Thread Manoj Samel (JIRA)
Manoj Samel created SLIDER-1151: --- Summary: Don't log Invalid port range values when there are no invalid ports specified Key: SLIDER-1151 URL: https://issues.apache.org/jira/browse/SLIDER-1151 Pr

[jira] [Updated] (SLIDER-1114) Provide option to run components as different user(s)

2016-06-23 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1114: Attachment: (was: run_components_as_different_users.pdf) > Provide option to run components

[jira] [Updated] (SLIDER-1114) Provide option to run components as different user(s)

2016-06-23 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1114: Attachment: run_component_as_different_users.pdf > Provide option to run components as differ

Need help on SLIDER-1114 - Run each component as different user

2016-06-15 Thread Manoj Samel
Hello, https://issues.apache.org/jira/browse/SLIDER-1114 Trying to implement a new feature where each component of a application is run as different linux user. I have so far implemented a way to start each component as different user and accurately keep track status of each component. The implem

[jira] [Commented] (SLIDER-1114) Provide option to run components as different user(s)

2016-06-13 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328475#comment-15328475 ] Manoj Samel commented on SLIDER-1114: - Hi, I attached a small document descri

[jira] [Updated] (SLIDER-1114) Provide option to run components as different user(s)

2016-06-13 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1114: Attachment: run_components_as_different_users.pdf > Provide option to run components as differ

[jira] [Commented] (SLIDER-875) Ability to create an Uber application package with capability to deploy and manage as a single business app

2016-06-13 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328342#comment-15328342 ] Manoj Samel commented on SLIDER-875: Hi [~gsaha], any ballpark timelines whe

[jira] [Commented] (SLIDER-875) Ability to create an Uber application package with capability to deploy and manage as a single business app

2016-06-13 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328301#comment-15328301 ] Manoj Samel commented on SLIDER-875: This would be a great feature to have. One p

[jira] [Created] (SLIDER-1137) Option to define port range per component & limit it to components only

2016-06-04 Thread Manoj Samel (JIRA)
Manoj Samel created SLIDER-1137: --- Summary: Option to define port range per component & limit it to components only Key: SLIDER-1137 URL: https://issues.apache.org/jira/browse/SLIDER-1137 Pro

Re: allowed.ports - Different use cases ...

2016-06-04 Thread Manoj Samel
assigned ports from the specified ranges. > > > On Jun 1, 2016, at 12:15 PM, Manoj Samel > wrote: > > > > Any thoughts ? > > > > On Wed, May 25, 2016 at 11:27 AM, Manoj Samel > > wrote: > > > >> Hi, > >> > >> At present th

Re: allowed.ports - Different use cases ...

2016-06-01 Thread Manoj Samel
Any thoughts ? On Wed, May 25, 2016 at 11:27 AM, Manoj Samel wrote: > Hi, > > At present the allowed.ports is applied to AM as well as containers > > While this may be a valid use case for certain deployments, there are > other use cases > > 1. The "allowed.por

Question on {PER_CONTAINER} flag

2016-05-25 Thread Manoj Samel
Hello, Just wanted to confirm my understanding of PER_CONTAINER for allocated ports. Looking @ AgentProviderService.java - if (!value.contains(PER_CONTAINER_TAG)) { // If the config property is shared then pass on the already allocated value // from any container If

allowed.ports - Different use cases ...

2016-05-25 Thread Manoj Samel
Hi, At present the allowed.ports is applied to AM as well as containers While this may be a valid use case for certain deployments, there are other use cases 1. The "allowed.ports" should be only applied to certain port(s) for components that are marked as {ALLOCATED_PORT}{PER_CONTAINER} etc.. T

[jira] [Commented] (SLIDER-1124) If unparsable port range is specified, Slider AM PortScanner.java setPortRange() should throw exception - add else part

2016-05-20 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15293722#comment-15293722 ] Manoj Samel commented on SLIDER-1124: - Thanks [~billie.rinaldi] for the fix

[jira] [Updated] (SLIDER-1124) If unparsable port range is specified, Slider AM PortScanner.java setPortRange() should throw exception - add else part

2016-05-19 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1124: Summary: If unparsable port range is specified, Slider AM PortScanner.java setPortRange() should

Re: What does PER_CONTAINER in port allocation does ? Facing AM start error ...

2016-05-19 Thread Manoj Samel
Filed https://issues.apache.org/jira/browse/SLIDER-1124 On Thu, May 19, 2016 at 7:18 AM, Billie Rinaldi wrote: > I agree, it would be good to throw an Exception if the format of the port > range is bad. > > On Wed, May 18, 2016 at 7:15 PM, Manoj Samel > wrote: > > &

[jira] [Updated] (SLIDER-1124) If unparsable port range is specified, Slider AM PortScanner.java setPortRange() should throw exception

2016-05-19 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1124: Description: {noformat} The issue was discovered when a JSON was generated with IDE and instead of

[jira] [Updated] (SLIDER-1124) If unparsable port range is specified, Slider AM PortScanner.java setPortRange() should throw exception

2016-05-19 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1124: Description: The issue was discovered when a JSON was generated with IDE and instead of "-&

[jira] [Updated] (SLIDER-1124) If unparsable port range is specified, Slider AM PortScanner.java setPortRange() should throw exception

2016-05-19 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1124: Description: The issue was discovered when a JSON was generated with IDE and instead of "-&

[jira] [Updated] (SLIDER-1124) If unparsable port range is specified, Slider AM PortScanner.java setPortRange() should throw exception

2016-05-19 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1124: Description: The issue was discovered when a JSON was generated with IDE and instead of "-&

[jira] [Updated] (SLIDER-1124) If unparsable port range is specified, Slider AM PortScanner.java setPortRange() should throw exception

2016-05-19 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1124: Description: The issue was discovered when a JSON was generated with IDE and instead of "-&

[jira] [Created] (SLIDER-1124) If unparsable port range is specified, Slider AM PortScanner.java setPortRange() should throw exception

2016-05-19 Thread Manoj Samel (JIRA)
Manoj Samel created SLIDER-1124: --- Summary: If unparsable port range is specified, Slider AM PortScanner.java setPortRange() should throw exception Key: SLIDER-1124 URL: https://issues.apache.org/jira/browse/SLIDER

Re: What does PER_CONTAINER in port allocation does ? Facing AM start error ...

2016-05-18 Thread Manoj Samel
nteger.parseInt(m.group())); } else { m = NUMBER_RANGE.matcher(range.trim()); if (m.find()) { } // else is missing . Add with a exception ??? Thoughts ? Manoj On Mon, May 16, 2016 at 6:45 PM, Manoj Samel wrote: > Here is slider.log for Slider AM. Note the port r

Re: What does PER_CONTAINER in port allocation does ? Facing AM start error ...

2016-05-16 Thread Manoj Samel
ks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:4

Re: What does PER_CONTAINER in port allocation does ? Facing AM start error ...

2016-05-16 Thread Manoj Samel
y > another application on the same box. > > Is your cluster idle or do you have other slider apps running? Do you have > more complete output of the logs and possibly the appConfig that you can > share? Are you sure it's the AM failing to start and not a service within

What does PER_CONTAINER in port allocation does ? Facing AM start error ...

2016-05-16 Thread Manoj Samel
Hello, When using ALLOCATED_PORT clause, there is a option "PER_CONTAINER". Can someone explain what does "PER_CONTAINER" option does ? It says keep port allocation private to container. What does that means ? If multiple containers are chosen to on same host machine, will this cause issue ? Whe

Re: Update : Run each component of application as different user working - except stop command

2016-05-04 Thread Manoj Samel
security in parity with what the traditional map-reduce does. I believe this will be important feature as slider gets more adoption for running custom services ( beyond Hbase etc. which could be run as single user) Thanks, Manoj On Wed, May 4, 2016 at 7:28 AM, Josh Elser wrote: > Manoj Samel wr

Update : Run each component of application as different user working - except stop command

2016-05-02 Thread Manoj Samel
ss, like the system command would. Perhaps if you used the system > command in your C code, that would produce a different result. > > Billie > > On Fri, Apr 22, 2016 at 12:17 PM, Manoj Samel > wrote: > > > Hello Again ! > > > > One more observation .. hopeful

[jira] [Comment Edited] (SLIDER-1114) Provide option to run components as different user(s)

2016-04-29 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264752#comment-15264752 ] Manoj Samel edited comment on SLIDER-1114 at 4/29/16 9:1

[jira] [Commented] (SLIDER-1114) Provide option to run components as different user(s)

2016-04-29 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264752#comment-15264752 ] Manoj Samel commented on SLIDER-1114: - Capturing d-list discussion for this t

[jira] [Updated] (SLIDER-1114) Provide option to run components as different user(s)

2016-04-29 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1114: Summary: Provide option to run components as different user(s) (was: Provide option to launch

[jira] [Updated] (SLIDER-1114) Provide option to launch components as different user(s)

2016-04-29 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Samel updated SLIDER-1114: Description: Environment is slider .80 on Hadoop 2.6 secured cluster A component is launched for

(2nd attempt) Need Help !: Run each component of application as different user

2016-04-22 Thread Manoj Samel
Thanks, Manoj ------ Forwarded message -- From: Manoj Samel Date: Thu, Apr 21, 2016 at 2:40 PM Subject: Need Help !: Run each component of application as different user To: dev@slider.incubator.apache.org Hi, See use case background below I have implemented option 2 mentioned below (as

[jira] [Created] (SLIDER-1114) Provide option to launch components as different user(s)

2016-04-21 Thread Manoj Samel (JIRA)
Manoj Samel created SLIDER-1114: --- Summary: Provide option to launch components as different user(s) Key: SLIDER-1114 URL: https://issues.apache.org/jira/browse/SLIDER-1114 Project: Slider

[jira] [Commented] (SLIDER-1063) duplicated port allocation when slider.allowed.ports is set

2016-04-21 Thread Manoj Samel (JIRA)
[ https://issues.apache.org/jira/browse/SLIDER-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15253002#comment-15253002 ] Manoj Samel commented on SLIDER-1063: - Any update on this issue ? > duplicat

Re: Need Help !: Run each component of application as different user

2016-04-21 Thread Manoj Samel
nt process which is "/bin/bash --login ..." and which launches a subprocess as specified in execute(). Can someone confirm this ? On Thu, Apr 21, 2016 at 2:40 PM, Manoj Samel wrote: > Hi, > > See use case background below > > I have implemented option 2 mentioned below

  1   2   >