Re: MapR problem. No FileSystem for scheme: maprfs

2017-02-17 Thread jonma...@gmail.com 
My guess is there's actually a need to have the MapR hadoop common jar (the
one I would assume has an updated file system implementation) be pre-pended
to the class path.  I don't have an installation in front of me, but I'd
look into modifying slider-env or the like to prepend at least that one jar
to the class path.

On Fri, Feb 17, 2017 at 12:19 PM, Nicolás Velásquez O. 
wrote:

> Hey,
> It's great to initiate MapR to slider!
> Unfortunately, following the suggestions did not work (remove the classpath
> modification on slider-env.sh and copied mapr's jars to slider's lib
> folder).
>
> I'm guessing support for parsing the fs.defaultFS string ("maprfs:///") has
> to be added to Slider's code. Something similar to
> https://issues.apache.org/jira/browse/FLINK-1006.
>
> Should I just open an incident on Jira?
>
> Thanks!
> Nicolas
>
>
>
> On Fri, Feb 17, 2017 at 4:40 PM, Billie Rinaldi 
> wrote:
>
> > I don't think anyone has tried running Slider on MapR yet, so you are
> > breaking new ground. I would not try to change CLASSPATH in
> slider-env.sh.
> > Slider has a standalone lib directory, meaning that all of its required
> > jars are in the lib dir (including hadoop jars). It might be the case
> that
> > more jars are needed from the mapr lib directory. I would try copying all
> > the jars from the mapr lib directory to the slider lib directory and see
> if
> > that makes a difference. If it works, then you might be able to remove
> some
> > of the jars.
> >
> > On Fri, Feb 17, 2017 at 3:32 AM, Nicolás Velásquez O. <
> gnico...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I'm having problems to run Solr using Apache Slider on MapR.
> > Specifically,
> > > when I try to install the slider package it fails because slider
> doesn't
> > > have support for maprfs:
> > > $ slider install-package --replacepkg --name solr --package
> > > ~/inst/solr/solr-slider/solr-on-yarn.zip
> > > 2017-02-17 12:08:28,635 [main] INFO  service.AbstractService - Service
> > > Slider Client failed in state INITED; cause: java.io.IOException: No
> > > FileSystem for scheme: maprfs
> > > java.io.IOException: No FileSystem for scheme: maprfs
> > > at
> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
> FileSystem.java:2644)
> > > at
> > > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
> > > at org.apache.hadoop.fs.FileSystem.access$200(
> > FileSystem.java:92)
> > > at
> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
> FileSystem.java:2687)
> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
> > FileSystem.java:2669)
> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
> > > at
> > > org.apache.slider.common.tools.CoreFileSystem.(
> > > CoreFileSystem.java:82)
> > > at
> > > org.apache.slider.common.tools.SliderFileSystem.(
> > > SliderFileSystem.java:38)
> > > at
> > > org.apache.slider.client.SliderClient.initHadoopBinding(
> > > SliderClient.java:498)
> > > at
> > > org.apache.slider.client.SliderClient.serviceInit(
> SliderClient.java:318)
> > > at
> > > org.apache.hadoop.service.AbstractService.init(
> AbstractService.java:163)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchService(
> > > ServiceLauncher.java:182)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
> > > ServiceLauncher.java:475)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
> > > ServiceLauncher.java:403)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
> > > ServiceLauncher.java:630)
> > > at org.apache.slider.Slider.main(Slider.java:49)
> > > Exception: java.io.IOException: No FileSystem for scheme: maprfs
> > > 2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher - Exception:
> > > java.io.IOException: No FileSystem for scheme: maprfs
> > > org.apache.hadoop.service.ServiceStateException: java.io.IOException:
> No
> > > FileSystem for scheme: maprfs
> > > at
> > > org.apache.hadoop.service.ServiceStateException.convert(
> > > ServiceStateException.java:59)
> > > at
> > > org.apache.hadoop.service.AbstractService.init(
> AbstractService.java:172)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchService(
> > > ServiceLauncher.java:182)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
> > > ServiceLauncher.java:475)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
> > > ServiceLauncher.java:403)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
> > > ServiceLauncher.java:630)
> > > at org.apache.slider.Slider.main(Slider.java:49)
> > > Caused by: java.io.IOException: No FileSystem for scheme: maprfs
> > > at
> > > org.apache.hadoop.f

Re: Apache slider (solr) on non-secured cluster

2017-02-21 Thread jonma...@gmail.com 
Can you provide your app config file?  Perhaps the combination of
properties you've specified is triggering an attempt to connect securely?
Are you specifiying the client properties in slider-client.xml or pointint
to the hadoop conf directory?  If the latter, are you certain that hadoop
authentication is not specified as kerberos?

On Tue, Feb 21, 2017 at 5:04 AM, Nicolás Velásquez O. 
wrote:

> Hi again,
> I'm trying to run solr via apache slider on an insecure mapr cluster, but
> for some reason the slider app is trying to use kerberos to authenticate
> against the zookeeper. I am using a standalone zookeeper (tried versions
> 3.4.6 and 3.4.9). This is the error I see on slider:
> java.security.PrivilegedActionException: javax.security.sasl.
> SaslException:
> Failure to initialize security context [Caused by GSSException: Invalid
> name provided (Mechanism level: KrbException: Cannot locate default realm)]
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslClient(
> ZooKeeperSaslClient.java:283)
> at
> org.apache.zookeeper.client.ZooKeeperSaslClient.(
> ZooKeeperSaslClient.java:131)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:
> 949)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
> Caused by: javax.security.sasl.SaslException: Failure to initialize
> security context [Caused by GSSException: Invalid name provided (Mechanism
> level: KrbException: Cannot locate default realm)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.(GssKrb5Client.java:149)
> at
> com.sun.security.sasl.gsskerb.FactoryImpl.createSaslClient(
> FactoryImpl.java:63)
> at javax.security.sasl.Sasl.createSaslClient(Sasl.java:372)
> at
> org.apache.zookeeper.client.ZooKeeperSaslClient$1.run(
> ZooKeeperSaslClient.java:288)
> at
> org.apache.zookeeper.client.ZooKeeperSaslClient$1.run(
> ZooKeeperSaslClient.java:283)
> ... 6 more
> Caused by: GSSException: Invalid name provided (Mechanism level:
> KrbException: Cannot locate default realm)
> at
> sun.security.jgss.krb5.Krb5NameElement.getInstance(
> Krb5NameElement.java:129)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getNameElement(
> Krb5MechFactory.java:95)
> at sun.security.jgss.GSSManagerImpl.getNameElement(
> GSSManagerImpl.java:202)
> at sun.security.jgss.GSSNameImpl.getElement(GSSNameImpl.java:476)
> at sun.security.jgss.GSSNameImpl.init(GSSNameImpl.java:201)
> at sun.security.jgss.GSSNameImpl.(GSSNameImpl.java:170)
> at sun.security.jgss.GSSManagerImpl.createName(GSSManagerImpl.java:137)
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.(GssKrb5Client.java:107)
>
> This is the relevant log part on the zookeeper:
> 2017-02-21 10:47:57,053 [myid:] - INFO [NIOServerCxn.Factory:
> 0.0.0.0/0.0.0.0:2181:ZooKeeperServer@928] - Client attempting to establish
> new session at /172.31.3.3:57138
> 2017-02-21 10:47:57,058 [myid:] - INFO [SyncThread:0:FileTxnLog@203] -
> Creating new log file: log.1
> 2017-02-21 10:47:57,112 [myid:] - INFO [SyncThread:0:ZooKeeperServer@673]
> -
> Established session 0x15a6010d826 with negotiated timeout 4 for
> client /172.31.3.3:57138
> 2017-02-21 10:48:02,138 [myid:] - WARN [NIOServerCxn.Factory:
> 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of stream exception
> EndOfStreamException: Unable to read additional data from client sessionid
> 0x15a6010d826, likely client has closed socket
> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> at
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(
> NIOServerCnxnFactory.java:203)
> at java.lang.Thread.run(Thread.java:745)
> 2017-02-21 10:48:02,143 [myid:] - INFO [NIOServerCxn.Factory:
> 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1008] - Closed socket connection for
> client /172.31.3.3:57138 which had sessionid 0x15a6010d826
> 2017-02-21 10:48:38,000 [myid:] - INFO [SessionTracker:ZooKeeperServer@358
> ]
> - Expiring session 0x15a6010d826, timeout of 4ms exceeded
> 2017-02-21 10:48:38,001 [myid:] - INFO [ProcessThread(sid:0
> cport:2181)::PrepRequestProcessor@487] - Processed session termination for
> sessionid: 0x15a6010d826
>
> I searched but couldn't find a way to tell slider that it is connecting to
> an insecure cluster. Any input is greatly appreciated!
>
> Thanks,
> Nicolas
>



-- 
Sent from some random computer

Re: slider diagnostics question

2017-03-03 Thread jonma...@gmail.com 
Should it actually be "APPNAME" instead of "APPNMAE"?

On Thu, Mar 2, 2017 at 11:29 PM, David.Serafini 
wrote:

> when I run :  slider diagnostics --application --name 
>
>   "global" : {
> "site.global.port" : "${APPNMAE.ALLOCATED_PORT}{PER_CONTAINER}",
>
>
> How do I find the actual value of this variable?
>
> thx,
> -david
>
>
>


-- 
Sent from some random computer

Re: [VOTE] Apache Slider (incubating) release 0.92.0-incubating-RC0

2017-03-12 Thread jonma...@gmail.com 
Anyone else attempted running the functional suite?  I'm getting some
errors against an HDP 2.4 cluster (2.4.2.0-258):

Failed tests:

AppsThroughAgentIT.testCreateFlex:86->CommandTestBase.expectLiveContainerCountReached:1367->SliderTestUtils.repeatUntilSuccess:1405->CommandTestBase.assertContainersLive:1327->SliderTestUtils.assertContainersLive:780
assert expected == actual
   ||  |
   2|  1
false

AppsUpgradeIT.testUpgrade:167->CommandTestBase.expectContainerRequestedCountReached:1300->SliderTestUtils.repeatUntilSuccess:1405->Assert.fail:88
expected request count of COMMAND_LOGGER = 1 not reached, actual: 0 after
9 ms

ExternalComponentIT.testExternalComponent:103->CommandTestBase.expectLiveContainerCountReached:1367->SliderTestUtils.repeatUntilSuccess:1405->CommandTestBase.assertContainersLive:1327->SliderTestUtils.assertContainersLive:780
assert expected == actual
   ||  |
   1|  0
false

UniqueComponentNamesIT.testCreateFlex:95->CommandTestBase.expectLiveContainerCountReached:1367->SliderTestUtils.repeatUntilSuccess:1405->CommandTestBase.assertContainersLive:1327->SliderTestUtils.assertContainersLive:780
assert expected == actual
   ||  |
   1|  0
false

Tests in error:

AgentWebPagesIT.testAgentWeb:131->awaitRestEndpointLive:218->SliderTestUtils.repeatUntilSuccess:1380
» UniformInterface

AgentPingSocketIT.testAgentPingSocket:91->CommandTestBase.awaitRegistryOutfileContains:1453->SliderTestUtils.repeatUntilSuccess:1405->CommandTestBase.slider:288
» Slider
  AMConfigPublishingIT.testCreate:130->CommandTestBase.slider:288 » Slider
Expec...

Tests run: 76, Failures: 4, Errors: 3, Skipped: 1

I'll dig in some more and debug to see if it's a config/setup issue, or
perhaps my cluster is simply less performant and is taking too long to
deploy certain apps, but wondering if anyone else has run these tests on
the release candidate?

-- Jon


On Fri, Mar 10, 2017 at 2:06 PM, Gour Saha  wrote:

> Apologies, fixing the broken link of the source artifacts this time -
> https://dist.apache.org/repos/dist/dev/incubator/slider/0.
> 92.0-incubating-RC0
>
> -Gour
>
> On 3/10/17, 10:53 AM, "Gour Saha"  gs...@hortonworks.com>> wrote:
>
> All,
> Kindly note that the Source Artifacts have been moved to -
> https://dist.apache.org/repos/dist/dev/incubator/slider/0.
> 92.0-incubating-R
> C0
>
> Please do not use the link from the original email which pointed to the
> release stage.
>
>
> Billie,
> Thank you for looking into the release candidate.
>
> My public key was already in the KEYS file. The key info was also added in
> the KEYS file before I staged the release. It was not committed to svn,
> which I did just now (right above my public key).
>
> I did notice the release vs dev staging link during the candidate build,
> but continued with the release as instructed in Slider's release process
> documentation. I am fixing the release process document in our site right
> now. I have also moved the source artifacts from the release to the dev
> stage.
>
> Let me know if you see anything else not followed as per standard.
>
> -Gour
>
>
> On 3/10/17, 9:44 AM, "Billie Rinaldi"  billie.rina...@gmail.com>> wrote:
>
> Gour, thanks for putting up a release candidate! Two important things with
> respect to release logistics: the release should be staged at
> https://dist.apache.org/repos/dist/dev/incubator/slider/
>  rather than
> https://dist.apache.org/repos/dist/release/incubator/slider/
>  ting-RC0>
>  ting-RC0>
> ... the "release" path means that the staged candidate is being
> distributed
> to all the mirrors. Maybe it would be possible to svn mv the
> 0.92.0-incubating-RC0 directory to the dev path? Also, please append your
> key to the KEYS file at
> https://dist.apache.org/repos/dist/release/incubator/slider/KEYS. There
> are
> instructions at the top of the file.
>
> I will continue reviewing RC0.
>
> On Thu, Mar 9, 2017 at 11:16 PM, Gour Saha  gs...@hortonworks.com>> wrote:
>
> Hello,
>
> This is a call for a vote on the Apache Slider (incubating) release
> 0.92.0-incubating-RC0
>
> Source artifacts:
>https://dist.apache.org/repos/dist/release/incubator/slider/
> 0.92.0-incubating-RC0
>
> Staged artifacts:
>
> https://repository.apache.org/content/repositories/orgapacheslider-1017/
>
> Git source:
>https://git-wip-us.apache.org/repos/asf?p=incubator-slider.g
> it;a=commit;h=c41b1c7051d02c02b0d78cab859c7e15d0af92c9
>
> Git commit SHA1: c41b1c7051d02c02b0d78cab859c7e15d0af92c9
>
> Issues fixed:
>https://issues.apache.org/jira/browse/SLIDER/fixforversion/12339846/
>
> Release Notes:
>https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> ctId=12315422&version=12339846
>
> PGP key:

Re: [VOTE] Apache Slider (incubating) release 0.92.0-incubating-RC0

2017-03-12 Thread jonma...@gmail.com 
Having trouble remembering:  is there a requirement for more than one node
manager for the functional tests?  Seeing the logging below during a test,
and my initial attempt at running the functional suite was against a single
node HDP installation.

2017-03-12 21:43:43,424 [AmExecutor-006] INFO  state.AppState
(AppState.java:reviewOneRole(2170)) - COMMAND_LOGGER: Asking for 1 more
nodes(s) for a total of 2
2017-03-12 21:43:43,424 [AmExecutor-006] DEBUG state.RoleHistory
(RoleHistory.java:findRecentNodeForNewInstance(606)) - There are 0 node(s)
to consider for COMMAND_LOGGER
2017-03-12 21:43:43,424 [AmExecutor-006] INFO  state.RoleHistory
(RoleHistory.java:findRecentNodeForNewInstance(625)) - No node found for
COMMAND_LOGGER


On Sun, Mar 12, 2017 at 6:17 PM, jonma...@gmail.com 
wrote:

> Anyone else attempted running the functional suite?  I'm getting some
> errors against an HDP 2.4 cluster (2.4.2.0-258):
>
> Failed tests:
>   AppsThroughAgentIT.testCreateFlex:86->CommandTestBase.
> expectLiveContainerCountReached:1367->SliderTestUtils.
> repeatUntilSuccess:1405->CommandTestBase.assertContainersLive:1327->
> SliderTestUtils.assertContainersLive:780 assert expected == actual
>||  |
>2|  1
> false
>   AppsUpgradeIT.testUpgrade:167->CommandTestBase.
> expectContainerRequestedCountReached:1300->SliderTestUtils.
> repeatUntilSuccess:1405->Assert.fail:88 expected request count of
> COMMAND_LOGGER = 1 not reached, actual: 0 after 9 ms
>   ExternalComponentIT.testExternalComponent:103->CommandTestBase.
> expectLiveContainerCountReached:1367->SliderTestUtils.
> repeatUntilSuccess:1405->CommandTestBase.assertContainersLive:1327->
> SliderTestUtils.assertContainersLive:780 assert expected == actual
>||  |
>1|  0
> false
>   UniqueComponentNamesIT.testCreateFlex:95->CommandTestBase.
> expectLiveContainerCountReached:1367->SliderTestUtils.
> repeatUntilSuccess:1405->CommandTestBase.assertContainersLive:1327->
> SliderTestUtils.assertContainersLive:780 assert expected == actual
>||  |
>1|  0
> false
>
> Tests in error:
>   AgentWebPagesIT.testAgentWeb:131->awaitRestEndpointLive:
> 218->SliderTestUtils.repeatUntilSuccess:1380 » UniformInterface
>   AgentPingSocketIT.testAgentPingSocket:91->CommandTestBase.
> awaitRegistryOutfileContains:1453->SliderTestUtils.
> repeatUntilSuccess:1405->CommandTestBase.slider:288 » Slider
>   AMConfigPublishingIT.testCreate:130->CommandTestBase.slider:288 »
> Slider Expec...
>
> Tests run: 76, Failures: 4, Errors: 3, Skipped: 1
>
> I'll dig in some more and debug to see if it's a config/setup issue, or
> perhaps my cluster is simply less performant and is taking too long to
> deploy certain apps, but wondering if anyone else has run these tests on
> the release candidate?
>
> -- Jon
>
>
> On Fri, Mar 10, 2017 at 2:06 PM, Gour Saha  wrote:
>
>> Apologies, fixing the broken link of the source artifacts this time -
>> https://dist.apache.org/repos/dist/dev/incubator/slider/0.92
>> .0-incubating-RC0
>>
>> -Gour
>>
>> On 3/10/17, 10:53 AM, "Gour Saha" > gs...@hortonworks.com>> wrote:
>>
>> All,
>> Kindly note that the Source Artifacts have been moved to -
>> https://dist.apache.org/repos/dist/dev/incubator/slider/0.92
>> .0-incubating-R
>> C0
>> <https://dist.apache.org/repos/dist/dev/incubator/slider/0.92.0-incubating-RC0>
>>
>> Please do not use the link from the original email which pointed to the
>> release stage.
>>
>>
>> Billie,
>> Thank you for looking into the release candidate.
>>
>> My public key was already in the KEYS file. The key info was also added in
>> the KEYS file before I staged the release. It was not committed to svn,
>> which I did just now (right above my public key).
>>
>> I did notice the release vs dev staging link during the candidate build,
>> but continued with the release as instructed in Slider's release process
>> documentation. I am fixing the release process document in our site right
>> now. I have also moved the source artifacts from the release to the dev
>> stage.
>>
>> Let me know if you see anything else not followed as per standard.
>>
>> -Gour
>>
>>
>> On 3/10/17, 9:44 AM, "Billie Rinaldi" > billie.rina...@gmail.com>> wrote:
>>
>> Gour, thanks for putting up a release candidate! Two important things with
>> respect to release logistics: the release should be staged at
>> http

Re: [VOTE] Apache Slider (incubating) release 0.92.0-incubating-RC0

2017-03-13 Thread jonma...@gmail.com 
Thanks!  I suspected environmental issues, and thought that disabling both
vmem and pmem checks might suffice, but they did not.  I may explore some
more to ascertain the reasons my failures (as an educational exercise ;) )

I had success with other validations:

- signatures and checksums
- builds from source
- unit tests

Given your validation of the functional tests, I feel comfortable
submitting a +1.


On Mon, Mar 13, 2017 at 5:13 PM, Gour Saha  wrote:

> Jon,
> Thank you for looking into the RC.
>
> I successfully ran the FTs against a HDP 2.4.x cluster. My cluster is a
> multi-node cluster but I don¹t think it has to be one (except if you want
> to run AppsThroughAgentQueueAndLabelsIT which needs a multi-node cluster
> with multiple labels). However as Billie mentioned you will need
> sufficient resources to run multiple containers required by few tests.
>
> I ran the tests that failed for you once again now and they all ran
> successfully.
>
> Note, I do see AASleepIT fail intermittently in my cluster.
>
> AppsThroughAgentIT
>
> ==
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 134.481
> sec - in org.apache.slider.funtest.lifecycle.AppsThroughAgentIT
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> AppsUpgradeIT
>
> =
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 229.602
> sec - in org.apache.slider.funtest.lifecycle.AppsUpgradeIT
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> ExternalComponentIT
>
> ===
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 266.571
> sec - in org.apache.slider.funtest.lifecycle.ExternalComponentIT
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> UniqueComponentNamesIT
>
> ==
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 131.196
> sec - in org.apache.slider.funtest.misc.UniqueComponentNamesIT
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> AgentWebPagesIT
>
> ===
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 80.773 sec
> - in org.apache.slider.funtest.lifecycle.AgentWebPagesIT
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> AgentPingSocketIT
>
> =
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 101.74 sec
> - in org.apache.slider.funtest.lifecycle.AgentPingSocketIT
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> AMConfigPublishingIT
>
> 
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.391 sec
> - in org.apache.slider.funtest.misc.AMConfigPublishingIT
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> -Gour
>
> On 3/13/17, 8:38 AM, "Billie Rinaldi"  wrote:
>
> >I haven't run the functional suite myself for this RC. I seem to recall
> >having run the tests on a single VM at some point in the past. Maybe the
> >VM
> >needs to have enough resources to run all the containers requested,
> >though?
> >
> >On Sun, Mar 12, 2017 at 7:05 PM, jonma...@gmail.com 
> >wrote:
> >
> >> Having trouble remembering:  is there a requirement for more than one
> >>node
> >> manager for the functional tests?  Seeing the logging below during a
> >>test,
> >> and my initial attempt at running the functional suite was against a
> >>single
> >> node HDP installation.
> >>
> >> 2017-03-12 21:43:43,424 [AmExecutor-006] INFO  state.AppState
> >> (AppState.java:reviewOneRole(2170)) - COMMAND_LOGGER: Asking for 1 more
> >> nodes(s) for a total of 2
> >> 2017-03-12 21:43:43,424 [AmExecutor-006] DEBUG state.RoleHistory
> >> (RoleHistory.java:findRecentNodeForNewInstance(606)) - There are 0
> >>node(s)
> >> to consider for COMMAND_LOGGER
> >> 2017-03-12 21:43:43,424 [AmExecutor-006] INFO  state.RoleHistory
> >> (RoleHistory.java:findRecentNodeForNewInstance(625)) - No node found
> for
> >> COMMAND_LOGGER
> >>
> >>
> >> On Sun, Mar 12, 2017 at 6:17 PM, jonma...@gmail.com  >
> >> wrote:
> >>
> >> > Anyone else attempted running the functional suite?  I'm getting some
> >> > errors against an HDP 2.4 cluster (2.4.2.0-258):
> >> >
> >> > Failed tests:
> >> >   AppsThroughAgentIT.testCreateFlex:86->CommandTestBase.
> >> > expectLiveContainerCountReached:1367-

Re: May report draft

2018-05-02 Thread jonma...@gmail.com 
+1

On Wed, May 2, 2018 at 1:13 PM, Gour Saha  wrote:

> +1
>
> -Gour
>
> > On May 2, 2018, at 9:12 AM, Ted Yu  wrote:
> >
> > +1
> >
> > On Wed, May 2, 2018 at 8:27 AM, Billie Rinaldi  >
> > wrote:
> >
> >> Hi all, here is a draft of our report due today. If there are no
> objections
> >> / suggestions, I'll post this by the end of the day.
> >>
> >> Slider
> >>
> >> Slider is a collection of tools and technologies to package, deploy, and
> >> manage long running applications on Apache Hadoop YARN clusters.
> >>
> >> Slider has been incubating since 2014-04-29.
> >>
> >> Three most important issues to address in the move towards graduation:
> >>
> >> N/A
> >>
> >> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> >> aware of?
> >>
> >> A discussion thread has begun on the retirement of the Slider podling.
> >> There has not been much activity on the thread so far. Concern was
> >> expressed that users who are unable to upgrade to Hadoop 3 will be
> >> stuck without Slider support. A question was posed about whether an
> >> additional release before retirement would help. Given the lack of
> >> involvement on the thread, it is likely that a retirement vote will be
> held
> >> soon.
> >>
> >> How has the community developed since the last report?
> >>
> >> No new community development.
> >>
> >> How has the project developed since the last report?
> >>
> >> Two commits have been made since the last report. One improved Slider's
> >> behavior in multi-homed environments and the other fixed a bug in how
> the
> >> Slider agent kills processes.
> >>
> >> How would you assess the podling's maturity?
> >> Please feel free to add your own commentary.
> >>
> >>  [ ] Initial setup
> >>  [ ] Working towards first release
> >>  [ ] Community building
> >>  [ ] Nearing graduation
> >>  [x] Other: Nearing retirement
> >>
> >> Date of last release:
> >>
> >>  2017-03-23 slider-0.92.0-incubating
> >>
> >> When were the last committers or PPMC members elected?
> >>
> >>  2018-02-21 New committer and PPMC member Kyungwan Nam
> >>
>
>


-- 
Sent from some random computer

Re: [VOTE] Retire Slider from incubation

2018-05-15 Thread jonma...@gmail.com 
+1

On Tue, May 15, 2018 at 10:56 AM, Billie Rinaldi 
wrote:

> Since there has been no additional activity on the discussion thread, it is
> time to call vote. Please vote on whether we should retire the Slider
> podling. Here is my +1.
>
> [ ] +1 Retire Slider
> [ ] +0 No opinion
> [ ] -1 Do not retire Slider because ...
>
> This vote will remain open for 72 hours.
>



-- 
Sent from some random computer

Re: [DISCUSS] YARN-5079 : Native YARN framework layer for services and Apache Slider

2016-07-19 Thread jonma...@gmail.com 
+1 - this approach definitely addresses an important need.

The project has gone through at least a couple of development/repo
approaches:

1)  git-flow with master/develop branches
2)  abandonment of the master branch and work/releases based on the develop
branch.

I have no strong objections to the approach above, but given the existence
of the two branches above, and some developer familiarity with the
'develop' branch, perhaps we could:

1)  Synch the master branch to the develop branch (not sure which git
mechanism would best achieve this)
2)  Leverage the develop branch to create the agent/app-package based
Slider distribution

In this way there'a a legacy branch available while develop continues to
proceed on the existing and familiar development branch.

Just a thought.  Like I said - no real strong feelings :)

On Mon, Jul 18, 2016 at 3:23 PM, Gour Saha  wrote:

> +1 forwarding from the discussion in Slider DL
>
> Note: On the Slider side, we plan to create a branch corresponding to this
> YARN branch. In this, we intend to remove all the pieces which will move
> to the YARN branch (Slider core, AM, client). We will retain the Agent and
> app-packages which will then depend on the new hadoop-slider module (in
> addition to the existing hadoop modules that it already depends on). This
> will create a single view of Slider codebase, exactly as it stands today,
> fully supporting the current app-packages. Slider can even release its
> future versions from this new branch, once the hadoop-sldier module
> reaches a matured state. It will be an easier path for existing Slider
> users/app-owners to move to the future stable state of Slider completely
> off of Hadoop YARN codebase. All that would be expected is to migrate the
> app-packages to the agent-less version. Of course the Slider community
> will do the migration for the current app-packages in the Slider codebase.
>
> -Gour
>
> On 7/14/16, 7:36 PM, "Vinod Kumar Vavilapalli"  wrote:
>
> >Hi, Hadoop YARN community!
> >
> >(Cross-posting across Hadoop and Slider communities)
> >
> >I opened a JIRA a little while ago to pursue a native YARN framework for
> >services: https://issues.apache.org/jira/browse/YARN-5079.
> >
> >It is part of a bigger effort that a bunch of us YARN community members
> >are interested in making progress on: YARN-4692 - [Umbrella] Simplified
> >and first-class support for services in YARN.
> >
> >The idea is that with our current attention on making services
> >first­-class, it's time to take a fresh look at how we can make Apache
> >Hadoop YARN support services well out of the box. I’ve been looking at
> >various possibilities - ranging from a custom new framework room scratch
> >to using one of the existing projects - and stopped at Apache Slider
> >(http://slider.incubator.apache.org) given its association with some of
> >the YARN community members (Steve Loughran, Devaraj Das, Arun C Murthy,
> >myself etc.).
> >
> >Slider client & AM already handles a great deal of the functionality that
> >we need. I posit that assimilating the client, ApplicationMaster etc of
> >an existing framework like Apache Slider can serve our purpose really
> >well. My early informal discussions about this with few Hadoop and Slider
> >community members yielded generally favourable feedback.
> >
> >The Apache Slider incubator community also discussed this and expressed
> >generally positive interest in YARN taking up Slider’s key pieces, you
> >can see that discussion here: https://s.apache.org/0hoh.
> >
> >So in summary, we are looking to the following
> >
> >- Code
> >   ― ‘Graduate' key pieces (Slider client, AM) of Apache Slider into
> >Apache Hadoop for providing a native services experiences in YARN
> >   ― Leave for now some of the pieces behind in Apache Slider - (a)
> >Slider agent as we won’t need it, (b) Slider packages that need more
> >deliberation in terms of where they will live in the long term.
> >   ― Create a branch in YARN, copy this code over into a new module(s),
> >and work towards completing a functioning app running on YARN.
> >
> >- Communities & releases
> >   ― Good thing is that many of Apache Slider community members are
> >already seasoned folk in the Apache Hadoop ecosystem projects. For those
> >committers & PMC in Slider that are not yet Hadoop committers / PMC,
> >without complicating things much, a proposed path forward is active
> >participation in the branch (as branch committers?) and eventually in
> >mainline YARN and thus go through a natural progression to committership
> >/ PMC. Given that most of the members are stalwarts in the Apache
> >communities, this should be a cinch IMO.
> >   ― The work on this new code can start, and depending on its state, and
> >assuming that the experiment succeeds, can be merged into trunk and later
> >picked up in the next nearest & feasible Apache Hadoop release.
> >   ― While the work on forked-over-code goes on till we have a
> >functioning app, the current Apache Slid