Re: registry / export question

AM, Gour Saha wrote: > It is REST style uri, so if you append the uri path with the export group > name you will get the info you are looking for. > > If that does not answer your question, can you give an example response > that you are expecting to see? > > -Gour > > >

Re: registry / export question

I've noticed the http://hostname/ws/v1/slider/publisher/exports/ only gives you the list of export values, but within each one the entries block are empty. Is it ok have them all embedded in one response so that you can get all

Re: Packaging new apps

Hi Jean, Thanks for the change, using instance tag(is it a new feature in the latest version? I didn't see it in the older slider versions) is a really good idea. it might be good for other's to have a template but not for kafka. Kafka is evolving in quite fast pace. I've seen many property key/v

Re: Component instance level configuration

nt will have their own > configuation? Only port can be different for instance of a component > nowadays. > > 2015-01-28 5:57 GMT+08:00 hsy...@gmail.com : > > > Hi, > > > > I'm just wondering if there is a way to support component instance level > > c

Component instance level configuration

Hi, I'm just wondering if there is a way to support component instance level configuration. For example, in kafka-on-yarn, kafka broker is a component, but each broker instance needs a unique id. It would be better to expose the instance id, so that different instance would override some of the

What configuration is mandatory in slider package?

Hi guys, Slider requires several configurations appConfig, metainfo, resources. What are necessary to be included in package what is only needed when launch the slider application. If I change appConfig do I have to update the package to make it run? Thanks!

Re: Is it able to specify the percentage of memory instead of # of MB

rom YARN and YARN only accepts numbers. > > > > > > > > % of total on each node also becomes an issue when nodes can have > > varying > > > > value of total memory available. This is common for heterogeneous > > > clusters > > > >

Is it able to specify the percentage of memory instead of # of MB

Hi guys, For each component instance, Is there a way to specify the % of total memory of each node instead of # of MB? Best, Siyuan

I can't specify user by calling Execute method

Hi I specified user when call Execute(..., user='hdp',...) But the container processes are all launched as *yarn* user

Re: Application configuration page is outdated?

and resources.json should be > enough. > > -Sumit > > On Tue, Nov 18, 2014 at 2:00 PM, Sumit Mohanty > wrote: > > > Is it possible for you to share the application package? I can browse > > through it to see what needs to change. > > > > On Tue, Nov 18, 2014 at 1

Application configuration page is outdated?

Hi guys, I just switch from 40 to 60 and I found the application configuration doesn't work any more. http://slider.incubator.apache.org/docs/slider_specs/application_instance_configuration.html For example : { "schema" : "http://example.org/specification/v2.0.0";, "metadata" : { }, "glo

Re: slider stop bug?

I'm using 0.60 On Tue, Nov 18, 2014 at 12:26 PM, Sumit Mohanty wrote: > You can use --force. > > Are you using the RC for 0.60 or 0.40? > > On Tue, Nov 18, 2014 at 12:00 PM, hsy...@gmail.com > wrote: > > > If the application is stuck in ACCEPT status, slider s

slider stop bug?

If the application is stuck in ACCEPT status, slider stop command doesn't stop the yarn application. I didn't setup the scheduler address correctly so the application stuck at ACCEPT status and I couldn't use slider stop to stop the application Best, Siyuan

HDP2.2?

Hi guys, Maybe it is not the right place to ask. If I want to switch to 0.60, it depends on hadoop 2.6 which is not generally available right now. Is there a easy way to setup a virtual cluster with hadoop2.6? I saw HDP2.2 preview is available with sandbox only. Best, Siyuan

Re: Do I have to repackage if I change appConfig.json/resources.json

"thaw" in 0.40) the application then it will use the hostnames that it remembered. What if I use *yarn application -kill* to stop the application? On Fri, Nov 14, 2014 at 5:09 PM, hsy...@gmail.com wrote: > The only problem is I have to upgrade to hadoop 2.6? > > On Fri, Nov 1

Re: Do I have to repackage if I change appConfig.json/resources.json

t;>> add more instances for example > > Flex command can be used while the application is running. > > >>> want the application recover from last snapshot > If you stop and then start (probably it is "freeze" and "thaw" in 0.40) the > application then

Question about behavior after container failures

Hi all, In the document, I got that slider will try to recover on container failure. But in my test application it doesn't I'm using the 0.40 release. build from source Here is what I get. *If I kill the child process * The agent check_process_status method will raise ComponentIsNotRunning and

Re: Do I have to repackage if I change appConfig.json/resources.json

at there are only one file to download for the users.* > > Which branch are you using? (for me to answer the next questions) > > On Fri, Nov 14, 2014 at 3:47 PM, hsy...@gmail.com > wrote: > > > I'm getting confused. After you edit your appConfig.json and > > re

How to change log level for Slider AM?

How to change log level for Slider AM? Thanks!

Re: Do I have to repackage if I change appConfig.json/resources.json

n package zip file > * Call create with --template appConfig.json --resources > resources.json ... > > -Sumit > > On Fri, Nov 14, 2014 at 2:08 PM, hsy...@gmail.com > wrote: > > > So actually you don't need appConfig.json and resources.json in the ZIP &

Re: Do I have to repackage if I change appConfig.json/resources.json

> instance referencing the new versions of the file from the command line: > > ./slider create —template —resources > > > the app config references the application package in HDFS, which can be > pre-seeded using “slider install-package” > > — Jon > > On Nov

Do I have to repackage if I change appConfig.json/resources.json

Everytime I change appConfig.json and resources.json. Do I have to repackage the zip file and redeploy the file to hdfs? Thanks!

Re: Q: How to define component configuration?

ake them available at the container? This seems to > be a good feature to support. > > -Sumit > > On Tue, Nov 11, 2014 at 2:42 PM, hsy...@gmail.com > wrote: > > > Thanks Steve, but I logged all the properties in params.py, I couldn't > > find any key named *myk

Re: Q: How to define component configuration?

rrides the global value, that override > is picked up > > It's essentially a form of prototype-based programming, except only of > properties, not methods: > http://en.wikipedia.org/wiki/Prototype-based_programming > > On 11 November 2014 01:30, hsy...@gmail.com wrote

Re: Q: How to define component configuration?

s": { "COMPONENT1": { *"mykey": "myvalue"* }, "slider-appmaster": { "jvm.heapsize": "256M" } } } Is it able to make the component1 read the value for *mykey* ？ Best, Siyuan On Mon, Nov 10, 2014 at 4:15

Re: Q: How to define component configuration?

If I want to have several instances of some component. But I want to set some of the properties to different value for different instances. How can I do it? Thanks! Best, Siyuan On Mon, Nov 10, 2014 at 1:26 PM, hsy...@gmail.com wrote: > Hi guys, > > Is there an example of

Q: How to define component configuration?

Hi guys, Is there an example of component configuration? Is there a way to give different value to same property for different instances? Siyuan

Re: Maven repository?

lider.git > > > You can checkout develop branch. > > > Cheers > > > On Wed, Nov 5, 2014 at 6:36 PM, hsy...@gmail.com wrote: > > > Is there a public maven repository that I can checkout the slider > library? > > > > Best, > > Siyuan > > >

Maven repository?

Is there a public maven repository that I can checkout the slider library? Best, Siyuan

Re: Is it able to make AM to restart from previous state?

from new node? What if you decrease the instance number? Thanks! Best, Siyuan On Wed, Nov 5, 2014 at 1:44 PM, hsy...@gmail.com wrote: > Thanks Steve, > > Is No 1 a new feature in YARN (Not released yet)? > > And you mentioned slider saves the location in history files. What ar

Re: Is it able to make AM to restart from previous state?

? Thanks On Wed, Nov 5, 2014 at 12:46 PM, Steve Loughran wrote: > On 5 November 2014 20:21, hsy...@gmail.com wrote: > > > Hi guys, > > > > I noticed in the code when a container fails it will try to relaunch from > > the same node. My question is if I restart whole app

Is it able to make AM to restart from previous state?

Hi guys, I noticed in the code when a container fails it will try to relaunch from the same node. My question is if I restart whole application(Ex. AM got killed, or manually restart the app). Does slider try to launch all containers from the nodes where it was running? Thanks! Best, Siyuan

Re: 2 questions

rt an fully functional embedded jetty server in the AM and set the tracking url to this {amhost:port}? Best, Siyuan On Fri, Oct 31, 2014 at 2:59 PM, Steve Loughran wrote: > On 31 October 2014 01:15, hsy...@gmail.com wrote: > > > Hi all, > > > > I have 2 questions, > > 1

2 questions

Hi all, I have 2 questions, 1. Is there a way to extend the REST API(add some customized api)? 2. Is there a way to add/remove component/component instance at the runtime? Thanks! Best, Siyuan

Re: Why command script is written in python?

opment. > > If you do not mind can you open a JIRA at > https://issues.apache.org/jira/browse/SLIDER describing the requirements > of > "debugging" during development time and post development. I will use that > JIRA to drive the issue. > > -Sumit > > On Thu, Oct

Re: Why command script is written in python?

fied to handle various > script types. Of course, today the only supported/tested one is python. > > On Thu, Oct 30, 2014 at 1:55 PM, hsy...@gmail.com > wrote: > > > May I ask why do you choose python to write the command script. And is > > there any other language? I find

How to inject my own web service handler into app master?

Hi, I noticed slider app master has an embedded web service server. Is there a way to inject my own web service call into that server? Another question how to you communication between containers(components) and AM(through agent code)? Is there a way to customize the communication/data between c

Why command script is written in python?

May I ask why do you choose python to write the command script. And is there any other language? I find difficult to debug because each time if I find an issue I have to change file package upload to hdfs, rerun and I'm not very familiar with python Best Siyuan

Re: Wrong FS: hdfs://localhost:9000/user/root/.slider/cluster/c100, expected: file:/// Issues deploying memcached using slider.

Sorry, my classpath should be $HADOOP_HOME/etc/hadoop/, thanks for you guys' help! On Wed, Oct 29, 2014 at 12:06 PM, hsy...@gmail.com wrote: > > > And all I have in slider-err.txt is > > log4j:WARN No appenders could be found for logger > (org.apache.slider.co

Re: Wrong FS: hdfs://localhost:9000/user/root/.slider/cluster/c100, expected: file:/// Issues deploying memcached using slider.

: hdfs://localhost:9000/user/siyuan/.slider/cluster/cl15, expected: file:/// Looks like there is no log4j log at all. How do I properly setup log4j? Best, Siyuan On Wed, Oct 29, 2014 at 11:47 AM, hsy...@gmail.com wrote: > Hi, > > I installed the apache hadoop with just unzip the tar ba

Re: Wrong FS: hdfs://localhost:9000/user/root/.slider/cluster/c100, expected: file:/// Issues deploying memcached using slider.

arn classpath variable. > > E.g. > > yarn.application.classpath > */etc/hadoop/conf*,/usr/hdp/current/hadoop-client/* > > On Wed, Oct 29, 2014 at 11:18 AM, hsy...@gmail.com > wrote: > > > @Pushkar > > > > Thanks, but it doesn&#x

Re: Wrong FS: hdfs://localhost:9000/user/root/.slider/cluster/c100, expected: file:/// Issues deploying memcached using slider.

@Pushkar Thanks, but it doesn't work for me. My slider-client.xml setting is yarn.application.classpath/home/siyuan/hadoop/current/etc/hadoop/*,/home/siyuan/hadoop/current/share/hadoop/tools/lib/*,/home/siyuan/hadoop/current/share/hadoop/mapreduce/lib/*,/home/siyuan/hadoop/current/share/ha

Re: Wrong FS: hdfs://localhost:9000/user/root/.slider/cluster/c100, expected: file:/// Issues deploying memcached using slider.

Another observation is this error has no stacktrace. I guess it does happen during launching the App master. It happens when app master try to launch container when it tries to localize the resource? Best, Siyuan On Tue, Oct 28, 2014 at 7:02 PM, hsy...@gmail.com wrote: > I try to run kafka

Re: Wrong FS: hdfs://localhost:9000/user/root/.slider/cluster/c100, expected: file:/// Issues deploying memcached using slider.

I try to run kafka as slider application On Tue, Oct 28, 2014 at 7:01 PM, hsy...@gmail.com wrote: > I had the same problem > > This is my appConfig.json > > { > "schema": "http://example.org/specification/v2.0.0";, > "metadata": { >

Re: Wrong FS: hdfs://localhost:9000/user/root/.slider/cluster/c100, expected: file:/// Issues deploying memcached using slider.

I had the same problem This is my appConfig.json { "schema": "http://example.org/specification/v2.0.0";, "metadata": { }, "global": { "application.def": "hdfs://localhost:9000/user/siyuan/slider_kafka.zip", "java_home": "/usr/lib/jvm/java-7-oracle/", "package_list": "files/kaf

NoClassDefFoundError ? Hadoop classpath

Hi guys, I'm new to slider. I tried to run a java application from slider and get the error. Hadoop classpath has been setup in client.xml Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/client/api/async/AMRMClientAsync$CallbackHandler Do I have to include the

Re: Is there a way to run application on certain subset of nodes?

e that the container can consume > most (not just > 50%) of the available memory. > > One anti-affinity is available or Slider adds such a feature itself, you > need not do use the work-around. > > > On Mon, Aug 4, 2014 at 2:04 PM, hsy...@gmail.com wrote: > > > Thanks Sum

Re: Is there a way to run application on certain subset of nodes?

half of available resource (e.g. memory) per host. > > Depending on the timeline of the availability of anti-affinity feature from > YARN, Slider can also add support for such a feature but the desirable path > is to get this through YARN. > > > On Mon, Aug 4, 2014 at 1:48 PM,

Re: Is there a way to run application on certain subset of nodes?

> > > https://issues.apache.org/jira/browse/YARN-796 > > > > > > On Mon, Aug 4, 2014 at 2:37 PM, hsy...@gmail.com > wrote: > > > > > Hi guys, > > > > > > I'm new to slider and try to convert some application into yarn app.

Is there a way to run application on certain subset of nodes?

Hi guys, I'm new to slider and try to convert some application into yarn app. I would like to ask is there a way to specify only a subset of nodes in the cluster to run my app and can slider guarantee every container(of that application) run on different nodes? Thank you very much! Best, Siyuan