Re: Monitoring at container level

sos clusters and f.ex. to detect that > some containers are having issues? > > -- > BR, > Michał Łowicki > -- Regards, Pradeep Chhetri

Re: Fetcher cache: caching even more while an executor is alive

Just a random thought, have you tried something like bittorrent based deployments. They are really efficient when you have to distribute big artifacts across cluster of machines. Following two projects might be helpful in achieving that: 1. http://erdgeist.org/arts/software/opentracker/ 2.

Re: Mesos Slave not registering or getting activated

5:33:18.412081 1622 process.cpp:1966] Failed to shutdown socket > with fd 10: Transport end W0520 05:33:18.108160 1616 slave.cpp:3484] > Master disconnected! Waiting for a new master to be elected > > E0520 05:33:18.412081 1622 process.cpp:1966] Failed to shutdown socket > with fd 10:

Re: Mesos Slave not registering or getting activated

ated all master nodes and then trying but isn’t > getting registered. > > > > *From:* Pradeep Chhetri [mailto:pradeep.chhetr...@gmail.com] > *Sent:* 19 May 2016 23:40 > > *To:* user@mesos.apache.org > *Subject:* Re: Mesos Slave not registering or getting activated > &

Re: Mesos Slave not registering or getting activated

gt; > > > > > Thanks > > > > > -- > > > This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise confidential information. If you have > received it in error, please notify the sender immediately and delete the > original. Any other use of the e-mail by you is prohibited. Where allowed > by local law, electronic communications with Accenture and its affiliates, > including e-mail and instant messaging (including content), may be scanned > by our systems for the purposes of information security and assessment of > internal compliance with Accenture policy. > > __ > > www.accenture.com > > > -- Regards, Pradeep Chhetri

Re: Cannot pull from private docker v1 registry

l > > and proprietary information and material for the sole use of the > > intended recipient(s). Any review, use or distribution that has not > > been expressly authorized by Stem, Inc. is strictly prohibited. If > > you are not the intended recipient, please contact the sender and > > delete all copies. Thank you. > > > > > > -- > The information contained in this message is for the intended addressee > only and may contain confidential and/or privileged information. If you are > not the intended addressee, please delete this message and notify the > sender; do not copy or distribute this message or disclose its contents to > anyone. Any views or opinions expressed in this message are those of the > author and do not necessarily represent those of Motortrak Limited or of > any of its associated companies. No reliance may be placed on this message > without written confirmation from an authorised representative of the > company. > > Registered in England 3098391 V.A.T. Registered No. 667463890 > -- Regards, Pradeep Chhetri

Re: FW: failure notice

Not sure though but my guess is that Mesos slave is unable to pull the docker image. Are you using private docker registry ? On Apr 26, 2016 5:24 PM, wrote: > Hi, > > I tried running python application through Marathon GUI ( >

Re: Pyspark Cluster Mode

erDispatcher on >>> Marathon? If so, can you please share your JSON configuration for the >>> application? >>> >>> >>> Thanks, >>> June Taylor >>> System Administrator, Minnesota Population Center >>> University of Minnesota

Re: Mesos Master and Slave on same server?

ll of our node servers are identical hardware. Is it reasonable for me >> to install the Mesos-Master and Mesos-Slave on the same physical hardware? >> >> Thanks, >> June Taylor >> System Administrator, Minnesota Population Center >> University of Minnesota >> > > -- Regards, Pradeep Chhetri

Re: Mesos Task History

utomatically > cleared out at some interval? > > Thanks, > June Taylor > System Administrator, Minnesota Population Center > University of Minnesota > -- Regards, Pradeep Chhetri

Re: Pyspark Cluster Mode

s the recommended method of doing this? > > Thanks, > June Taylor > System Administrator, Minnesota Population Center > University of Minnesota > -- Regards, Pradeep Chhetri

Re: orphaned_tasks cleanup and prevention method

or > System Administrator, Minnesota Population Center > University of Minnesota > > On Thu, Apr 7, 2016 at 9:46 AM, haosdent <haosd...@gmail.com> wrote: > > > pyspark executors hanging around and consuming resources marked as Idle > in mesos Web UI > > Do you have some logs about this? > > >is there an API call I can make to kill these orphans? > > As I know, mesos agent would try to clean orphan containers when restart. > But I not sure the orphan I mean here is same with yours. > > On Thu, Apr 7, 2016 at 10:21 PM, June Taylor <j...@umn.edu> wrote: > > Greetings mesos users! > > I am debugging an issue with pyspark executors hanging around and > consuming resources marked as Idle in mesos Web UI. These tasks also show > up in the orphaned_tasks key in `mesos state`. > > I'm first wondering how to clear them out - is there an API call I can > make to kill these orphans? Secondly, how it happened at all. > > Thanks, > June Taylor > System Administrator, Minnesota Population Center > University of Minnesota > > > > > -- > Best Regards, > Haosdent Huang > > > > > > -- > Best Regards, > Haosdent Huang > > > > > > -- > Best Regards, > Haosdent Huang > > > > > > -- > Best Regards, > Haosdent Huang > > > > > > > > -- Regards, Pradeep Chhetri

Re: Set LIBPROCESS_IP for frameworks launched with marathon

marathon (Kafka and elastic search). However, in order for >>> the frameworks to receive resource offers I need to set the LIBPROCESS_IP >>> environment variable to the hosts IP address for the docker container >>> running the frameworks. Currently I am working around me this by using a >>> constraint to hard wire the slave that the framework gets launched on, so >>> then I can put the slaves ip in the marathon json file. >>> >>> Obviously this is not ideal. Is there a better way to define the host ip >>> Inside the docker container? >>> >>> Sent from my iPad >>> >>> >>> >> >> >> -- >> Best Regards, >> Haosdent Huang >> > > -- Regards, Pradeep Chhetri

Re: Question on slave recovery

led. Are > you using cgroup isolation? Can you include logs? > > On Fri, Mar 25, 2016 at 6:17 AM, Pradeep Chhetri < > pradeep.chhetr...@gmail.com> wrote: > >> >> Hello, >> >> I remember when i was running some older mesos version (maybe 0.23.0), >&g

Re: Agent won't start

ally you should be able to leave the work_dir alone and let >> the Mesos agent manage it for you. >> >> In any case, I would recommend that you set the work_dir to something >> outside of /tmp; /var/lib/mesos is a commonly-used location. >> >> Cheers, >> Greg >> > > -- Regards, Pradeep Chhetri

Re: Agent won't start

erval="1secs" --revocable_cpu_low_priority="true" > --strict="true" --switch_user="true" --version="false" > --work_dir="/tmp/mesos" > I0329 14:19:39.616835 5870 slave.cpp:354] Slave resources: cpus(*):4; > mem(*):23089; disk(*):122517; ports(*):[31000-32000] > I0329 14:19:39.617032 5870 slave.cpp:384] Slave hostname: 71.100.202.193 > I0329 14:19:39.617046 5870 slave.cpp:389] Slave checkpoint: true > I0329 14:19:39.618841 5894 state.cpp:36] Recovering state from > '/tmp/mesos/meta' > I0329 14:19:39.618872 5894 state.cpp:672] Failed to find resources file > '/tmp/mesos/meta/resources/resources.info' > I0329 14:19:39.619730 5898 group.cpp:313] Group process (group(1)@ > 71.100.202.193:5051) connected to ZooKeeper > I0329 14:19:39.619760 5898 group.cpp:787] Syncing group operations: queue > size (joins, cancels, datas) = (0, 0, 0) > I0329 14:19:39.619773 5898 group.cpp:385] Trying to create path '/mesos' > in ZooKeeper > > -- Regards, Pradeep Chhetri

Re: Port Resource Offers

phemeral-port-range/237543#237543 > > - Erik > > On Tue, Mar 29, 2016 at 3:07 AM, Pradeep Chhetri < > pradeep.chhetr...@gmail.com> wrote: > >> Hi Klaus, >> >> Thank you for the quick reply. >> >> One quick question: >> >> I have some of th

Re: Port Resource Offers

So I guess, I should always define ports in ranges rather than individual > port. > > It will be helpful if someone can confirm if it is the expected behaviour > and my configuration is wrong. > > -- > Regards, > Pradeep Chhetri > -- Regards, Pradeep Chhetri

Question on slave recovery

at i deleted all the information related to checkpointing by cleaning up /tmp/mesos. -- Regards, Pradeep Chhetri

Re: HTTP Basic Auth for Mesos UI

, >>>> >>>> Is there any built-in support for it in Mesos UI (port 5050 by >>>> default)? There is --authenticate_http but it seems to apply to HTTP >>>> API? Found MESOS-1923 >>>> <https://issues.apache.org/jira/browse/MESOS-1923> but ticket is very >>>> old. >>>> >>>> -- >>>> BR, >>>> Michał Łowicki >>>> >>> >>> >>> >>> -- >>> Best Regards, >>> Haosdent Huang >>> >> >> > > > -- > BR, > Michał Łowicki > -- Regards, Pradeep Chhetri

Re: Resource Isolation per Framework

e > would share cross all frameworks. > > On Fri, Mar 18, 2016 at 9:43 PM, Pradeep Chhetri < > pradeep.chhetr...@gmail.com> wrote: > >> Hello All, >> >> I am looking for ways to isolate resources per framework basis. Basically >> what i want

Resource Isolation per Framework

} But I don't think if we can register a framework to mesos master with multiple roles. I can see this open issue regarding this: https://issues.apache.org/jira/browse/MESOS-1763 Can someone provide me some alternative way regarding what i am trying to achieve. -- Regards, Pradeep Chhetri

Re: What are the invalid-user.log files?

the case for the mesos master. That explains > the invalid-user, but why isn't these processes picking up $USER? > > > On 03/18/2016 07:31 AM, Pradeep Chhetri wrote: > > I can see that USER environment variable is not set for mesos-slave > process from /proc//environ. > > On F

Re: What are the invalid-user.log files?

I can see that USER environment variable is not set for mesos-slave process from /proc//environ. On Fri, Mar 18, 2016 at 2:19 PM, Pradeep Chhetri < pradeep.chhetr...@gmail.com> wrote: > My mesos cluster also produces logs > like mesos-slave.ip-172-31-45-33.invalid-user.log.ERROR.

Re: What are the invalid-user.log files?

>>>>> >>>>> mesos-slave..invalid-user.log.INFO.20160317-062640.918 >>>>> >>>>> I don't see any errors in the logs themselves, but the fact that >>>>> "invalid-user" is part of the name makes me think I'm missing something in >>>>> my config. The mesos-master.WARNING log has the message >>>>> >>>>> W0317 06:26:41.178268 915 authenticator.cpp:511] No credentials >>>>> provided, authentication requests will be refused >>>>> >>>>> which I'm thinking might be related. What am I missing? >>>>> >>>>> Peter >>>>> >>>>> >>>> >>>> >>>> -- >>>> Best Regards, >>>> Haosdent Huang >>>> >>> >>> >>> >>> -- >>> Best Regards, >>> Haosdent Huang >>> >>> >>> >> >> >> -- >> Best Regards, >> Haosdent Huang >> > > > > -- > Best Regards, > Haosdent Huang > > > -- Regards, Pradeep Chhetri

Re: What are the invalid-user.log files?

I installed mesos using mesosphere debian repository. On Fri, Mar 18, 2016 at 4:10 PM, haosdent <haosd...@gmail.com> wrote: > cool, you install mesos by deb or build it from source directly? > On Mar 19, 2016 12:06 AM, "Pradeep Chhetri" <pradeep.chhetr...@gmail.c

Re: Apache Spark Over Mesos

that jar from S3. Thanks. On Tue, Mar 15, 2016 at 5:17 PM, Pradeep Chhetri < pradeep.chhetr...@gmail.com> wrote: > TIm, sorry I am wrong above. > > The above config is from spark dispatcher container and those > configuration is not being propagated to the driver. > > I

Re: Apache Spark Over Mesos

TIm, sorry I am wrong above. The above config is from spark dispatcher container and those configuration is not being propagated to the driver. I will try the workaround you told and let you know how it goes. Thanks On Tue, Mar 15, 2016 at 4:42 PM, Pradeep Chhetri < pradeep.chh

Re: Apache Spark Over Mesos

driver is launched. > > Tim > > On Tue, Mar 15, 2016 at 9:26 AM, Pradeep Chhetri < > pradeep.chhetr...@gmail.com> wrote: > >> Hello Timothy, >> >> I am setting spark.mesos.executor.docker.image. In my case, the driver >> is actually started as a docker con

Re: Apache Spark Over Mesos

t; available but that too will download a copy to the sandbox. > > I'd prefer the former. > > Sent from Outlook Mobile <https://aka.ms/qtex0l> > > _ > From: Pradeep Chhetri <pradeep.chhetr...@gmail.com> > Sent: Tuesday, March 15, 2016 4:41 pm >

Re: Apache Spark Over Mesos

that too will download a copy to the sandbox. > > I'd prefer the former. > > Sent from Outlook Mobile <https://aka.ms/qtex0l> > > _ > From: Pradeep Chhetri <pradeep.chhetr...@gmail.com> > Sent: Tuesday, March 15, 2016 4:41 pm > Subject: A

Re: 0.27.2 RPMs

Hello all, I still don't see any Mesos 0.27.2 deb/rpm artifacts in the mesosphere repository. Is this expected ? Thanks. On Mar 9, 2016 3:46 PM, "Rad Gruchalski" wrote: > Oh, cool, I was expecting some info here. > Going to build one now :) > > Kind regards, > Radek

Re: Question regarding docker containerizer

ssage. > > Thanks, > > Guangya > > On Fri, Feb 26, 2016 at 6:08 PM, Pradeep Chhetri < > pradeep.chhetr...@gmail.com> wrote: > >> Actually the reason i asked this question is because i am unable to >> generate these warning messages by setting MEMORY &l

Re: Question regarding docker containerizer

master/validation.cpp#L397-L425 So i was trying to figure out whether https://github.com/apache/mesos/blob/4d4d7166414f0ebd8d5e40df34070446098a3c91/src/master/validation.cpp#L399 is resulting in true or false in case of docker containers started using marathon. On Fri, Feb 26, 2016 at 8:20 AM, Pradeep C

Re: Question regarding docker containerizer

ithub.com/apache/mesos/blob/master/src/slave/slave.cpp#L3672 > > -jojy > > > On Feb 25, 2016, at 4:10 PM, Pradeep Chhetri <pradeep.chhetr...@gmail.com> > wrote: > > Hi Jie, > > I see executor_id as empty. Does this means that it is running as task and > there is

Re: Question regarding docker containerizer

t;172.17.0.11"}]}]}}],"container": {"type": "DOCKER","docker": { "image": "kibana:4.3.1","network": "BRIDGE","port_mappings": [{"host_port": 31050,"container_port": 5601,&q

Re: Question regarding docker containerizer

ke a look at the TaskInfo. If the TaskInfo does not have > ExecutorInfo set, then it's a task. Otherwise, Mesos will launch the > executor and send the task to the executor. > > - Jie > > On Thu, Feb 25, 2016 at 2:50 PM, Pradeep Chhetri < > pradeep.chhetr...@gmail.com>

Question regarding docker containerizer

Hello, >From docker containerizer documentation ( http://mesos.apache.org/documentation/latest/docker-containerizer/), *"Users can either launch a Docker image as a Task, or as an Executor."* How can i identify whether a docker container started by lets say marathon is running as a task or as

Re: Mesos metrics -> influxdb

influxdb? > > I've found a couple of options, a collectd plugin which doesn't appear to > work with version 0.24.x and a more up to date containerised option which > randomly crashes regularly and doesn't appear to actually post any stats. > > Anyone have any good solutions? > > >

Re: external storage with dvdi module

is exposed by the > storage component itself. For example, concerning Openstack Cinder the > pre-emptive mount is not currently supported and there is a blueprint > (approved but not implemented yet) for enabling the multiple attachments of > a single volume. The roadmap for such addition in cinder is not clear to me > yet…what will be the behavior of the dvdi module (and the openstack storage > driver) in the meanwhile? > > > > … > > > -- Pradeep Chhetri In the world of Linux, who needs Windows and Gates...

Re: Changing mesos slave configuration

uch like it if Mesos supported such dynamic >>> changes. I suppose this would require that the agent consult its default >>> file on demand, rather than once at start-up. >>> >>> Cordially, >>> >>> Paul >>> >>> On Wed, Sep 23, 20

Setup environment variables on storm supervisors

Hello All, I am trying out Storm deployment over Mesos. I need to setup some environment variables like KAFKA_BROKER_HOSTS on supervisors. I was unable to figure out how can i do that. Can someone help me out. Thank you. -- Pradeep

Docker Registry 2.0 + Mesos + Marathon

hour debugging it, so thought that it might be useful for others as well :) -- Pradeep Chhetri

Re: Running storm over mesos

, Tim Chen wrote: Hi Pradeep, Without any more information it's quite impossible to know what's going on. What's in the slave logs and storm framework logs? Tim On Fri, Jul 3, 2015 at 10:06 AM, Pradeep Chhetri pradeep.chhetr...@gmail.com mailto:pradeep.chhetr...@gmail.com wrote: Hello

Running storm over mesos

as well. Can someone help in figuring out the problem. Thank you. -- Pradeep Chhetri