Re: distributed file systems

Does EFS count? :-) https://aws.amazon.com/efs/ \-- **Rodrick Brown** / Systems Engineer +1 917 445 6839 / [rodr...@orchardplatform.com](mailto:char...@orchardplatform.com) **Orchard Platform** 101 5th Avenue, 4th Floor, New York, NY 10003 [http://www.orchardplatform.com](htt

distributed file systems

Hello, Has anyone customized/compiled mesos and successfully deployed a mesos cluster to run on cephfs, orangefs [1], or any other distributed file systems? If so, some detail on your setup would be appreciated. [1] http://www.phoronix.com/scan.php?page=news_item&px=OrangeFS-Lands-Linux-

Re: Enable s3a for fetcher

Ok. Thanks Joseph. I will figure out how to get a more recent hadoop onto my dcos agents then. Jamie From: Joseph Wu mailto:jos...@mesosphere.io>> Reply-To: "user@mesos.apache.org" mailto:user@mesos.apache.org>> Date: Tuesday, May 10, 2016 at 1:40 PM To: user mailt

Re: Enable s3a for fetcher

I can't speak to what DCOS does or will do (you can ask on the associated mailing list: us...@dcos.io). We will be maintaining existing functionality for the fetcher, which means supporting the schemes: * file * http, https, ftp, ftps * hdfs, hftp, s3, s3n <-- These rely on hadoop. And we will

Re: Enable s3a for fetcher

This is the mesos latest documentation: If the requested URI is based on some other protocol, then the fetcher tries to utilise a local Hadoop client and hence supports any protocol supported by the Hadoop client, e.g., HDFS, S3. See the slave configuration documentation

Re: Enable s3a for fetcher

Mesos does not explicitly support HDFS and S3. Rather, Mesos will assume you have a hadoop binary and use it (blindly) for certain types of URIs. If the hadoop binary is not present, the mesos-fetcher will fail to fetch your HDFS or S3 URIs. Mesos does not ship/package hadoop, so these URIs are n

Re: Enable s3a for fetcher

I’m happy to have default IAM role on the box that can read-only fetch from my s3 bucket. s3a gets the credentials from AWS instance metadata. It works. If hadoop is gone, does that mean that hfds: URIs don’t work either? Are you saying dcos and mesos are diverging? Mesos explicitly supports hdf

Re: Enable s3a for fetcher

The s3 fetcher stuff inside of DC/OS is not supported. The `hadoop` binary has been entirely removed from DC/OS 1.8 already. There have been various proposals to make it so the mesos fetcher is much more pluggable / extensible (https://issues.apache.org/jira/browse/MESOS-2731 for instance). Genera

Re: Configuring failover timeouts?

This is the health check of MySQL. { "protocol": "TCP", "portIndex": 0, "gracePeriodSeconds": 100, "intervalSeconds": 10, "timeoutSeconds": 20, "maxConsecutiveFailures": 3, "ignoreHttp1xx": false } The DCOS is detecting it fairly fast. Marathon is taking quite a lot

Re: Configuring failover timeouts?

Hi What is configuration of health-check. If it's configured to have 5 min interval and allow 3 consequential fails you end up with 15 min delay between marking task unhealthy and spawning new one. Best Tomek wt., 10.05.2016 o 18:54 użytkownik Ruturaj Dhekane napisał: > Hi all, > > I have a Me

Enable s3a for fetcher

I want to use s3a: urls in fetcher. I’m using dcos 1.7 which has hadoop 2.5 on its agents. This version has the necessary hadoop-aws and aws-sdk: hadoop--afadb46fe64d0ee7ce23dbe769e44bfb0767a8b9]$ ls usr/share/hadoop/tools/lib/ | grep aws aws-java-sdk-1.7.4.jar hadoop-aws-2.5.0-cdh5.3.3.jar Wha

Configuring failover timeouts?

Hi all, I have a Mesos+Marathon setup deployed on 3 master 6 slaves mode. I have deployed a MySQL docker container and another front end container. I have health check enabled on the MySQL container. When i turn off the slave that contains the MySQL, DCOS detects bad health, marks the slave as u

Re: Should "read-only" HTTP endpoints allow other request methods than "GET"?

There was some discussion of this between mpark and I in relation to the v1 operator API. The idea was to have a base class for endpoints that implement GET/POST/DELETE/PUT/etc... functions that return an error by default. You can then override the specific subset of them that that each endpoint s

Re: Should "read-only" HTTP endpoints allow other request methods than "GET"?

+1 to only allow GET for read only @vinodkone > On May 10, 2016, at 6:37 AM, Jan Schlicht wrote: > > Hi guys, > > while working on HTTP endpoint authorization for Mesos, I found some > interesting behavior: It's the responsibility of the HTTP endpoint handlers > to validate the HTTP request me

Should "read-only" HTTP endpoints allow other request methods than "GET"?

Hi guys, while working on HTTP endpoint authorization for Mesos, I found some interesting behavior: It's the responsibility of the HTTP endpoint handlers to validate the HTTP request method they've been called with. Many "read-only" endpoints (e.g. "/flags", "/state") don't do this at the moment.

RE: Marathon GUI -Scaling An Application Issue.

Hi, When I deploy the docker image for conatiners, it’s able to bring up the container on one of the slave. I believe the resource is being utilized and therefore the marathon tries to deploy the container on the other machine but it gets an error of “Abnormal executor termination” failed. The