Re: scp with mesos fetcher

That's so cool, otherwise I am going to contribute it. Cheers guys On Wed, Jul 6, 2016 at 10:47 AM, Jie Yu wrote: > Bryan, not yet. But we are trying to make fetcher pluggable so that we can > easily extend it to support scp. You can monitor the progress on this epic: > https://issues.apache.or

Re: scp with mesos fetcher

Bryan, not yet. But we are trying to make fetcher pluggable so that we can easily extend it to support scp. You can monitor the progress on this epic: https://issues.apache.org/jira/browse/MESOS-3918 - Jie On Tue, Jul 5, 2016 at 7:42 PM, haosdent wrote: > Hi, @Bryan. So far mesos fetcher only s

Re: scp with mesos fetcher

Hi, @Bryan. So far mesos fetcher only support * ftps,ftp,https,http (via libcurl) * hdfs, hftp, s3, s3n (via hadoop client) On Wed, Jul 6, 2016 at 9:41 AM, Bryan Fok wrote: > Hi all > > Is it possible to do scp over the mesos fetcher? > > BR > Bryan > -- Best Regards, Haosdent Huang

Re: Return a vector value from custom executor

Understood, moving to that direction now. Thank you for your advance. BR Bryan On Wed, Jul 6, 2016 at 1:29 AM, haosdent wrote: > I afraid protobuf don't allow so large message size? I think you may store > your result in HDFS or other distributed filesystem. And executor just > notify framework

scp with mesos fetcher

Hi all Is it possible to do scp over the mesos fetcher? BR Bryan

Re: A Redis Framework for Apache Mesos

Thanks Avinash, It is done now. On Mon, Jul 4, 2016 at 12:01 PM, Avinash Sridharan wrote: > > > On Mon, Jul 4, 2016 at 12:11 AM, DhilipKumar Sankaranarayanan < > s.dhilipku...@gmail.com> wrote: > >> *"* >> *Would be nice to get on the frameworks page:* >> >> *http://mesos.apache.org/documentatio

Allocator slack channel

Hi, I created an #allocator slack channel in mesos.slack.com, please join this if you want to have some discussion for allocator related issues, such as allocator performance, optimistic offer, revocable resource etc. @bmahler and @vinodkone, I also posted a RR here https://reviews.apache.org/r/

Re: Documentation for ACCEPT HTTP API

- the list Hi Giulio, You probably remember me, I used to work with Predrag and Jakob and we met when BenH was at CERN. Please let me know how it goes with using HTTP API. The documentation is indeed rough on edges so I am not surprised you're having questions. I'll be happy to setup a google ha

Re: On a whim I sync'd my local master branch

You're welcome! Kudos to Gilbert and Jie for landing this :). On Wednesday, June 29, 2016, Charles Allen wrote: > And ran the unit tests (through jenkins) > > [--] Global test environment tear-down > [==] 1209 tests from 140 test cases ran. (360352 ms total) > [ PASSED ] 1209 t

Re: Documentation for ACCEPT HTTP API

Hi Giulio, You can find more details on the JSON format here: https://github.com/apache/mesos/blob/master/include/mesos/v1/scheduler/scheduler.proto#L221 i.e., the JSON accepted by the /scheduler endpoint is equivalent to the protobuf message format described in that file. Neil On Mon, Jul 4,

Re: ScyllaDB

+1 Le 5 juil. 2016 17:38, "José Guilherme Vanz" a écrit : > Hi guys! > > Have you ever consider make ScyllaDB run on > Mesos? > I took a look on it, it seems very powerful and cool! It would be nice run > it on Mesos =) > > Thanks >

Re: Return a vector value from custom executor

I afraid protobuf don't allow so large message size? I think you may store your result in HDFS or other distributed filesystem. And executor just notify framework the progress of store. On Tue, Jul 5, 2016 at 2:17 PM, Bryan Fok wrote: > Hi all > > What is the best practice to have a custom execu

ScyllaDB

Hi guys! Have you ever consider make ScyllaDB run on Mesos? I took a look on it, it seems very powerful and cool! It would be nice run it on Mesos =) Thanks

Re: Fetcher cache: caching even more while an executor is alive

Thanks, it looks promising to me - I was aware of persistent volumes because I thought the use case was different, like for databases. I'll try it on. As the document says > persistent volumes are associated with roles, this makes failure handling a little bit difficult - As my framework is not

RE: Fetcher cache: caching even more while an executor is alive

As you're writing the framework, have you looked at reserving persistent volumes? I think it might help in your use case: http://mesos.apache.org/documentation/latest/persistent-volume/ Aaron -- Aaron Carey Production Engineer - Cloud Pipeline Industrial Light & Magic London 020 3751 9150 ___

Fetcher cache: caching even more while an executor is alive

Hi, I'm developing my own framework - that distributes >100 independent tasks across the cluster and just run them arbitrarily. My problem is, each task execution environment is a bit large tarball (2~6GB, mostly application jar files) and task itself finishes within 1~200 seconds, while tarball ex