Hi.
I'm looking at existing open source workflow engines we can use for
scheduling spark jobs with intricate dependencies on a datastax cassandra
cluster. Currently we are using crontab to schedule jobs and want to move
to something which is more robust and highly available.
There are 2 main
I have limited knowledge of Azkaban/Spark, but this sounds like a good fit for
the Chronos (or Aurora) schedulers.
Mesos itself doesn't provide the scheduling logic (ie the dependency
information, the 'run this task at x time' info etc). Mesos is the framework,
which Chronos or Aurora run on
Greetings, friends.
I'm adam-mesos, and I'm a Distributed Systems Architect at Mesosphere and
an Apache Mesos committer. My current areas of focus are security
https://mesoscon2015.sched.org/event/7c88ed02112a47935292554102f6d25a,
storage, and stateful services
Hi all,
I am Ankur Chauhan. I am a Sr. Software engineer with the Reporting and
Analytics team
at Brightcove Inc. I have been evaluating, tinkering, developing with mesos for
about an year
now. My latest adventure has been in the spark mesos integration and writing
the new apache flink -
mesos
Hi Gidon,
just to make sure, you mean static reservations on mesos agents (via
--resources flag) and not dynamic reservations, right?
Let me first try to explain, why you get the TASK_ERROR message. The
built-in allocator merges '*' and reserved resources, hinting master to
create a single
Hi Alex,
Yep, this setup is using static reservations in agents.
I haven't tried running a big task with two or more resources (reserved
and unreserved), but guess it is quite intuitive for a developer - a
framework is offered two resource objects, and launches a task specifying
these
if there were an api for splitting a resource object
I think it's a good idea, resource math is something that each framework
re-implements. We were discussing the idea of providing a framework kit,
but AFAIK there has been no work done in this direction yet. Mind filing a
JIRA ticket?
sending
We've a 20 node mesos cluster running mesos v0.21.1, We run marathon on top
of this setup without any problems for ~4 months now. I'm now trying
to get hadoop
mesos https://github.com/mesos/hadoop/ integration working but I see the
TaskTrackers that gets launched are failing with the following
Hello, python mesos.native depends mesos.so currently. Java also depends on
mesos.so to community with Mesos now. And mesos.so link curl-nss library in
its dependency. Maybe mesos would remove native dependency in the future (
Hello Everyone,
I am Sharma Podila, senior software engineer at Netflix. It is exciting to
be a part of MesosCon again this year.
We developed a cloud native Mesos framework to run a mix of service, batch,
and stream processing workloads. To which end we created a reusable
plug-ins based
Hi,
I'm Nic Grayson, Software Engineer at Banno/Jack Henry Associates. I'm
excited to return to mesoscon this year. I'll be bringing more of our team
with me this year, 7 in total.
We've been hard at work automating deployments with terraform, marathon,
and mesos. I’m excited to see the
Hi,
I'm Alex, I'm working on a distributed stream processor in c++ (concord.io).
Looking forward to connecting with all of you. Would be great to meet with
people doing large cluster load testing on mesos :)
I'll be at the hackathon with some coworkers as well.
On Mon, Aug 17, 2015 at 1:48
Sounds good, will do.
Regards,
Gidon
From: Alex Rukletsov a...@mesosphere.com
To: user@mesos.apache.org
Date: 17/08/2015 05:30 PM
Subject:Re: Launching tasks with reserved resources
if there were an api for splitting a resource object
I think it's a good idea,
Apologies for not making myself clear. I agree that libcurl4-nss-dev
has been listed earlier as a required dependency. In fact, we did have
it listed in our build scripts as well. What's new in 0.23 is that
there is now a hard runtime dependency in MesosExecutorDriver that we
have not experienced
Hi All,
I'm Aaron and I work as a production/rd engineer at Industrial Light and
Magic. We've been experimenting with Mesos and Docker over the last 4 or 5
months for a variety of purposes.
I'm really looking forward to hearing more on scheduling algorithms and per
container IP applications.
I’m Mark Eijsermans, Sr Software Engineer at Hootsuite on the platform team.
Currently running Mesos for our build (Jenkins), some internal tooling and
looking to move our stateless dockerized scala services in the future. Really
excited to meet everyone and hear people’s experiences
Howdy all!
I’m Joe Smith, Site Reliability Engineer for Aurora/Mesos at Twitter. I’ve been
running our Aurora/Mesos clusters for over three years, so I’ve got lots of war
stories around migrations, pitching teams/organizations, and operations +
maintenance.
I’m really excited to share our
Hey everyone! I'm Tom Arnfeld, a software engineer working at DueDil (in
London). We've been running Mesos for almost 18 months now, for large scale
batch and stream data processing applications. Give me a shout if you want to
talk data stuff! We're also experimenting with Mesos to deploy long
Hello MessoCon Attendees! I'm Deshi Xiao, CTO at an Beijing Cloud startup:
Beijing DataMan Inc. our startup 's core tech is based on apache mesos
project, and I'd love to talk about mesos use case in China or about waht
you're building.
You can find me on Twitter at @xds2000
2015-08-16 20:21
19 matches
Mail list logo
