I'm hoping to move it into production this week. Ran into some major
difficulties with etcd on Thursday which I *think* I've ironed out but I
need to be sure. etcd has a few gotchas in the current version but I will
say they've got one of the best IRC rooms I've ever seen! Tons of help
there if anyone else considers going this same path.
On Saturday, October 4, 2014 3:09:24 PM UTC-6, Rafał Krzewski wrote:
>
> Sounds great! I'm looking forward to the blog post :)
>
> Cheers,
> Rafał
>
> W dniu czwartek, 2 października 2014 16:46:04 UTC+2 użytkownik Ryan Tanner
> napisał:
>>
>> I've been working on this the past few weeks. Here's what I'm currently
>> doing.
>>
>> Background:
>> * CoreOS as the host OS
>> * Using etcd/fleet for service discovery and scheduling
>> * Using SkyDNS in order to address nodes by their logical name, also
>> allows nodes to bind to that name rather than the IP of the CoreOS host or
>> Docker container
>>
>> Every CoreOS host has a Docker container running SkyDNS. SkyDNS acts as
>> an authoritative DNS server for "conspire.local" (company name is Conspire,
>> use whatever you like) and it then falls back to 8.8.8.8 for other DNS
>> queries. SkyDNS watches specific paths in etcd in order to resolve those
>> queries to our services. For instance, if I place "{'host':10.0.1.5,
>> 'port':2552}" at the path
>> /skydns/local/conspire/us-east-1/dev/backend/supervisor in etcd, a DNS
>> query for supervisor.backend.dev.us-east-1.conspire.local will resolve
>> to 10.0.1.5, where 10.0.1.5 is the EC2 private IP of the CoreOS node
>> hosting the Docker container running an Akka node named "supervisor".
>>
>> The Docker containers running our Akka nodes link to the SkyDNS container
>> (--link skydns:skydns) and use it as their DNS server (SKYDNS_IP=$(docker
>> inspect skydns | grep IPAddress | cut -f4 -d'"'); --dns $SKYDNS_IP).
>> The hostname of the Docker container is the hostname we publish to etcd (-h
>> supervisor.backend.dev.us-east-1.conspire.local).
>>
>> Every container running an Akka node has a sidekick service (scheduled by
>> fleet) which publishes the appropriate network location into etcd for
>> skydns. It's just a bash script in a while loop doing etcdctl set
>> /path/to/hostname "{'host':$COREOS_PRIVATE_IPV4}". We set a TTL on the
>> etcd entry so that if the Akka node dies, the sidekick will die with it and
>> the entry will expire.
>>
>> Seed nodes are listed in another location in etcd and then resolved
>> against their SkyDNS entry. That's handled by a second sidekick. The DNS
>> sidekick runs *before* the Akka node is started, so that Akka can bind to
>> its logical hostname rather than container or host IP, the seed sidekick
>> advertises it as an available seed *after* Akka has started. If the two
>> aren't separated there's a nasty race condition.
>>
>> This lets us easily (modulo etcd difficulties) run Akka in a multi-host
>> Docker environment with service discovery handled by etcd. Node locations
>> are advertised in etcd so no configuration is needed in advance. CoreOS
>> lets us easily (modulo etcd difficulties) add and remove EC2 nodes and
>> fleet decides where things are deployed. We're hoping to expand this
>> deployment strategy beyond our Akka cluster. We have plans to split some
>> services out of our cluster and also deploy Kafka. Ideally this will be our
>> unified deployment strategy for all these services.
>>
>> We also use etcd for all of our application config parameters (DB hosts,
>> etc). I wrote a small Ruby script that plucks those out of etcd and outputs
>> a config.json which we pass in with -Dconfig.file=...
>>
>> The caveat is that I spent all of yesterday battling etcd failures,
>> hoping all this work wasn't for nothing. ;)
>>
>> I'm working on a blog post to detail all of this with code samples, I'll
>> try to post it next week.
>>
>> On Wednesday, October 1, 2014 12:14:13 PM UTC-6, Alessandro Vermeulen
>> wrote:
>>>
>>> Dear all,
>>>
>>> When running our Akka cluster in docker we need to bind netty to 0.0.0.0
>>> to make it accessible to the outside of the container and binding to the
>>> external ip of the docker host is not possible. However, the seed node uri
>>> needs to point to the external ip in order to connect to the seed nodes.
>>> Consequently, the seed node drops the messages from joining nodes because
>>> it thinks they are not meant for him.
>>>
>>> Is there an existing solution to this issue? One of the solutions would
>>> be to use host names and change the ip addresses through the hosts file.
>>> This would be very cumbersome for us to maintain.
>>>
>>> Would it be sensible to include a different variable that is used to
>>> configure the identifier of the node such that it is decoupled from the
>>> hostname used to bind the network interface?
>>>
>>> Cheers,
>>> Alessandro
>>>
>>>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
