Well, the hostname reported by EC2 machines isn't even always resolvable by other machines in the same region, so hostnames have to be taken with a pretty big grain of salt.
It should be noted that the public names resolve to internal address inside EC2. That means that traffic runs as cheaply as possible. Regarding the identification of a host, the idea that there is a DNS name on the far side of the connections that matches anything on the local side is just incorrect. A single machine has to be considered to have multiple interfaces, multiple versions of host name and multiple names that resolve to addresses that reach it from other machines. That these don't match up very well is only to be expected. What is needed is a sense of self on the part of a host and it has to advertise what it thinks of itself when sending and receiving RPC's. If that is done, then it doesn't take too long to build a relatively comprehensive dictionary of "when you say this, you connect to that" on each node. It is possible to build systems that assume a single network interface, that IP addresses are the same on both ends of the connection and that the same names resolve everywhere the right way. But if you do that, you have a relatively delicate system subject to violation of these assumptions. On Wed, Aug 17, 2011 at 8:15 PM, Matei Zaharia <[email protected]>wrote: > The problem with using the public hostname everywhere is that it's not the > hostname the machine reports for itself, so applications running on Mesos > that care about data placement, such as Spark and Hadoop, will not know > which data is on the node. (That is, HDFS will see the node as > ec2-foo.internal, and Mesos will say it's ec2-bar.amazonaws.com. If you > wanted to use a single hostname, you'd have to use the internal one and lose > the usability for EC2. > > However, I don't see what would be wrong with just having a webui_url > property on each slave. That could then use the public hostname and > incorporate the port as well. > > Matei > > On Aug 17, 2011, at 7:58 PM, Benjamin Hindman wrote: > > >> No, up until now I haven't specified a webui_port ... > > > > Yeah, that's what I expected (given the TODO in the code). The first work > around will be using webui_port=8081 for your slaves. However, this doesn't > explain why the slave webui isn't loading. But, if you can drop the result > of doing: > > > > $ curl http://slave_hostname:port/slave/state.json > > > > Then I can try and debug why that JSON doesn't appear to be getting > generated correctly. > > > >> I believe that talking to public addresses instead of private ones means > you > >> pay for the traffic ... > > > > > > This sounds like it's just across availability zones, and that you would > pay for the traffic across availability zones regardless of if you used a > private or a public address. > > > > > > Thanks Michael! > > > > > > > >> > >> Michael > > > >
