Re: blog with some out of the box pains

2014-10-18 Thread Joe Stein
1) Kafka Gradle thing would be great to figure out. Samza, Aurora (maybe a few other Apache projects) would benefit too. We need a better way to bootstrap the gradle-wrapper.jar.I created https://issues.apache.org/jira/browse/KAFKA-1714 to track that. 2) I have seen, more than a few times, a

blog with some out of the box pains

2014-10-17 Thread Jay Kreps
This guy documented a few struggles getting going with Kafka. Not sure if there is anything we can do to make it better? http://ispyker.blogspot.com/2014/10/kafka-part-1.html 1. Would be great to figure out the apache/gradle thing. 2. The problem of having Kafka advertise localhost on AWS is

Re: blog with some out of the box pains

2014-10-17 Thread Gwen Shapira
In #2, do you refer to advertising the internal hostname instead of the external one? In this case, will it be enough to use getCanonicalHostName (which uses a name service)? Note that I think the problem the blog reported (wrong name advertised) is somewhat orthogonal to the question of which

Re: blog with some out of the box pains

2014-10-17 Thread Jay Kreps
Hmm, yes, actually I don't think I actually understand the issue. Basically as I understand it we do InetAddress.getLocalHost.getHostAddress which on AWS picks the wrong hostname/ip and then the producer can't connect. People eventually find this FAQ, but I was hoping there was a more automatic

Re: blog with some out of the box pains

2014-10-17 Thread Gwen Shapira
Basically, the issue (or at least one of very many possible network issues...) is that the server has localhost hardcoded as its canonical name in /etc/hosts: [root@Billc-cent70x64 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 Billc-cent70x64

Re: blog with some out of the box pains

2014-10-17 Thread Ewen Cheslack-Postava
The first issue he runs into is one I also find frustrating -- with cloud providers pushing SSDs, you have to use a pretty large instance type to get a reasonable test setup. I'm not sure if he couldn't launch an older type like m1.large (I think some newer AWS accounts aren't able to) or if he