I don't think it necessarily has to be. I was thinking about that while I was typing it out, and I realized that as well. With a special broker port, my biggest concern is making sure that nothing other than a broker uses it (and so cannot bypass security controls like authentication and authorization). Outside of that assurance, I don't see a reason to add overhead like TLS to the broker communication. Someone else might have one, though.
-Todd On Tue, Dec 2, 2014 at 4:12 PM, Jun Rao <j...@confluent.io> wrote: > Todd, > > Does that imply the intra-broker protocol is always plaintext? > > Thanks, > > Jun > > On Tue, Dec 2, 2014 at 3:31 PM, Todd Palino <tpal...@gmail.com> wrote: > > > Thanks. Just to add more detail as to why I think it's a good idea to be > > able to segregate traffic like that... > > > > One reason would just be to separate out the intra-cluster communication > to > > a separate port. This can allow you to run it over a different interface > > (for example, you could have dedicated links for the brokers to do > > replication), though you could do that with a wildcard bind and multiple > > interfaces with a little care on the broker config. It also allows for > > firewalling off clients from a cluster while leaving the broker > > communication intact. We've run into this situation a couple times where > it > > was advantageous to do this during recovery to prevent things like > runaway > > file descriptor allocation. It also gives the ability to use QOS tools to > > work with networking guarantees for broker traffic. > > > > Maybe it's enough to be able to specify a special protocol name to do > this. > > Meaning you could configure a port with protocol "broker" (or something > > like that) which could be used by the brokers if it exists. Otherwise > they > > would default back to something else. > > > > -Todd > > > > -TOdd > > > > On Tue, Dec 2, 2014 at 3:23 PM, Gwen Shapira <gshap...@cloudera.com> > > wrote: > > > > > The good news is that I'm not using a map to represent protocol list. > > > > > > The bad news is that as mentioned in the wiki: producers, consumers > > > and broker configuration specify "security protocol", so we'll know > > > which host/port pair to use for communication. This implicitly assumes > > > there will be only one host/port per protocol. > > > > > > I'll think a bit on how this assumption can be relaxed. > > > > > > Gwen > > > > > > On Tue, Dec 2, 2014 at 3:14 PM, Todd Palino <tpal...@gmail.com> wrote: > > > > Gwen - just my reading of what we could expect from what you had > > > described > > > > so far. Without having gone into implementation details, there didn't > > > seem > > > > to be anything that would block the ability to run two ports with the > > > same > > > > protocol configuration, at least from the way you proposed to > represent > > > it > > > > in Zookeeper. I'd just like to not go down the path of using > something > > > like > > > > a map for representing the protocol list that would eliminate this > > > > possibility, unless there's a pretty good reason to do so. > > > > > > > > -Todd > > > > > > > > On Tue, Dec 2, 2014 at 3:00 PM, Gwen Shapira <gshap...@cloudera.com> > > > wrote: > > > > > > > >> Hey Todd, > > > >> > > > >> You say "lose the ability" - you mean this ability actually exist > now? > > > >> Or is this something you hope the new patch will support? > > > >> > > > >> Gwen > > > >> > > > >> On Tue, Dec 2, 2014 at 2:08 PM, Todd Palino <tpal...@gmail.com> > > wrote: > > > >> > Leaving aside the rest of this, on #1, while I consider being able > > to > > > >> > advertise the ports a good idea, I don't want to lose the ability > > for > > > >> > maintaining multiple ports with the same protocol. For example, > > being > > > >> able > > > >> > to have 2 plaintext ports, one that only brokers communicate over, > > and > > > >> one > > > >> > that general clients use. The ability to segregate this traffic is > > > useful > > > >> > in a number of situations, over and above other controls like > > quotas, > > > and > > > >> > is relatively easy to do once we support multiple ports. > > > >> > > > > >> > -Todd > > > >> > > > > >> > > > > >> > On Tue, Dec 2, 2014 at 1:58 PM, Jun Rao <jun...@gmail.com> wrote: > > > >> > > > > >> >> Hi, Gwen, > > > >> >> > > > >> >> Thanks for writing up the wiki. Some comments below. > > > >> >> > > > >> >> 1. To make it more general, should we support a binding and an > > > >> advertised > > > >> >> host for each protocol (e.g. plaintext, ssl, etc)? We will also > > need > > > to > > > >> >> figure out how to specify the wildcard binding host. > > > >> >> > > > >> >> 2. Broker format change in ZK > > > >> >> The broker registration in ZK needs to store the host/port for > all > > > >> >> protocols. We will need to bump up the version of the broker > > > >> registration > > > >> >> data. Since this is an intra-cluster protocol change, we need an > > > extra > > > >> >> config for rolling upgrades. So, in the first step, each broker > is > > > >> upgraded > > > >> >> and is ready to parse brokers registered in the new format, but > not > > > >> >> registering using the new format yet. In the second step, when > that > > > new > > > >> >> config is enabled, the broker will register using the new format. > > > >> >> > > > >> >> 3. Wire protocol changes. Currently, the broker info is used in > the > > > >> >> following requests/responses: TopicMetadataResponse , > > > >> >> ConsumerMetadataResponse, LeaderAndIsrRequest and > > > >> UpdateMetadataRequest. > > > >> >> 3.1 TopicMetadataResponse and ConsumerMetadataResponse: > > > >> >> These two are used between the clients and the broker. I am not > > sure > > > >> that > > > >> >> we need to make a wire protocol change for them. Currently, the > > > protocol > > > >> >> includes a single host/port pair in those responses. Based on the > > > type > > > >> of > > > >> >> the port on which the request is sent, it seems that we can just > > pick > > > >> the > > > >> >> corresponding host and port to include in the response. > > > >> >> 3.2 UpdateMetadataRequest: > > > >> >> This is used between the controller and the broker. Since each > > broker > > > >> needs > > > >> >> to cache the host/port of all protocols, we need to make a wire > > > protocol > > > >> >> change. We also need to change the broker format in MetadataCache > > > >> >> accordingly. This is also an intra-cluster protocol change. So > the > > > >> upgrade > > > >> >> path will need to follow that in item 2. > > > >> >> 3.3 LeaderAndIsrRequest: > > > >> >> This is also used between the controller and the broker. The > > > receiving > > > >> >> broker uses the host/port of the leader replica to send the fetch > > > >> request. > > > >> >> I am not sure if we need a wire protocol change in this case. I > was > > > >> >> imagining that we will just add a new broker config, sth like > > > >> >> replication.socket.protocol. Base on this config, the controller > > will > > > >> pick > > > >> >> the right host/port to include in the request. > > > >> >> > > > >> >> 4. Should we plan to support security just on the new java > clients? > > > >> >> Supporting security in both the old and the new clients adds more > > > work > > > >> and > > > >> >> gives people less incentive to migrate off the old clients. > > > >> >> > > > >> >> Thanks, > > > >> >> > > > >> >> Jun > > > >> >> > > > >> >> > > > >> >> On Tue, Nov 25, 2014 at 11:13 AM, Gwen Shapira < > > > gshap...@cloudera.com> > > > >> >> wrote: > > > >> >> > > > >> >> > Hi Everyone, > > > >> >> > > > > >> >> > One of the pre-requisites we have for supporting multiple > > security > > > >> >> > protocols (SSL, Kerberos) is to support them on separate ports. > > > >> >> > > > > >> >> > This is done in KAFKA-1684 (The SSL Patch), but that patch > > > addresses > > > >> >> > several different issues - Multiple ports, enriching the > > channels, > > > SSL > > > >> >> > implementation - which makes it more challenging to review and > to > > > >> test. > > > >> >> > > > > >> >> > I'd like to split this into 3 separate patches: multi-port > > brokers, > > > >> >> > enriching SocketChannel, and the actual security > > implementations. > > > >> >> > > > > >> >> > Since even just adding support for multiple listeners per > broker > > is > > > >> >> > somewhat involved and touches multiple components, I wrote a > > short > > > >> design > > > >> >> > document that covers the necessary changes and the upgrade > > process: > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > >> > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Multiple+Listeners+for+Kafka+Brokers > > > >> >> > > > > >> >> > Comments are more than welcome :) > > > >> >> > > > > >> >> > If this is acceptable, hope to have a patch ready in few days. > > > >> >> > > > > >> >> > Gwen Shapira > > > >> >> > > > > >> >> > > > >> > > > > > >
