Hey Jay, thanks for reviewing the proposal. Answers inline. On Mon, Oct 12, 2015 at 10:53 AM, Jay Kreps <j...@confluent.io> wrote:
> Hey guys, > > I think this is an important feature and one we've talked about for a > while. I really think trying to invent a new nomenclature is going to make > it hard for people to understand, though. As such I recommend we call > namespaces "directories" and denote them with '/'--this will make the > feature 1000x more understandable to people. Essentially you are suggesting two things here. 1. Use "Directory" instead of "Namespace" as it is more intuitive. I agree. 2. Make '/' as delimiter instead of ':'. Fine with me and I agree if we call these directories, '/' is the way to go. I think we should inheret the > semantics of normal unix fs in so far as it makes sense. > > In this approach we get rid of topics entirely, instead we really just have > partitions which are the equivalent of a file and retain their numeric > names, and the existing topic concept is just the first directory level but > we generalize to allow arbitrarily many more levels of nesting. This allows > categorization of data, such as /datacenter1/user-events/page-views/3 and > you can subscribe, apply configs or permissions at any level of the > hierarchy. > +1. This actually requires just a minor change to existing proposal, i.e., "some:namespace:topic" becomes "some/namespace/topic". > > I'm actually not 100% such what the semantics of accessing data in > differing namespaces is in the current proposal, maybe you can clarify > Ashish? I will add more info to KIP on this, however I think a client should be able to access data in any namespace as long as following conditions are satisfied. 1. Namespace, the client is trying to access, exists. 2. The client has sufficient permissions on the namespace for type of operation the client is trying to perform on a topic within that namespace. 3. The client has sufficient permissions on the topic for type of operation the client is trying to perform on that topic. If we choose to go with what you suggested earlier that just have hierarchy of directories, then step 3 will actually be covered in step 2. In the current proposal, consumers will subscribe to a topic in a namespace by specifying <namespace>:<topic> as the topic name. They can subscribe to topics from multiple namespaces. Let me know if I totally missed your question. Since the point of Kafka is sharing data I think it is really > important that the grouping be just for convenience/permissions/config/etc > and that it remain possible to access multiple directories/namespaces from > the same client. > Totally agree with you. > > -Jay > > On Fri, Oct 9, 2015 at 6:32 PM, Ashish Singh <asi...@cloudera.com> wrote: > > > Hey Guys, > > > > I just created KIP-37 for adding namespaces to Kafka. > > > > KIP-37 > > < > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-37+-+Add+Namespaces+to+Kafka > > > > > tracks the proposal. > > > > The idea is to make Kafka support multi-tenancy via namespaces. > > > > Feedback and comments are welcome. > > > > -- > > > > Regards, > > Ashish > > > -- Regards, Ashish
