Re: NIO and Threading implementation

2015-01-22 Thread Jay Kreps
Hey Chittaranjan, Yeah I think at a high level our goal is that classes are either threadsafe or not and if threadsafe their safety doesn't depend on the details of their current usage(s) since that often changes. In other words the synchronization should be encapsulated in the class. So that's th

Re: NIO and Threading implementation

2015-01-21 Thread Chittaranjan Hota
Thanks for your comments Jay. Quote "Technically startup is not called from multiple threads but the classes correctness should not depended on the current usage so it should work correctly if it were." --> If this were a requirement then one can see that many methods are NOT thread safe while the

Re: NIO and Threading implementation

2015-01-21 Thread Jay Kreps
1. a. I think startup is a public method on KafkaServer so for people embedding Kafka in some way this helps guarantee correctness. b. I think KafkaScheduler tries to be a bit too clever, there is a patch out there that just moves to global synchronization for the whole class which is easier to rea

NIO and Threading implementation

2015-01-21 Thread Chittaranjan Hota
Hello, Congratulations to the folks behind kafka. Its has been a smooth ride dealing with multi TB data when the same set up in JMS fell apart often. Although I have been using kafka for more than a few days now, started looking into the code base since yesterday and already have doubts at the ver