Given that we need super control over the serialization process in some conditions and do not any in most other situations I am not sure what is the best solution here. Maybe a hybrid and that would be crazy. Fundamentally I think this is a triviality and this mechanism is really brain dead simple. Your point about tests is very very well taken. In our defense if we had religiously written tests I do not think we would have been where we are w.r.t to where the system is now :). We have a slew of tests that we do have but they are super well written but rather ad-hoc and I do not mean the main() that you see in most other places. Avinash
On Sat, Mar 28, 2009 at 10:39 AM, Avinash Lakshman < [email protected]> wrote: > For eg Thrift will definitely not help in the messages that we use for the > membership protocol. Because there we need to control how big the serialized > messages are - we make sure we serialize a part of the object such that it > fits into an ethernet MTU packet. We do this so that we don't get bitten by > UDP fragmentation. I don't think you could do operations like that in Thrift > based serialization mechanism. We need more control over the serialization > mechanism. > > So I don't know if this is something that is insanely important in any > capacity in my opinion. I am sure there are bunch of other reasons I can > come up with - we went through this exercise 2 year back. Of course if you > want to investigate the efficacy I can't stop you from doing so :). > > Avinash > > > On Sat, Mar 28, 2009 at 9:48 AM, Jonathan Ellis <[email protected]> wrote: > >> One point of clarification -- I don't understand why looking up by >> string is better than using an enum, for instance. java will autobox >> enums for use in a hashmap lookup. >> >> On Sat, Mar 28, 2009 at 10:34 AM, Avinash Lakshman >> <[email protected]> wrote: >> > Why is it ad-hoc? And it uses a factory pattern and I don't think it >> hard >> > once you get a hang of it. Consumers of the system are not even going to >> > know about these details. Personally I am never a fan of fixing anything >> > that is not broken - in this case it has been working really well for >> us. >> > This is now just a matter of what one might prefer. Thrift is something >> that >> > I would not like to see anywhere apart from the entry point. With >> regards to >> > the using the string to lookup the handlers it was done because if you >> don't >> > do that then you will have to resort to RPC style instead of message >> passing >> > or find the handlers based on the kind of messages i.e if-else >> branching. We >> > use non-blocking I/O for all our internal messaging and Thrift using >> > blocking I/O. There is big difference in throughput and also Thrift >> > non-blocking I/O from what I hear is horrendous in performance and >> > stability. My friend you don't have my vote for this :). >> > Avinash >> > >> > On Sat, Mar 28, 2009 at 8:11 AM, Jonathan Ellis <[email protected]> >> wrote: >> > >> >> we have a Message class that mostly represents a bunch of bytes (but >> >> sometimes does not, which in some cases causes bugs) and a bunch of >> >> other *Message classes that are not Message subclasses but generate >> >> Message objects (so you have the amusingly redundant Message message = >> >> readResponseMessage.makeReadResponseMessage() in places). >> >> >> >> I think we can replace these ad-hoc and tedious-to-write Message >> >> factories with generated thrift code. Thrift is good at this and >> >> efficient (currently our message identifiers are inefficient strings >> >> like "ROW-READ-VERB-HANDLER"). >> >> >> >> Any objections to investigating replacing the hand-coded messages with >> >> thrift? >> >> >> > >> > >
