cool, thanks
> Date: Thu, 21 Feb 2008 18:40:05 -0800> From: [EMAIL PROTECTED]> To: [EMAIL
> PROTECTED]> CC: [email protected]> Subject: Re: Alternative Binary
> Protocol idea for memcached.> > This list is the best.> > Then there's
> #memcached on irc.freenode.net> > Then there's the wiki linked from
> danga.com/memcached/> > Development and user assistance both happen on this
> list. It's> relatively low volume. I don't know of any web forums...> > joe
> schmo wrote:> > > > Does anyone know of a good site to post questions on
> regarding problems> > with memcahced?> > > > > > > > > >
> ------------------------------------------------------------------------> >
> Date: Thu, 21 Feb 2008 02:42:10 +0000> > From: [EMAIL PROTECTED]> > To:
> [email protected]> > Subject: Re: Alternative Binary Protocol idea
> for memcached.> > > > Alright, I've started looking at the code. > > > >
> Would anyone object if I first started working on abstracting the> > protocol
> handling routines so that they are handled by callback> > instead of the
> current if/else setup?> > > > Then I was going to put all the protocol stuff
> in prot_ascii.h|c and> > prot_binary.h|c respectively.> > > > This way we
> could potentially have hundreds of protocols handled> > without having a
> lengthy if/else comparison.> > > > On Wed, Feb 20, 2008 at 10:06 AM, Clint
> Webb <[EMAIL PROTECTED]> > <mailto:[EMAIL PROTECTED]>> wrote:> > > > I wasn't
> expecting such a quick reply. Good point about> > allowing multiple
> protocols. I might pull out some of my old> > code and see how easy it is to
> drop in.> > > > I thought I'd give a little background on myself and this> >
> protocol style. I used to work in the controls and automation> > industry. If
> you've ever checked your luggage into an airport,> > or sent anything thru
> USPS, UPS or FedEX, or bought anything> > online from amazon, b&n and other
> large online stores, or from> > Walmart, kmart, target, etc... then your
> product has likely had> > some experience with my coding somewhere along the
> line.> > > > I developed this protocol for a tiny little side-project for
> one> > of the above mentioned companies. It was basically a connector> > that
> took information from a large number of different systems> > and passed it to
> another. The requirements changed a lot, so I> > developed something that
> could be fast, but had to be flexible. > > If I added a feature to the
> server, I didn't want to be forced> > to update all the clients as well.
> Also, some of the clients> > were tiny little embedded controllers, so it had
> to be pretty> > simple too.> > > > This solution was VERY fast, as all the
> commands are a single> > byte and could be easily mapped to an array of
> callback> > routines. This protocol also had to run on a real-time system> >
> also, so we had to ensure that all operations preformed in a> > predictable
> fashion.> > > > I seperated the commands by their parameter type. 0-31 had
> no> > parameter. 32-63 had a single byte parameter. 64-95 had a> > 32-bit
> parameter. 96-127 had a single-byte paramter which was> > the length of a
> stream of data to follow (short-string). > > 128-159 had a 32-bit integer
> that was the length of a stream of> > data to follow. This was our 5
> different parameter types. A> > command could only have one parameter.> > > >
> This way, the IO handling code could retrieve all the necessary> > data and
> then pass it off to a callback routine for that command.> > > > Each socket
> connection would have a set of buffers set aside for> > the incoming data. In
> this case we would want a buffer set> > aside to hold the key and value
> data.> > > > To speed up processing and ensure that the minimum data set has>
> > been provided, we used a 32-bit (or was it 64-bit?) word as> > flags. Each
> operation would set or clear a flag(s). So when> > a GO command is received,
> it can quickly determine what 'action'> > needs to take place, and which
> 'parameters' have been provided.> > > > If we ran out of room having to
> handle more than 256 commands,> > we would use a command of 0xFF which would
> expect that the next> > byte would be another command (from a set different
> to the> > first). I never actually implemented it though. The most> >
> commands I ever used was about 100 or so.> > > > I cant imagine that a
> variable-length structured protocol could> > be much faster than that. Still
> the emphasis of this protocol> > is not so much on speed, but on flexibility
> to add functionality> > to the protocol (by adding commands) without breaking
> existing> > clients (and without having to handle multiple versions of the> >
> protocol).> > > > The 'noreply' stuff that I have seen around the list could>
> > probably benefit from this protocol. I haven't looked close> > enough at
> the CAS stuff either, but I suspect that would be easy> > to implement too.>
> > > > Also, those that want to shave off a few extra bytes in their> >
> client, have the option of sending a request that only includes> > the bits
> they want. If you care about expiry leave it out, same> > with flags, tags,
> cas id's, and anything else. Plus you can> > stream-line some of your
> requests by not using the CLEAR> > command, and re-using the state.> > > >
> Dang, if I had a little more time on my hands right now, I'd be> > really
> tempted to implement it. I don't actually have a *need*> > for this protocol
> in memcached, it was purely an intellectual> > itch ever since I saw people
> complaining about the existing> > protocols being difficult to expand.> > > >
> > > > > On Feb 20, 2008 4:14 PM, Dustin Sallings <[EMAIL PROTECTED]> >
> <mailto:[EMAIL PROTECTED]>> wrote:> > > > > > On Feb 19, 2008, at 22:20,
> Clint Webb wrote:> > > > > I know a considerable amount of thought and work
> has gone> > into the> > > existing flavour of the binary protocol, and I dont
> expect> > that work> > > to be discarded, I'm really only mentioning this
> new> > concept now as> > > an alternative for the future if we ever find the
> current> > binary> > > protocol to be too restrictive and inflexible. And> >
> something to> > > think about, or even use elsewhere.> > > > The is certainly
> interesting. The first step of> > doing the binary> > protocol implementation
> was to create a change that allowed> > multiple> > protocols to coexist. It
> would be possible to implement> > this to run> > in parallel with the
> existing protocols in the 1.3 codebase.> > > > Intuitively, it doesn't seem
> as efficient to process> > as what we've> > got now, but I like being proven
> wrong, so I'd welcome> > another front-> > end protocol. :)> > > > Of course,
> I wrote the majority of the current binary> > protocol code> > about six
> months ago, so I'd really like to at least have> > one in more> > people's
> hands.> > > > --> > Dustin Sallings> > > > > > > > > > -- > > "Be excellent
> to each other" > > > > > > > > > > -- > > "Be excellent to each other" > > >
> > > >
> ------------------------------------------------------------------------>
_________________________________________________________________
