[zeromq-dev] Getting started with ruby bindings

2010-04-14 Thread Brian Candler
Hello, I'm just starting with zeromq and would like to give some feedback on the experience from a newcomer's point of view. * Compiling the C/C++ code was completely hassle-free - excellent! (zeromq-2.0.6 tarball, under Ubuntu Hardy i386) * I installed the gem version of the ruby bindings, whi

[zeromq-dev] Implementing persistent queues

2010-04-14 Thread Brian Candler
I am interested in local persistent queueing of messages in the case where the remote server is unreachable. The application I have in mind is a RADIUS server which forwards RADIUS accounting packets to a central data warehouse for processing. However the data warehouse and/or the intervening net

Re: [zeromq-dev] Getting started with ruby bindings

2010-04-14 Thread Brian Candler
On Wed, Apr 14, 2010 at 05:48:57PM +0200, Wolfgang Barth wrote: > And a line ending with ';' like > > s = ctx.socket(ZMQ::REP); > > is not common, but valid ruby code in 1.8.7 and 1.9.1, so you don't need to > patch such things. Sure - it's just if I had to change the line anyway, I couldn't

Re: [zeromq-dev] Implementing persistent queues

2010-04-15 Thread Brian Candler
On Wed, Apr 14, 2010 at 05:21:22PM +0200, Pieter Hintjens wrote: > Right now our simplest conceptual design is for point-to-point > reliability, with store/forward handled in the sender as a separate > thread or process. You'll see the sketch on that blog page. Sounds sensible to me. > Our goal

Re: [zeromq-dev] Implementing persistent queues

2010-04-17 Thread Brian Candler
On Thu, Apr 15, 2010 at 06:19:00PM +0200, Pieter Hintjens wrote: > > Sorry, that was more of a braindump than I was intending :-) > > Any chance you can post this to that page? Done; I've also given a brief review of RADIUS and MQTT reliable delivery mechanisms (although perhaps they belong in a

Re: [zeromq-dev] RPC design pattern

2010-04-18 Thread Brian Candler
On Sun, Apr 18, 2010 at 02:28:04AM -0500, Joe Holloway wrote: > I guess this is an age old problem of fault tolerant service > discovery. The service registry can't know whether or not a service > is reachable until a client has tried to connect. I am a newbie (so take what I say with a pinch of

Re: [zeromq-dev] Implementing persistent queues

2010-04-18 Thread Brian Candler
On Sun, Apr 18, 2010 at 12:14:00AM +0200, Pieter Hintjens wrote: > > Done; I've also given a brief review of RADIUS and MQTT reliable delivery > > mechanisms (although perhaps they belong in a separate wiki page) > > Perhaps. Well, this makes everything easier to find, at least. Fair enough. How

Re: [zeromq-dev] RPC design pattern

2010-04-19 Thread Brian Candler
On Sun, Apr 18, 2010 at 12:54:12PM +0200, Martin Sustrik wrote: > >The API docs don't make it clear what happens if you call zmq_connect > >multiple times on the same socket. > > Yes, you pretty much right. > > You can connect your client to multiple instances of a service: > > zmq_connect (c, "

Re: [zeromq-dev] RPC design pattern

2010-04-19 Thread Brian Candler
On Mon, Apr 19, 2010 at 11:01:24AM +0200, Martin Sustrik wrote: > Yes. This can be achieved in two ways: > > 1. On top of 0MQ. Each client simply chooses a random server to connect to. But I didn't think the 0MQ API provided a way for the application to detect whether its chosen endpoint is reach

Re: [zeromq-dev] equivalent of accept

2010-04-20 Thread Brian Candler
On Tue, Apr 20, 2010 at 06:45:16PM +0200, Martin Sustrik wrote: > 0MQ does this for you. Actual networking stuff is done in I/O > thread(s). Your application thread is free to do the actual > processing. Could you also clarify for my benefit: if you have (say) five worker threads which want to han

Re: [zeromq-dev] Getting started with ruby bindings

2010-04-20 Thread Brian Candler
On Wed, Apr 14, 2010 at 03:20:01PM +0100, Brian Candler wrote: > What about in Ruby, given that Ruby (1.8) has > only green threads, i.e. a single POSIX thread? To answer my own question: rbzmq does _not_ play nicely with ruby 1.8 threads. As soon as you call s.recv, it locks up the

Re: [zeromq-dev] equivalent of accept

2010-04-21 Thread Brian Candler
On Wed, Apr 21, 2010 at 08:25:51AM +0800, Steven McCoy wrote: > Could you also clarify for my benefit: if you have (say) five worker > threads > which want to handle incoming REQs from clients, should the five > threads all > call zmq_recv on the same socket, or should the

Re: [zeromq-dev] Getting started with ruby bindings

2010-04-21 Thread Brian Candler
On Tue, Apr 20, 2010 at 09:57:42PM +0200, Pieter Hintjens wrote: > Perhaps a general solution is to move the 0MQ work to a separate > process and talk to that. It can run at full speed and queue stuff up > for the Ruby app to take when it needs it. That is logical. However, then you need to desig

[zeromq-dev] Connection setup

2010-04-21 Thread Brian Candler
Just a few more newcomer comments after having a bit of a play with 0MQ. I like the idea of having different messaging models bound to different connections. It avoids having to tag each individual message as "this is a PUB with a topic", or "this is a REQ which expects a REP", thus saving those

Re: [zeromq-dev] Getting started with ruby bindings

2010-04-21 Thread Brian Candler
On Wed, Apr 21, 2010 at 01:28:06PM +0200, Martin Sustrik wrote: > The main problem here (and that's a reoccurring issue) is the 0MQ > sockets cannot be polled on because they aren't real file > descriptors. > > I can imagine a small wrapper library on top of 0MQ that would > intercept the system B

Re: [zeromq-dev] Connection setup

2010-04-21 Thread Brian Candler
On Wed, Apr 21, 2010 at 02:30:03PM +0200, Martin Sustrik wrote: > >There is an existing initial handshake, which at the moment does nothing > >more than exchange peer identities (*). Maybe it should also say "I'm a > >REQ" or "I'm a P2P"? Then the connection could be dropped if it was an > >inapp

Re: [zeromq-dev] equivalent of accept

2010-04-26 Thread Brian Candler
> > On Tue, Apr 20, 2010 at 06:45:16PM +0200, Martin Sustrik wrote: > >> 0MQ does this for you. Actual networking stuff is done in I/O > >> thread(s). Your application thread is free to do the actual > >> processing. > > > > Could you also clarify for my benefit: if you have (say) five worker thre

Re: [zeromq-dev] Multicore Magic

2010-04-27 Thread Brian Candler
On Mon, Apr 26, 2010 at 11:00:39PM +0200, Pieter Hintjens wrote: > On Mon, Apr 26, 2010 at 10:57 PM, Martin Sustrik wrote: > > >> Are signals as fast as, or slower than inproc? > > > > They are slower. > > Any other mechanism for inter-process communication? > > Seems so unfair that processes a

Re: [zeromq-dev] Multicore Magic

2010-04-28 Thread Brian Candler
On Tue, Apr 27, 2010 at 10:39:52AM -0700, Brian Granger wrote: > BUT, the other part is scalability and this is where I am still > hopeful. Maybe we will be stuck with horrible latency in Python > because of this issue. But if that latency scales well (is constant) > with the total number of proc