Re: [zeromq-dev] Identities & PUB/SUB

2011-04-04 Thread Pieter Hintjens
+1 for this change. On 5 Apr 2011 07:45, "Martin Sustrik" wrote: > On 04/04/2011 11:32 PM, Paul Colomiets wrote: > >>> 2. Publish/subscribe. In this case we assume infinite feed of messages >>> that individual subscribers consume. When shutting down, the subscriber >>> has to cut off the feed at s

Re: [zeromq-dev] LRU broker queue in intuitive way on 3.0

2011-04-04 Thread Martin Sustrik
On 04/05/2011 05:35 AM, Andrew Hume wrote: > i like creative ways to solve problems as much as the next sapient, > but this problem (optimal job scheduling) is much more complicated > than that, and to run well, requires adaptive scheduling stuff in realtime. > you can't really do it with just load

[zeromq-dev] Identities & PUB/SUB

2011-04-04 Thread Martin Sustrik
On 04/04/2011 11:32 PM, Paul Colomiets wrote: >> 2. Publish/subscribe. In this case we assume infinite feed of messages >> that individual subscribers consume. When shutting down, the subscriber >> has to cut off the feed at some point. Whether it cuts off immediately, >> dropping the messages on

Re: [zeromq-dev] LRU broker queue in intuitive way on 3.0

2011-04-04 Thread Andrew Hume
i like creative ways to solve problems as much as the next sapient, but this problem (optimal job scheduling) is much more complicated than that, and to run well, requires adaptive scheduling stuff in realtime. you can't really do it with just load balancing and back pressure. how accurately can y

Re: [zeromq-dev] LRU broker queue in intuitive way on 3.0

2011-04-04 Thread Henry Baragar
On April 4, 2011 06:30:06 pm Pieter Hintjens wrote: > On Tue, Apr 5, 2011 at 12:16 AM, Paul Colomiets wrote: > > There are two patterns in the guide for request/reply broker. One > > with using XREQ on router and REP on worker. And the other is > > based on XREP on router REQ on worker. The proble

Re: [zeromq-dev] LRU broker queue in intuitive way on 3.0

2011-04-04 Thread Ian Barber
On Mon, Apr 4, 2011 at 11:30 PM, Pieter Hintjens wrote: > > The BOSS socket keeps a list of ready workers, which signal presence > to the BOSS socket, when they connect, and when they reply to a > request. They work synchronously, and look just like REP sockets to > applications. They could fan-in

Re: [zeromq-dev] LRU broker queue in intuitive way on 3.0

2011-04-04 Thread Pieter Hintjens
On Tue, Apr 5, 2011 at 12:16 AM, Paul Colomiets wrote: > There are two patterns in the guide for request/reply broker. One > with using XREQ on router and REP on worker. And the other is > based on XREP on router REQ on worker. The problem is that > first is being obvious but basically does not w

[zeromq-dev] LRU broker queue in intuitive way on 3.0

2011-04-04 Thread Paul Colomiets
Hi, Discussing incompatible changes for zeromq 3.0, I have recalled the old issue were uneasy for me to understand when I was new to zeromq. There are two patterns in the guide for request/reply broker. One with using XREQ on router and REP on worker. And the other is based on XREP on router REQ

Re: [zeromq-dev] Important: backward incompatible changes for 0MQ/3.0!

2011-04-04 Thread Pieter Hintjens
2011/4/4 Paul Colomiets : This is a great thread :) >> 3. Pipeline (push/pull). This is the interesting case. The communication >> is uni-directional, meaning that we can't timeout and resend while at >> the same time we want every message to be delivered. JMS defines three levels of acks for th

Re: [zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

2011-04-04 Thread Pedro Borges
On Mon, Apr 4, 2011 at 7:38 PM, Brian Ledger wrote: > You are mistaken on the int64 issue.  The OCaml Int64 API states that > arithmetic is carried out modulo 2^64, just like an unsigned int, which does > not have to use a sign bit from it's allocation.  However, the Int64 still > supports negati

Re: [zeromq-dev] Important: backward incompatible changes for 0MQ/3.0!

2011-04-04 Thread Paul Colomiets
Hi Martin, 04.04.2011, 09:16, "Martin Sustrik" : > Hi Paul, > >>  The documentation is actually a bit misleading. After >>  you call shutdown(s, SHUT_RD) you *can* read, up to the point >>  when shutdown called. It means everything already buffered will >>  be read, and you will read until 0 (zero

Re: [zeromq-dev] [PATCH] pkg-config

2011-04-04 Thread Pieter Hintjens
Applied to 2.1 and 2.2, thanks! On Mon, Apr 4, 2011 at 10:53 PM, Martin Sustrik wrote: > Hi Mikko, > >> after fiddling around a bit I think the attached patch works in making >> the pkg-config dependency optional rather than mandatory. The >> conditional dependency makes things a lot easier when

Re: [zeromq-dev] [PATCH] pkg-config

2011-04-04 Thread Martin Sustrik
Hi Mikko, > after fiddling around a bit I think the attached patch works in making > the pkg-config dependency optional rather than mandatory. The > conditional dependency makes things a lot easier when building on Win > using MinGW(32|64). Applied to master. Thanks! Martin _

[zeromq-dev] [PATCH] pkg-config

2011-04-04 Thread Mikko Koppanen
Hi, after fiddling around a bit I think the attached patch works in making the pkg-config dependency optional rather than mandatory. The conditional dependency makes things a lot easier when building on Win using MinGW(32|64). -- Mikko Koppanen 0001-Make-pkg-config-dependency-conditional.patch

Re: [zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

2011-04-04 Thread Brian Ledger
>I'm almost convinced to fork Caravan add whatever is missing, and make an higher level api on top that would be distributed with Caravan (in the spirit of ZMQ utils). The only conditions I make is that the socket types must be interoperable (without casting functions) between the two apis and the

Re: [zeromq-dev] Zero-copy message API

2011-04-04 Thread Pieter Hintjens
On Mon, Apr 4, 2011 at 8:23 PM, Martin Pales wrote: > TBH, attach sounds like I could call it multiple times to add several > "attachments". It's similar to "add". What about a simple zmq_msg_set_data() > or zmq_msg_set_buffer() ? Indeed. > Or just to have one function(zmq_msg_create or zmq_msg

Re: [zeromq-dev] Zero-copy message API

2011-04-04 Thread Martin Sustrik
On 04/04/2011 08:23 PM, Martin Pales wrote: > zmq_msg_create(zmq_msg_t *msg); // zmq_msg_init(); > zmq_msg_create(zmq_msg_t *msg, size_t size); // zmq_msg_init_size(); > zmq_msg_create(zmq_msg_t *msg, size_t size, void *data, zmq_free_fn > *ffn, void *hint); No polymorphism in C :( Martin _

Re: [zeromq-dev] Zero-copy message API

2011-04-04 Thread Martin Pales
On Mon, Apr 4, 2011 at 6:40 PM, Ian Barber wrote: > > > On Mon, Apr 4, 2011 at 5:29 PM, Martin Sustrik wrote: > >> What about calling it zmq_msg_attach()? >> >> It is kind of explicit about the fact that it attaches the message to >> existing buffer. >> >> Martin >> > > I like attach, it's clear

Re: [zeromq-dev] Zero-copy message API

2011-04-04 Thread Ian Barber
On Mon, Apr 4, 2011 at 5:29 PM, Martin Sustrik wrote: > What about calling it zmq_msg_attach()? > > It is kind of explicit about the fact that it attaches the message to > existing buffer. > > Martin > I like attach, it's clear and obvious. Wrap I think is potentially confusing because of simila

Re: [zeromq-dev] Zero-copy message API

2011-04-04 Thread Pieter Hintjens
On Mon, Apr 4, 2011 at 6:31 PM, Martin Sustrik wrote: >> What about calling it zmq_msg_attach()? > > Or zmq_msg_wrap()? As that it wraps the existing buffer into zmq_msg_t > object? I'm using zmsg_wrap() in libzapi to mean "add an envelope to message" so that would be confusing. -Pieter ___

Re: [zeromq-dev] Zero-copy message API

2011-04-04 Thread Martin Sustrik
On 04/04/2011 06:29 PM, Martin Sustrik wrote: > What about calling it zmq_msg_attach()? > > It is kind of explicit about the fact that it attaches the message to > existing buffer. Or zmq_msg_wrap()? As that it wraps the existing buffer into zmq_msg_t object? Martin

Re: [zeromq-dev] Zero-copy message API

2011-04-04 Thread Martin Sustrik
On 04/04/2011 09:31 AM, Ian Barber wrote: > > For my 2p, most of the people I see doing 0MQ with C/C++ are using > init_data for exactly this, with some simple slab type memory scheme > going on, and seeing notable speed improvements, so I think the practice > is pretty widespread. What about cal

Re: [zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

2011-04-04 Thread Guillaume Yziquel
Le Monday 04 Apr 2011 à 13:24:17 (+0100), Pedro Borges a écrit : > On Mon, Apr 4, 2011 at 12:31 PM, Guillaume Yziquel > wrote: > > Le Monday 04 Apr 2011 à 12:17:01 (+0100), Pedro Borges a écrit : > > If I allocate a buffer outside the heap, the buffer from ocaml (a > string) would have to be memc

Re: [zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

2011-04-04 Thread Guillaume Yziquel
Le Monday 04 Apr 2011 à 13:24:17 (+0100), Pedro Borges a écrit : > On Mon, Apr 4, 2011 at 12:31 PM, Guillaume Yziquel > wrote: > > Le Monday 04 Apr 2011 à 12:17:01 (+0100), Pedro Borges a écrit : > > > You also the cute caml_register_generational_global_root function if you > > want buffers > >

Re: [zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

2011-04-04 Thread Guillaume Yziquel
Le Monday 04 Apr 2011 à 13:24:17 (+0100), Pedro Borges a écrit : > On Mon, Apr 4, 2011 at 12:31 PM, Guillaume Yziquel > wrote: > > Le Monday 04 Apr 2011 à 12:17:01 (+0100), Pedro Borges a écrit : > > >> Unfortunately implementing zero-copy in ocaml is impossible, AFAIK, > >> without blocking ever

Re: [zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

2011-04-04 Thread Pedro Borges
On Mon, Apr 4, 2011 at 12:31 PM, Guillaume Yziquel wrote: > Le Monday 04 Apr 2011 à 12:17:01 (+0100), Pedro Borges a écrit : >> Unfortunately implementing zero-copy in ocaml is impossible, AFAIK, >> without blocking every other thread. If another thread makes an >> allocation the GC might move th

Re: [zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

2011-04-04 Thread Guillaume Yziquel
Le Monday 04 Apr 2011 à 12:17:01 (+0100), Pedro Borges a écrit : > On Mon, Apr 4, 2011 at 5:51 AM, Pieter Hintjens wrote: > > On Mon, Apr 4, 2011 at 4:24 AM, Brian Ledger wrote: > > > > * It's profitable to map libzmq's API closely (including to keep > > zero-copy functionality) but it's also p

[zeromq-dev] [PATCH] Another error handling issue on Win32 solved

2011-04-04 Thread Martin Sustrik
>From 8203c4dbb2e727ce608590e41c9e0d990c015b3e Mon Sep 17 00:00:00 2001 From: Martin Sustrik Date: Mon, 4 Apr 2011 13:12:46 +0200 Subject: [PATCH] Another error handling issue on Win32 solved Signed-off-by: Martin Sustrik --- perf/inproc_lat.cpp |2 +- perf/inproc_thr.cpp |2 +- 2 file

Re: [zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

2011-04-04 Thread Martin Sustrik
On 04/04/2011 01:17 PM, Pedro Borges wrote: > Unfortunately implementing zero-copy in ocaml is impossible, AFAIK, > without blocking every other thread. If another thread makes an > allocation the GC might move the string you are trying to send. That's true of almost all the bindings. One spectac

Re: [zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

2011-04-04 Thread Pedro Borges
On Mon, Apr 4, 2011 at 5:51 AM, Pieter Hintjens wrote: > On Mon, Apr 4, 2011 at 4:24 AM, Brian Ledger wrote: > * It's profitable to map libzmq's API closely (including to keep > zero-copy functionality) but it's also profitable to build > abstractions on top. Unfortunately implementing zero-co

[zeromq-dev] [PATCH] Issue with error checking on Win32 platform fixed

2011-04-04 Thread Martin Sustrik
>From 475ba3849dd6563ce7b70c4db4d0ac85ac83658d Mon Sep 17 00:00:00 2001 From: Martin Sustrik Date: Mon, 4 Apr 2011 12:49:39 +0200 Subject: [PATCH] Issue with error checking on Win32 platform fixed Signed-off-by: Martin Sustrik --- src/thread.cpp |2 +- 1 files changed, 1 insertions(+), 1 d

Re: [zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

2011-04-04 Thread Guillaume Yziquel
Le Monday 04 Apr 2011 à 11:11:33 (+0100), Pedro Borges a écrit : > On Mon, Apr 4, 2011 at 3:24 AM, Brian Ledger wrote: > > >> Well using Uint keeps the user from passing negative arguments > >> plus respects the type of the original options so I thought it > >> was necessary. Why do you think it

[zeromq-dev] [PATCH] Unreferenced variable removed

2011-04-04 Thread Martin Sustrik
>From 1842628b9058b9243ebe037a6818c27ea1c8cd8d Mon Sep 17 00:00:00 2001 From: Martin Sustrik Date: Mon, 4 Apr 2011 12:12:06 +0200 Subject: [PATCH] Unreferenced variable removed Signed-off-by: Martin Sustrik --- src/tcp_socket.cpp |1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff

Re: [zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

2011-04-04 Thread Pedro Borges
On Mon, Apr 4, 2011 at 3:24 AM, Brian Ledger wrote: >> Well using Uint keeps the user from passing negative arguments >> plus respects the type of the original options so I thought it >> was necessary. Why do you think it is unnecessary? > Ocaml's implementation of the Int64 type explicitly alloc

Re: [zeromq-dev] [PATCH] multiple properties exchanged on connection initiation

2011-04-04 Thread Martin Sustrik
On 04/04/2011 09:23 AM, Ian Barber wrote: > Good idea! Is the plan for all properties to be in sent in one message > when starting the connection, or over several? Currently it's one multipart message. One property per message part. > Just wondering if it's > worth having something saying number

Re: [zeromq-dev] Zero-copy message API

2011-04-04 Thread Ian Barber
On Sun, Apr 3, 2011 at 10:31 AM, Martin Sustrik wrote: > > As for renaming zmq_msg_data to zmq_msg_zerocopy, I would say that the > actual semantics of the command is not "zero copy" but "user defined > allocator". This has to do with the custom allocation on recv() side (as > discussed recently

Re: [zeromq-dev] [PATCH] multiple properties exchanged on connection initiation

2011-04-04 Thread Ian Barber
On Sun, Apr 3, 2011 at 8:44 AM, Martin Sustrik wrote: > Hi all, > > To follow the process more closely from now on I am going to post my > patches to the mailing list for review same way as everyone else does. Feel > free to comment! > > This patch to master is initial work on multiple-property c