Re: [zeromq-dev] Adding workers dynamically ...

[Andrew Hume]

yes.
this is one of the great cool things about zeromq.

On Oct 10, 2011, at 11:25 PM, James Ladd wrote:

> Is it possible to add workers at runtime without re-starting clients or 
> brokers?
> ie: a box fails and we bring another online with 'n' workers on it.
> 
> 
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev


--
Andrew Hume  (best -> Telework) +1 623-551-2845
and...@research.att.com  (Work) +1 973-236-2014
AT&T Labs - Research; member of USENIX and LOPSA




___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Adding workers dynamically ...

[James Ladd]

Is it possible to add workers at runtime without re-starting clients or
brokers?
ie: a box fails and we bring another online with 'n' workers on it.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] OpenPGM 5.1.118 - maintenance release

[Mikko Koppanen]

On Mon, Oct 10, 2011 at 8:16 PM, Steven McCoy  wrote:
> A repackaging with Debian 7 based Autotools instead of OSX 10.6.8, this
> might fix some platform specific issues that have arisen.
> http://code.google.com/p/openpgm/downloads/detail?name=libpgm-5.1.118-1~dfsg.tar.gz
> Normally the "configure" scripts are built by running "./bootstrap.sh" on a
> clean SVN checkout.

This package seems to fix silent rules and building on solaris 10
using solaris studio 12.2.

-- Mikko
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] epgm example

[Roberto Palmer]

Thankyou Steve for your explanation.
BTW using the interface name (em2 on my system)
still doesn't work and the program ends with a
slightly different error:

Segmentation fault (core dumped)

So my only choice is to use the ip address/network address method.

Rgds,
Roberto

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] pyzmq on uwsgi process question

[Pieter Hintjens]

On Mon, Oct 10, 2011 at 9:46 AM, 郭梁  wrote:

> Independent testing everything looks normal, but on the uwsgi process, i got
> the error code: http 502

HTTP error 502 (bad gateway) means something is wrong in your web
server configuration, presumably before the request ever hits your
application. I'd recommend testing with a hello world app just to make
sure your server is properly configured.

-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] OpenPGM 5.1.118 - maintenance release

[Steven McCoy]

A repackaging with Debian 7 based Autotools instead of OSX 10.6.8, this
might fix some platform specific issues that have arisen.

http://code.google.com/p/openpgm/downloads/detail?name=libpgm-5.1.118-1~dfsg.tar.gz

Normally the "configure" scripts are built by running "./bootstrap.sh" on a
clean SVN checkout.

-- 
Steve-o

On 27 September 2011 14:39, Steven McCoy  wrote:

> Bug fix release for interface enumeration with hostnames in network
> parameters.
>
> *Please note:*
>
> OSX Lion is not supported in this version, OpenPGM 5.2 is required due to
> API changes.
>
> *Changes in 5.1.118 (r1419):*
>
>- Update C89 patch.
>
>
>- Additional Windows Sockets 2 notes.
>
>
>- Document purpose for _WIN32_WINNT=0x0501 on Windows platforms.
>
>
>- Fix nullptr usage from getaddrinfo lookups for single result sets.
>
>
>- Backport fixes from 5.2 development branch:
>   - r1372 - Link pthread for non-Ubuntu Scons.
>   - r1379 - Fix member name in IP_MSFILTER_SIZE usage.
>   - r1370 - Fix off-tree builds with Autoconf prefix
>
>
> *Changes in 5.1.117 (r1366):
>
>- Default recv socket always non-blocking, i.e. PGM_NOBLOCK sets state
>of underlying send sockets only.
>- Build system changes for RPM packaging (Mikko Koppanen).
>- Remove gcc & make from RPM packaging as per
>
> *
>
> http://fedoraproject.org/wiki/PackagingGuidelines#Exceptions_2
>
>
>- Avoid using IP_RECVDSTADDR on Windows XP despite undocumented
>definition in MSVC100.
>
>
> *Changes in 5.1.116 (r1361):
>
>- Remove incorrect PGM_GNUC_INTERNAL decoration to send API that causes
>libtool to mark hidden.
>- Fix consistency of malloc wrapper usage to permit interchanging of
>the default allocator.
>
> *
>
> Please consider +1'ing the project in Google search:
>
> http://www.google.com/search?q=openpgm
>
> --
> Steve-o
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Windows Developer Preview = Windows 8?

[Steven McCoy]

I just noticed a new suite of IP stack APIs in the new look MSDN, called
"Registered I/O":

RIOCloseCompletionQueue
>
> 
> RIOCreateCompletionQueue
> 
> RIOCreateRequestQueue
>
> 
> RIODequeueCompletion
> 
> RIODeregisterBuffer
>
> 
> RIONotify
> 
> RIOReceive
>
> 
> RIOReceiveEx
> 
> RIORegisterBuffer
>
> 
> RIOResizeCompletionQueue
> 
> RIOResizeRequestQueue
>
> 
> RIOSend
> RIOSendEx


>From what I can interpret these are a new zero-copy set of APIs that push
VirtualAlloc locked pages to WinSock to prevent the buffer churn that
currently occurs.  Interestingly completions are in that list but IOCP does
need registered buffers unless the particular difference is user-kernel
space locked pages so this new API accelerates the IP stack by not requiring
the page locking.  I don't think there is a direct equivalent in Linux.

VirtualAlloc is a really slow API already.

Some more details here, noting the interesting "a new fast path loopback
allows high-speed apps to achieve a higher level of performance".  That
might be Windows 8 adding TCP Fusion from Solaris.

http://channel9.msdn.com/Events/BUILD/BUILD2011/SAC-593T

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] epgm example

[Steven McCoy]

On 10 October 2011 11:57, Roberto Palmer  wrote:

> Hello all,
> I have a problem while trying to make a simple example with epgm.
> I wrote the following simple C program, but it makes a core dump
> with the following error:
>
>Assertion failed: rc == 0 (connect_session.cpp:84).
>Aborted (core dumped)
>

Looks like the infamously vague error handling problems interpreting the
network parameter:


>rc = zmq_connect(s, "epgm://239.192.1.1:");
>

The network parameter here does not specify the interface and it is likely
the system is presenting multiple options and cannot decide between them.
 Solution is to explicitly define the interface to use.  You have a choice
between hostname, IP address, network address, or network name, for example:

   rc = zmq_connect(s, "epgm://eth0;239.192.1.1:");
>

Would likely fail if you have IPv6 enabled too.

   rc = zmq_connect(s, "epgm://192.168.1.1;239.192.1.1:");
>

More explicit.  To have some abstraction you can use DNS or the NSS naming
system for networks.  The NSS subsystem does not support IPv6 and so OpenPGM
provides an internal implementation if necessary.

   rc = zmq_connect(s, "epgm://myhostname-eth0;239.192.1.1:");
   rc = zmq_connect(s, "epgm://mynetwork;239.192.1.1:");

/etc/hosts
  192.168.1.1   myhostname-eth0

/etc/networks:
  mynetwork 192.168.1


Alternative compatibility NSS:

/tmp/mynetworks:
  mynetwork 192.168.1

PGM_NETDB=/tmp/networks ./myØMQapp ...

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] epgm example

[Roberto Palmer]

Hello all,
I have a problem while trying to make a simple example with epgm.
I wrote the following simple C program, but it makes a core dump
with the following error:

Assertion failed: rc == 0 (connect_session.cpp:84).
Aborted (core dumped)

I'm using zmq 2.1.10 with F15. 

What I'm missing ?
Thanks for any help.

R.



#include 
#include 

#include 


int
main(int argc, char** argv)
{
void *ctx;
void *s;
zmq_msg_t msg;
int rc;


ctx = zmq_init (1);
assert(ctx);

s = zmq_socket(ctx, ZMQ_PUB);

rc = zmq_connect(s, "epgm://239.192.1.1:");
assert (rc == 0);

rc = zmq_msg_init_size (&msg, 4);
assert (rc == 0);

memset(zmq_msg_data (&msg), 'A', 4);

rc = zmq_send(s, &msg, 0);
assert (rc == 0);

zmq_msg_close(&msg);
assert (rc == 0);

sleep(3);
return 0;
}

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] push/pull -- unusual workflow

[Whit Armstrong]

Thanks, Martin.

It was just bad programming and bad debugging on my part.  I did
manage to get this working in most recent version of the rzmq package
(not pushed yet).

I'm going to roll the server portion up into a public ami, so others
can try it out.  It's amazing what one can accomplish in only one
hundred lines of code using zmq.

-Whit


On Mon, Oct 10, 2011 at 6:03 AM, Martin Sustrik  wrote:
> Hi Whit,
>
>> The problem that I am having is that the pull socket on the client is
>> not receiving all of the results back.  My guess is that the pull
>> socket on the client is attempting to fair queue among the connected
>> workers, and if on worker takes a larger share of the jobs, then
>> potentially the pull socket blocks while waiting for more messages
>> from the other node.  Is that possible?
>
> No. If there are no messages from a particular peer, the fair-queueing
> algorithm simply moves on to the another peer.
>
> If you are missing messages, it's probably a bug. Please, create a minimal
> test case and a ticket in the bug tracker. Thanks!
>
>> Additionally, is it possible to print state information for a socket
>> (I need to determine a good way to prove or disprove this theory).
>
> What kind of state are you interested in?
>
> Martin
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZeroMQ binding for PHP not working on WAMPSERVER

[Mikko Koppanen]

On Sun, Oct 9, 2011 at 9:31 PM, lanre lawal  wrote:
> I want to use the ZMQ library for a PHP project. I have downloaded the files
> and done a build with visual studio. When I type the pear command to get the
> pecl extension i get an error that says: ERROR: The DSP zmq.dsp does not
> exist. Also when I try the windows build from source, at the stage when you
> do configure --enable-zmq, it says the zmq option is invalid. Please i need
> urgent help on this

Hello,

as I mentioned yesterday the option is called '--with-zmq' rather than
'--enable-zmq'. You also need to make sure that you have got the
required dependencies built (zeromq). The configure line probably
looks something like this:

configure --disable-all --enable-cli --with-zmq=shared
--with-extra-includes=C:\path\to\libmzq2-1\include
--with-extra-libs=C:\path\to\libmzq2-1\lib

-- 
Mikko Koppanen
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] push/pull -- unusual workflow

[Martin Sustrik]

Hi Whit,

> The problem that I am having is that the pull socket on the client is
> not receiving all of the results back.  My guess is that the pull
> socket on the client is attempting to fair queue among the connected
> workers, and if on worker takes a larger share of the jobs, then
> potentially the pull socket blocks while waiting for more messages
> from the other node.  Is that possible?

No. If there are no messages from a particular peer, the fair-queueing 
algorithm simply moves on to the another peer.

If you are missing messages, it's probably a bug. Please, create a 
minimal test case and a ticket in the bug tracker. Thanks!

> Additionally, is it possible to print state information for a socket
> (I need to determine a good way to prove or disprove this theory).

What kind of state are you interested in?

Martin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] LMAX/Distruptor code project

[Martin Sustrik]

On 10/07/2011 06:06 PM, asif saeed wrote:

> Can you guys post a comparison between Disruptor and 0MQ?

Disruptor is a queueing algorithm. 0MQ is messaging software. They are 
different things. There's a class inside 0MQ (ypipe_t) that roughly 
corresponds to the disruptor.

Martin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] pyzmq on uwsgi process question

[郭梁]

oops! my mistake !
http://lists.unbit.it/pipermail/uwsgi/2011-January/001269.html


2011/10/10 郭梁 

> hello everyone
>
> I have an application based on zeromq-2.1.10 & pyzmq-2.1.9, the
> architecture looks like this:
> 
>nginx(uwsgi) REQ
>  \ | /
>  / | \
> backend worker1 worker2  workerN  REP
> 
> for some reasons, I do not intend to use the XREP/XREQ model, and each
> back-end worker has its own address, the front-end through a certain
> algorithm to determine the back-end workers, by "ipc://..." way
> communication.
> Independent testing everything looks normal, but on the uwsgi process, i
> got the error code: http 502
> The following is a simplified front-end code:
> 
> #!/usr/bin/env python
>
> import random
> import zmq
>
> num_workers = 32
> socket_workers = []
> context = zmq.Context()
>
> for i in range(num_workers):
>  socket = context.socket(zmq.REQ)
> socket.connect("ipc:///dev/shm/april/worker%d.ipc" % i)
>  socket_workers.append(socket)
>
> def run(environ):
> i = random.randint(0, 32)
>  socket_workers[i].send_pyobj({})
> return str(socket_workers[i].recv_json())
>
> def application(environ, start_response):
> status = '200 OK'
> output = run(environ)
> response_headers = [('Content-type', 'text/plain'), ('Content-Length',
> str(len(output)))]
> start_response(status, response_headers)
> return [output]
>
> # for Test
> if __name__ == "__main__":
> print run(None)
> 
> can anyone share their experience and thoughts on this?
> regards
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] pyzmq on uwsgi process question

[郭梁]

hello everyone

I have an application based on zeromq-2.1.10 & pyzmq-2.1.9, the architecture
looks like this:

   nginx(uwsgi) REQ
 \ | /
 / | \
backend worker1 worker2  workerN  REP

for some reasons, I do not intend to use the XREP/XREQ model, and each
back-end worker has its own address, the front-end through a certain
algorithm to determine the back-end workers, by "ipc://..." way
communication.
Independent testing everything looks normal, but on the uwsgi process, i got
the error code: http 502
The following is a simplified front-end code:

#!/usr/bin/env python

import random
import zmq

num_workers = 32
socket_workers = []
context = zmq.Context()

for i in range(num_workers):
socket = context.socket(zmq.REQ)
socket.connect("ipc:///dev/shm/april/worker%d.ipc" % i)
socket_workers.append(socket)

def run(environ):
i = random.randint(0, 32)
socket_workers[i].send_pyobj({})
return str(socket_workers[i].recv_json())

def application(environ, start_response):
status = '200 OK'
output = run(environ)
response_headers = [('Content-type', 'text/plain'), ('Content-Length',
str(len(output)))]
start_response(status, response_headers)
return [output]

# for Test
if __name__ == "__main__":
print run(None)

can anyone share their experience and thoughts on this?
regards
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev