there is a queue in zmq that you can use as template for own mods.
I took it years ago for following functionality:
- creating a queue
device, supplying two endpoints that others can connect to (one bcast,
one rep/req)
- creating two internal endpoints, one for broadcasting,
one for req/rep
Hi all and a merry christmas!
I recently switched from ZMQ V2
(self compiled C++) to V4 installed as part of Linux-Distribution.
After several days of fixing the modified interfaces (classes formerly
delivered as "helper.cpp" and now are separately packaged etc.) the
system compiled fine and
Hi and thx for your fast answer.
I just this minute fixed some
further things for getting to 64bit and found the fixings for the (still
segfaulting) 64bit system, when brought back to the 32bit system to
segfault there also.
Poking in some senseless sleep()s just before the
segfaulting parts
Hi * !
I recently switched from ZMQ2 (pretty old) to ZMQ 4 and
since then have some problems in debugging with EINTR.
Following code:
do {
rc = zmsg_send (&zrep, clsocket_);
if (rc<0) {
if (errno ==
EINTR || errno == EAGAIN) {
logWarn("temporary failure in zmq send()
... will be tried
another related thing buthers me in this context:
When zmsg_send()
indeed returns with rc==-1 and a NULLed message (data is definitely
lost) I have a chance to check for this lost message (simply asserting
on rc==-1 && message==NULL).
BUT (!!) what is with zmsg_recv() ?
Situation: I successf
Sorry for spamming with the third mail today ...
I had a look into
your implementation of zmsg_send() and there indeed is alogical bug:
sorry for sending the third mail today ...
I had a look into your
implementation of zmsg_send() and indeed found a logical bug.
The
following code assur
the referenced fix should work for zmsg_send() but the same bug is
in zmsg_recv() and there the application does not (yet) have any data.
zmsg_recv() destroyes properly received frames 1 to n-1 if receiving
frame n failes. Since the data is ONLY inside ZMQ at this time, the
application cannot
I get that error only during debuging inside Eclipse C++ (gdb).
It
seems irrelevant, if there are breakpoints set or not (but if there are
any breakpoints, that stop the program, the number of EINTR is
significantly higher).
I also wonder about getting EINTR because of a
debugger. Thet is no
I now use some code doublicate of your cmzq code, that does frame
send()ing with REUSE and retries in case of EINTR:
I copied
zmsg_recv() and wrapped the frame receiving in a loop checking EINTR.
zmsg_t *zmsg_recv (void *source)
{
assert (source);
zmsg_t *self =
zmsg_new ();
if (!self)
Hey, I had a comparable idea for shutting down.
BUT: a shutdown
command in a system that stops and respawns during normal runtime is
never ever something that can be treated as a typical
bcast/fire-and-forget and so should never be able to reach some worker
that is not meant to get it.
Such
Hi *!
I found the "bug" that I >>>thought<<< to be caused by
EINTR:
The EINTR-thingy was gone, when I fixed some illegal refference
usages, that the gcc compiler didn't complain about (but "clang" of LLVM
did ;o)
... the lost messages problem was still there.
BUT: when I
switched from ZM
Hi *!
I just read through this thread and would like to share some
thoughts about several things in discussion:
I picked ZMQ to have a
straight forward lib that is easy to handle and makes it easy to both
stick to high level things (application concerned) as well as going into
details where
Hi all!
after migration from ZMQ2 to ZMQ4 I had problems with
vanished messages (on REP/REQ as well as on PUB/SUB).
I now MANAGED TO
BUILD SOME DIRTY "MINIMAL" TESTCASE (see below) displaying at least
>one< of my problems.
The testcase does following:
- create a router
binding to one SUB
provement.
Also I checked with tdmalloc, valgrind, efence and duma, but they all say that
I'm fine with memory access. (or does zmq somehow work arround those memory
libs?)
mit freundlichen Grüßen
Sven Koebnick
E = mc² ± 2dB
Oops ... I overlooked, that there's no time between connect and send
in client ;o)
please excuse this silly thing.
Result: I still don't
have an easy test case for my vanished messages ;o((
Am 2015-03-07
13:41, schrieb Sven Koebnick:
> Thanks for the fast response ;
Hi again !
I worked on some demontrating testcase and build up the
router code below (see end of this mail).
Please excuse the ugly look
of the code, it was typed without planing and worsened during test ;o)
It is doing the following:
* create a router (binding 2 ZMQ_ROUTER
socke
Hi @ all!
I'm developing against ZMQ for quite some time now and the distributed
service architechture is ready and working stable (also under heavy
load).
(No, I din't update to the latest version because the behaviour doesn't
look like an accidential bug)
While hardening the system, a queer
In the
last months I nearly finished implementing my service
concept using zmq (instead of an ugly selfwritten msglib) and there are
some things, I'd like to discuss here ... maybe s'one has an idea to
solve one
of the issues or needs the same feature ... or tells me, I'm just blind,
not sseing t
ed libstdc.
mit freundlichen Grüßen
Sven Koebnick
-
E = mc² ± 2dBA - everything is relative
-
-Original Message-
Date: Mon, 27 Sep 2010 16:56:32 +0200
Subject: Re: [
IMHO it
should be decided by the developer of the application, if different
message type/streams can sensefully share one stream. this should not be
decided by the developer of a used lib (e.g. zmq).if the feature
is simply not implemented (yet) it's OK, but I also think it's a fine
feature.But it
when I
stripped the system down for a short demo, I found a bug (of mine) that
caused a second "poll" when it's been clear (by the first
poll), that data had arrived.
==> Sorry, my fault!
^5
Sven
-
E = mc² ± 2dBA - everything is relati
that
reminds me of my recently reported problem, that poll returns wrong
codes.I thought, that it's been completely my fault, because I
accidentially called poll() twice on a socket before recv()ing, BUT:the second call to poll() should still tell me, that data is
availably, which does not happen (
Imho you shouldnt invest too much time in getting dists sound and safe
for everybody. S'one that needs zmq surely has no prob in upgrading
one or two pkgs. Think about the typical help text in kernal config:
"if you dont know what this is, you propably dont need it"
^5
Sven
==
Hi
all!I have some tricky problem, which I guess is caused by
the balancing algorhythm of the XREQ sockets in a queue device.following situation:The system has several service types, each
having several worker threads on one queue device per service type.One of these service types is a dispatcher,
-- everything is relative
-
-Original Message-
Date: Wed, 20 Oct 2010 09:32:12 +0200
Subject: Re: [zeromq-dev] loadbalancing problem?
From: Martin Sustrik
To: Sven Koebnick , ZeroMQ development
list
Hi Sven,
Conclusion: (correct me, if I'm wron
pipes.swap (current, active);
< else
current = 0; }
-Original Message-
Date: Wed, 20 Oct 2010 11:37:23 +0200
Subject: Re: [zeromq-dev] loadbalancing problem?
From: Martin Sustrik
To: Sven Koebnick , ZeroMQ development
list
Hi Sven,
> Yes, I'm aware of the
Hi
Nir!
I had a look at the LRU method and found that it propably destroys the
routing information.Data runs like this:- someone send new
msg to dispatcher queue device- queue device has several workers
and send msg to one- the dispatcher worker forwards the message to
the proper service (again so
c² ± 2dBA - everything is relative
-
-Original Message-
Date: Wed, 20 Oct 2010 15:47:15 +0200
Subject: Re: [zeromq-dev] loadbalancing problem?
From: Martin Sustrik
To: Sven Koebnick , ZeroMQ development
list
Sven,
> I saw, there already is some technique in the loadbalancer to take
a
just a
thought: isn't subscription forwarding more or less, what multicast hubs
do? sounds to me like reinventing the wheel under a new name.
rgdssven
-
E = mc² ± 2dBA - everything is relative
--
have a
look at the overloaded socket_t::operator void* (). I also stumbled over
this tiny difference when modifying a device.
sven
-
E = mc² ± 2dBA - everything is relative
-
--
Jaime,I stumbled over the same thing, but alas, at
a state, where I already had several thousand lines of code arround my
service frame ;o(The sockets do not really make balancing, they
"round robin" through all connected servers. When I asked
about this (here), I got this REALLY helpful link:
http
Hey,
great! just a few hours ago, I tagged a "make a shell tool" to
my TODOs. I'm gonna post my extension to your code since I'm using
google protocol buffers as data format.Sven
-
E = mc² ± 2dBA - everything is relative
---
wouldn't
it be easiest to just subscribe to regex's instead of fixed prefix
strings?that shouldn't be too much effort to build into the
subscription filter and would also allow a big deal of other fine
stuff.
^5sven
-
E = mc² ± 2dBA -
what's
wrong in just pluging in some additional functions on lowest protocol
level implementing ipsec stuff ?surely, this does not deliver
application level security, but this should anyway (IMHO) be implemented
clearly on top of zmq (well, there could be some (also ontop) libs
helping zmq-apps to
maybe
this is helpful:in my system, I have dedicated dispatcher
services, that just know, where to forward the different messages.for shutdown, I send a "shut up" message to all
services, which afterwards refuse do do anything. refused messages are
written to DB by the dispatchers, as well as fresh
talking
about Java-bindings, I wonder (expecting you to think about roasting me
slowly on a small fire ;-) how much work it would be to write a PURE
java implementation ... without JNI or other ways to break an applet.my current (already accepted) design has a small translater that
works a gateway
a few
years ago, I read some artical about cheating IPC into Windows. it
worked with using parts of the paging file as a shared datafile and
thereby simulating shared memory.Maybe this can be wrapped to just
plug it into where IPC sits on real operating systems.
^5
Sven
---
did you
set socket ID on both ends? req AND rep?
^5
Sven
-
E = mc² ± 2dBA - everything is relative
-
-Original Message-
Date: Sat, 04 Dec 2010 13:59:45 +0100
Subject: [ze
such
extreme performance drops should not be too difficult to find.My
first guess would be a wrong NIC-config. There is (since years) a kind
of quirk, that (seemly intensionally) guesses the doublex setting wrong
in some constellations. this would perfectly fit to your problem with
fast onw way tra
hi
toni,i had a comparable scenario and got this link:
http://zguide.zeromq.org/chapter:all#toc45 (LRU-Queue)
it descrives a technique to know about available workers. It looks a bit
complicated, but only on the first view. the described concept works
fine for me.
^5sven
--
Hi
all!
I have a queer behaviour in my services build on zmq. I tried to
write a minimal testcase, but failed to rebuild this bug in a smaller
program ;o(Hopefully my description is enough to find the
cause:I have a PUB/SUB socket, where several messages
already have passed successfully. As the las
17:50:22 +0100
Subject: [zeromq-dev] poll does not return on a SUB socket
From: "Sven Koebnick"
To: zeromq-dev@lists.zeromq.org
Hi all! I have a queer
behaviour in my services build on zmq. I tried to write a minimal
testcase, but failed to rebuild this bug in a smaller program ;o(Ho
Hi Martin,
the main issue is, that poll(... ,-1) does not at all return despite
the fact that there is a message available!
The usage of timeout value is a current workaround, because the second
call to poll (the first returned zero) successfully returns a number
greater one.
Sven
=
I'll try
again, but the system ist already rather complex and it will be much
effort to strip it down (maybe impossible). So I'd have to build the
testcase from scratch.
Current Scenario is the following:normal devices:- a
modified queue device, implemented as LRU queue (not exactly as
described in
you have to:
- create a zmq msg with sufficient size
- copy the struct into the msg buffer
This means, that you need a func, that knows the struct and copies the data of
the inner pointers etc.
In short: you need to write a serialization function for your struct.
^5
Sven
===
sent
45 matches
Mail list logo
