[zeromq-dev] how to reliably wait for SUBscribers to connect before sending messages

[KIU Shueng Chuan]

Message: 1
Date: Fri, 5 Jan 2018 10:03:17 +0100
From: Francesco 
To: zeromq-dev 
Subject: [zeromq-dev] how to reliably wait for SUBscribers to connect
before sending messages
Message-ID:

Content-Type: text/plain; charset="utf-8"

Hi all,
I have some unit tests around the ZMQ wrappers I wrote for PUB/SUB sockets
(these wrappers integrate functionalities specific to my use case).

In these unit tests I spawn a thread that creates a PUB socket and another
one creating a SUB socket. In the PUB thread I do zmq_bind(), then sleep
for a SETTLE_TIME and then I start sending messages with zmq_msg_send().
In the SUB I just do zmq_connect() and then immediately zmq_msg_recv().

The problem is when I run such unit tests under valgrind. In that case I
noticed that randomly my settle time of 1sec is not enough: the unit tests
fail because the SUB receives 0 messages instead of N>0.
A simple fix would be to increase the settle time. However since I repeat
that kind of tests hundreds of times, that means increasing testing time a
lot.

I think my problem is that I need to wait after the zmq_bind() and before
zmq_msg_send() some time to allow ZMQ background threads to actually
connect the PUB and SUB sockets together.
Is there a better way to test if there are pending connections in ZMQ
background threads rather than waiting a (random) amount of time?

--


If you could change to use XPUB socket, after binding you could wait for
the subscription message to know that the subscriber had connected.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Zmq 4.2.0 aligned memory

[KIU Shueng Chuan]

A common use case for me is sending an array of floats.

First message part is some small metadata. Second message part is the float
array.

On reception, zmq_msg_data is cast to float* and accessed directly.

This non-alignment would be problematic.
Or perhaps there never was any alignment guarantee?

On 15 Nov 2016 3:34 a.m., "Jens Auer"  wrote:

> Hi,
>
> I think I have an idea why you are seeing unaligned messages, but this
> only applies to messages where the payload is not stored in the msg_t
> object itself. I think the threshold for this is 64 bytes. In ZeroMQ 4.1,
> receiving messages was done by first receiving from the socket into a
> static 8kb buffer, and then a new message object was created that allocated
> memory externally by calling malloc.  The payload was then copied from the
> receive buffer to the message buffer. The malloced message buffer was
> aligned probably.
>
> In ZeroMQ 4.2, this is changed to reduce the number of malloc calls and
> copy operations. The receive buffer is now dynamically allocated as a 8kb
> block, and messages are constructed as zero-copy messages using the part of
> the receive buffer containing the payload. This saves malloc calls and copy
> operations and increases performance. However, the payload may now start at
> basically arbitrary addresses. As an example, let's assume that we receive
> a small message of 10 bytes and a large message of 1kb, both received in a
> single call to recv on the socket. The engine allocates a new buffer of
> 8kb, calls recv(socket, buffer) and the data is written to the buffer. A
> small message is then created which contains the data from byte 2-11 in the
> msg_t, byte 1 contains the header. At byte 12 starts the header of the next
> message, and at byte 22(?) starts the payload. The large message is created
> as a zero-copy message using the pointer to byte 22 as storage. This is not
> aligned to a 4-byte address.
>
> Could you provide some more information about the sizes of the messages
> that you receive? How do you decode the buffer content?
>
> Best wishes,
> Jens
>
> -Ursprüngliche Nachricht-
> Von: zeromq-dev [mailto:zeromq-dev-boun...@lists.zeromq.org] Im Auftrag
> von Emmanuel Taurel
> Gesendet: Montag, 14. November 2016 16:49
> An: zeromq-dev@lists.zeromq.org
> Betreff: [zeromq-dev] Zmq 4.2.0 aligned memory
>
> Hello all,
>
> We are using zeromq since years now without troubles. We have recently
> tried our software using Zmq 4.2.0 (on linux hosts).
> For our application, we are using multipart messages with 4 parts in
> publish/subscribe mode.
> With Zmq 4.0.5, on the subscriber side, when we get the last message part,
> the received buffer was memory aligned (at least on 0x4 border).
> Unfortunately, with Zmq 4.2.0, the buffer is not aligned any more.
>  For instance with Zmq 4.0.5, the buffer was at address xxx08 while with
> Zmq 4.2.0, it is at address xxx23.
>
> I don't know if it is relevant but our messages are relatively small
> messages (few tens of bytes) This is a problem for us in decoding the
> buffer content.
>
> Is there something to be done to have memory aligned buffers?
>
> Thank's in advance for your answers
>
>  Emmanuel
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> ---
> Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
> https://www.avast.com/antivirus
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Lost message with PUSH/PULL

[KIU Shueng Chuan]

I have attached some minimal code here where doing a TCP connect to
the PUSH bind socket makes the latter writable.

Tested on 4.1.4

On 30 June 2016 at 23:11, Doron Somech  wrote:
> I think it might be the ZeroMQ behavior (which is probably a bug), once the
> connection is open ZeroMQ start to queue messages for the connection, even
> if handshake is not yet completed, so when you do telnet the message is not
> discarded but queue for the telnet connection.
>
> I think only PUSH is affected by this behavior, also fixing it might be
> hard, it also might be security issue, right now PUSH socket type is not
> safe for internet use because of this.
>
> If you can I suggest you reverse the bind/connection so PUSH will connect
> and PULL will bind. If not an option try to use ROUTER instead and have some
> kind of handshake.
>
> Nice catch, I still want to make sure that is really what happen and see if
> it possible to fix this easily..
>
> On Thu, Jun 30, 2016 at 11:32 AM, 王运来  wrote:
>>
>> Hi every guys:
>>I got a problem which ZMQ will lost some messages with PUSH/PULL
>> ZMQ socket.
>>The scene like this:
>>A: PUSH socket, bind address "tcp://*.1209"
>>B: PULL socket, connect to "tcp://localhost:1209"
>>
>>Run the command "telnet localhost 1209" while A sending message to
>> B.
>>
>>The result is B will miss  messages even if I set the option of
>> ZMQ_IMMEDIATE to 1 like this:
>>int immediate = 1;
>>zmq_setsockopt(pSock, ZMQ_IMMEDIATE, , sizeof(immediate));
>>
>>   Is it right in this scene or is it should be?
>>
>>
>>
>>
>>
>> ___
>> zeromq-dev mailing list
>> zeromq-dev@lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
#include 
#include 

#include 
#include 
#include 
#include 

#include 

int main()
{
void *zctx = zmq_ctx_new();
void *zsock = zmq_socket(zctx, ZMQ_PUSH);
int rc = zmq_bind(zsock, "tcp://127.0.0.1:34567");
assert(rc!=-1);

rc = zmq_send(zsock, "ABC", 3, ZMQ_DONTWAIT);
assert(rc==-1); // expected not writable

int sockfd = socket(AF_INET, SOCK_STREAM, 0);
struct sockaddr_in saddr = {AF_INET, htons(34567), htonl(INADDR_LOOPBACK)};
rc = connect(sockfd, (struct sockaddr*), sizeof(saddr));
assert(rc==0);

zmq_pollitem_t item = {zsock, 0, ZMQ_POLLOUT};
rc = zmq_poll(, 1, 100);
assert(rc==1);  // oops, became writable
   
rc = zmq_send(zsock, "ABC", 3, ZMQ_DONTWAIT);
assert(rc==3);  // write succeeds despite no PULL socket connecting

close(sockfd);

int linger = 0;
rc = zmq_setsockopt(zsock, ZMQ_LINGER, , sizeof(linger));
assert(rc==0);

rc = zmq_close(zsock);
assert(rc==0);

// sometimes this blocks...
printf("destroying\n");
zmq_ctx_destroy(zctx);
}

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] What are the “serious caveats” with ZMQ_FD?

[KIU Shueng Chuan]

In pseudocode, it might look like this:

void check_events(zsock)
{
  int revents = zsock.getsockopt(ZMQ_EVENTS);
  if (revents & ZMQ_POLLIN) {
  // "call_later" is some function that processes the call in the
  // next iteration of your event loop.
call_later(on_readable);
  }
}

// call this callback when the ZMQ_FD is readable
void on_readable(zsock)
{
  int revents = zsock.getsockopt(ZMQ_EVENTS);
  if (revents & ZMQ_POLLIN) {
// call zmq_recv() repeatedly to receive all frames of a multipart message
  }
  check_events(zsock);
}

void do_send(zsock, msg)
{
  zsock.send(msg);
  check_events(zsock);
}


On 2 May 2016 4:25 pm, "Kalle Huttunen" <khut...@gmail.com> wrote:

So basically I will just write (I use the C++ bindings):

mySocket.send(myMsg);
mySocket.getsockopt(ZMQ_EVENTS); // Update the file descriptor state

?

-- 
Kalle Huttunen


ma 2. toukokuuta 2016 klo 11.15 Justin Karneges <jus...@affinix.com> kirjoitti:
>
> "update the state" is strange wording but it means you need to query 
> ZMQ_EVENTS.
>
> And yes, you should assume that even a call to zmq_recv() might yield a 
> situation where there is more to read, but the fd doesn't become readable. 
> Basically, check ZMQ_EVENTS after every call to zmq_send or zmq_recv, and 
> whenever the fd becomes readable, regardless of which events you are 
> interested in.
>
> On Mon, May 2, 2016, at 01:07 AM, Kalle Huttunen wrote:
>
> Okay, this one I missed because I was looking at an old version of the man 
> page (http://api.zeromq.org/master:zmq-getsockopt takes you to 2.2.0 man 
> page).
>
> Full quote from the 4.1.5 man page (http://api.zeromq.org/4-1:zmq-getsockopt):
>
> "The returned file descriptor is also used internally by the zmq_send and 
> zmq_recv functions. As the descriptor is edge triggered, applications must 
> update the state of ZMQ_EVENTS after each invocation of zmq_send or 
> zmq_recv.To be more explicit: after calling zmq_send the socket may become 
> readable (and vice versa) without triggering a read event on the file 
> descriptor."
>
> This definitely sounds like a caveat I should handle. The quote raises some 
> questions, though:
>
> - What exactly does it mean to "update the state of ZMQ_EVENTS"?
>
> - Do both zmq_send() and zmq_recv() reset all events in the FD? So if I'm 
> only interested on events about incoming messages (I check only ZMQ_POLLIN 
> from ZMQ_EVENTS), do I need to "update the state of ZMQ_EVENTS" after both 
> zmq_send() and zmq_recv(), or is it enough to do it only after zmq_send()?
>
> --
> Kalle Huttunen
>
> ma 2. toukokuuta 2016 klo 4.16 KIU Shueng Chuan <nixch...@gmail.com> 
> kirjoitti:
>
> From the man page for zmq_getsockopt:
>
> "after calling zmq_send the socket may become readable (and vice versa) 
> without triggering a read event on the file descriptor."
>
>
> On 1 May 2016 3:33 am, "Kalle Huttunen" <khut...@gmail.com> wrote:
>
> The ZeroMQ FAQ (http://zeromq.org/area:faq#toc5) states in the "Why can't I 
> use standard I/O multiplexing functions such as select() or poll() on ZeroMQ 
> sockets?" question:
>
> > Note that there's a way to retrieve a file descriptor from ZeroMQ socket 
> > (ZMQ_FD socket option) that you can poll on from version 2.1 onwards, 
> > however, there are some serious caveats when using it. Check the 
> > documentation carefully before using this feature.
>
> I've prototyped integrating ZeroMQ socket receiving to Qt's and custom 
> select() based event loops, and on the first glance everything seems to work.
>
> From the documentation I have identified two "caveats" that I handle in my 
> code:
>
> 1. The ability to read from the returned file descriptor does not necessarily 
> indicate that messages are available to be read from the socket
>
> This I have solved by checking ZMQ_EVENTS before reading from the socket.
>
> 2. Events are signaled in edge-triggered fashion
>
> This one I have solved by always receiving all the messages from the socket 
> when the file descriptor signals.
>
> Are there some caveats that I'm missing?
>
> --
> Kalle Huttunen
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-

Re: [zeromq-dev] What are the “serious caveats” with ZMQ_FD?

[KIU Shueng Chuan]

>From the man page for zmq_getsockopt:

"after calling zmq_send the socket may become readable (and vice versa)
without triggering a read event on the file descriptor."
On 1 May 2016 3:33 am, "Kalle Huttunen"  wrote:

> The ZeroMQ FAQ (http://zeromq.org/area:faq#toc5) states in the "Why can't
> I use standard I/O multiplexing functions such as select() or poll() on
> ZeroMQ sockets?" question:
>
> > Note that there's a way to retrieve a file descriptor from ZeroMQ socket
> (ZMQ_FD socket option) that you can poll on from version 2.1 onwards,
> however, there are some serious caveats when using it. Check the
> documentation carefully before using this feature.
>
> I've prototyped integrating ZeroMQ socket receiving to Qt's and custom
> select() based event loops, and on the first glance everything seems to
> work.
>
> From the documentation I have identified two "caveats" that I handle in my
> code:
>
> 1. The ability to read from the returned file descriptor does not
> necessarily indicate that messages are available to be read from the socket
>
> This I have solved by checking ZMQ_EVENTS before reading from the socket.
>
> 2. Events are signaled in edge-triggered fashion
>
> This one I have solved by always receiving all the messages from the
> socket when the file descriptor signals.
>
> Are there some caveats that I'm missing?
>
> --
> Kalle Huttunen
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Empty message does not close ZMQ_STREAM socket?

[KIU Shueng Chuan]

Hi Jens,

I did some more experimentation, and the following sequence works to cause
the client disconnection:

```cpp
// send client some data
server.send(id.data(), id.size(), ZMQ_SNDMORE);
server.send("A", 1, ZMQ_SNDMORE);
// disconnect client
server.send(id.data(), id.size(), ZMQ_SNDMORE);
server.send("", 0, ZMQ_SNDMORE);
```

In your example code, the server was immediately disconnecting the client
without having sent it any other bytes.
So it would seem that you have hit a corner case in ZMQ_STREAM.

Should this be considered a zmq bug?


On Tue, Mar 22, 2016 at 11:11 PM, Auer, Jens <jens.a...@cgi.com> wrote:

> Hi,
>
> ZMQ_STREAM sockets are special and do not follow the normal behavior w.r.t
> ZMQ_SNDMORE. You always have to send two messages, the first one is
> discarded as the identity frame. If you omit ZMQ_SNDMORE once, you will not
> be able to send any messages over the socket again.
>
> I also tried with and without ZMQ_SNDMORE for the empty message. It does
> not make a difference, but sending it twice does.
>
> Cheers,
>   Jens
>
> --
> *Jens Auer *| CGI | Software-Engineer
> CGI (Germany) GmbH & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
> T: +49 6151 36860 154
> *jens.a...@cgi.com* <jens.a...@cgi.com>
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter
> *de.cgi.com/pflichtangaben* <http://de.cgi.com/pflichtangaben>.
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
> --
> *Von:* zeromq-dev-boun...@lists.zeromq.org [
> zeromq-dev-boun...@lists.zeromq.org]" im Auftrag von "Osiris Pedroso [
> opedr...@gmail.com]
> *Gesendet:* Dienstag, 22. März 2016 15:03
> *An:* ZeroMQ development list; KIU Shueng Chuan
> *Betreff:* Re: [zeromq-dev] Empty message does not close ZMQ_STREAM
> socket?
>
> If I am completely off here, please let me know (still learning ZMQ):
>
> server.send(id, ZMQ_SNDMORE);
> server.send(empty, ZMQ_SNDMORE );
>
> But I don't think you should have the ZMQ_SENDMORE flag on the second
> send() call.
>
> Otherwise the message will not be sent, instead waiting for another call
> to send() without the flag which only then commits the message to be sent
> by underlying code.
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Empty message does not close ZMQ_STREAM socket?

[KIU Shueng Chuan]

Hi Jens,

Because the client is using ZMQ_STREAM, when it gets disconnected it
immediately reattempts a reconnect.

So the first empty message is for connect. Second empty message is for
disconnect. And the 3rd empty message is for the reconnect. And so on.
On 22 Mar 2016 12:07 am, "Auer, Jens" <jens.a...@cgi.com> wrote:

> Hi,
>
> I tried that before and thought that sending the close message
> repetitively would do the trick. To better see if the socket is really
> closed, I set the reconnect interval to ten minutes in the client by setting
> int tenminutes = 6;
> client.setsockopt(ZMQ_RECONNECT_IVL, , sizeof(tenminutes));
>
> Now, when I use the following server which is basically taken from the
> manpage, I can close the socket by sending the close message twice.
> Removing the ZMQ_SNDMORE flag does not change anything.
>
> int main(int argc, char* argv[]) {
> int n = std::atoi(argv[1]);
> void *ctx = zmq_ctx_new ();
> /* Create ZMQ_STREAM socket */
> void *socket = zmq_socket (ctx, ZMQ_STREAM);
>
> int rc = zmq_bind (socket, "tcp://127.0.0.1:5000");
> /* Data structure to hold the ZMQ_STREAM ID */
> uint8_t id [256];
> size_t id_size = 256;
> /* Data structure to hold the ZMQ_STREAM received data */
> uint8_t raw [256];
> size_t raw_size = 256;
> for(int i=0; i != n; ++i)
> {
> /*  Get HTTP request; ID frame and then request */
> id_size = zmq_recv (socket, id, 256, 0);
> assert (id_size > 0);
> std::cout << id_size << std::endl;
> raw_size = zmq_recv (socket, raw, 256, 0);
>
> /* Closes the connection by sending the ID frame followed by a
> zero response */
> zmq_send (socket, id, id_size, ZMQ_SNDMORE);
> zmq_send (socket, 0, 0, 0);
> /* NOTE: If we don't use ZMQ_SNDMORE, then we won't be able to
> send more */
> /* message to any client */
> }
> std::cout << "Done" << std::endl;
> int i;
> std::cin >> i;
> zmq_close (socket); zmq_ctx_destroy (ctx);
> return 0;
> }
>
> Cheers,
>   Jens
>
> --
> *Jens Auer *| CGI | Software-Engineer
> CGI (Germany) GmbH & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
> T: +49 6151 36860 154
> *jens.a...@cgi.com* <jens.a...@cgi.com>
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter
> *de.cgi.com/pflichtangaben* <http://de.cgi.com/pflichtangaben>.
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
> --
> *Von:* zeromq-dev-boun...@lists.zeromq.org [
> zeromq-dev-boun...@lists.zeromq.org]" im Auftrag von "KIU Shueng Chuan [
> nixch...@gmail.com]
> *Gesendet:* Montag, 21. März 2016 15:38
> *An:* ZeroMQ development list
> *Betreff:* Re: [zeromq-dev] Empty message does not close ZMQ_STREAM
> socket?
>
> Hi Jens,
>
> I tried out the pair of programs in your original post. Indeed, I got the
> behavior that you described using ZeroMQ 4.1.2.
>
> Putting the server code into a while loop did the trick.
> ```c++
> server.bind(address);
> while (1)
> {
>// recv connection
>// send disconnection
> }
> int i;
> std::cin >> i;
>
> ```
>
> I think in my own code, I have only ever used it in this forever loop
> fashion. Hmm...
>
>
> On Mon, Mar 21, 2016 at 9:11 PM, Auer, Jens <jens.a...@cgi.com> wrote:
>
>> Hi,
>>
>>
>>
>> Thanks for the suggestion. I have modified my example to copy the id into
>> a new message, but it did not change anything. I have also tried a second
>> version of the server side which is basically the example provided at the
>> bottom of the manual page:
>>
>> int main2(int argc, char* argv[]) {
>>
>> int n = std::atoi(argv[1]);
>>
>> void *ctx = zmq_ctx_new ();
>>
>> /* Create ZMQ_STREAM socket */
>>
>> void *socket = zmq_socket (ctx, ZMQ_STREAM);
>>
>>
>>
>> int rc = zmq_bind (socket, "tcp://127.0.0.1:5000");
>>
>> /* Data structure to hold the ZMQ_STRE

Re: [zeromq-dev] Empty message does not close ZMQ_STREAM socket?

[KIU Shueng Chuan]

Hi Jens,

I tried out the pair of programs in your original post. Indeed, I got the
behavior that you described using ZeroMQ 4.1.2.

Putting the server code into a while loop did the trick.
```c++
server.bind(address);
while (1)
{
   // recv connection
   // send disconnection
}
int i;
std::cin >> i;

```

I think in my own code, I have only ever used it in this forever loop
fashion. Hmm...


On Mon, Mar 21, 2016 at 9:11 PM, Auer, Jens <jens.a...@cgi.com> wrote:

> Hi,
>
>
>
> Thanks for the suggestion. I have modified my example to copy the id into
> a new message, but it did not change anything. I have also tried a second
> version of the server side which is basically the example provided at the
> bottom of the manual page:
>
> int main2(int argc, char* argv[]) {
>
> int n = std::atoi(argv[1]);
>
> void *ctx = zmq_ctx_new ();
>
> /* Create ZMQ_STREAM socket */
>
> void *socket = zmq_socket (ctx, ZMQ_STREAM);
>
>
>
> int rc = zmq_bind (socket, "tcp://127.0.0.1:5000");
>
> /* Data structure to hold the ZMQ_STREAM ID */
>
> uint8_t id [256];
>
> size_t id_size = 256;
>
> /* Data structure to hold the ZMQ_STREAM received data */
>
> uint8_t raw [256];
>
> size_t raw_size = 256;
>
> for(int i=0; i != n; ++i)
>
> {
>
> /*  Get HTTP request; ID frame and then request */
>
> id_size = zmq_recv (socket, id, 256, 0);
>
> assert (id_size > 0);
>
> std::cout << id_size << std::endl;
>
> raw_size = zmq_recv (socket, raw, 256, 0);
>
>
>
> /* Closes the connection by sending the ID frame followed
> by a zero response */
>
> zmq_send (socket, id, id_size, ZMQ_SNDMORE);
>
> zmq_send (socket, 0, 0, ZMQ_SNDMORE);
>
> /* NOTE: If we don't use ZMQ_SNDMORE, then we won't be
> able to send more */
>
> /* message to any client */
>
> }
>
> std::cout << "Done" << std::endl;
>
> int i;
>
> std::cin >> i;
>
> zmq_close (socket); zmq_ctx_destroy (ctx);
>
> return 0;
>
> }
>
>
>
> This has the same problem as described before that the socket is not
> closed and I even receive an empty message at the client’s side.
>
>
>
> Cheers,
>
>   Jens
>
>
>
> *--*
>
> *Dr. Jens Auer *| CGI | Software Engineer
>
> CGI Deutschland Ltd. & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
>
> T: +49 6151 36860 154
>
> jens.a...@cgi.com
>
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
> unter de.cgi.com/pflichtangaben.
>
>
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
>
>
>
> *From:* zeromq-dev-boun...@lists.zeromq.org [mailto:
> zeromq-dev-boun...@lists.zeromq.org] *On Behalf Of *KIU Shueng Chuan
> *Sent:* 21 March 2016 11:43
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] Empty message does not close ZMQ_STREAM
> socket?
>
>
>
> There's something in the manpage that your code doesn't do.
>
> > To open a connection to a server, use the zmq_connect call, and then
> fetch the socket identity using the ZMQ_IDENTITY zmq_getsockopt call.
>
> I don't recall if this was strictly necessary but I see in my code that I
> did fetch the identity but immediately discarded it.
>
> I have some python scripts here where the server code disconnects the
> client.
> https://github.com/pijyoi/test_zmqstream
> See zmqstream_publisher.py and zmqstream_subscriber.py
>
> On 21 Mar 2016 5:07 pm, "Auer, Jens" <jens.a...@cgi.com> wrote:
>
> Hi,
>
> I am trying to close a TCP socket over a ZMQ_STREAM by sending an identity
> frame followed by an empty message, as described in the manual ("To close a
> specific connection, send the identity frame followed by a zero-length
> message"). I have a simple client/server example:
> #include 
> #include 
> #include 
>
> int main2() {
> std::string addres

Re: [zeromq-dev] Empty message does not close ZMQ_STREAM socket?

[KIU Shueng Chuan]

There's something in the manpage that your code doesn't do.

> To open a connection to a server, use the zmq_connect call, and then
fetch the socket identity using the ZMQ_IDENTITY zmq_getsockopt call.

I don't recall if this was strictly necessary but I see in my code that I
did fetch the identity but immediately discarded it.

I have some python scripts here where the server code disconnects the
client.
https://github.com/pijyoi/test_zmqstream
See zmqstream_publisher.py and zmqstream_subscriber.py

On 21 Mar 2016 5:07 pm, "Auer, Jens"  wrote:

> Hi,
>
> I am trying to close a TCP socket over a ZMQ_STREAM by sending an identity
> frame followed by an empty message, as described in the manual ("To close a
> specific connection, send the identity frame followed by a zero-length
> message"). I have a simple client/server example:
> #include 
> #include 
> #include 
>
> int main2() {
> std::string address = "tcp://127.0.0.1:5000";
>
> zmq::context_t zmq;
> zmq::socket_t server(zmq, ZMQ_STREAM);
>
> server.bind(address);
>
> zmq::message_t id;
> zmq::message_t m;
> server.recv();
> server.recv();
> std::cout << "Connection received: " << id.size() << std::endl;
>
> zmq::message_t empty{};
>
> server.send(id, ZMQ_SNDMORE);
> server.send(empty, ZMQ_SNDMORE );
>
> int i;
> std::cin >> i;
>
> return 0;
> }
>
> #include 
> #include 
>
> int main() {
> std::string address = "tcp://127.0.0.1:5000";
>
> zmq::context_t zmq;
> zmq::socket_t client(zmq, ZMQ_STREAM);
>
> client.connect(address);
>
> zmq::message_t m3;
> zmq::message_t m4;
> client.recv();
> client.recv();
>
> std::cout << "Connection established" << std::endl;
>
> {
> zmq::message_t m5;
> zmq::message_t m6;
> client.recv();
> client.recv();
> }
>
> std::cout << "Connection closed" << std::endl;
> int i;
> std::cin >> i;
>
> return 0;
> }
>
> When I start the server and then the client, the server will receive the
> connection message, but the empty message does not close the socket:
> ./stream_s 2
> Connection received: 5
> /tmp/stream_c
> Connection established
>
> tcp0  0 127.0.0.1:5000  0.0.0.0:*
> LISTEN  32615/./stream_s
> tcp0  0 127.0.0.1:5000  127.0.0.1:34270
> ESTABLISHED 32615/./stream_s
> tcp0  0 127.0.0.1:34270 127.0.0.1:5000
> ESTABLISHED 32618/stream_c
>
> I have also created a second example where the server sends the close
> message twice. Here, the client actually receives the close message as an
> empty message. Why is this even possible over a ZMQ_STREAM socket?
>
> I am using ZeroMQ 4.1.4, so the notification messages should be enabled by
> default without setting ZMQ_STREAM_NOTIFY, and I have also tested 4.1.2 to
> check for a regression.
>
> Best wishes,
>   Jens
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] level-triggered FD

[KIU Shueng Chuan]

I don't see how this can be the case since zmq_poll() internally uses
ZMQ_FD to implement level triggering for zmq sockets.

https://github.com/zeromq/zeromq4-1/blob/master/src/zmq.cpp
On 27 Jan 2016 02:59, "Justin Karneges"  wrote:

> Also beware that inproc sockets are broken with ZMQ_FD.
> https://github.com/zeromq/libzmq/issues/1434
>
> If you decide to work on level-triggered FDs, you might ensure that it
> works for all socket types.
>
> On Tue, Jan 26, 2016, at 09:29 AM, Doron Somech wrote:
>
> I will give it some time in the ZeroMQ hackathon :-)
>
> On Tue, Jan 26, 2016 at 5:51 PM, MinRK  wrote:
>
>
>
> On Fri, Jan 22, 2016 at 5:30 PM, Doron Somech  wrote:
>
> the FD must be read-only, it might be possible in some OS but I won't be
> portable.
>
> Regarding the Command FD, it must be used, otherwise the Recv/Send FD
> won't work.
>
> So in your case you need to be add the event-loop both the command FD
> (which is the regular FD) and Recv/Send FD.
>
> When command FD is signaled you must call zmq_process_comands, which
> currently doesn't exist.
> When recv/send FD is signaled you can call recv/send.
> zmq_process_command it what causing the other FDs to get signaled.
>
> The bottom line, this is kind of syntactic sugar, it will be the
> equivalent of calling has_in or has_out immediately after FD is signaled
> and only then call recv/send.
>
>
>
> Gotcha, thanks for the explanation. I think this will still be a huge
> improvement.
>
>
> -MinRK
>
>
>
>
>
>
>
>
> On Fri, Jan 22, 2016 at 5:54 PM, MinRK  wrote:
>
>
>
> On Fri, Jan 22, 2016 at 4:19 PM, Doron Somech  wrote:
>
> The FD today is signalled when ever a command should be processes. What we
> can do is split it to 3 different FD:
>
> * Command FD : The one being used right now, this still must be used, when
> ever signalled call process commands (which we should expose in API).
> * Recv FD: use as level triggered to receive.
> * Send FD: use as level triggered to send.
>
> Only issue with this solution, you should include in your event loop
> minimum two FD, one for processing commands and one for send/ recv.
>
>
> I think two FDs would be fine; certainly better than what we have now. It
> would eliminate the significant problem of one signal for separate events.
> Perhaps this is a naïve question: Is it not possible to have an FD signal
> writable when the socket becomes writable and readable when the socket
> becomes readable? If they both have to be read-only FDs, that seems fine,
> as long as the signaling for send and recv are separated somehow. I'm not
> sure what users would do with the Command FD.
>
>
> -MinRK
>
>
>
> For thread safe sockets this is a little simpler as we can make one FD for
> all sockets for processing commands.
> On Jan 22, 2016 2:52 PM, "Pieter Hintjens"  wrote:
>
> Yes, the edge triggered FD in libzmq has been a constant source of
> annoyance. Maybe someone on this list knows how to fix it.
>
> On Fri, Jan 22, 2016 at 1:40 PM, MinRK  wrote:
> > Hi all,
> >
> > I've implemented yet another eventloop integration in pyzmq (asyncio,
> this
> > time), and this is only nontrivial because of the edge-triggered
> read-only
> > zmq.FD. Integrating into existing eventloops would be much easier if we
> had
> > a more traditional level-triggered FD to work with.
> >
> > Is there a technical reason why we can't add a zmq.LEVEL_FD that would
> > behave in a more conventional manner:
> >
> > - level-triggered
> > - signal write when socket is writable
> > - signal read when socket is readable
> >
> > I would work on this myself, but unfortunately I don't think I have the
> > relevant expertise.
> >
> > -MinRK
> >
> > ___
> > zeromq-dev mailing list
> > zeromq-dev@lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> *___*
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> 

Re: [zeromq-dev] ZeroMQ 4.1.4 stable is now available

[KIU Shueng Chuan]

I encountered this problem yesterday too.
It's been fixed in:
https://github.com/zeromq/zeromq4-1/commit/b9dbc5b8caea5491c7330d891e93e2791d84c906
So 4.1.4 will no longer work on WinXP.
On 16 Jan 2016 03:08, "Tom Quarendon"  wrote:

> OK, I'll try that out.
> Thanks.
>
> -Original Message-
> From: zeromq-dev-boun...@lists.zeromq.org [mailto:
> zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Pieter Hintjens
> Sent: 15 January 2016 17:37
> To: ZeroMQ development list 
> Subject: Re: [zeromq-dev] ZeroMQ 4.1.4 stable is now available
>
> Can you try on libzmq master? I fixed a number of issues in libzmq for
> Windows in December.
>
> On Fri, Jan 15, 2016 at 4:12 PM, Tom Quarendon <
> tom.quaren...@teamwpc.co.uk> wrote:
> > I'm having trouble compiling this on Windows.
> > I'm getting "if_nametoindex : identifier not found" from tcp_address.cpp
> line 440.
> > This is with VS2012 or VS2013, and indeed VS2015 (well actually I'm
> using 2015, but with those compiler toolsets).
> > The function appears to be available on Windows,
> >
> > This seems to be a change in 4.1.4. I've been using 4.1.3 previously
> without issue, compiling it myself.
> > (Note that I'm actually trying to cure issue #1144, which as far as I
> can see still exists in 4.1.3).
> >
> > Any ideas?
> > Thanks.
> >
> > -Original Message-
> > From: zeromq-dev-boun...@lists.zeromq.org
> > [mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Pieter
> > Hintjens
> > Sent: 18 December 2015 08:40
> > To: zeromq-dev@lists.zeromq.org
> > Subject: [zeromq-dev] ZeroMQ 4.1.4 stable is now available
> >
> > Hi all,
> >
> > There's a new stable release of libzmq on line now.
> >
> > Changes:
> >
> > * Fixed #1315 - socket monitor hangs if bind/setsockopt failed.
> > * Fixed #1399 - assertion failure in tcp.cpp after network reconnect.
> > * Fixed #1632 - build failure using latest libsodium.
> > * Fixed #1644 - assertion failure in msg.cpp:390 on STREAM sockets.
> > * Fixed #1661 - does not handle IPv6 link local addresses.
> >
> > At http://zeromq.org/intro:get-the-software as usual.
> >
> > -Pieter
> > ___
> > zeromq-dev mailing list
> > zeromq-dev@lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> > ___
> > zeromq-dev mailing list
> > zeromq-dev@lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] 64bit Issue discovered with zmq_getsockopt(..., ZMQ_RCVMORE, ...

[KIU Shueng Chuan]

The type of that socket option was changed to "int" since version 3.2.x

Rather than extracting the more flag using getsockopt, you can also do:

int more = zmq_msg_more ();
On 13 Dec 2015 03:01, "Jeff Shanab"  wrote:

> Using ZMQ 4.0.4 on windows 64bit in Visual studio with a simple REQ-REP
> with multipart messages.
>
> All examples and guide show code like
>
> int64_t more;
> size_t more_size = sizeof more;
> do {
>  /* Create an empty ØMQ message to hold the message part */
>  zmq_msg_t part;
>  int rc = zmq_msg_init ();
>  assert (rc == 0);
>  /* Block until a message is available to be received from socket */
>  rc = zmq_msg_recv (, socket, 0);
>  assert (rc != -1);
>  /* Determine if more message parts are to follow */
>  rc = zmq_getsockopt (socket, ZMQ_RCVMORE, , _size);
>  assert (rc == 0);
>  zmq_msg_close (); } while (more);
>
>
> I found that the value of more is a ridiculously large negative integer
> and fails to detect the last frame unless I initialize it to zero.
>
> (note Debug in GCC zeros memory, Visual Studio does not.)
>
> Inspection of the value in hex may reveal a 64 bit porting problem,
>
> The values is 0x
>
> Which is setting it to a int const without the ull specifier 
>
> Anyway I change "int64_t more;" to "int64_t more = 0;" and operation is 
> restored.
>
>
>
>
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_msg_recv not blocking

[KIU Shueng Chuan]

Note that zmq_msg_recv() returns >= 0 on success.
On 23 Nov 2015 19:54, "Jeff Shanab"  wrote:

> I have used a previous version of zmq with google protocol buffers a few
> years ago but am trying a simpler project with the newest zmq and have run
> into a problem.
>
> Imagine the weather server and client converted to the multipart message
> code in the guide. But when I call zmq_msg_recv(, subscriber,0), it
> does not block!
>
> do {
> /* Create an empty ØMQ message to hold the message part */
> zmq_msg_t part;
> int rc = zmq_msg_init ();
> assert (rc == 0);
> /* Block until a message is available to be received from socket */
> rc = zmq_msg_recv (, subscriber,0);
>
> if (rc != 0)
> {
> std::cout << "First msg Recv error :" << zmq_errno() <<
> zmq_strerror (zmq_errno()) << std::endl;
>
> boost::this_thread::sleep(boost::posix_time::milliseconds(250));
> continue;
> };
>
> rc = zmq_getsockopt (subscriber, ZMQ_RCVMORE, , _size);
> assert (rc == 0);
>
> 
>
> With this code, I get
>
> First msg Recv error :35Resource temporarily unavailable
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed out
> First msg Recv error :60Operation timed o ...
>
> This is on OSX. I am going to try windows and linux after work tonight.
>
> Are there any known issues with 4.1.3, zmq_msg_recv and OSX ?
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_poll when socket is closed

[KIU Shueng Chuan]

I didn't run your example but isn't the zmq_poll() call going to return
immediately before the 5 seconds are up?

POLLOUT indicates writability which is true for zmq sockets until the hwm
is reached.
On 13 Oct 2015 23:50, "Auer, Jens"  wrote:

> Hi,
>
> what is the semantics of zmq_poll when the remote side of the socket is
> closed? My sample program
> int main()
> {
> zmq::context_t ctx;
> zmq::socket_t s1{ctx, ZMQ_PAIR};
> zmq::socket_t s2{ctx, ZMQ_PAIR};
>
> s1.bind("inproc://1");
> s2.connect("inproc://1");
>
> auto t = std::thread( [&]() {
> std::this_thread::sleep_for( std::chrono::seconds(5) );
> s1.close();
> });
>
> zmq::pollitem_t pollitems[] = { {static_cast(s2), -1,
> ZMQ_POLLIN + ZMQ_POLLERR + ZMQ_POLLOUT, 0} };
>
> auto rc = zmq::poll(pollitems, 1);
>
> std::cout << rc << std::endl;
> std::cout << pollitems[0].revents;
>
> t.join();
>
> return 0;
> }
>
> prints 2 which is ZMQ_POLLOUT. However, I cannot see this in the
> documentation.
>
> Best wishes,
>   Jens
>
> --
> *Jens Auer *| CGI | Software-Engineer
> CGI (Germany) GmbH & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
> T: +49 6151 36860 154
> *jens.a...@cgi.com* 
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter
> *de.cgi.com/pflichtangaben* .
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zeroMQ version with ZMQ_STREAM_NOTIFY

[KIU Shueng Chuan]

The page you referenced says:

a) ZMQ_STREAM sockets now return an empty message upon client
connect and/or disconnect.
b) This no longer happens unless you set the ZMQ_STREAM_NOTIFY option on
the socket.

"a" is correct going from 4.0.x to 4.1.x so there shouldn't have been a
strikeout.
"b" *was* the case in 4.2 master but not any more.
On Sep 24, 2015 5:53 PM, "Auer, Jens" <jens.a...@cgi.com> wrote:

> That would explain my confusion, because the page
> http://zeromq.org/docs:4-1-upgrade gave me the impression that it has
> been introduced in 4.1.
>
>
>
>
>
> *--*
>
> *Dr. Jens Auer *| CGI | Software Engineer
>
> CGI Deutschland Ltd. & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
>
> T: +49 6151 36860 154
>
> jens.a...@cgi.com
>
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
> unter de.cgi.com/pflichtangaben.
>
>
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
>
>
>
> *From:* zeromq-dev-boun...@lists.zeromq.org [mailto:
> zeromq-dev-boun...@lists.zeromq.org] *On Behalf Of *KIU Shueng Chuan
> *Sent:* 24 September 2015 11:17
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] zeroMQ version with ZMQ_STREAM_NOTIFY
>
>
>
> It appeared in 4.2.x
>
> The summary here may be useful:
> https://github.com/zeromq/libzmq/pull/1487
>
> On 24 Sep 2015 16:38, "Auer, Jens" <jens.a...@cgi.com> wrote:
>
> Hi,
>
> I am writing an application which uses ZMQ_STREAM sockets to provide an
> interface to non-zeromq applications on raw sockets. I know that there has
> been a change in the semantics of ZMQ_STREAM and a new option 
> ZMQ_STREAM_NOTIFY
> has been introduced. I want to check the version in my code and set the
> option if necessary. Which version introduced the option? I guess it is
> 4.1.2.
>
> Cheers,
>   Jens
>
>
>
>
>
> *--*
>
> *Dr. Jens Auer *| CGI | Software Engineer
>
> CGI Deutschland Ltd. & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
>
> T: +49 6151 36860 154
>
> jens.a...@cgi.com
>
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
> unter de.cgi.com/pflichtangaben.
>
>
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
>
>
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zeroMQ version with ZMQ_STREAM_NOTIFY

[KIU Shueng Chuan]

It appeared in 4.2.x

The summary here may be useful:
https://github.com/zeromq/libzmq/pull/1487
On 24 Sep 2015 16:38, "Auer, Jens"  wrote:

> Hi,
>
> I am writing an application which uses ZMQ_STREAM sockets to provide an
> interface to non-zeromq applications on raw sockets. I know that there
> has been a change in the semantics of ZMQ_STREAM and a new option 
> ZMQ_STREAM_NOTIFY
> has been introduced. I want to check the version in my code and set the
> option if necessary. Which version introduced the option? I guess it is
> 4.1.2.
>
> Cheers,
>   Jens
>
>
>
>
>
> *--*
>
> *Dr. Jens Auer *| CGI | Software Engineer
>
> CGI Deutschland Ltd. & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
>
> T: +49 6151 36860 154
>
> jens.a...@cgi.com
>
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
> unter de.cgi.com/pflichtangaben.
>
>
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
>
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Disable resend of Publisher after reconnection

[KIU Shueng Chuan]

TCP connect follows a exponential backoff. You have to wait a while after
replugging your cable. The a while depends on how long your cable was
unplugged.

In libzmq master, there is an option ZMQ_CONNECT_TIMEOUT to reduce this
long wait.
On 27 Aug 2015 21:29, Bachmair Florian - flexSolution GmbH 
florian.bachm...@flexsolution.eu wrote:

 I have done this:
 I do not get the packages, but a reconnect is also not established...


 int alive = 1, alive_idle = 1, alive_probes = 1, alive_intvl = 1;
 int rc = zmq_setsockopt(publisher, ZMQ_TCP_KEEPALIVE, alive, sizeof
 alive);
 assert(rc == 0);
 rc = zmq_setsockopt(publisher, ZMQ_TCP_KEEPALIVE_IDLE, alive_idle, sizeof
 alive_idle);
 assert(rc == 0);
 rc = zmq_setsockopt(publisher, ZMQ_TCP_KEEPALIVE_CNT, alive_probes,
 sizeof alive_probes);
 assert(rc == 0);
 rc = zmq_setsockopt(publisher, ZMQ_TCP_KEEPALIVE_INTVL, alive_intvl,
 sizeof alive_intvl);
 assert(rc == 0);

 Am 27.08.2015 um 15:13 schrieb KIU Shueng Chuan:

 You could use the tcp keepalive socket options to make the tcp connection
 drop faster once you have unplugged your cable. That should cause the
 queued packets to be dropped.

 Or your subscriber could drain the socket of messages by reading it until
 it would block and only process the last one.
 On Aug 27, 2015 20:42, Bachmair Florian - flexSolution GmbH 
 florian.bachm...@flexsolution.eu wrote:

 ok, thank you!

 I have to think about that, because I want to use curve as well, and
 therefore I would need tcp...

 Am 27.08.2015 um 14:12 schrieb Sergey Zinov:

 Seems like that should be the case. TCP is a reliable protocol, it
 buffers all data till it acknowledged by receiver and resend when needed.
 Switch to the UDP if don't need that reliability.

 On 27.08.2015 14:07, Bachmair Florian - flexSolution GmbH wrote:

 This does not work either :/

 I'm unsing PUB/SUB via tcp.

 I have captured the tcp connection with tcpdump and I saw that after I
 unplug the cable I get tcp retransmission packets.

 May this be more a behaviour of tcp itself than of zmq?


 Am 27.08.2015 um 13:39 schrieb Sergey Zinov:

 Hi,

 Try to set ZMQ_SNDHWM to 1. By the way which transport are you using?

 On 27.08.2015 11:52, Bachmair Florian - flexSolution GmbH wrote:

 I tried it, but it does not work!

 Don't know If I get that right, but even If I set ZMQ_RECONNECT_IV to
 -1 my PUB and SUB Sockets are reconnecting to each other after I replug
 the cable

 Am 27.08.2015 um 10:59 schrieb Doron Somech:

 I'm not sure, but try to set  ZMQ_IMMEDIATE option.
 On Aug 27, 2015 11:38 AM, Bachmair Florian - flexSolution GmbH 
 florian.bachm...@flexsolution.eu wrote:

 Hello!

 Is there a opportunity to tell the publisher not to resend messages?

 I have two stations A and B

 A publishes all the time
 B receives it

 A-PUB 10
 B-SUB 10
 A-PUB 20
 B-SUB 20
 A-PUB 30
 B-SUB 30
 A-PUB 40
 B-SUB 40
 ---unplug cable
 A-PUB 50
 A-PUB 60
 A-PUB 70
 ---plug cable again
 B-SUB 50
 B-SUB 60
 B-SUB 70
 


 B does some routines after it gets those values. But I'm not interested
 in 50 and 60 any more, I just want the 70 after reconnection.
 I do Heartbeating so I detect connection lost on my own and I can
 request the last value as well. So is this possible to disable the
 resend mechanism?

 Thx
 Florian
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev




 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev



 --
 Sergey Zinov

 Wissenschaftlicher Mitarbeiter
 Fachbereich Elektrotechnik, Maschinenbau und Wirtschaftsingenieurwesen
 Hochschule Anhalt
 /
 Research Assistant
 Department of Electrical, Mechanical and Industrial Engineering
 Anhalt University of Applied Sciences

 Phone: +4915738145172



 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev




 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev



 --
 Sergey Zinov

 Wissenschaftlicher Mitarbeiter
 Fachbereich Elektrotechnik, Maschinenbau und Wirtschaftsingenieurwesen
 Hochschule Anhalt
 /
 Research Assistant
 Department of Electrical, Mechanical and Industrial Engineering
 Anhalt University of Applied Sciences

 Phone: +4915738145172



 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Disable resend of Publisher after reconnection

[KIU Shueng Chuan]

You could use the tcp keepalive socket options to make the tcp connection
drop faster once you have unplugged your cable. That should cause the
queued packets to be dropped.

Or your subscriber could drain the socket of messages by reading it until
it would block and only process the last one.
On Aug 27, 2015 20:42, Bachmair Florian - flexSolution GmbH 
florian.bachm...@flexsolution.eu wrote:

 ok, thank you!

 I have to think about that, because I want to use curve as well, and
 therefore I would need tcp...

 Am 27.08.2015 um 14:12 schrieb Sergey Zinov:

 Seems like that should be the case. TCP is a reliable protocol, it buffers
 all data till it acknowledged by receiver and resend when needed. Switch to
 the UDP if don't need that reliability.

 On 27.08.2015 14:07, Bachmair Florian - flexSolution GmbH wrote:

 This does not work either :/

 I'm unsing PUB/SUB via tcp.

 I have captured the tcp connection with tcpdump and I saw that after I
 unplug the cable I get tcp retransmission packets.

 May this be more a behaviour of tcp itself than of zmq?


 Am 27.08.2015 um 13:39 schrieb Sergey Zinov:

 Hi,

 Try to set ZMQ_SNDHWM to 1. By the way which transport are you using?

 On 27.08.2015 11:52, Bachmair Florian - flexSolution GmbH wrote:

 I tried it, but it does not work!

 Don't know If I get that right, but even If I set ZMQ_RECONNECT_IV to
 -1 my PUB and SUB Sockets are reconnecting to each other after I replug
 the cable

 Am 27.08.2015 um 10:59 schrieb Doron Somech:

 I'm not sure, but try to set  ZMQ_IMMEDIATE option.
 On Aug 27, 2015 11:38 AM, Bachmair Florian - flexSolution GmbH 
 florian.bachm...@flexsolution.eu wrote:

 Hello!

 Is there a opportunity to tell the publisher not to resend messages?

 I have two stations A and B

 A publishes all the time
 B receives it

 A-PUB 10
 B-SUB 10
 A-PUB 20
 B-SUB 20
 A-PUB 30
 B-SUB 30
 A-PUB 40
 B-SUB 40
 ---unplug cable
 A-PUB 50
 A-PUB 60
 A-PUB 70
 ---plug cable again
 B-SUB 50
 B-SUB 60
 B-SUB 70
 


 B does some routines after it gets those values. But I'm not interested
 in 50 and 60 any more, I just want the 70 after reconnection.
 I do Heartbeating so I detect connection lost on my own and I can
 request the last value as well. So is this possible to disable the
 resend mechanism?

 Thx
 Florian
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev




 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev



 --
 Sergey Zinov

 Wissenschaftlicher Mitarbeiter
 Fachbereich Elektrotechnik, Maschinenbau und Wirtschaftsingenieurwesen
 Hochschule Anhalt
 /
 Research Assistant
 Department of Electrical, Mechanical and Industrial Engineering
 Anhalt University of Applied Sciences

 Phone: +4915738145172



 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev




 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev



 --
 Sergey Zinov

 Wissenschaftlicher Mitarbeiter
 Fachbereich Elektrotechnik, Maschinenbau und Wirtschaftsingenieurwesen
 Hochschule Anhalt
 /
 Research Assistant
 Department of Electrical, Mechanical and Industrial Engineering
 Anhalt University of Applied Sciences

 Phone: +4915738145172



 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Building using ZeroMQ with Visual Studio

[KIU Shueng Chuan]

Could you try editing src/config.hpp, change the line signaler_port from
5905 to 0 and recompile?
On 20 Aug 2015 14:13, Matt Bolger matt.bolger...@gmail.com wrote:

 Some further digging in case this mean something to someone:

 I *think* the issue can be traced back to zmq::signaler_t::make_fdpair() -
 line 447

 rc = bind (listener, (const struct sockaddr*) addr, sizeof addr);

 this fails with error 10030 (WSAEACCES)


 https://msdn.microsoft.com/en-us/library/windows/desktop/ms740668(v=vs.85).aspx#WSAEACCES

 So make_fdpair() in the signaler_t constructor never produces anything
 other than invalid file descriptors. This seems to go through without
 anything saying this is an issue until these descriptors are passed to
 select() for the first time.


 On Thu, Aug 20, 2015 at 4:00 AM, Bob Clarke optiongu...@gmail.com wrote:

 I have two projects using 0MQ. I built 4.1.2 from source but, frankly,
 didn't run all the test programs. My apps are simple (PUB/SUB, REQ/REP,
 PAIR and PUSH/PULL) and have only encountered one problem with PAIR
 sockets, which may be of my own doing.

 Bob

 On Tue, Aug 18, 2015 at 6:08 PM, Matt Bolger matt.bolger...@gmail.com
 wrote:

 Hi All,



 Anyone out there successfully building/using ZeroMQ under Windows (MSVC)?



 I'm trying to build/use the current master in GitHub (
 https://github.com/zeromq/libzmq) which has Visual Studio projects as
 well as CMake files. I've built the CMake project (after fixing an issue in
 the MSVC specific stuff - https://github.com/zeromq/libzmq/issues/1517)
 which also builds all the tests but the tests fail the same way my simple
 test program does.



 The first call to WinSock select() always results in WSAENOTSOCK and the
 app bails. I've now tried this on a 64bit Win7 and Win8 machines with both
 VS2010 and VS2013 for debug and release builds with no luck :(



 I get similar aborts when trying to build against the pre-built binaries
 on the ZeroMQ site or building my own with the included Visual Studio
 project files included in the repository.



 Maybe I'm missing something obvious but any help would be great.


 Thanks

 Matt

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] CZMQ and Server and Client Sockets

[KIU Shueng Chuan]

Is this new pollfd level triggered or edge triggered?
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] CZMQ and Server and Client Sockets

[KIU Shueng Chuan]

If the pollfd is level triggered, then we could just use the system
poll/select directly. Then check all thread safe zsocks attached to the
pollfd for ZMQ_EVENTS.

It would thus be trivial to use thread safe zsocks with other event loops.
If pollfd is level triggered, that is.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] CZMQ related travis failures

[KIU Shueng Chuan]

I think it's because zmq_pollitem_t has very recently gained a new poller
field. Perhaps that new field should be moved to the end?
On 16 Aug 2015 22:06, Brian Knox bk...@digitalocean.com wrote:

 Joe - that's what I'm thinking too.

 On Sun, Aug 16, 2015 at 9:59 AM, Joe McIlvain joe.eli@gmail.com
 wrote:

 Could be that Travis updated its software packages so that a different
 set of warning/error settings are active for the compiler.

 On Sun, Aug 16, 2015 at 5:02 AM, Brian Knox bk...@digitalocean.com
 wrote:

 CZMQ is currently failing to build on travis-ci for GoCZMQ, just wanted
 to see if anyone else has run into this.  As far as I can tell the code
 involved hasn't changed in over a year, so not sure why it's causing travis
 issues now:

 src/zbeacon.c: In function ‘zbeacon’:
 src/zbeacon.c:281:13: error: initialization makes pointer from integer 
 without a cast [-Werror]
 src/zbeacon.c:281:13: error: (near initialization for 
 ‘pollitems[1].poller’) [-Werror]
 cc1: all warnings being treated as errors
 make[1]: *** [src/src_libczmq_la-zbeacon.lo] Error 1



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] CZMQ and Server and Client Sockets

[KIU Shueng Chuan]

Hello, the new definition for zmq_pollitem_t would break code like the
following:

zmq_pollitem_t pollitems [] = {
{ zsock, 0, ZMQ_POLLIN, 0 },
{ NULL, fd, ZMQ_POLLIN, 0 }
};
On 16 Aug 2015 19:10, Doron Somech somdo...@gmail.com wrote:

 So I made a pull request which add polling on multiple sockets:

 https://github.com/zeromq/libzmq/pull/1525

 Polling on thread safe sockets is a little different then regular socket,
 take a look at the gist:

 https://gist.github.com/somdoron/902169bf115d3534bd24

 Next is making a pull request to CZMQ to use the new ability.


 On Sun, Aug 16, 2015 at 12:53 AM, Brian Knox bk...@digitalocean.com
 wrote:

 Makes sense to me!  The API is close enough to the other poller
 implementations that there's no surprises.

 Cheers!
 Brian

 On Sat, Aug 15, 2015 at 5:21 PM, Doron Somech somdo...@gmail.com wrote:

 Polling on multiple thread safe socket is a little bit different,
 because thread safe doesn't have a FD, so we need to create one for all
 thread safe sockets for each thread before calling the zmq_poll.

 So I want to make it very close to the current API, this what I have so
 far:

 https://gist.github.com/somdoron/902169bf115d3534bd24

 zmq_poller_t is actually a FD, when added to the thread safe socket the
 socket will signal it once a command is ready, multiple sockets can use the
 same poller. When signalled the zmq_poll will check all sockets for events.

 What do you think?



 On Sat, Aug 15, 2015 at 5:54 PM, Brian Knox bk...@digitalocean.com
 wrote:

 Doron - as a heads up, being able to poll multiple sockets from zpoller
 would be of great interest to me (I use zpoller in goczmq quite a bit).

 Cheers,
 Brian

 On Sat, Aug 15, 2015 at 3:48 AM, Doron Somech somdo...@gmail.com
 wrote:

 Andrew are you using CZMQ? which class do you use for multiple
 polling, zloop or zpoller?

 On Fri, Aug 14, 2015 at 10:36 PM, Andrew Simpson 
 simpsona...@yahoo.com wrote:

 this sounds really excellent!  I am building an application that
 would greatly benefit from this over a standard Router/Dealer setup.  The
 only thing that will hold me back right now is the lack of polling on
 multiple client/server sockets.  I definitely need that.

 Good stuff!



 On Friday, August 14, 2015 9:09 AM, Doron Somech somdo...@gmail.com
 wrote:



 Hi All,

 I added server and client sockets support to CZMQ, you can take a
 look at the change at the following pull request:

 https://github.com/zeromq/czmq/pull/1059

 Server socket is like router socket except you don't have an identity
 frame, each message also include routing id which is an int (vs byte
 array). So each message coming from a server socket include a routing id
 which can be retrieve by calling zframe_routing_id. When sending a 
 message
 you must set the routing id by calling zframe_set_routing_id. You can use
 zframe_send_reply with both the destination frame and the source frame
 (which include the routing id), the method copy the routing id from the
 source frame to the destination frame and then send the message.

 Client socket is same as dealer socket. Client and Server can only
 talk to each other.

 Following is a small example on how to use the new client and server
 sockets:
 https://gist.github.com/somdoron/542b74922f652d229566

 Client and server socket are thread safe (currently only support
 single frame messages but that might change, I think) so if your protocol
 is single frame you can use the server and client sockets from multiple
 threads.

 Polling on multiple client or server sockets is not supported yet.

 In the coming week I plan to also add zproto support and complete the
 polling on multiple sockets.

 Doron


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] CZMQ and Server and Client Sockets

[KIU Shueng Chuan]

Do you think it would be worthwhile to preserve ABI compatibility of the
zmq_pollitem_t structure?

Perhaps for threadsafe sockets, the user could extract the fd of the poller
object and populate the pollitem fd field himself?

e.g.
socket!=NULL  fd!=0 == threadsafe zmq socket
socket!=NULL  fd==0 == regular zmq socket
socket==NULL == regular TCP socket (or any fd on unix)


On Sun, Aug 16, 2015 at 11:06 PM, Doron Somech somdo...@gmail.com wrote:

 so, I made a pull request to fix the issue, moving field to the end of
 struct:

 https://github.com/zeromq/libzmq/pull/1526

 On Sun, Aug 16, 2015 at 5:54 PM, Doron Somech somdo...@gmail.com wrote:

 I will try to fix that...

 On Sun, Aug 16, 2015 at 5:43 PM, KIU Shueng Chuan nixch...@gmail.com
 wrote:

 Hello, the new definition for zmq_pollitem_t would break code like the
 following:

 zmq_pollitem_t pollitems [] = {
 { zsock, 0, ZMQ_POLLIN, 0 },
 { NULL, fd, ZMQ_POLLIN, 0 }
 };
 On 16 Aug 2015 19:10, Doron Somech somdo...@gmail.com wrote:

 So I made a pull request which add polling on multiple sockets:

 https://github.com/zeromq/libzmq/pull/1525

 Polling on thread safe sockets is a little different then regular
 socket, take a look at the gist:

 https://gist.github.com/somdoron/902169bf115d3534bd24

 Next is making a pull request to CZMQ to use the new ability.


 On Sun, Aug 16, 2015 at 12:53 AM, Brian Knox bk...@digitalocean.com
 wrote:

 Makes sense to me!  The API is close enough to the other poller
 implementations that there's no surprises.

 Cheers!
 Brian

 On Sat, Aug 15, 2015 at 5:21 PM, Doron Somech somdo...@gmail.com
 wrote:

 Polling on multiple thread safe socket is a little bit different,
 because thread safe doesn't have a FD, so we need to create one for all
 thread safe sockets for each thread before calling the zmq_poll.

 So I want to make it very close to the current API, this what I have
 so far:

 https://gist.github.com/somdoron/902169bf115d3534bd24

 zmq_poller_t is actually a FD, when added to the thread safe socket
 the socket will signal it once a command is ready, multiple sockets can 
 use
 the same poller. When signalled the zmq_poll will check all sockets for
 events.

 What do you think?



 On Sat, Aug 15, 2015 at 5:54 PM, Brian Knox bk...@digitalocean.com
 wrote:

 Doron - as a heads up, being able to poll multiple sockets from
 zpoller would be of great interest to me (I use zpoller in goczmq quite 
 a
 bit).

 Cheers,
 Brian

 On Sat, Aug 15, 2015 at 3:48 AM, Doron Somech somdo...@gmail.com
 wrote:

 Andrew are you using CZMQ? which class do you use for multiple
 polling, zloop or zpoller?

 On Fri, Aug 14, 2015 at 10:36 PM, Andrew Simpson 
 simpsona...@yahoo.com wrote:

 this sounds really excellent!  I am building an application that
 would greatly benefit from this over a standard Router/Dealer setup.  
 The
 only thing that will hold me back right now is the lack of polling on
 multiple client/server sockets.  I definitely need that.

 Good stuff!



 On Friday, August 14, 2015 9:09 AM, Doron Somech 
 somdo...@gmail.com wrote:



 Hi All,

 I added server and client sockets support to CZMQ, you can take a
 look at the change at the following pull request:

 https://github.com/zeromq/czmq/pull/1059

 Server socket is like router socket except you don't have an
 identity frame, each message also include routing id which is an int 
 (vs
 byte array). So each message coming from a server socket include a 
 routing
 id which can be retrieve by calling zframe_routing_id. When sending a
 message you must set the routing id by calling zframe_set_routing_id. 
 You
 can use zframe_send_reply with both the destination frame and the 
 source
 frame (which include the routing id), the method copy the routing id 
 from
 the source frame to the destination frame and then send the message.

 Client socket is same as dealer socket. Client and Server can only
 talk to each other.

 Following is a small example on how to use the new client and
 server sockets:
 https://gist.github.com/somdoron/542b74922f652d229566

 Client and server socket are thread safe (currently only support
 single frame messages but that might change, I think) so if your 
 protocol
 is single frame you can use the server and client sockets from 
 multiple
 threads.

 Polling on multiple client or server sockets is not supported yet.

 In the coming week I plan to also add zproto support and complete
 the polling on multiple sockets.

 Doron


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZeroMQ Raspberry Pi (Debian)

[KIU Shueng Chuan]

Fwiw, I recently tried zeromq 4.1.2 with raspberry pi B. How about
compiling zeromq from source and running the test suite?

One test failed intermittently but that was due to the slowness of the pi,
I think.
On 15 Aug 2015 00:00, Kelly Beard kenverybigl...@gmail.com wrote:

 I tried compiling a couple of samples from the examples and was getting
 illegal instruction at run-time.  I then tried Jame's Chapman's push/pull
 examples (I did have to comment out a call to disconnect()).  Same result.
 I'd really like to have a chance at using this because it fits a need.

 System is the latest Pi 2.  OS is Raspbian/Debian

 uname -a
 Linux qfuel-dev 3.12.22+ #691 PREEMPT Wed Jun 18 18:29:58 BST 2014 armv6l
 GNU/Linux

 I have the following packages installed

 libzmq-dbg:armhf  2.2.0+dfsg-2
 libzmq1:armhf 2.2.0+dfsg-2
 libzmq3:armhf 3.2.3+dfsg-2~bpo70+1
 libzmq3-dev:armhf 3.2.3+dfsg-2~bpo70+1

 g++ -std=c++0x zeromq_push.cpp -o zeromq_push -lzmq
 g++ -std=c++0x zeromq_pull.cpp -o zeromq_pul -lzmq

 Output from strace -o output.txt zeromq_pull

 execve(/home/kelly/C++/zeromq_pull, [zeromq_pull], [/* 17 vars */]) = 0
 brk(0)  = 0x1ebe000
 uname({sys=Linux, node=qfuel-dev, ...}) = 0
 access(/etc/ld.so.nohwcap, F_OK)  = -1 ENOENT (No such file or
 directory)
 mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
 = 0xb6f2b000
 access(/etc/ld.so.preload, R_OK)  = 0
 open(/etc/ld.so.preload, O_RDONLY|O_CLOEXEC) = 3
 fstat64(3, {st_mode=S_IFREG|0644, st_size=44, ...}) = 0
 mmap2(NULL, 44, PROT_READ|PROT_WRITE, MAP_PRIVATE, 3, 0) = 0xb6f2a000
 close(3)= 0
 open(/usr/lib/arm-linux-gnueabihf/libcofi_rpi.so, O_RDONLY|O_CLOEXEC) = 3
 read(3,
 \177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\270\4\0\0004\0\0\0...,
 512) = 512
 lseek(3, 7276, SEEK_SET)= 7276
 read(3,
 \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0...,
 1080) = 1080
 lseek(3, 7001, SEEK_SET)= 7001
 read(3,
 A.\0\0\0aeabi\0\1$\0\0\0\0056\0\6\6\10\1\t\1\n\2\22\4\24\1\25..., 47) = 47
 fstat64(3, {st_mode=S_IFREG|0755, st_size=10170, ...}) = 0
 mmap2(NULL, 39740, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
 0xb6f2
 mprotect(0xb6f22000, 28672, PROT_NONE)  = 0
 mmap2(0xb6f29000, 4096, PROT_READ|PROT_WRITE,
 MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb6f29000
 close(3)= 0
 munmap(0xb6f2a000, 44)  = 0
 open(/etc/ld.so.cache, O_RDONLY|O_CLOEXEC) = 3
 fstat64(3, {st_mode=S_IFREG|0644, st_size=49745, ...}) = 0
 mmap2(NULL, 49745, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb6ef2000
 close(3)= 0
 access(/etc/ld.so.nohwcap, F_OK)  = -1 ENOENT (No such file or
 directory)
 open(/usr/lib/arm-linux-gnueabihf/libzmq.so.3, O_RDONLY|O_CLOEXEC) = 3
 read(3,
 \177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\0\206\0\0004\0\0\0...,
 512) = 512
 lseek(3, 181420, SEEK_SET)  = 181420
 read(3,
 \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0...,
 1160) = 1160
 lseek(3, 181056, SEEK_SET)  = 181056
 read(3, A4\0\0\0aeabi\0\1*\0\0\0\0057-A\0\6\n\7A\10\1\t\2\n\4\22..., 53)
 = 53
 fstat64(3, {st_mode=S_IFREG|0644, st_size=182580, ...}) = 0
 mmap2(NULL, 213848, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0)
 = 0xb6ebd000
 mprotect(0xb6ee7000, 28672, PROT_NONE)  = 0
 mmap2(0xb6eee000, 16384, PROT_READ|PROT_WRITE,
 MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x29) = 0xb6eee000
 close(3)= 0
 access(/etc/ld.so.nohwcap, F_OK)  = -1 ENOENT (No such file or
 directory)
 open(/usr/lib/arm-linux-gnueabihf/libstdc++.so.6, O_RDONLY|O_CLOEXEC) = 3
 read(3,
 \177ELF\1\1\1\3\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0xk\4\0004\0\0\0..., 512) =
 512
 lseek(3, 808332, SEEK_SET)  = 808332
 read(3,
 \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0...,
 1280) = 1280
 lseek(3, 807760, SEEK_SET)  = 807760
 read(3,
 A.\0\0\0aeabi\0\1$\0\0\0\0056\0\6\6\10\1\t\1\n\2\22\4\24\1\25..., 47) = 47
 fstat64(3, {st_mode=S_IFREG|0644, st_size=809612, ...}) = 0
 mmap2(NULL, 900808, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0)
 = 0xb6de1000
 mprotect(0xb6ea2000, 61440, PROT_NONE)  = 0
 mmap2(0xb6eb1000, 24576, PROT_READ|PROT_WRITE,
 MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc0) = 0xb6eb1000
 mmap2(0xb6eb7000, 24264, PROT_READ|PROT_WRITE,
 MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb6eb7000
 close(3)= 0
 access(/etc/ld.so.nohwcap, F_OK)  = -1 ENOENT (No such file or
 directory)
 open(/lib/arm-linux-gnueabihf/libm.so.6, O_RDONLY|O_CLOEXEC) = 3
 read(3,
 \177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0p\0\0004\0\0\0..., 512) =
 512
 lseek(3, 434644, SEEK_SET)  = 434644
 read(3,
 \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0...,
 1160) = 1160
 lseek(3, 434312, SEEK_SET)  = 434312
 

Re: [zeromq-dev] cannot send big chunk via PUSH/PULL by pyzmq

[KIU Shueng Chuan]

Each element of the list is sent as a zeromq message frame. Each (short)
frame has a header overhead of 2 bytes. track=True also causes more
overhead per frame.

['0' for x in range(N)] is sent as N message frames.
['0'*N] is sent as 1 message frame. (Did you want to do this instead?)


On Tue, May 12, 2015 at 12:40 PM, Li Ma skywalker.n...@gmail.com wrote:

 I just created a list filled by 2 million zero and sended it out. Any
 problems?

 On Sun, Apr 26, 2015 at 8:27 PM, KIU Shueng Chuan nixch...@gmail.com
 wrote:
  I read your code as sending a multipart message composed of 2 million
 parts
  each of size 1 byte. Is that right?
 
  On Apr 26, 2015 16:50, Li Ma skywalker.n...@gmail.com wrote:
 
  On Thu, Apr 23, 2015 at 7:50 PM, Pieter Hintjens p...@imatix.com wrote:
   - please provide your test case as a minimal program
 
  Sure.
 
  Server part:
  context = zmq.Context(1)
  socket = context.socket(zmq.PULL)
  socket.bind(tcp://*:)
  while True:
  #  Wait for next request from client
  message = socket.recv_multipart()
  print hello: %s, time: %s\n % (len(message), datetime.utcnow())
 
  Client part:
  context = zmq.Context(1)
 
  def send(size):
  buffer = []
  for i in range(size):
  buffer.append('0')
 
  socket = context.socket(zmq.PUSH)
  socket.connect(tcp://localhost:)
  print(Sending request %s ... % datetime.utcnow())
  tracker = socket.send_multipart(buffer, copy=False, track=True)
  while(True):
  if tracker.done:
  socket.close(-1)
  return '\nDONE, %s\n' % datetime.utcnow()
  else:
  time.sleep(.01)
 
  for i in range(3):
  print send(200)
 
   - don't do any TCP tuning unless you know why you're doing it
  
 
  I use the distro-default configuration, but it doesn't work.
 
   On Thu, Apr 23, 2015 at 8:35 AM, Li Ma skywalker.n...@gmail.com
 wrote:
   Hi all,
  
   I wanna test a simple application using PUSH/PULL to send a big chunk
   of data.
   The application is straightforward.
  
   Server part is to open a PULL socket while client part is sending
 1MB
   message per time via PUSH.
  
   The socket is listened on the localhost only.
  
   Any messages less than 200KB are working well, but messages greater
   than 500KB is not working. The socket is stuck after handshake
 without
   any exceptions.
  
   By the way, I set a large size (1MB) for SND/RCV BUF and tcp window
   scaling is set to 1.
  
   I'd appreciate any hints to make it work.
  
   --
  
   Li Ma (Nick)
   Email: skywalker.n...@gmail.com
   ___
   zeromq-dev mailing list
   zeromq-dev@lists.zeromq.org
   http://lists.zeromq.org/mailman/listinfo/zeromq-dev
   ___
   zeromq-dev mailing list
   zeromq-dev@lists.zeromq.org
   http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 
 
  --
 
  Li Ma (Nick)
  Email: skywalker.n...@gmail.com
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 



 --

 Li Ma (Nick)
 Email: skywalker.n...@gmail.com
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZeroMQ with Python and a basic task

[KIU Shueng Chuan]

Isn't the system python in redhat 7 python 2.7? Are you installing
libraries in python 2.7 and trying to run your script in python 2.6.6?

yum install python-devel would be installing for python 2.7 for instance.

If you don't need to use zeromq 4.x, redhat's repository should include
both pyzmq (for python 2.7) and zeromq 3.2.x.
On 8 Mar 2015 07:28, Michael Cuggy mcu...@gmail.com wrote:

 I am trying to run the Hello World python program taken from this link:

 http://zguide.zeromq.org/py:hwclient

 I cannot get it to run.  I get an error about the import zeromq line.
 I lifted the code verbatim.  Maybe I installed ZeroMQ incorrectly?

 I am using Python 2.6.6 and RedHat Linux version 7. I installed ZeroMQ
 by I running these commands as root:

 tar -xvf zeromq-4.0.5.tar.gz

 cd zeromq-4.0.5

 ./configure

 make

 make install

 ldconfig

 yum install python-devel

 easy_install pyzmq

 When I try to run Hello.py I get this error:

 Traceback (most recent call last): File Hello.py, line 7, in import
 zmq ImportError: No module named zmq

 How do I get the module zmq installed? pip install pyzmq indicates
 that the requirement has already been satisfied. python -v foo.py
 didn't show me anything meaningful.
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZMQ vs SPI: FD shenanigans

[KIU Shueng Chuan]

Are you sure that the ioctl call failed? You are printing out errno without
having checked the return code of the ioctl call.
 Hello,

Am 20.01.2015 22:02, schrieb Arnaud Kapp:
-Snipp-
 On Tue, Jan 20, 2015 at 11:32 AM, Olaf Mandel o.man...@menlosystems.com
 wrote:
-Snipp-
 Good point, my machine and version numbers are:

 CPU:   Freescale i.MX537 (Cortex-A8, NEON)
 GCC:   4.8.1, cross-compiling
 Linux: 3.10.28 + many platform patches
 ZMQ:   zeromq/libzmq.git @ be23e699c

-Snipp-
 I see two things to try:
   * Try with a more stable version of libzmq -
 https://github.com/zeromq/zeromq4-1
   * Upgrading to a more recent kernel and to gcc 4.9 if those are
available.


this took a while, but I have now recompiled the complete system
(including kernel) with GCC 4.9.2. I will not be able to go for a newer
kernel: that has quite a few patches and porting those is a somewhat
daunting task for me.

The behaviour seems the same with the following combinations of ZeroMQ
and GCC:

ZeroMQ w/ GCC:
4.0.5  w/ 4.8.1
HEAD   w/ 4.8.1
HEAD   w/ 4.9.2
4.1.0  w/ 4.9.2

I want to check a bit why C and C++ behave differently tomorrow. Any
further suggestions anyone could make?

Best regards,
Olaf Mandel
--
Olaf Mandel
phone: +49-89-189166-250
fax:   +49-89-189166-111
Menlo Systems GmbH
Am Klopferspitz 19a, D-82152 Martinsried
Amtsgericht München HRB 138145
Geschäftsführung: Dr Michael Mei, Dr Ronald Holzwarth
USt-IdNr. DE217772017, St.-Nr. 14316170324


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] CZMQ selftest build fails.

[KIU Shueng Chuan]

By the way, the reason such defines appear is to attempt to distinguish
between mingw32 and mingw64. Both define __MINGW32__ but unfortunately
their implementation differs slightly usually in the header files.

Sometimes people patch the code to work with mingw64 but that breaks
mingw32 (and vice versa.)

On 7 Jan 2015 19:28, Pieter Hintjens p...@imatix.com wrote:

 I believe this condition is bogus:

 || (defined (__MINGW32__)  defined (__IS_64BIT__)) \

 Try removing it.

 On Wed, Jan 7, 2015 at 11:48 AM, Riskybiz riskybizl...@live.com wrote:
  Finally I'm able to get CZMQ to build on Windows 7 using the Visual
Studio
  2013 solution.  However there are a couple of issues;
 
 
 
  Without this section commented out of czmq_prelude.h then the build gave
  many errors relating to int8_t redefinition. N.B. This computer has
Mingw
  installed if that makes any difference.
 
 
 
  #   if ((!defined (__MINGW32__) \
 
  || (defined (__MINGW32__)  defined (__IS_64BIT__))) \
 
   !defined (ZMQ_DEFINED_STDINT))
 
  typedef __int8 int8_t;
 
  typedef __int16 int16_t;
 
  typedef __int32 int32_t;
 
  typedef __int64 int64_t;
 
  typedef unsigned __int8 uint8_t;
 
  typedef unsigned __int16 uint16_t;
 
  typedef unsigned __int32 uint32_t;
 
  typedef unsigned __int64 uint64_t;
 
  #   endif
 
 
 
  The other issue is that the czmq_selftest build fails like so:
 
 
 
  -- Build started: Project: czmq_selftest, Configuration: ReleaseDEXE
  Win32 --
 
  1  ConfigurationType : Application
 
  1  Configuration : ReleaseDEXE
 
  1  PlatformToolset   : v120
 
  1  TargetPath:
 
C:\zeromq4-1\czmq\builds\msvc\vs2013\czmq_selftest\..\..\..\..\bin\Win32\Release\v120\dynamic\czmq_selftest.exe
 
  1  Linkage-czmq  : dynamic
 
  1  Linkage-libzmq: dynamic
 
  1  Linkage-libsodium : dynamic
 
  1czmq_selftest.obj : error LNK2001: unresolved external symbol
  __imp__zsys_allocs
 
 
1C:\zeromq4-1\czmq\builds\msvc\vs2013\czmq_selftest\..\..\..\..\bin\Win32\Release\v120\dynamic\czmq_selftest.exe
  : fatal error LNK1120: 1 unresolved externals
 
  == Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped
==
 
 
 
  Does anyone please know how to fix this?
 
 
 
  Thanks,
 
 
 
  Riskybiz.
 
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] How to use ZeroMQ context with zauth?

[KIU Shueng Chuan]

zctx_t *ctx_ptr =
zctx_shadow_zmq_ctx ((void *)context);

should do what you want.
On 16 Dec 2014 06:29, Check Peck comptechge...@gmail.com wrote:

 I need to use ZeroMQ context from C++ binding to C way of ZeroMQ context
 so when I do it like this, I get a Segmentation fault

 zctx_t* ctx_ptr = static_castzctx_t*((void*)context);
 m_auth = zauth_new (ctx_ptr);

 I am using Strawhouse security pattern by following this link -
 http://hintjens.com/blog:49


 On Mon, Dec 15, 2014 at 1:20 PM, Check Peck comptechge...@gmail.com
 wrote:

 I am using C++ bindings for ZeroMQ and I have declared context as -

 zmq::context_t context
 zauth_t *m_auth;

 Now I am trying to use Strawhouse security pattern in my application and
 for which I need to czmq which is a C binding not C++ binding.

 Now when I try to initialize zauth like below, it always fail during
 compilation and I am not sure what's wrong?

 m_auth = zauth_new (*context);

 Below is the compilation error -

 In file included from
 /usr/local/include/boost/detail/container_fwd.hpp:98:0,
  from
 /usr/local/include/boost/functional/hash/extensions.hpp:17,
  from
 /usr/local/include/boost/functional/hash/hash.hpp:529,
  from /usr/local/include/boost/functional/hash.hpp:6,
  from
 /usr/local/include/boost/regex/v4/basic_regex.hpp:23,
  from /usr/local/include/boost/regex/v4/regex.hpp:67,
  from /usr/local/include/boost/regex.hpp:31,
  from /home/david/ZeroMQTester/test_queue.cpp:7:
 /usr/include/c++/4.7/complex:379:5: note: templateclass _Tp
 std::complex_Tp std::operator*(const std::complex_Tp, const
 std::complex_Tp)
 /usr/include/c++/4.7/complex:379:5: note:   template argument
 deduction/substitution failed:
 /home/david/ZeroMQTester/test_queue.cpp:126:26: note:   âzmq::context_tâ
 is not derived from âconst std::complex_Tpâ


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] How to convert socket_t* to void *zocket so that I can integrate Strawhouse czmq to my c++ ZeroMQ project?

[KIU Shueng Chuan]

Try:
zsocket_set_zap_domain ((void*)(*m_socket_ptr), global);

Both the zmq::context_t and zmq::socket_t implement operator void*() which
return the underlying libzmq context and socket void* handles.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] EOF on file transfer

[KIU Shueng Chuan]

range (offsets+1) should just be range (offsets).
filename.seek (offset) should be filename.seek (offset * chunksize)

Try doing a local file copy with your loop first. The problem in your code
is not with zeromq.
On 10 Dec 2014 00:47, Tiago Hillebrandt tiagohillebra...@ubuntu.com
wrote:

 Hey guys,

 I am using the below Python code to transfer big files between a server
 and a client.

 *Implementation to send file, server*

 CHUNK_SIZE = 25

 message = pair.recv() # message is the path to the file

 filename = open(message, 'rb')
 filesize = os.path.getsize(message)

 offsets = (int(ceil(filesize / CHUNK_SIZE)), 0)[filesize = CHUNK_SIZE]

 for offset in range(offsets + 1):
 filename.seek(offset)

 chunksize = CHUNK_SIZE

 if offset == offsets:
 chunksize = filesize - (CHUNK_SIZE * (offset - 1)) # calculate 
 the size of the last chunk

 data = filename.read(chunksize)

 pair.send(data)

 pair.send(b'')

 *Implementation to receive file, client*

 while True:
 data = pairs.recv()

 if data is not '':
 target.write(data)
 else:
 break

 However, after transfer a big file using this implementation, for some
 reason an extra data is being added at end of the file:

 *File server side*

 $ stat file.zip
   File: `file.zip'
   Size: 1503656416  Blocks: 2936840IO Block: 4096   regular file

 *Client side*

 $ stat file.zip
   File: `file.zip'
   Size: 1503906416  Blocks: 2937328IO Block: 4096   regular file

 The size and blocks are different between them.

 That said, do you have any suggestions to calculate/send the end of file
 properly?

 Thanks!
 --
 *Tiago Hillebrandt*
 Ubuntu Member
 Ubuntu Brazilian Community Council Member

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] libzmq mingw32 build fails

[KIU Shueng Chuan]

There's an old thread about cross compiling with mingw32 here:
https://github.com/zeromq/czmq/issues/104
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_getsockopt issue with ZMQ_FD

[KIU Shueng Chuan]

rc != -1 usually means success.

Your code snippet also doesn't initialize fd_size.

On Tue, Nov 4, 2014 at 6:54 AM, Saurav Dasgupta saurav.dasgupt...@gmail.com
 wrote:

 I am facing some problem in retrieving the socket associated with the
 subscriber.
 In my case, the subscriber is spawned before the Publisher.
 rc = zmq_getsockopt(subscriber, ZMQ_FD, z_fd, fd_size); is returning -1
 although the errorno is giving as SUCCESS.

 Here is the code snippet

 context = zmq_ctx_new();
 subscriber = zmq_socket(context, ZMQ_SUB);
 rc = zmq_connect(subscriber,tcp://localhost:5556);
 if (rc != 0) {
 printf(error in zmq_connect: %s\n, zmq_strerror (errno));
  }
  zmq_setsockopt(subscriber,ZMQ_SUBSCRIBE,,0);
  z_fd = malloc(sizeof(int));
  rc = zmq_getsockopt(subscriber, ZMQ_FD, z_fd, fd_size);
  if (rc != -1) {
 printf(error in zmq_getsockopt: %s\n, zmq_strerror (errno));
  }

 Is there any known issue with zmq_getsockopt.. Also i using a Linux
 environment.

 Regards,
 Saurav

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] API break in 4.1.x

[KIU Shueng Chuan]

http://lists.zeromq.org/pipermail/zeromq-dev/2014-April/025977.html

See referred thread. Without the reconnection notification, it would not be
possible to receive and assemble large messages from the server.
On 26 Oct 2014 03:09, Pieter Hintjens p...@imatix.com wrote:

 Hi all,

 We've noticed this commit broke the API in 4.1.x:

 mmit afb24b53e6d1fc39cd1b73be706b335ed5c7b6fb
 Author: Goswin von Brederlow goswin-...@web.de
 Date:   Fri Jan 17 23:21:42 2014 +0100

 Add STREAM connect notification.
 Adjust test cases to connection notification.
 Increase error checking in test cases.

 This change breaks existing applications. Does anyone have an
 objection to either removing it, or making it a configurable socket
 option?

 -Pieter

 Ps. also, volunteers to clean this up?  :-)
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Lazy subscriber, only receive last message

[KIU Shueng Chuan]

Would the zmq4 socket option ZMQ_CONFLATE do what you want?
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Python to C++ PUB-SUB Socket

[KIU Shueng Chuan]

You didn't seem to have set any subscriptions on your SUB sockets.

I also seem to recall that binding to localhost doesn't work. Bind to
127.0.0.1 instead. Connecting to localhost does work though.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event

[KIU Shueng Chuan]

Your code could be stripped down a lot:
1) PUB socket is only for sending. There's no need to test it for
ZMQ_POLLIN.
2) PUB socket never blocks when sending. There's no need to test it for
ZMQ_POLLOUT.
3) SUB socket is only for receiving. There's no need to test it for
ZMQ_POLLOUT.
4) The ZMQ_FDs are only to be tested for readability. They are NOT to be
tested for writability (and in fact should always test positive for
writability.)



On Thu, Sep 25, 2014 at 5:09 PM, Dorvin dor...@tlen.pl wrote:

 Thank you for your replies so far. I'll clarify if it does matter that
 I'm using Windows and libzmq 4.0.4.

 W dniu 2014-09-25 04:38, Goswin von Brederlow pisze:
 the ØMQ library
   shall signal any pending events on the socket in an edge-triggered
   fashion by making the file descriptor become ready for reading.
 This is exactly what bothers me. According to documentation I'm supposed
 to get most socket events when FD gets ready for reading.

 But when I do:
 sel_ret = select(sub_fd + 1, sub_set, NULL, NULL, time );
 if (sel_ret == 1)
 {
  zmq_getsockopt(subscriber, ZMQ_EVENTS, zevents, zevents_len);
  if(zevents  ZMQ_POLLIN)
  {
  //retrieve and do something with message
  }
 }

 I receive only some or none messages at all. You may see it in testcase.

 For now I found 3 options to work this out:
 1. Use zmq_poll() periodically - this seems to get read and write
 notifications at the same time and therefore not lose events. This does
 not look like event-driven way of doing things, though.
 2. Use getsockopt with ZMQ_EVENTS after each send or receive - this is
 like small version of zmq_poll invoked almost constantly.
 3. Keeping write notifier active all time which is not a good idea
 because it fires on every pass of event loop. In first reply I already
 was discouraged from this option.

 If you would be so kind I would ask you to execute my code, see results
 and tell me if it's expected behavior (some kind of optimization or
 something), I miss something in how 0mq sockets should work, or if I
 should file bug report.

 You may remove line if (occurences  0) {break;}; to see that what I
 describe is not single event but it repeats.


 With regards,
 Jarek

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event

[KIU Shueng Chuan]

Don't check the FD for writability. You are only meant to check for its
readability.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Trouble sending from ROUTER to DEALER

[KIU Shueng Chuan]

Here's a simple zmq router example using the same endpoint as your
zmqrouter.cpp

https://gist.github.com/pijyoi/9088034e36f85e0c0aac
https://github.com/pijyoi/msg_parts/blob/master/msg_parts.hpp

On Thu, Sep 18, 2014 at 12:26 AM, Riskybiz riskybizl...@live.com wrote:

 Ok.  I've been trying to get those 5 bytes of the ROUTER generated
 identity into a C++ variable so that it can be stored in memory and used
 to prepend an outbound message at some later time.  Can anyone suggest how
 the identity data stored in a zmq_msg_t object and pointed to by:



 zmq_msg_data(messageIn)



 could be stored in a variable which;



 1.   Is easily printable on the console.

 2.   Can be simply copied into a zmq_msg_t object to form the
 identity frame of a message sent via a ROUTER socket.

 3.   Works without knowing that auto generated identities are of any
 specific size.



 Stuck.



 Riskybiz.



 Date: Tue, 16 Sep 2014 22:45:52 +0800

 From: KIU Shueng Chuan nixch...@gmail.com

 Subject: Re: [zeromq-dev] Trouble sending from ROUTER to DEALER

 To: ZeroMQ development list zeromq-dev@lists.zeromq.org

 Message-ID:

 
 CAP2skc-a8DXP+v2UhLoo=7c8dCoBk6K69zZVX=6lpszknrn...@mail.gmail.com

 Content-Type: text/plain; charset=iso-8859-1



 In your multipartmsg.h file, you have code that does the following:

 int32_t id = *(static_castint*(zmq_msg_data(messageIn)))



 You have assumed that the auto generated identity is 4 bytes long, whereas
 it is in fact 5 bytes long.

 So when sending a reply, you would have sent a truncated identity, which
 the router socket know nothing about.



 More generally, you shouldn't need to write your code to know that auto
 generated identities are of any specific size.

 -- next part --

 An HTML attachment was scrubbed...

 URL:
 http://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140916/3819b97f/attachment-0001.htm



 --



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Trouble sending from ROUTER to DEALER

[KIU Shueng Chuan]

zmq_msg_init_size(messageOut, sizeof id );

memcpy(zmq_msg_data(messageOut), id, sizeof id);

In the above 2 lines in your multipartmsg.h, replace sizeof id with
id.size() and id with id.data().
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Trouble sending from ROUTER to DEALER

[KIU Shueng Chuan]

You already use a std::string to store the non-identity parts of the
multi-part message. Why not use the same data type for the identity part?

The identity and non-identity parts are just data blobs.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zsocket_set_identity(), crash!

[KIU Shueng Chuan]

Try zsock_set_identity() instead of zsocket_set_identity().
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Trouble sending from ROUTER to DEALER

[KIU Shueng Chuan]

In your multipartmsg.h file, you have code that does the following:
int32_t id = *(static_castint*(zmq_msg_data(messageIn)))

You have assumed that the auto generated identity is 4 bytes long, whereas
it is in fact 5 bytes long.
So when sending a reply, you would have sent a truncated identity, which
the router socket know nothing about.

More generally, you shouldn't need to write your code to know that auto
generated identities are of any specific size.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Trouble sending from ROUTER to DEALER

[KIU Shueng Chuan]

Well on my system the following line
printf(identity frame size: %ld bytes\n, zmq_msg_size(messageIn));

inserted before your line in multiPartMsg.h
id = *(static_castint*(zmq_msg_data(messageIn)));

yields 5 bytes.



The code at
https://github.com/zeromq/libzmq/blob/master/src/router.cpp#L465
seems to indicate that automatically generated identities are 5-bytes in
length.

On Mon, Sep 15, 2014 at 7:16 PM, Riskybiz riskybizl...@live.com wrote:

 I made a quick modification to check the size of the received identity
 frame.  It shows as 32 bits. Console output is:



 router: Preparing

 router: Ready to receive

 --- Message Received ---

 Originator Identity: 32768 Size of identity frame: 32bits

 Frame Vector Index: 0 Frame Data: Frame1

 Frame Vector Index: 1 Frame Data: Frame2

 Frame Vector Index: 2 Frame Data: Frame3

 Frame Vector Index: 3 Frame Data: Frame4

 

 MultiPartMessage: Sending to connection identity: 32768 Size of identity:
 32bits



 router: Ready to receive





 Date: Mon, 15 Sep 2014 10:44:46 +0800

 From: KIU Shueng Chuan nixch...@gmail.com

 Subject: Re: [zeromq-dev] Trouble sending from ROUTER to DEALER

 To: ZeroMQ development list zeromq-dev@lists.zeromq.org

 Message-ID:

 
 CAP2skc-L543rQTFv1+n_=fV31M60FPQfVpFpZsV8B=zn554=h...@mail.gmail.com

 Content-Type: text/plain; charset=iso-8859-1



 You may want to check whether the received identity frame is only 32-bits
 in length.

 On 15 Sep 2014 02:30, Riskybiz riskybizl...@live.com wrote:



  Dear zeromq developers,

 

 

 

  I've created a couple of simple Windows C++ console programs

  zmqDealerClient http://pastebin.com/1AqWXbTG  zmqRouter

  http://pastebin.com/jD4LsUKU  to understand zeromq (4.0.4)

  ROUTER-DEALER connections.  Both use the header file multiPartMessage

  http://pastebin.com/hhKDamzt to handle the preparation, sending and

  receiving of multipart messages. I can get a message to pass from the

  DEALER to the ROUTER with the resultant console output of:

 

 

 

  router: Preparing

 

  router: Ready to receive

 

  --- Message Received

  ---

 

  Originator Identity: 32768

 

  Frame Vector Index: 0 Frame Data: Frame1

 

  Frame Vector Index: 1 Frame Data: Frame2

 

  Frame Vector Index: 2 Frame Data: Frame3

 

  Frame Vector Index: 3 Frame Data: Frame4

 

  --

  --

 

  MultiPartMessage: Sending to connection identity: 32768

 

  router: Ready to receive

 

 

 

  However when zmqRouter attempts to send a reply from ROUTER back to

  the originating DEALER then the message is not getting through

  properly and the zmqDealerClient output stalls at;

 

 

 

  dealerClient: Preparing

 

  dealerClient: Dealer Socket Ready

 

  dealerClient: MultiPartMessage Sent

 

  dealerClient: Awaiting Reply

 

 

 

  Along with the following error messages from zmqRouter;

 

 

 

  [9668] MultiPartMessage: Error Sending Identity Message Part: Unknown

  error

 

  [9668] MultiPartMessage: Error Sending Regular Message Part: Unknown

  error

 

  [9668] MultiPartMessage: Error Sending Regular Message Part: Unknown

  error

 

  [9668] MultiPartMessage: Error Sending Regular Message Part: Unknown

  error

 

 

 

  The multi part message being sent via the ROUTER to the DEALER is

  prepended with the identity of the DEALER socket, I'm unsure of the

  problem here; can anyone explain what I've done wrong?

 

 

 

  Also I notice that if I run two instances of zmqDealerClient in

  separate console windows, both connecting to the same instance of

  zmqRouter, then both clients appear to have the same identity.  The

  code I've written uses the default situation where the ROUTER socket

  sets the identities internally.  I expected each instance to have a

  unique identity.  Anyone know why this is not the case?

 

 

 

  With thanks,

 

 

 

  Riskybiz.

 



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Trouble sending from ROUTER to DEALER

[KIU Shueng Chuan]

You may want to check whether the received identity frame is only 32-bits
in length.
On 15 Sep 2014 02:30, Riskybiz riskybizl...@live.com wrote:

 Dear zeromq developers,



 I've created a couple of simple Windows C++ console programs
 zmqDealerClient http://pastebin.com/1AqWXbTG  zmqRouter
 http://pastebin.com/jD4LsUKU  to understand zeromq (4.0.4)
 ROUTER-DEALER connections.  Both use the header file multiPartMessage
 http://pastebin.com/hhKDamzt to handle the preparation, sending and
 receiving of multipart messages. I can get a message to pass from the
 DEALER to the ROUTER with the resultant console output of:



 router: Preparing

 router: Ready to receive

 --- Message Received ---

 Originator Identity: 32768

 Frame Vector Index: 0 Frame Data: Frame1

 Frame Vector Index: 1 Frame Data: Frame2

 Frame Vector Index: 2 Frame Data: Frame3

 Frame Vector Index: 3 Frame Data: Frame4

 

 MultiPartMessage: Sending to connection identity: 32768

 router: Ready to receive



 However when zmqRouter attempts to send a reply from ROUTER back to the
 originating DEALER then the message is not getting through properly and the
 zmqDealerClient output stalls at;



 dealerClient: Preparing

 dealerClient: Dealer Socket Ready

 dealerClient: MultiPartMessage Sent

 dealerClient: Awaiting Reply



 Along with the following error messages from zmqRouter;



 [9668] MultiPartMessage: Error Sending Identity Message Part: Unknown error

 [9668] MultiPartMessage: Error Sending Regular Message Part: Unknown error

 [9668] MultiPartMessage: Error Sending Regular Message Part: Unknown error

 [9668] MultiPartMessage: Error Sending Regular Message Part: Unknown error



 The multi part message being sent via the ROUTER to the DEALER is
 prepended with the identity of the DEALER socket, I'm unsure of the problem
 here; can anyone explain what I've done wrong?



 Also I notice that if I run two instances of zmqDealerClient in separate
 console windows, both connecting to the same instance of zmqRouter, then
 both clients appear to have the same identity.  The code I've written uses
 the default situation where the ROUTER socket sets the identities
 internally.  I expected each instance to have a unique identity.  Anyone
 know why this is not the case?



 With thanks,



 Riskybiz.

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zyre and zmq_poll problem

[KIU Shueng Chuan]

zmq_poll() takes libzmq void* sockets not zsock_t* that is the type of your
nodesock.

A web search of errno 88 says it's ENOTSOCK.
On 11 Sep 2014 20:04, Sridhar Anandakrishnan sx...@psu.edu wrote:


 Hi, I am trying to poll on a zyre socket and it returns immediately with
 errno set to 88.

 zyre version is 1.1.0 and libzmq is 4.1.0
 Here is the output of the code (snippet below):
 ---
 zyre: 1 1 0
 zmq: 4 1 0
 I: 14-09-10 11:01:57 (pine07) JOIN group=GLOBAL
 zmq_poll err, 88
 ---

 If I use zpoller the poller works properly for the zyre socket alone, but
 according to the docs, in order to use both a zmq socket and a file
 descriptor, I need to use zmq_poll()...

 Any help appreciated.
 Sincerely,
 Sridhar


 serfd opens /dev/null in this case for testing...

 zyre_version(major, minor, patch);
 fprintf(stderr, zyre: %d %d %d\n, major, minor, patch);
 zmq_version(major, minor, patch);
 fprintf(stderr, zmq: %d %d %d\n, major, minor, patch);

 zyre_t *node = zyre_new(pine07);
 assert(node);
 zyre_set_verbose(node);
 //zyre_set_name(node, pine07);
 ret = zyre_start(node);
 assert(ret==0);
 zyre_join(node, GLOBAL);

 zsock_t *nodesock = zyre_socket(node);

 zmq_pollitem_t items[] = {
 { NULL, serfd, ZMQ_POLLIN, 0},
 { nodesock, 0, ZMQ_POLLIN, 0}
 };
 int npoll = sizeof(items) / sizeof(items[0]);
 int timeout = 6; // FIXME - define?

 while(1) {
 static char *sender, *ipaddr, *base = NULL, *hdr, *name;
 char *msgstring;
 zmsg_t *msg;
 ret=zmq_poll(items, npoll, timeout);
 if(ret0) {
 fprintf(stderr, zmq_poll err, %d\n, errno);
 }
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Paranoid Pirate

[KIU Shueng Chuan]

Okay, from the zguide, As a historical note, ZMQ v2.2 and earlier use
UUIDs as identities, and ZMQ v3.0 and later use short integers.

And looking closer at zmsg.hpp, the recv() method automatically converts
message parts that look like uuids to printable c_strs. The send() method
does the reverse.

So now that identities are no longer uuids, they no longer get auto
converted to c_strs and this doesn't work well with zmsg which only deals
with c_strs.

In short, you are using example code that relies on identities being UUIDs.

Even if the example were to work, it wouldn't be very useful that only
c_strs could be used, would it?
On Aug 26, 2014 9:29 PM, Riskybiz riskybizl...@live.com wrote:

 Is it these lines which are the problem with c_str() in  the Paranoid
 Pirate
 Queue?

 msg.push_front((char*)identity.c_str());

 msg.wrap (it-identity.c_str(), NULL);

 What can be done to fix the problem, not using uuid identifiers?  Replacing
 uuids in the Paranoid Pirate Queue code with calls to s_set_id ()?

 But it begs the question; how should the uuid identifiers be properly
 handled and passed around in code?  Surely Paranoid Pirate pattern must
 have
 worked with uuids at some point in time otherwise how would it have passed
 muster for the zeromq guide?

 Confused.

 Riskybiz.


 Message: 8
 Date: Tue, 26 Aug 2014 06:15:54 +0800
 From: KIU Shueng Chuan nixch...@gmail.com
 Subject: Re: [zeromq-dev] Paranoid Pirate
 To: ZeroMQ development list zeromq-dev@lists.zeromq.org
 Message-ID:
 CAP2skc-=
 rlz92jxpxsavmlfqrgapjmojh9qn+b1yulaxfny...@mail.gmail.com
 Content-Type: text/plain; charset=iso-8859-1

 The zmsg class api takes in char* and the pirate queue has code that passes
 it identities as c_str(). This doesn't work since identities are not nul
 terminated c strings and may contain nuls.

 It works for the workers only because they explicitly set their own id to
 something containing only characters. See s_set_id ().
 On 26 Aug 2014 04:33, Riskybiz riskybizl...@live.com wrote:

  Dear zeromq developers,
 
 
 
  I'm trying to get the Paranoid Pirate pattern to
  operate on Debian Linux using zeromq-4.0.4.  (When I get it working on
  Linux then I will turn my attention to making it work on Windows).
  The Paranoid Pirate Queue http://pastebin.com/KTsn4Yq8 and the
  Paranoid Pirate Worker http://pastebin.com/hLHRC2LB are
  communicating and demonstrably heartbeating.  The problem is that the
  Lazy Pirate Client http://pastebin.com/Ekd1ZGQF does not appear to
  communicate properly or receive a reply message. Is anyone able to
 identify what is wrong here?
  There are a couple of other necessary files; zhelpers.h
  http://pastebin.com/ir8bkQaU and zmsg.hpp
  http://pastebin.com/4KYir507 .
 
  I have changed int64_t more = 0; to  int more = 0; as kindly pointed
  out by KIU Shueng Chuan as being necessary for zeromq-3.2.x and
  higher.  Also added are some console print statements to trace what is
  actually going on.  In order to correct very long waiting times I have
  modified the delay periods built in to the pattern from those provided
  by the stock example in the zeromq guide.
 
  Hope someone is able to help.
 
  Thanks,
 
  Riskybiz.
 
 
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 
 -- next part --
 An HTML attachment was scrubbed...
 URL:

 http://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140826/03acd436/a
 ttachment-0001.htm

 --

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


 End of zeromq-dev Digest, Vol 80, Issue 24
 **

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Paranoid Pirate

[KIU Shueng Chuan]

The zmsg class api takes in char* and the pirate queue has code that passes
it identities as c_str(). This doesn't work since identities are not nul
terminated c strings and may contain nuls.

It works for the workers only because they explicitly set their own id to
something containing only characters. See s_set_id ().
On 26 Aug 2014 04:33, Riskybiz riskybizl...@live.com wrote:

 Dear zeromq developers,



 I'm trying to get the Paranoid Pirate pattern to operate
 on Debian Linux using zeromq-4.0.4.  (When I get it working on Linux then I
 will turn my attention to making it work on Windows).  The Paranoid
 Pirate Queue http://pastebin.com/KTsn4Yq8 and the Paranoid Pirate Worker
 http://pastebin.com/hLHRC2LB are communicating and demonstrably
 heartbeating.  The problem is that the Lazy Pirate Client
 http://pastebin.com/Ekd1ZGQF does not appear to communicate properly or
 receive a reply message. Is anyone able to identify what is wrong here?
 There are a couple of other necessary files; zhelpers.h
 http://pastebin.com/ir8bkQaU and zmsg.hpp http://pastebin.com/4KYir507
 .

 I have changed int64_t more = 0; to  int more = 0; as kindly pointed out
 by KIU Shueng Chuan as being necessary for zeromq-3.2.x and higher.  Also
 added are some console print statements to trace what is actually going
 on.  In order to correct very long waiting times I have modified the delay
 periods built in to the pattern from those provided by the stock example in
 the zeromq guide.

 Hope someone is able to help.

 Thanks,

 Riskybiz.



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Zeromq Paranoid Pirate Majordomo examples.

[KIU Shueng Chuan]

In your req2Router.cpp file, the RCVMORE socket option uses int64_t. It
changed from int64_t in zmq 2.2 to int in zmq 3.2.x and later.
On 24 Aug 2014 04:53, Riskybiz riskybizl...@live.com wrote:

 Dear 0MQ developers,



 For some months I've been trying to get the Paranoid Pirate  Majordomo
 patterns working so that I may study their operation and implement the
 principles in some new development work I have in mind.  What I would like
 to achieve is relatively simple.



 · Using 0MQ networking I would like several sources (REQ?) of
 time-series data to pass any accumulated time series data history to a
 central point (ROUTER?).

 · When the accumulated data history has been passed then the
 sources would pass real time updates to the central point; as and when the
 new data are generated.

 · I would like the data flows to be asynchronous and that the
 data sources 'push' data to the central point.

 · It is important that no data is lost due to slow joiner
 syndrome.  It is also important that the data arrives in the correct order;
 otherwise the data will be gobbledegook.

 · A facility to detect when data sources have died/gone away,
 heartbeating.

 · Preferably to I'd like to work in C++; it's familiar.

 · Windows 7 compatibility is essential, I am tied to using
 Windows it is where the data sources will run.



 In working through the problems getting Paranoid Pirate  Majordomo to
 work  I came to suspect that the problem lay in the REQ-ROUTER connection
 which is present in both the Paranoid Pirate and Majordomo patterns.  This
 stackoverflow question explains some of approaches I've tried:  Stackoverflow
 question
 http://stackoverflow.com/questions/24597251/zeromq-issues-with-example-networking-patterns-paranoid-pirate-and-majordomo



 I even tried building CZMQ on Windows 7 so that I could try the C language
 original versions of the Paranoid Pirate  Majordomo patterns.  Have you
 tried building CZMQ on Windows; it's just a nightmare?  The provided Visual
 Studio solution file just didn't work and the instructions are practically
 non-existent.  Searching the internet for help found  rocket scientists
 http://lists.zeromq.org/pipermail/zeromq-dev/2013-October/022981.html
  proposing cross compilation as the solution; but where to start with
 that?  Really should it be that difficult?  All I could see there was
 another month of lost weekends.  So all in all that was a dead end.



 Seeking answers  further investigating REQ-ROUTER I discovered this
 example: simple req2Router example
 http://thisthread.blogspot.co.uk/2012/05/very-simple-req-router-example.html
 and built it into a working Windows program. The cpp   header file code
 for it are here  req2Router cpp file http://pastebin.com/g66mzGm9 and zmq2.h
 header file http://pastebin.com/yti8jUEc .



 What I discovered was that this simple req2Router program would function
 properly when linked against zeromq-2.2.0 but would *not* function when
 linked against zeromq-3.2.3 or zeromq-4.0.4.  N.B. When running tests I
 ensured that the corresponding libzmq.dll file was used from the relevant
 version of zeromq. Could this be the problem in getting the Paranoid Pirate
  Majordomo examples to work; that the example code is zeromq version
 dependant?  Does anyone know why this might be the case? What has changed
 between versions which would affect the REQ-ROUTER functionality?



 I really am stuck and getting tangled with this and am losing time to it.
 I cannot move forward with the plan I have in mind until I can get working
 implementations of Paranoid Pirate,  Majordomo  Asynchronous Majordomo on
 Windows as a foundation on which to build.  It's doubly difficult because I
 want to learn from the examples but debugging them at the same time is
 testing to say the least.



 Is anyone able to help get these examples working with the latest versions
 of zeromq on Windows, not just for my sake but also for the next person who
 tries?  Surely I can't be the only person who needs or wants to try this?



 With thanks,



 Riskybiz.

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Query on zmq_poll API

[KIU Shueng Chuan]

There's a lbbroker example in the zguide. The main complexity seems to be
in framing a proper response to the REQ sockets.

http://zguide.zeromq.org/c:lbbroker
On 19 Aug 2014 11:37, Badhrinath Manoharan badh...@gmail.com wrote:

 Hi Kiu,

 That was just a copy paste error in mail. I do have poll_items[1]
 initialized.
 Do you see any issue why zmq_poll API just returns only the first message
 received on each socket.

 Thanks
 Badhri

 Sent from my iPhone

 On Aug 18, 2014, at 5:31 PM, KIU Shueng Chuan nixch...@gmail.com wrote:

 Your snippet neither initializes nor checks poll_items[1]. But no, revents
 field doesn't need to be reset (nor initialized).


 On Tue, Aug 19, 2014 at 8:14 AM, Badhrinath Manoharan badh...@gmail.com
 wrote:

 Hi,

 I have the following topology

 Client --- Broker  Server

 Both the client and server are sockets of type ZMQ_REQ while the Broker
 has a socket connected to client and another Socket Connected to Server
 both are of type ZMQ_BROKER.

 void *frontend = zmq_socket (context, ZMQ_ROUTER);
 void *backend = zmq_socket (context, ZMQ_ROUTER);

 zmq_bind(frontend, ipc:///users/badmanoh/frontend.ipc);
 zmq_bind(backend, ipc:///users/badmanoh/backend.ipc);

 poll_items[0].socket = frontend;
 poll_items[0].fd = 0;
 poll_items[0].events = ZMQ_POLLIN;
 poll_items[0].revents = 0;

 poll_items[0].socket = backend;
 poll_items[0].fd = 0;
 poll_items[0].events = ZMQ_POLLIN;
 poll_items[0].revents = 0;

 while (1) {
 ret = zmq_poll(poll_items, 2, -1);
 if (ret == -1) {
 printf(zmq_poll returned -1. Error: %d\n, errno);
 return 0;
 }
 if (poll_items[0].revents  ZMQ_POLLIN) {
 }
 if (poll_items[0].revents  ZMQ_POLLIN) {
 }
 }

 On the broker code, I have a zmq_poll(poll_items, 2, -1) on a while loop.
 I see the zmq_poll notifying the first message from each socket. However
 subsequent messages from both the client or server sockets are not at all
 returned and the zmq_poll just stays in an infinite loop. Could you let us
 know if I am missing anything? Do I need to reset the revents value as part
 of the first notification?

 Thanks
 Badhri

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Query on zmq_poll API

[KIU Shueng Chuan]

Your snippet neither initializes nor checks poll_items[1]. But no, revents
field doesn't need to be reset (nor initialized).


On Tue, Aug 19, 2014 at 8:14 AM, Badhrinath Manoharan badh...@gmail.com
wrote:

 Hi,

 I have the following topology

 Client --- Broker  Server

 Both the client and server are sockets of type ZMQ_REQ while the Broker
 has a socket connected to client and another Socket Connected to Server
 both are of type ZMQ_BROKER.

 void *frontend = zmq_socket (context, ZMQ_ROUTER);
 void *backend = zmq_socket (context, ZMQ_ROUTER);

 zmq_bind(frontend, ipc:///users/badmanoh/frontend.ipc);
 zmq_bind(backend, ipc:///users/badmanoh/backend.ipc);

 poll_items[0].socket = frontend;
 poll_items[0].fd = 0;
 poll_items[0].events = ZMQ_POLLIN;
 poll_items[0].revents = 0;

 poll_items[0].socket = backend;
 poll_items[0].fd = 0;
 poll_items[0].events = ZMQ_POLLIN;
 poll_items[0].revents = 0;

 while (1) {
 ret = zmq_poll(poll_items, 2, -1);
 if (ret == -1) {
 printf(zmq_poll returned -1. Error: %d\n, errno);
 return 0;
 }
 if (poll_items[0].revents  ZMQ_POLLIN) {
 }
 if (poll_items[0].revents  ZMQ_POLLIN) {
 }
 }

 On the broker code, I have a zmq_poll(poll_items, 2, -1) on a while loop.
 I see the zmq_poll notifying the first message from each socket. However
 subsequent messages from both the client or server sockets are not at all
 returned and the zmq_poll just stays in an infinite loop. Could you let us
 know if I am missing anything? Do I need to reset the revents value as part
 of the first notification?

 Thanks
 Badhri

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq send question

[KIU Shueng Chuan]

It resets the zmq_msg_t structure to an initialized state.
On 11 Aug 2014 05:59, Chrys Mischa Piva chrys.p...@googlemail.com wrote:

 hello,

 I believe I read somewhere that zeromq deletes the memory for a message
 after calling send ...

 can someone either tell me I am wrong or point me to the documentation for
 this ?

 regards,
 Chrys

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Extending zmq_msg_t API

[KIU Shueng Chuan]

Couldn't the refcount change after you have obtained its value?
E.g.
Make a copy
Send the 1st
Read the refcount (2)
Background io releases 1st copy
 On 9 Jul 2014 18:21, Thomas Rodgers rodg...@twrodgers.com wrote:

 zmq_msg_get() could be extended to give either the refcount or an
 indicator on whether a message was share; based on other refcounted designs
 I'm hesitant to promote surfacing the actual count.  Similarly,
 zmq_msg_set() could allow 'unsharing' by adding a ZMQ_SHARED property
 #define and setting it's value to 0 (no effect on non-shared messages).

 So the only API surface area change is an additional message property.
  This seems the cleanest to me.

 On Wednesday, July 9, 2014, Goswin von Brederlow goswin-...@web.de
 wrote:

 On Tue, Jul 08, 2014 at 10:42:41AM -0500, Thomas Rodgers wrote:
  tl;dr; Is there any objection to adding some sort of accessor to the
 API to
  determine if a given zmq_msg_t is_shared()?
 
  Background/Rationale:
 
  Something I encountered while writing a high level C++ wrapper for
  zmq_msg_t and it's API is the following set of behaviors -
 
  zmq_msg_init(msg_vsm, 20);
 
  Results in a type_vsm message, the body of which is held entirely within
  the space allocated to zmq_msg_t
 
  zmq_msg_init(msg_lmsg, 1024);
 
  Results in a type_lmsg message, the body is held as a reference to a
 block
  of size bytes.
 
  memcpy(zmq_msg_data(msg_vsm), VSM, 3);
  memcpy(zmq_msg_data(msg_lmsg), LMSG, 4);
 
  So far so good.  Now copy -
 
  zmq_msg_copy(msg_vsm2, msg_vsm);
  zmq_msg_copy(msg_lmsg2, msg_lmsg);
 
  Now change contents -
 
  memcpy(zmq_msg_data(msg_vsm2), vsm, 3);
  memcpy(zmq_msg_data(msg_lmsg2), lmsg, 4);
 
  assert(memcmp(msg_vsm, msg_vsm2, 3) != 0); // ok
  assert(memcmp(msg_lmsg, msg_lmsg2, 4) != 0); // fail
 
  This happens by design (lmsg's are refcounted on copy, not deep copied).
  But it results in a situation where a zmq_msg_t is sometimes a Value and
  sometimes a Reference.  This could lead to astonishment for the unwary.
 
  From the perspective of a wrapper (particularly one that takes a strong
  stand on value semantics and local reasoning), this behavior is ungood.
  So
  my options are deep copy always or implement copy-on-write.
 
  For efficiency I prefer the latter approach in the case of type_lmsg
  messages.  I have implemented the copy-on-write logic through a horrible
  brittle hack that examines the last byte of zmq_msg_t.  I would prefer a
  less brittle solution.

 lmsg's are refcounted on copy Can't you access the refcount?
 Or is that the API call you want to add?

 Maybe instead of is_shared() an unshare() call would be more usefull,
 which would copy the message payload if it is shared. Or both?

 MfG
 Goswin
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] PeriodicCallback timeout?

[KIU Shueng Chuan]

Zmq sockets are only to be manipulated by their creating thread.

The add_callback example thus assumes that stream was created by the same
thread as the one running the event loop and that your producer was in
another thread.
On 4 Jul 2014 23:55, Indradhanush Gupta indradhanush.gu...@gmail.com
wrote:

 I had asked out on irc about this problem and charliebone made me
 realise that I should directly queue the messages on the socket itself.




 On Thu, Jul 3, 2014 at 9:44 PM, MinRK benjami...@gmail.com wrote:

 Why not just queue sends directly? What is the goal of the Queue? For
 threadsafety, you should use the `add_callback` functionality of IOLoop:

 loop = IOLoop.instance()

  loop.add_callback(lambda : stream.send_multipart(msg))

 Then you don't need any polling, it will just send messages when there
 are messages to be sent.


 However, I don't get this part. How is it different than directly invoking
 stream.send_multipart(msg) ?



 -MinRK



 On Thu, Jul 3, 2014 at 7:07 AM, KIU Shueng Chuan nixch...@gmail.com
 wrote:

 How about replacing the Queue with zmq sockets? Then you could just add
 the consumer socket to the event loop.
 On 3 Jul 2014 16:15, Indradhanush Gupta indradhanush.gu...@gmail.com
 wrote:

 Hello,

 I am using a PeriodicCallback in pyzmq, that checks a python Queue for
 data that needs to be sent.

 It looks like this -

 self.to_client = Queue.Queue()

 def check_to_client(self):
 while not self.to_client.empty():
 data = self.to_client.get()
 self.frontend.send(data)

 So, going by the current callback definition, it sends each element of
 the Queue until it is empty. I am wondering if this is good. It appears,
 that this callback might block if the Queue is very large.

 I have two approaches in mind.

 1. Send n items, then return, where n = a number that will not block.
 So is there an idea on what might be a good number?

 2. Use a timeout on check_to_client, such that it runs for n
 milliseconds and returns. Again what would be a good enough number for
 this? But also, is this even possible?

 Any other solution that someone has used before? This appears to be a
 very common problem.

 Thanks,
 --
 Indradhanush Gupta


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev




 --
 Indradhanush Gupta


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] PeriodicCallback timeout?

[KIU Shueng Chuan]

How about replacing the Queue with zmq sockets? Then you could just add the
consumer socket to the event loop.
On 3 Jul 2014 16:15, Indradhanush Gupta indradhanush.gu...@gmail.com
wrote:

 Hello,

 I am using a PeriodicCallback in pyzmq, that checks a python Queue for
 data that needs to be sent.

 It looks like this -

 self.to_client = Queue.Queue()

 def check_to_client(self):
 while not self.to_client.empty():
 data = self.to_client.get()
 self.frontend.send(data)

 So, going by the current callback definition, it sends each element of the
 Queue until it is empty. I am wondering if this is good. It appears, that
 this callback might block if the Queue is very large.

 I have two approaches in mind.

 1. Send n items, then return, where n = a number that will not block. So
 is there an idea on what might be a good number?

 2. Use a timeout on check_to_client, such that it runs for n milliseconds
 and returns. Again what would be a good enough number for this? But also,
 is this even possible?

 Any other solution that someone has used before? This appears to be a very
 common problem.

 Thanks,
 --
 Indradhanush Gupta


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Message initialisation

[KIU Shueng Chuan]

 zmq_msg_close() does not return with an error code when applied to an
 empty zmq_msg_t, neither one that has just been sent nor one that has
 been initialised as empty.  However, if I call it *again* on the same
 message, it fails:

  zmq_msg_t m;
  zmq_msg_init(m);
  assert(zmq_msg_close(m) == 0); // Passes
  assert(zmq_msg_close(m) == 0); // Fails

 In other words, closing an empty message is apparently not a no-op.  So,
 what is the difference between an empty message and a closed message?

A closed message is the same as an uninitialized zmq_msg_t. It becomes just
a blob of memory.

Implementation-wise, close scrubs away some magic id that identifies it as
being initialized or not. That's why the second close fails.

It would be simpler just to ignore the part about send nullifying the
message on success. Just pair every init with a close.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] 0MQ 4 - how to detect closed stream socket ?

[KIU Shueng Chuan]

It turns out that this functionality was added in January to libzmq.
An empty message is sent both on connect and also upon disconnect.

The net effect for my use case is that one receives 2 empty messages when
the connection breaks and then when it gets reconnected.


A pair of python scripts that demonstrate the behavior (A tcp publisher
subscribed using zmq_stream):
https://github.com/pijyoi/test_zmqstream



On Thu, Apr 24, 2014 at 6:40 PM, Pieter Hintjens p...@imatix.com wrote:

 On Thu, Apr 24, 2014 at 12:35 PM, KIU Shueng Chuan nixch...@gmail.com
 wrote:

  Sadly, I am dealing with publisher TCP-servers that start sending data
  upon client connection. The protocol is some small fixed header followed
 by
  a variable length message.
 
  A ZMQ_STREAM server already uses sending of an empty message in order to
  disconnect a client. Would be nice if an empty message could be received
 by
  the ZMQ_STREAM client upon disconnection.

 This sounds familiar, so may already be implemented. I've not used
 ZMQ_STREAM much. Signaling the end of a connection with an empty
 message is definitely a decent idea.

 -Pieter
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] 0MQ 4 - how to detect closed stream socket ?

[KIU Shueng Chuan]

Being informed about both connects and disconnects might be useful for
other use cases.

Looks like one could practically avoid using bsd tcp sockets now.
On 26 Apr 2014 07:27, Pieter Hintjens p...@imatix.com wrote:

 Is that what you'd expect? It seems a good minimal solution.

 On Fri, Apr 25, 2014 at 11:52 PM, KIU Shueng Chuan nixch...@gmail.com
 wrote:
  It turns out that this functionality was added in January to libzmq.
  An empty message is sent both on connect and also upon disconnect.
 
  The net effect for my use case is that one receives 2 empty messages when
  the connection breaks and then when it gets reconnected.
 
 
  A pair of python scripts that demonstrate the behavior (A tcp publisher
  subscribed using zmq_stream):
  https://github.com/pijyoi/test_zmqstream
 
 
 
  On Thu, Apr 24, 2014 at 6:40 PM, Pieter Hintjens p...@imatix.com wrote:
 
  On Thu, Apr 24, 2014 at 12:35 PM, KIU Shueng Chuan nixch...@gmail.com
  wrote:
 
   Sadly, I am dealing with publisher TCP-servers that start sending
 data
   upon client connection. The protocol is some small fixed header
 followed
   by
   a variable length message.
  
   A ZMQ_STREAM server already uses sending of an empty message in order
 to
   disconnect a client. Would be nice if an empty message could be
 received
   by
   the ZMQ_STREAM client upon disconnection.
 
  This sounds familiar, so may already be implemented. I've not used
  ZMQ_STREAM much. Signaling the end of a connection with an empty
  message is definitely a decent idea.
 
  -Pieter
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] 0MQ 4 - how to detect closed stream socket ?

[KIU Shueng Chuan]

Sadly, I am dealing with publisher TCP-servers that start sending data
upon client connection. The protocol is some small fixed header followed by
a variable length message.

A ZMQ_STREAM server already uses sending of an empty message in order to
disconnect a client. Would be nice if an empty message could be received by
the ZMQ_STREAM client upon disconnection.



On Thu, Apr 24, 2014 at 6:11 PM, Pieter Hintjens p...@imatix.com wrote:

 This is indeed lacking in the client side use of ZMQ_STREAM. However
 most conventional TCP protocols don't return arbitrary data from the
 server without a request from the client first. That is, if the server
 is sending 10K bytes and the connection is broken, the client will get
 nothing more until it decides the connection is dead and then
 reconnects.

 -Pieter

 On Thu, Apr 24, 2014 at 3:09 AM, KIU Shueng Chuan nixch...@gmail.com
 wrote:
  I am trying out ZMQ_STREAM and it's great that (re)-connects are handled
  asynchronously. However...
 
  Say a TCP server sends out 1-byte messages such that multiple recv()s
  are required to get the whole message. A ZMQ_STREAM client receives 1500
  bytes and then the connection gets broken. (E.g. server gets killed and
  restarted) How would the client know to reset its message assembling
 state
  if it doesn't know about the disconnection?
 
 
  On Sat, Oct 12, 2013 at 11:49 PM, Pieter Hintjens p...@imatix.com wrote:
 
  Indeed... ZMQ_STREAM like all sockets will reconnect opportunistically
  so you don't get a formal notification of a closed socket.
 
  On Fri, Oct 11, 2013 at 9:03 PM, Charles Remes li...@chuckremes.com
  wrote:
   Via a timeout.
  
   On Oct 11, 2013, at 12:15 PM, itli...@schrievkrom.de wrote:
  
   How can I detect, that the server has closed a stream client socket ?
  
   Marten
   ___
   zeromq-dev mailing list
   zeromq-dev@lists.zeromq.org
   http://lists.zeromq.org/mailman/listinfo/zeromq-dev
  
   ___
   zeromq-dev mailing list
   zeromq-dev@lists.zeromq.org
   http://lists.zeromq.org/mailman/listinfo/zeromq-dev
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] 0MQ 4 - how to detect closed stream socket ?

[KIU Shueng Chuan]

I am trying out ZMQ_STREAM and it's great that (re)-connects are handled
asynchronously. However...

Say a TCP server sends out 1-byte messages such that multiple recv()s
are required to get the whole message. A ZMQ_STREAM client receives 1500
bytes and then the connection gets broken. (E.g. server gets killed and
restarted) How would the client know to reset its message assembling state
if it doesn't know about the disconnection?


On Sat, Oct 12, 2013 at 11:49 PM, Pieter Hintjens p...@imatix.com wrote:

 Indeed... ZMQ_STREAM like all sockets will reconnect opportunistically
 so you don't get a formal notification of a closed socket.

 On Fri, Oct 11, 2013 at 9:03 PM, Charles Remes li...@chuckremes.com
 wrote:
  Via a timeout.
 
  On Oct 11, 2013, at 12:15 PM, itli...@schrievkrom.de wrote:
 
  How can I detect, that the server has closed a stream client socket ?
 
  Marten
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Try to run a very simple example wih pyzmq and multiprocessing

[KIU Shueng Chuan]

Don't pass context to the child process. Create another context in the
child. ZeroMQ objects don't work well across forks.



On Thu, Mar 27, 2014 at 10:42 PM, SUDRES Xavier xavier.sud...@atos.netwrote:

  I'm trying to run a very simple example code using pyzmq and
 multiprocessing. I just want to have two threads communicating through an
 zmq socket (inproc or ipc).

 I'm using the following code:

 import zmq
 from multiprocessing import Process

 def main():

 uri = ipc:///tmp/FORK
 context = zmq.Context()

 pipe = context.socket(zmq.PAIR)
 print(bind, uri)
 pipe.bind(uri)

 p = Process(target=thread, args=(context, uri, zmq.PAIR))
 p.start()

 print(Fork done, receiving...)
 print(pipe.recv_string())
 print(Received !!!)

 p.join()
 del pipe
 del context

 def thread(context, uri, socketType):
 pipe = context.socket(socketType)
 print(connect, uri)
 pipe.connect(uri)

 print(Thread send Hello...)
 pipe.send_string(Hello)

 main()

 Screen output =
 bind ipc:///tmp/FORK
 Forked, receiving...
 connect ipc:///tmp/FORK
 Thread send Hello...

 I'have detected a little error while installing the version of
 pyzmq(master) using cython 0.20 :
 Configure: Autodetecting ZMQ settings...
 Custom ZMQ dir:
 /home_local/xsudres/dev/src/LPISIS/COTS/ZEROMQ/work/zeromq
 creating build/temp.linux-x86_64-3.4/scratch/tmp
 cc -c /tmp/timer_create6cc83401.c -o
 build/temp.linux-x86_64-3.4/scratch/tmp/timer_create6cc83401.o
 cc build/temp.linux-x86_64-3.4/scratch/tmp/timer_create6cc83401.o -o
 build/temp.linux-x86_64-3.4/scratch/a.out
 build/temp.linux-x86_64-3.4/scratch/tmp/timer_create6cc83401.o: In
 function `main':
 timer_create6cc83401.c:(.text+0x15): undefined reference to `timer_create'
 collect2: ld returned 1 exit status
 gcc -pthread -Werror=declaration-after-statement -DNDEBUG -g -fwrapv -O3
 -Wall -Wstrict-prototypes -fPIC
 -I/home_local/xsudres/dev/src/LPISIS/COTS/ZEROMQ/work/zeromq/include
 -Izmq/utils -Izmq/backend/cython -Izmq/devices -c
 build/temp.linux-x86_64-3.4/scratch/vers.c -o
 build/temp.linux-x86_64-3.4/scratch/vers.o
 gcc -pthread build/temp.linux-x86_64-3.4/scratch/vers.o
 -L/home_local/xsudres/dev/src/LPISIS/COTS/ZEROMQ/work/zeromq/lib
 -Wl,--enable-new-dtags,-R/home_local/xsudres/dev/src/LPISIS/COTS/ZEROMQ/work/zeromq/lib
 -lzmq -lrt -o build/temp.linux-x86_64-3.4/scratch/vers
 ZMQ version detected: 4.0.4

 = undefined reference to `timer_create', is it important ?

 Thanks for your help,

 Xavier SUDRES
  --

 Ce message et les pièces jointes sont confidentiels et réservés à l'usage
 exclusif de ses destinataires. Il peut également être protégé par le secret
 professionnel. Si vous recevez ce message par erreur, merci d'en avertir
 immédiatement l'expéditeur et de le détruire. L'intégrité du message ne
 pouvant être assurée sur Internet, la responsabilité du groupe Atos ne
 pourra être engagée quant au contenu de ce message. Bien que les meilleurs
 efforts soient faits pour maintenir cette transmission exempte de tout
 virus, l'expéditeur ne donne aucune garantie à cet égard et sa
 responsabilité ne saurait être engagée pour tout dommage résultant d'un
 virus transmis.

 This e-mail and the documents attached are confidential and intended
 solely for the addressee; it may also be privileged. If you receive this
 e-mail in error, please notify the sender immediately and destroy it. As
 its integrity cannot be secured on the Internet, the Atos group liability
 cannot be triggered for the message content. Although the sender endeavors
 to maintain a computer virus-free network, the sender does not warrant that
 this transmission is virus-free and will not be liable for any damages
 resulting from any virus transmitted.

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] assertion failed Access Denied in (bundled\zeromq\src\signaler.cpp:318)

[KIU Shueng Chuan]

The assertion on line 318 is due to failure to open an Event used for
synchronization. While I do not know why that would be the case, if you are
able to use libzmq master, the code there allows you to avoid using this
synchronization Event, which might help in your case.

In libzmq master, edit config.hpp and change signaler_port from 5905 to 0.
On Mar 15, 2014 7:01 PM, christophe chappet christophe.chap...@gmail.com
wrote:

 Hi,
 My configuration :
 Win Seven
 libzmq-4.0.4
 pyzmq-14.1.0
 Python-2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)]

 The context:
 Several programs using zmq successfully launched as windows services that
 run simultaneously on the PC.

 The error:
 when too many programs (about 10) using zmq run and I want to launch a new
 one, I get:
 Assertion failed: Accès refusé.
 (bundled\zeromq\src\signaler.cpp:318)
 when I call zmq.Context().

 Already tried without success:
 - Increasing Windows TCP socket limits by defining:
 MaxUserPort: and TcpTimedWaitDelay as explained here:
 http://support.uptimesoftware.com/article.php?id=271

 The problem is probably related to ephemeral port but I dont' know how
 to fix it.

 Thanks for any help.

 Regards,
 Christophe

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Assertion fails in zmq.hpp line 267 when running C++ hello world client and server

[KIU Shueng Chuan]

Try instantiating context_t with context(1, 1023). If that still fails try
context(1, 64).
On 13 Mar 2014 03:35, Johnathon Ender johnathon.en...@gmail.com wrote:

 Hello,



 I have been attempting to run a C++ instance of the hello world client and
 server in Visual Studio 2010 on a Windows 7 machine. I have taken both
 examples from the git source, and both compile but fail when attempting to
 debug. The failure is an assertion at line 267 of the zmq.hpp file where rc
 == 0. I have created a stackoverflow thread attempting to figure out what I
 am doing wrong, and you can reference the details at




 http://stackoverflow.com/questions/22355457/is-changing-a-c-project-to-a-c-project-as-simple-as-including-the-related-hpp



 I would like to note I was able to compile and run the C versions of both
 these pieces of code without issue.



 Thank you for your time and assistance,



 Johnathon Ender

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Assertion fails in zmq.hpp line 267 when running C++ hello world client and server

[KIU Shueng Chuan]

Sorry, the second one should have been context (1, 63) instead of 64.

A better solution would be to fix zmq.hpp to not set max sockets option if
the user didn't pass in a 2nd parameter to the constructor.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] How should sockets be destroyed?

[KIU Shueng Chuan]

Is this related to the following issue?
https://github.com/zeromq/libzmq/issues/792

In the issue, the problem only occurs with inproc transport, as also used
by you.
On 25 Feb 2014 16:36, Laurent Alebarde l.aleba...@free.fr wrote:

  You are right. Possibly a bug in the lib ?

 Le 24/02/2014 18:13, Olaf Mandel a écrit :

 Am 24.02.2014 18:04, schrieb Laurent Alebarde:

  By default, you are allowed 1024 sockets per context. After having
 created the context, you shall increase the quantity of sockets allowed:

 rc = zmq_ctx_set (ctx, ZMQ_MAX_SOCKETS, 1);
 assert (rc == 0);


  -Snipp-

 Hello Laurent,

 maybe I am just unclear on the documentation and intended behaviour: is
 ZMQ_MAX_SOCKETS intended to be the number of sockets for the context at
 any one time or is it the absolute number of sockets that may be created
 over the life-time of the context?

 In the latter case I don't think REQ-REP with e.g. the Lazy Pirate
 Pattern work for long-running programs: they would eventually run out of
 sockets to create.

 Best regards,
 Olaf Mandel





 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] default MAX_SOCKETS on Windows set to 63?

[KIU Shueng Chuan]

Looking at the ctx.cpp source, setting of MAX_SOCKETS goes through
clipped_maxsocket() which clips it to max_fds()-1, where max_fds() is
equal to FD_SETSIZE as returned by zmq::select_t::max_fds(). So your issue
does appear to be related to FD_SETSIZE.



I tried it out again on zeromq4-x 571c668fa2e0b101f607b4c4a34c30da2f174499.

Cross-compile on Linux using MinGW32:
 CPPFLAGS=-DFD_SETSIZE=1024 ./configure --host=i686-pc-mingw32

 grep CPPFLAGS Makefile
CPPFLAGS = -pedantic -Werror -Wall -Wno-long-long -D_REENTRANT
-D_THREAD_SAFE -DFD_SETSIZE=1024 -DZMQ_FORCE_SELECT

Adding the following snippet in one of the source files shows that it does
get picked up.
#if FD_SETSIZE!=1024
#error FD_SETSIZE wrong
#endif



If you are compiling directly on Windows and do not want to install MSYS
(using MSYS may yield other quirks!), an alternative to using configure is
to go to the subdir builds/mingw32.
There is a simple Makefile.mingw32 there.
Just run mingw32-make Makefile.mingw32
Haven't tried this for some time and it is supposed to be automatically
generated nowadays.



On Tue, Feb 18, 2014 at 12:27 AM, Charles Remes li...@chuckremes.comwrote:

 I've tried a few different ways of providing FD_SETSIZE but nothing has
 worked for me so far.

 I have tried:

 * CFLAGS=-DFD_SETSIZE=1024 ./configure

 * CPPFLAGS=-DFD_SETSIZE=1024 ./configure

 Neither one is picked up properly by the system. What else should I try?

 On Feb 14, 2014, at 5:56 PM, KIU Shueng Chuan nixch...@gmail.com wrote:

 https://github.com/zeromq/czmq/issues/104

 If compiling using the configure script you need to provide FD_SETSIZE.
 On 15 Feb 2014 01:28, Charles Remes li...@chuckremes.com wrote:

 I received a bug report from a user who compiled zeromq 4.0.3 on Windows
 using 32-bit mingw. The code that blew up was:

 context = LibZMQ.zmq_ctx_new
 rc = LibZMQ.zmq_ctx_set(context, ZMQ::MAX_SOCKETS, 1023)

 At this point, rc was returning -1 and the error message is Invalid
 Argument. After playing around with this a bit I discovered that the
 default MAX_SOCKETS was 63 on this platform.

 context = LibZMQ.zmq_ctx_new
 print LibZMQ.zmq_ctx_set(context, ZMQ::MAX_SOCKETS) # = prints 63

 So, trying to set a value above 63 always fails.

 For further confirmation, I ran make check and noted that
 test_ctx_options also fails (though it doesn't tell me why).

 Does anyone know why MAX_SOCKETS is set to 63 when compiling on Windows
 using mingw? When I look at the source, MAX_SOCKETS_DFLT is set to 1023!

 cr

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] default MAX_SOCKETS on Windows set to 63?

[KIU Shueng Chuan]

Oops. Should have been mingw32-make -f Makefile.mingw32
On 18 Feb 2014 11:04, Charles Remes li...@chuckremes.com wrote:

 I am using an msys shell to do the compilation directly on Windows (no
 cross-compiling). I just tried to run mingw32-make Makefile.mingw32 from
 the proper directory but it just said Nothing to be done for
 Makefile.mingw32 and exited.

 I wish I could figure out the secret incantation to get my FD_SETSIZE
 override to work. Better yet, it would be useful if the generated Makefile
 automatically set it to that value when run in a mingw32 build host.


 On Feb 17, 2014, at 5:42 PM, KIU Shueng Chuan nixch...@gmail.com wrote:

 Looking at the ctx.cpp source, setting of MAX_SOCKETS goes through
 clipped_maxsocket() which clips it to max_fds()-1, where max_fds() is
 equal to FD_SETSIZE as returned by zmq::select_t::max_fds(). So your issue
 does appear to be related to FD_SETSIZE.



 I tried it out again on zeromq4-x 571c668fa2e0b101f607b4c4a34c30da2f174499.

 Cross-compile on Linux using MinGW32:
  CPPFLAGS=-DFD_SETSIZE=1024 ./configure --host=i686-pc-mingw32

  grep CPPFLAGS Makefile
 CPPFLAGS = -pedantic -Werror -Wall -Wno-long-long -D_REENTRANT
 -D_THREAD_SAFE -DFD_SETSIZE=1024 -DZMQ_FORCE_SELECT

 Adding the following snippet in one of the source files shows that it does
 get picked up.
 #if FD_SETSIZE!=1024
 #error FD_SETSIZE wrong
 #endif



 If you are compiling directly on Windows and do not want to install MSYS
 (using MSYS may yield other quirks!), an alternative to using configure is
 to go to the subdir builds/mingw32.
 There is a simple Makefile.mingw32 there.
 Just run mingw32-make Makefile.mingw32
 Haven't tried this for some time and it is supposed to be automatically
 generated nowadays.



 On Tue, Feb 18, 2014 at 12:27 AM, Charles Remes li...@chuckremes.comwrote:

 I've tried a few different ways of providing FD_SETSIZE but nothing has
 worked for me so far.

 I have tried:

 * CFLAGS=-DFD_SETSIZE=1024 ./configure

 * CPPFLAGS=-DFD_SETSIZE=1024 ./configure

 Neither one is picked up properly by the system. What else should I try?

 On Feb 14, 2014, at 5:56 PM, KIU Shueng Chuan nixch...@gmail.com wrote:

 https://github.com/zeromq/czmq/issues/104

 If compiling using the configure script you need to provide FD_SETSIZE.
 On 15 Feb 2014 01:28, Charles Remes li...@chuckremes.com wrote:

 I received a bug report from a user who compiled zeromq 4.0.3 on Windows
 using 32-bit mingw. The code that blew up was:

 context = LibZMQ.zmq_ctx_new
 rc = LibZMQ.zmq_ctx_set(context, ZMQ::MAX_SOCKETS, 1023)

 At this point, rc was returning -1 and the error message is Invalid
 Argument. After playing around with this a bit I discovered that the
 default MAX_SOCKETS was 63 on this platform.

 context = LibZMQ.zmq_ctx_new
 print LibZMQ.zmq_ctx_set(context, ZMQ::MAX_SOCKETS) # = prints 63

 So, trying to set a value above 63 always fails.

 For further confirmation, I ran make check and noted that
 test_ctx_options also fails (though it doesn't tell me why).

 Does anyone know why MAX_SOCKETS is set to 63 when compiling on Windows
 using mingw? When I look at the source, MAX_SOCKETS_DFLT is set to 1023!

 cr

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] default MAX_SOCKETS on Windows set to 63?

[KIU Shueng Chuan]

https://github.com/zeromq/czmq/issues/104

If compiling using the configure script you need to provide FD_SETSIZE.
On 15 Feb 2014 01:28, Charles Remes li...@chuckremes.com wrote:

 I received a bug report from a user who compiled zeromq 4.0.3 on Windows
 using 32-bit mingw. The code that blew up was:

 context = LibZMQ.zmq_ctx_new
 rc = LibZMQ.zmq_ctx_set(context, ZMQ::MAX_SOCKETS, 1023)

 At this point, rc was returning -1 and the error message is Invalid
 Argument. After playing around with this a bit I discovered that the
 default MAX_SOCKETS was 63 on this platform.

 context = LibZMQ.zmq_ctx_new
 print LibZMQ.zmq_ctx_set(context, ZMQ::MAX_SOCKETS) # = prints 63

 So, trying to set a value above 63 always fails.

 For further confirmation, I ran make check and noted that
 test_ctx_options also fails (though it doesn't tell me why).

 Does anyone know why MAX_SOCKETS is set to 63 when compiling on Windows
 using mingw? When I look at the source, MAX_SOCKETS_DFLT is set to 1023!

 cr

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Inter thread communication for scalability

[KIU Shueng Chuan]

During socket shutdown, with linger set to 0, messages in-flight would be
dropped?

I use synchronized queues to hold the buffer pointers like the OP but use
zeromq to send a signal to the consumer thread to pop a buffer from the
queue.
On 15 Jan 2014 22:06, Goswin von Brederlow goswin-...@web.de wrote:

 On Tue, Jan 14, 2014 at 03:40:17PM -0500, Lindley French wrote:
  I'm going to caution you about passing pointers through inproc. It may be
  possible to do safely, but I haven't yet figured out how to manage
  ownership semantics in an environment where messages (pointers) can be
  silently dropped.

 Does a PUSH/PULL inproc socket ever drop messages?

 MfG
 Goswin
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Heartbeating using TCP keepalives

[KIU Shueng Chuan]

What should be the behavior if the user specifies 1500ms but the underlying
OS only supports seconds granularity?
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Problem with zeromq and large messages

[KIU Shueng Chuan]

BTW, zmq_recv is implemented using zmq_msg_recv and memcpy, so you get the
same number of mallocs and frees in addition to an extra copy.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Bug report for zeromq

[KIU Shueng Chuan]

Mkstemp() opens the file so there is a file descriptor leak in the patch.

Not sure if you are allowed to bind to an existing regular file either.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Blocking after 2000 sent messages

[KIU Shueng Chuan]

For inproc, the effective hwm is the sum of the send and recv hwm. Default
of 1000 per side gives 2000.

To set hwm to unlimited, you would need to set both send and recv hwm to 0.
You didn't say at which number it blocks after you set only the send hwm to
0.
On Dec 19, 2013 5:20 AM, Garrett Smith g...@rre.tt wrote:

 I'm using czmq. This code:

 zctx_t *ctx = zctx_new ();

 void *output = zsocket_new (ctx, ZMQ_PAIR);
 zsocket_bind (output, inproc://zstr.test);
 void *input = zsocket_new (ctx, ZMQ_PAIR);
 zsocket_connect (input, inproc://zstr.test);

 int i;
 for (i = 0; i  3000; i++) {
 fprintf (stderr,  %i, i);
 zstr_send (output, this is string %d, i);
 }

 will print up to 2000 and then block.

 As this behavior is (seems to be) governed by sndhwm - I set it to 0
 to on output disable it:

 zsocket_set_sndhwm (output, 0);

 But I'm seeing the same behavior.

 I've read The Guide, bought Pieter's book, given presentations on 0MQ
 at conferences and *even* read the API docs. Now I'm trying crowd
 sourcing -- is there an obvious explanation to someone?

 Garrett
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Blocking after 2000 sent messages

[KIU Shueng Chuan]

Okay I tried it.
Set the socket option *before* the bind *and* connect.

Note: I was wrong about having to set *both* sndhwm (of output) and rcvhwm
(of input) to 0 in order to get infinite hwm behaviour.
Setting either to 0 is sufficient. The logic is in socket_base.cpp.


On Thu, Dec 19, 2013 at 7:09 AM, Garrett Smith g...@rre.tt wrote:

 I omitted these details, I actually have this brutally set:

 zsocket_set_sndhwm(output, 0);
 zsocket_set_rcvhwm(output, 0);
 zsocket_set_sndhwm(input, 0);
 zsocket_set_rcvhwm(input, 0);

 The output ends at 2000 (blocking send) - same as when the HWMs aren't
 set.

 On Wed, Dec 18, 2013 at 4:03 PM, KIU Shueng Chuan nixch...@gmail.com
 wrote:
  For inproc, the effective hwm is the sum of the send and recv hwm.
 Default
  of 1000 per side gives 2000.
 
  To set hwm to unlimited, you would need to set both send and recv hwm to
 0.
  You didn't say at which number it blocks after you set only the send hwm
 to
  0.
 
  On Dec 19, 2013 5:20 AM, Garrett Smith g...@rre.tt wrote:
 
  I'm using czmq. This code:
 
  zctx_t *ctx = zctx_new ();
 
  void *output = zsocket_new (ctx, ZMQ_PAIR);
  zsocket_bind (output, inproc://zstr.test);
  void *input = zsocket_new (ctx, ZMQ_PAIR);
  zsocket_connect (input, inproc://zstr.test);
 
  int i;
  for (i = 0; i  3000; i++) {
  fprintf (stderr,  %i, i);
  zstr_send (output, this is string %d, i);
  }
 
  will print up to 2000 and then block.
 
  As this behavior is (seems to be) governed by sndhwm - I set it to 0
  to on output disable it:
 
  zsocket_set_sndhwm (output, 0);
 
  But I'm seeing the same behavior.
 
  I've read The Guide, bought Pieter's book, given presentations on 0MQ
  at conferences and *even* read the API docs. Now I'm trying crowd
  sourcing -- is there an obvious explanation to someone?
 
  Garrett
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] signaler::send() called after signaler::~signaler()

[KIU Shueng Chuan]

Hi All,

While trying to get mingw32-compiled czmq selftest to run under wine, I
occasionally get a Bad file number assertion triggered by zauth selftest.

The assertion occurs in signaler_t::send(). Adding some printfs indicate
that the send() method is being called *after* the signaler object has
already been destructed.

Is there a synchronization problem somewhere in ctx.cpp? I don't think it
has anything to do with czmq zauth selftest, nor even that I was running
the Windows codepath.


Note: I observe this for both libzmq master and zeromq4-x.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Blocking issues with signaler_t::make_fdpair

[KIU Shueng Chuan]

I think the code could be modified such that if the signaler port is 5905,
then the old event mechanism is used, else if it is not 0, then a new mutex
mechanism would be used.

Signaler port equal to 0 would be the new default. People who still prefer
fixed port would choose their own port of choice other than 5905.

It's still a compile time choice though. If the default use of ephemeral
ports works, don't think it's worth the effort to make it runtime
configurable.
On Dec 11, 2013 12:43 AM, Koby Boyango kob...@mce-sys.com wrote:

 Sorry for my late reply, been sick for a few days. I've done some tests
 using the make_fdpair from the master, and it seems like using the
 ephemeral port support and avoiding the locking solved it. Thanks!
 But I do believe that if supporting a fixed signaler port is still
 desired, we should better protect against the scenarios I've described in
 my first mail. What do you think?

 Koby


 On Tue, Dec 10, 2013 at 12:37 AM, KIU Shueng Chuan nixch...@gmail.comwrote:

 I believe no permission is needed to do a pull request. :)

 Upon rereading Koby's mail more closely, his problem can be reproduced by
 having one background program use version 3.2.2. The leaked event handle
 ensures that the global event stays alive and doesn't get recreated each
 time by Windows.
  On Dec 10, 2013 2:44 AM, Felipe Farinon 
 felipe.fari...@powersyslab.com wrote:

  As Koby didn't answered, and I am not able to reproduce the problem
 anymore, could I make the modification even being unable to reproduce the
 problem (indirectly it will be tested, since I am going to run the
 modification in the same environment where the problem was happening)?

 Em 01/12/2013 21:27, KIU Shueng Chuan escreveu:

 In master, you can switch to using ephemeral ports by modifying
 signaler_port to 0 in config.hpp. A new ephemeral port is used per
 make_fdpair call and no critical section is used.

 Could you try that and see if it solves your problems?
 On Dec 1, 2013 9:39 PM, Koby Boyango kob...@mce-sys.com wrote:

   Hi
  I'm fairly new to ZeroMQ, and have been working on integrating it
 using czmq in several projects, Windows only.
  I've opened an issue on GitHub*, *#767, and to Pieter's request I'm
 moving the discussion here. So here is what I've written there:
 While trying to integrate ZeroMQ in different modules\processes
 (Windows only), I've encountered a problem where in some situations a
 ZeroMQ call blocks - forever. After debugging the issue, I've found out
 that zmq_init wasn't returning, and after further debugging and digging
 through the code I've found out that the problem was in
 signaler_t::make_fdpair, where the WaitForSingleObject on the
 zmq-signaler-port-sync didn't return.
 Initially i wasn't sure in which situations it occurs. So I did some
 further investigation and found out that in my case:

- For some reason, when I close a test program with Ctrl+C, the
event stays un-signaled. Not sure why yet, will need further debugging.
- I had a node.js script, which uses ZeroMQ, running in the
background. Because it uses version 3.2.2 of libzmq, which leaks the 
 event
handle, the existing event wasn't deleted, and stayed in an un-signaled
state.
- Basically, from that point no one on the system can use ZeroMQ.

 I find make_fdpair to be very problematic on Windows:

- If one call exits without signaling the event, while someone else
is holding a handle to the event - All further calls on the system will
block. It can happen, for example, if an assertion fails, and the 
 process
crashes because of the exception raised.
- It can also happen if an assertion has failed, an exception was
raised, but caught by the caller using a __try  __except block (SEH). 
 We
can't simply rely on the exception to crash the process (for example, a
program might wrap calls to its plugins with __try  __except, so a 
 faulty
plugin won't crash the while program).
- So it basically means that one faulty program can cause other,
unrelated programs, to block.

 I suggest:

- No matter which synchronization mechanism is used, wrap the code
with __try  __finally, and release the lock in the finally block. This
will make sure that we'll release in case of an exception (In my case,
though, I tried it and it didn't help. the thread might be terminated
during the call).
- If possible, don't use a global, system wide, lock. From my
understanding, it is used in order to reuse the signaler port. So either
use a random, available, port, or make the port libzmq instance 
 specific
(the first calls binds on a random port, further calls will reuse the 
 port)
and protect it with critical section. This will at least limit the 
 problems
to the same process.
- If the system wide lock is really needed, I suggest using a mutex
instead of the event. When using a mutex, if the owning thread dies 
 without
releasing

Re: [zeromq-dev] czmq_selftest fails after fresh clone

[KIU Shueng Chuan]

You should be right that the 64-bit alignment assertion is bogus. Even the
openssl manpage of sha1_update doesn't mention any alignment requirements.
The czmq included version doesn't seem to have any requirement either.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_msg_move

[KIU Shueng Chuan]

1) yes, needed.
2) yes, it should. After the move, the origin message returns to an init-ed
state. You could get away with not closing, but in my opinion that relies
on knowing the underlying implementation.
3) yes. However in this case, closing the origin message is mandatory.
(Unless you also know that very short messages are copied and not shared)

It would be simpler to remember to match all inits with a corresponding
close.

I implemented a simple multi part message c++ wrapper using the move and
copy functions.
https://github.com/pijyoi/msg_parts/blob/master/msg_parts.hpp
 On Dec 9, 2013 9:35 PM, Laurent Alebarde l.aleba...@free.fr wrote:

  Hi Devs,

 1) Should zmq_msg_move be preceded by a zmq_msg_init of the dest ? Since No
 actual copying of message content is performed, *dest* is simply updated
 to reference the new content., I expect the answer is no.

  If true, a small documentation update to precise it would be nice.

 2) Should the origin message be zmq_msg_close-d after zmq_msg_move ?

 3) I assume it is the same for zmq_msg_copy ?

 Cheers,


 Laurent

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Blocking issues with signaler_t::make_fdpair

[KIU Shueng Chuan]

I believe no permission is needed to do a pull request. :)

Upon rereading Koby's mail more closely, his problem can be reproduced by
having one background program use version 3.2.2. The leaked event handle
ensures that the global event stays alive and doesn't get recreated each
time by Windows.
On Dec 10, 2013 2:44 AM, Felipe Farinon felipe.fari...@powersyslab.com
wrote:

  As Koby didn't answered, and I am not able to reproduce the problem
 anymore, could I make the modification even being unable to reproduce the
 problem (indirectly it will be tested, since I am going to run the
 modification in the same environment where the problem was happening)?

 Em 01/12/2013 21:27, KIU Shueng Chuan escreveu:

 In master, you can switch to using ephemeral ports by modifying
 signaler_port to 0 in config.hpp. A new ephemeral port is used per
 make_fdpair call and no critical section is used.

 Could you try that and see if it solves your problems?
 On Dec 1, 2013 9:39 PM, Koby Boyango kob...@mce-sys.com wrote:

   Hi
  I'm fairly new to ZeroMQ, and have been working on integrating it using
 czmq in several projects, Windows only.
  I've opened an issue on GitHub*, *#767, and to Pieter's request I'm
 moving the discussion here. So here is what I've written there:
 While trying to integrate ZeroMQ in different modules\processes (Windows
 only), I've encountered a problem where in some situations a ZeroMQ call
 blocks - forever. After debugging the issue, I've found out that zmq_init
 wasn't returning, and after further debugging and digging through the code
 I've found out that the problem was in signaler_t::make_fdpair, where the
 WaitForSingleObject on the zmq-signaler-port-sync didn't return.
 Initially i wasn't sure in which situations it occurs. So I did some
 further investigation and found out that in my case:

- For some reason, when I close a test program with Ctrl+C, the event
stays un-signaled. Not sure why yet, will need further debugging.
- I had a node.js script, which uses ZeroMQ, running in the
background. Because it uses version 3.2.2 of libzmq, which leaks the event
handle, the existing event wasn't deleted, and stayed in an un-signaled
state.
- Basically, from that point no one on the system can use ZeroMQ.

 I find make_fdpair to be very problematic on Windows:

- If one call exits without signaling the event, while someone else
is holding a handle to the event - All further calls on the system will
block. It can happen, for example, if an assertion fails, and the process
crashes because of the exception raised.
- It can also happen if an assertion has failed, an exception was
raised, but caught by the caller using a __try  __except block (SEH). We
can't simply rely on the exception to crash the process (for example, a
program might wrap calls to its plugins with __try  __except, so a faulty
plugin won't crash the while program).
- So it basically means that one faulty program can cause other,
unrelated programs, to block.

 I suggest:

- No matter which synchronization mechanism is used, wrap the code
with __try  __finally, and release the lock in the finally block. This
will make sure that we'll release in case of an exception (In my case,
though, I tried it and it didn't help. the thread might be terminated
during the call).
- If possible, don't use a global, system wide, lock. From my
understanding, it is used in order to reuse the signaler port. So either
use a random, available, port, or make the port libzmq instance specific
(the first calls binds on a random port, further calls will reuse the 
 port)
and protect it with critical section. This will at least limit the 
 problems
to the same process.
- If the system wide lock is really needed, I suggest using a mutex
instead of the event. When using a mutex, if the owning thread dies 
 without
releasing it, Windows automatically releases it and the next call to
WaitForSingleObject will return WAIT_ABANDONED, and do not block. We can
than check if the port was left in a listening state, close it if
necessary, and re-listen with a new socket.

 I'm using libzmq 4.0.1 with czmq 2.0.2. I saw that the make_fdpair was
 improved in the master, but I believe it still doesn't entirely solve it.
  What do you say?

 Koby

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org

Re: [zeromq-dev] czmq_selftest fails after fresh clone

[KIU Shueng Chuan]

The assertion is raised in the selftest of zfile.
The problem is in zchunk, used by zfile_digest() which uses
zdigest_update().

On a 32-bit platform, sizeof(zchunk_t) is 12-bytes. Based on the memory
allocation method in zchunk_new(), the data buffer would then only be
32-bit aligned.

I suppose the appropriate fix depends on whether zchunk should be making
any guarantees about the alignment of its internal buffer.



On Mon, Dec 9, 2013 at 4:10 AM, Pieter Hintjens p...@imatix.com wrote:

 On Sun, Dec 8, 2013 at 5:59 PM, Ingo Koinzer i...@koinzer.net wrote:

  I cloned the czmq repository like described for the FileMQ
  implementation (https://github.com/zeromq/filemq/blob/master/README.md).
  By doing ./configure  make check I hit the error lt-czmq_selftest:
  zdigest.c:92: zdigest_update: Assertion `((long) buffer  7L) == 0'
  failed. when running the selftests. Following the command line output.
  I'm new to ZeroMQ so please tell me if I did something wrong.

 Interesting. What OS/architecture are you on? Try removing that
 assertion, just comment the line out, and see what happens when you
 run make check?

 -Pieter
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Blocking issues with signaler_t::make_fdpair

[KIU Shueng Chuan]

In master, you can switch to using ephemeral ports by modifying
signaler_port to 0 in config.hpp. A new ephemeral port is used per
make_fdpair call and no critical section is used.

Could you try that and see if it solves your problems?
On Dec 1, 2013 9:39 PM, Koby Boyango kob...@mce-sys.com wrote:

 Hi
 I'm fairly new to ZeroMQ, and have been working on integrating it using
 czmq in several projects, Windows only.
 I've opened an issue on GitHub*, *#767, and to Pieter's request I'm
 moving the discussion here. So here is what I've written there:
 While trying to integrate ZeroMQ in different modules\processes (Windows
 only), I've encountered a problem where in some situations a ZeroMQ call
 blocks - forever. After debugging the issue, I've found out that zmq_init
 wasn't returning, and after further debugging and digging through the code
 I've found out that the problem was in signaler_t::make_fdpair, where the
 WaitForSingleObject on the zmq-signaler-port-sync didn't return.
 Initially i wasn't sure in which situations it occurs. So I did some
 further investigation and found out that in my case:

- For some reason, when I close a test program with Ctrl+C, the event
stays un-signaled. Not sure why yet, will need further debugging.
- I had a node.js script, which uses ZeroMQ, running in the
background. Because it uses version 3.2.2 of libzmq, which leaks the event
handle, the existing event wasn't deleted, and stayed in an un-signaled
state.
- Basically, from that point no one on the system can use ZeroMQ.

 I find make_fdpair to be very problematic on Windows:

- If one call exits without signaling the event, while someone else is
holding a handle to the event - All further calls on the system will block.
It can happen, for example, if an assertion fails, and the process crashes
because of the exception raised.
- It can also happen if an assertion has failed, an exception was
raised, but caught by the caller using a __try  __except block (SEH). We
can't simply rely on the exception to crash the process (for example, a
program might wrap calls to its plugins with __try  __except, so a faulty
plugin won't crash the while program).
- So it basically means that one faulty program can cause other,
unrelated programs, to block.

 I suggest:

- No matter which synchronization mechanism is used, wrap the code
with __try  __finally, and release the lock in the finally block. This
will make sure that we'll release in case of an exception (In my case,
though, I tried it and it didn't help. the thread might be terminated
during the call).
- If possible, don't use a global, system wide, lock. From my
understanding, it is used in order to reuse the signaler port. So either
use a random, available, port, or make the port libzmq instance specific
(the first calls binds on a random port, further calls will reuse the port)
and protect it with critical section. This will at least limit the problems
to the same process.
- If the system wide lock is really needed, I suggest using a mutex
instead of the event. When using a mutex, if the owning thread dies without
releasing it, Windows automatically releases it and the next call to
WaitForSingleObject will return WAIT_ABANDONED, and do not block. We can
than check if the port was left in a listening state, close it if
necessary, and re-listen with a new socket.

 I'm using libzmq 4.0.1 with czmq 2.0.2. I saw that the make_fdpair was
 improved in the master, but I believe it still doesn't entirely solve it.
 What do you say?

 Koby

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] pyzmq use ioloop the threading version work and the multiprocessing can't work

[KIU Shueng Chuan]

In threading module, the __init__() method is executed by the parent
thread. Only the run() method executes in the newly spawned thread.

So you should create the context in the run() method.

Since multiprocessing module follows the threading module interface, I
suppose the same thing applies.
On Nov 18, 2013 12:22 PM, Mo Jia life.130...@gmail.com wrote:

 Hi all,

  I ask a question in stackoverflow about using pyzmq to write a sample
 server.
  When a change threading to multiprocessing , it can't work any more.

 the mulitprocessing got error on ZMQError: Interrupted system call. while
 the thread version work fine. You can see i create context in the process
 's init func .

 Here is the link .
 http://stackoverflow.com/questions/20031530/zmq-zmqerror-interrupted-system-call-on-multiprocessing-while-the-threading-i
  .



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq::signaler_t::make_fdpair hangs application on crash

[KIU Shueng Chuan]

Which versions of Windows and ZeroMQ does it happen on? Does your
application open and close lots of ZeroMQ sockets? How about with libzmq
master?

Actually you could submit the change to mutex and port yourself?
On Nov 13, 2013 3:02 AM, Felipe Farinon felipe.fari...@powersyslab.com
wrote:

  My application is hanging frequently in one co-worker's machine. Whenever
 I diagnose the machine, I see the event still set with no application
 running. I'm afraid that when the application ships the bug happens much
 more.

 We wouldn't be taking one more port, just changingthe old one.

 Em 12/11/2013 10:52, KIU Shueng Chuan escreveu:

 I thought of the same thing before, port 5906 too. But I don't feel
 comfortable taking up one more port without some consensus. Maybe if there
 were more people reporting that their ZeroMQ applications (with heavy
 socket creation!) were hanging on Windows...

  For now, I have modified signaler.cpp to not assert within the critical
 section.

 https://github.com/zeromq/libzmq/commit/4a7f07a19ae226fe92c3c7320bd425f9a18d0c79


 On Mon, Nov 11, 2013 at 11:16 PM, Felipe Farinon 
 felipe.fari...@powersyslab.com wrote:

  Maybe we could change signaler_port to another value and define that
 port 5905 is protected by the Event and the new port (e.g. 5906) is
 protected by Mutex. This way we don't need to check if the Event is present.

 Em 11/11/2013 11:16, Felipe Farinon escreveu:

 Ok.

 Seems reasonable to me and I think that a 4 seconds timeout is fine. The
 only scenario where I could imagine that this would break is if some heavy
 socket creation is going on and IFF the WaitForSingleObject wakeup order
 for Events is not FIFO.

 Em 11/11/2013 11:02, KIU Shueng Chuan escreveu:

 Realistically, I think only bind, accept and connect have a chance of
 failing. The rest of the asserts just test for programming errors. Accept
 and connect are already handled. What's left is handling bind error.


 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev




 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev




 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq::signaler_t::make_fdpair hangs application on crash

[KIU Shueng Chuan]

I thought of the same thing before, port 5906 too. But I don't feel
comfortable taking up one more port without some consensus. Maybe if there
were more people reporting that their ZeroMQ applications (with heavy
socket creation!) were hanging on Windows...

For now, I have modified signaler.cpp to not assert within the critical
section.
https://github.com/zeromq/libzmq/commit/4a7f07a19ae226fe92c3c7320bd425f9a18d0c79


On Mon, Nov 11, 2013 at 11:16 PM, Felipe Farinon 
felipe.fari...@powersyslab.com wrote:

  Maybe we could change signaler_port to another value and define that port
 5905 is protected by the Event and the new port (e.g. 5906) is protected by
 Mutex. This way we don't need to check if the Event is present.

 Em 11/11/2013 11:16, Felipe Farinon escreveu:

 Ok.

 Seems reasonable to me and I think that a 4 seconds timeout is fine. The
 only scenario where I could imagine that this would break is if some heavy
 socket creation is going on and IFF the WaitForSingleObject wakeup order
 for Events is not FIFO.

 Em 11/11/2013 11:02, KIU Shueng Chuan escreveu:

 Realistically, I think only bind, accept and connect have a chance of
 failing. The rest of the asserts just test for programming errors. Accept
 and connect are already handled. What's left is handling bind error.


 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev




 ___
 zeromq-dev mailing 
 listzeromq-dev@lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq::signaler_t::make_fdpair hangs application on crash

[KIU Shueng Chuan]

Realistically, I think only bind, accept and connect have a chance of
failing. The rest of the asserts just test for programming errors. Accept
and connect are already handled. What's left is handling bind error.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq::signaler_t::make_fdpair hangs application on crash

[KIU Shueng Chuan]

The problem with simply switching to using Mutex is that it would break
compatibility with older versions using an Event.

I tried coding a backwards compatible version.
https://github.com/pijyoi/libzmq/blob/eventhang/src/signaler.cpp
It first locks using a Mutex, then locks again using the old Event
mechanism for backwards compatibility. However, if locking using the Event
times out (1 sec), it assumes that the Event mechanism has hung and
proceeds as though it had acquired the Event (and will release the Event at
the end)

I am not so sure if the assumption that a 1 sec timeout means that it has
hung is valid in all situations though.


On Sat, Nov 9, 2013 at 5:36 PM, Pieter Hintjens p...@imatix.com wrote:

 Hi Felipe,

 Can you reproduce the case? If you can make a patch and test case, we
 can backport this to 4.0 and 3.2 stable if it applies there.

 -Pieter

 On Fri, Nov 8, 2013 at 8:46 PM, Felipe Farinon
 felipe.fari...@powersyslab.com wrote:
  In windows, the code that creates a mutual exclusive section uses the
  'Event' mechanism.
 
   HANDLE sync = CreateEvent (sa, FALSE, TRUE, TEXT
  (Global\\zmq-signaler-port-sync));
   if (sync == NULL  GetLastError () == ERROR_ACCESS_DENIED)
 sync = OpenEvent (SYNCHRONIZE | EVENT_MODIFY_STATE, FALSE, TEXT
  (Global\\zmq-signaler-port-sync));
 
   win_assert (sync != NULL);
 
   //  Enter the critical section.
   DWORD dwrc = WaitForSingleObject (sync, INFINITE);
   zmq_assert (dwrc == WAIT_OBJECT_0);
 
   // Some code here...
 
   brc = SetEvent (sync);
   brc = CloseHandle (sync);
 
 
  win_assert and zmq_asserts abort the program if the conditions passed to
  them aren't met. If the program is aborted before calling 'SetEvent' on
  sync, the event is unsetted forever and all zeromq applications that
  call this code will block on 'WaitForSingleObject'. A preferred way to
  handle this is using 'Mutex' since if this scenario happens,
  WaitForSingleObject will not freeze forever but will return
 WAIT_ABANDONED.
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 --
 -
 Pieter Hintjens
 CEO of iMatix.com
 Founder of ZeroMQ community
 blog: http://hintjens.com
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Question about zmq_recv len and truncate

[KIU Shueng Chuan]

zmq_recv() doesn't really receive directly into your buffer, it just does
the copying from a zmq_msg_t to your buffer.
It was probably created to mimic the BSD sockets API.

You could prevent out-of-memory with socket option of ZMQ_MAXMSGSIZE.


On Sat, Oct 12, 2013 at 4:45 AM, Bruno D. Rodrigues 
bruno.rodrig...@litux.org wrote:

 Hello all,

 I'm curious about a comment on a certain behaviour of zmq_recv that
 affected me when using it over the Java bindings.

 The sentence in question is the following:
 http://api.zeromq.org/3-2:zmq-recv
 The zmq_recv() function shall return number of bytes in the message if
 successful. Note that the value can exceed the value of the len parameter
 in case the message was truncated.

 and on the code, zmq.cpp 493
 //  At the moment an oversized message is silently truncated.

 The context I was trying was a PUB server where I inject random messages,
 which I can't control (JSONs with some hundred bytes to several hundred
 kilobytes), and on the other side a SUB consuming them. This was over the
 Java bindings, using zmq git master (or 4.0.1) and jzmq 3 git master (or
 3.0.1).

 If I use the simpler memcpy heavy versions (send(byte[]) and byte[] x =
 socket.recv()), everything works as expected, but I was trying the other
 alternatives that allows to use direct ByteBuffers and avoid the memcpy
 operations.
 Or explicitly, the recv(byte[] buffer, int len) and the recv(no
 copy)(ByteBuffer but, int len)

 What happens is that unless I pre-create a buffer of the maximum expected
 size (like 16MB), messages bigger than my buffer size will not only be
 truncated, but will also break on java when the callback tries to set the
 ByteBuffer position to a value larger than the limit.
 jzmq Socket.cpp
 703 setByteBufferPosition(env, buffer, rc);
 or
 728 env-CallVoidMethod(buffer, setPositionHandle, pos + rc);

 On top of this, as my C/C++ is quite forgotten since I was forced to move
 to Java, I couldn't follow exactly where the zmq_msg_t content is filled
 up, but a grep by malloc scared me as it seems possible to telnet into the
 server, send the right header prefix, and then announce an incoming message
 of several GB, which is blindly followed by the malloc, to then be
 discarded by the truncate reported above. (this can be mitigated by setting
 a maxMessageSize somewhere depending on the bindings, it seems, but hadn't
 tested it yet)

 In simpler words:
 - is it possible to receive a message of unknown size using the methods
 that receive my own buffers? If not, why do those methods exist at all?
 - is it possible to crash a server with out of memory by sending a couple
 message headers with a huge len value?

 I'd like to help fixing whatever I can do, but I'm lost now on where to
 pick stuff up, architecturally speaking. could the recv(buffer) return the
 next part of the same message, as I expected?

 Thanks in advance

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] CZMQ Visual Studio

[KIU Shueng Chuan]

Hi, I just tried creating an x86 MSVC import lib for a libzmq.dll generated
by MINGW32 on 32-bit Windows.  I did get it to work. I imagine
cross-compiled, 64-bit versions and libczmq.dll would work too.

There's no C++ name mangling issue since libzmq exports a C interface.
The errors you encountered when compiling czmq with MSVC is probably due to
the C99 style used in czmq.

Follow instructions on the web to create a .def file:
nm libzmq.dll | findstr /C: T _z  libzmq.def

then edit the .def file so that it looks like this:
EXPORTS
zmq_bind
zmq_close
...

How does your .def file look like?

Generate the .lib file:
lib.exe /machine:x86 /def:libzmq.def

Tested it out with a simple zeromq program compiled with MSVC.


On Mon, Oct 7, 2013 at 1:43 PM, HG Choi c.hog...@gmail.com wrote:

 To Developers,

 Even though I do not enjoy dealing with Windows, I must do so at this
 point in time. I have successfully cross-compiled libzmq, libczmq, and
 libfmq for x86_64 Windows using Mingw under Linux. All three were tested
 using the built selftests.exe. So now I know it is possible to use all
 three libraries in Windows.

 But the issue is that the current QT project that I am tying to integrate
 FileMQ (zmq, czmq) into is a MSVC project. Thus my cross compiled mingw c++
 libs are not compatible with the msvc compiler. I have been told this is
 due to C++ name mangling issues.

 http://www.mingw.org/wiki/MixingCompilers

 I have tried to create mvsc compatible libs through DLLs using the visual
 studio dumpbin, dlltool, and def file method, but also no luck. Maybe
 someone has succeeded?

 Cross compiliation with x86_64-w64-mingw32 toolset to Windows definitely
 works, but as far as I know, for MSVC, I am not sure. I have tried using
 the i586-mingw32msvc but without success..

 For my situation, the ideal method of building these libraries is to use
 the msvc solution files on a Windows host, but I could only get libzmq to
 build cleanly without errors.

 Below is the errors I receive from a fresh czmq-master on Visual Studio
 2010.
 http://pastebin.com/7piDizJA

 If anyone has any clues to what is causing the errors or successfully
 built CZMQ and FMQ using Visual Studio for Windows x86_64, I would really
 appreciate the guidance.

 Thank you for reading,

 Ho-Gyun Choi

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] CZMQ Windows: zmutex_try_lock()

[KIU Shueng Chuan]

I made a comment on this at:
https://github.com/zeromq/czmq/commit/d44e4dbbfda10c8458d4d48e1c8925f3b6fd354c
 On Oct 4, 2013 3:32 PM, HG Choi c.hog...@gmail.com wrote:

 To Developers,

 I have recently just built the latest ZMQ and CZMQ for Windows x86_64
 using x86_64_w64-mingw32 toolset from an Ubuntu Linux machine. I have run
 into a small problem when running czmqselftest.exe. The zmutex test seems
 to fail with an assertion at line 159 at the zmutex_try_lock() function. If
 anyone could shed some light on the issue, I would much appreciate it.

 Thank you,

 HG Choi

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Version 4.0 release fork

[KIU Shueng Chuan]

You could generate the import library from the provided libsodium-4.def
file per the instructions at http://www.mingw.org/wiki/MSVC_and_MinGW_DLLs.

libsodium seems to build fine using MinGW's msys, yielding the same output
as what comes in the pre-compiled tarball.
make check passes.



On Mon, Sep 23, 2013 at 10:58 PM, Steven McCoy steven.mc...@miru.hk wrote:

 On 23 September 2013 10:38, AJ Lewis aj.le...@quantum.com wrote:

  I have installers for Windows built sans *libsodium *but mushrooming to
   30MB to include static libraries.  Investigating *libsodium*, looks
 more
  tedium as the developers are not conducive to anything other than
  *Autoconf* although
  there is a *CMake *pull request that was rejected and that may be
 utilised.
 
  https://github.com/jedisct1/libsodium/pull/74/files

 That's a little depressing - does it look like it'll build without a ton
 of
 tweaking if the build system is fixed up?


 Maybe, the libsodium people says their binary release should work with
 MSVC but there are no import libraries and no 64-bit builds.  I only had a
 brief time to investigate, hopefully someone else can have a look this week.

 --
 Steve-o

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Trouble building CZMQ master with libzmq installed to ${prefix}/lib64

[KIU Shueng Chuan]

Hi,

Using OpenSUSE 12.3 x64 with distribution packaged ZeroMQ 3.2.2 installed
to /usr/lib64.

Built libzmq master with:
./configure --prefix=/home/user/local
Libraries get installed to /home/user/local/lib64
(The same problem occurs if I don't supply the prefix, it just gets
installed to /usr/local/lib64 instead)

Build czmq master with
./configure --with-libzmq=/home/user/local
Build fails with:

/bin/sh ../libtool --tag=CC   --mode=link gcc -std=gnu99 -g
-I/home/user/local/include -g -O2  -L/home/user/local/lib  -o czmq_selftest
czmq_selftest.o libczmq.la -lzmq -lsodium -lpthread
libtool: link: gcc -std=gnu99 -g -I/home/user/local/include -g -O2 -o
.libs/czmq_selftest czmq_selftest.o  -L/home/user/local/lib
./.libs/libczmq.so -L/usr/local/lib -lzmq -lsodium -lpthread
./.libs/libczmq.so: undefined reference to `zmq_z85_decode'
./.libs/libczmq.so: undefined reference to `zmq_z85_encode'

Problem is that the Makefile provides
-L/home/user/local/lib
instead of
-L/home/user/local/lib64

Not familiar with autotools to provide a fix for this.

Note: Build of libzmq tests link fine with libsodium installed to
/usr/local/lib64
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Trouble building CZMQ master with libzmq installed to ${prefix}/lib64

[KIU Shueng Chuan]

According to
http://stackoverflow.com/questions/4791699/gcc-looks-for-headers-in-usr-local-include-when-compiling-but-not-for-librarie,
it seems it's going to be a problem to have distro packaged zmq development
headers in /usr and libzmq master in /usr/local

Current workaround, install libzmq master to home user directory:
./configure --prefix=/home/user/local

To compile CZMQ against system installed ZeroMQ 3.2.2:
./configure

To compile CZMQ against user's own libzmq
CFLAGS=-I/home/user/local/include LDFLAGS=-L/home/user/local/lib64
./configure

I couldn't get the following options to do what I wanted:
--with-libzmq, --with-libzmq-include-dir, --with-libzmq-lib-dir



On another note, shouldn't the libzmq installed libraries get their version
bumped to libzmq.so.4?
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Trouble building CZMQ master with libzmq installed to ${prefix}/lib64

[KIU Shueng Chuan]

Correction:
./configure --with-libzmq=/home/user/local
does work, but I needed to link lib64 to lib
ln -s /home/user/local/lib64 /home/user/local/lib



On Sun, Sep 22, 2013 at 11:20 AM, KIU Shueng Chuan nixch...@gmail.comwrote:

 According to
 http://stackoverflow.com/questions/4791699/gcc-looks-for-headers-in-usr-local-include-when-compiling-but-not-for-librarie,
 it seems it's going to be a problem to have distro packaged zmq development
 headers in /usr and libzmq master in /usr/local

 Current workaround, install libzmq master to home user directory:
 ./configure --prefix=/home/user/local

 To compile CZMQ against system installed ZeroMQ 3.2.2:
 ./configure

 To compile CZMQ against user's own libzmq
 CFLAGS=-I/home/user/local/include LDFLAGS=-L/home/user/local/lib64
 ./configure

 I couldn't get the following options to do what I wanted:
 --with-libzmq, --with-libzmq-include-dir, --with-libzmq-lib-dir



 On another note, shouldn't the libzmq installed libraries get their
 version bumped to libzmq.so.4?

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Simple questions

[KIU Shueng Chuan]

Was looking at the following updated zmq_msg_close documentation:
Note that this is NOT necessary after a successful _zmq_msg_send()_.

As a test, I tried calling zmq_msg_close() twice after a send.
The first returns 0 while the second returns -1.
This seems to imply that send does not fully invalidate the msg?



On Sat, Apr 27, 2013 at 6:46 PM, Pieter Hintjens p...@imatix.com wrote:

 On Sat, Apr 27, 2013 at 10:11 AM, Alexey Melnichuk mi...@newmail.ru
 wrote:

  I expect this too. But is it documented behaviour or implementation
 detail?

 It's not documented properly but IMO it does not make sense that _send
 takes ownership and then the caller still has to call _close... I'll
 patch the documentation so this is clear.

  And one more question.
  Do i need call zmq_msg_init before copy/move?

 Yes, I guess you do.

 -Pieter
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] SUB socket can fail to subscribe if PUB socket is reset

[KIU Shueng Chuan]

The 0x00 0x00 sent by both pub and sub is the 0-length identity frame. I.e.
no explicitly set identity.

I made a mistake in my previous mail, final-short is 2 bytes long, the
length is part of it.
On Aug 13, 2013 6:00 PM, Shan Wang shan.w...@ig.com wrote:

 Thanks,

 ** **

 That document makes sense. But do you know in what situation the SUB
 socket can send 0x00 0x00 to PUB? Does that mean a final short with length
 0?

 ** **

 Also why PUB send 0x00 0x00 to SUB after the signature? 

 ** **

 One thing I forgot to mention, once the SUB socket is broken, it can not
 recover, means if I keep on restarting PUB sockets, the SUB will keep on
 receiving wrong sequence of messages and no other data.

 ** **

 ** **

 *From:* zeromq-dev-boun...@lists.zeromq.org [mailto:
 zeromq-dev-boun...@lists.zeromq.org] *On Behalf Of *KIU Shueng Chuan
 *Sent:* 12 August 2013 22:56
 *To:* ZeroMQ development list
 *Subject:* Re: [zeromq-dev] SUB socket can fail to subscribe if PUB
 socket is reset

 ** **

 You should read http://rfc.zeromq.org/spec:15 instead.

 The five bytes are:
 2 bytes identity frame.
 3 bytes subscription frame
   0: final-short
   1: payload length
   1: code for subscribe

 On Aug 12, 2013 7:05 AM, Shan Wang shan.w...@ig.com wrote:

 Hi All,

 I'm trying to use 0MQ in our project, but recently I found a very tricky
 problem in the pub/sub pattern.

 the setup is like this:

 version: 3.2.3, c++ API.

 transport protocol: tcp

 one SUB socket, subscribe to everything, high water mark = 1
 one PUB socket, high water mark = 1

 there is application level heartbeat between pub and sub, so if SUB socket
 discover there is no heartbeat from PUB, it assumes PUB is gone and will
 reset itself.
 This means whenever PUB is shutdown, SUB will close the socket, create it
 again, reconnect, and resubscribe.
 (The heartbeat is not necessary on normal PUB shutdown, but is essential
 in situation like PUB server power down or cable cut.)

 What I found is, if I keep on restarting PUB server, ie. once every 5
 seconds, the SUB socket can fail to subscribe and never receive anything
 from PUB.
 This is very hard to reproduce and happens once in a few hundred times.

 Unfortunately we are not allowed to use tcpdump here, so all I can do is
 use strace on the io thread to monitor socket functions.
 When it's broken, the messages SUB exchange with PUB after reconnect is
 like this:

 recvfrom(53, \377\0\0\0\0\0\0\0\1\177, 12, 0, NULL, NULL) = 10
 | 0  ff 00 00 00 00 00 00 00  01 7f
 recvfrom(53, 0x7f881000b9db, 2, 0, 0, 0) = -1 EAGAIN (Resource temporarily
 unavailable)
 sendto(53, \377\0\0\0\0\0\0\0\1\177\1\2, 12, 0, NULL, 0) = 12
 | 0  ff 00 00 00 00 00 00 00  01 7f 01 02
 recvfrom(53, \1\1, 2, 0, NULL, NULL)  = 2
 | 0  01 01
 recvfrom(53, \0\0, 8192, 0, NULL, NULL) = 2
 | 0  00 00
 sendto(53, \0\0, 2, 0, NULL, 0)   = 2

 The last line is what broke it, when it's normal, the last line look like
 this:
 sendto(54, \0\0\0\1\1, 5, 0, NULL, 0) = 5

 After reading this ZMTP document:
 http://rfc.zeromq.org/spec:23
 I still can't crack the exact meanings of these messages, but I attempt to
 explain these messages like this:
 After connected, SUB tries to read 12 bytes from PUB, it got 10 initially,
 but received the last two eventually as 01 01.(I guess the 12 bytes are
 some signature)
 Then SUB sends 12 byts to PUB, with the first 10 bytes the same, but last
 two bytes as 01 02.
 Then SUB tries to read 8192 bytes, but only got two, which are 00 00
 Then SUB decides to send 00 00 back to PUB, and that's it, there will be
 nothing coming from this SUB socket.
 The TCP connection is still there, so there must be something wrong with
 the subscription.

 And in the normal case, SUB will send 00 00 00 11 11 instead of 00 00, I
 guess this must be the subscription message, but i don't understand why
 it's 5 bytes?

 So can someone please explain the exchange sequence on PUB/SUB, and maybe
 shed some lights on what is broken here?

 Thanks very much
 Shan


 The information contained in this email is strictly confidential and for
 the use of the addressee only, unless otherwise indicated. If you are not
 the intended recipient, please do not read, copy, use or disclose to others
 this message or any attachment. Please also notify the sender by replying
 to this email or by telephone (+44(020 7896 0011) and then delete the email
 and any copies of it. Opinions, conclusion (etc) that do not relate to the
 official business of this company shall be understood as neither given nor
 endorsed by it. IG is a trading name of IG Markets Limited (a company
 registered in England and Wales, company number 04008957) and IG Index
 Limited (a company registered in England and Wales, company number
 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
 London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG
 Index Limited (register number 114059

Re: [zeromq-dev] SUB socket can fail to subscribe if PUB socket is reset

[KIU Shueng Chuan]

You should read http://rfc.zeromq.org/spec:15 instead.

The five bytes are:
2 bytes identity frame.
3 bytes subscription frame
  0: final-short
  1: payload length
  1: code for subscribe
 On Aug 12, 2013 7:05 AM, Shan Wang shan.w...@ig.com wrote:

 Hi All,

 I'm trying to use 0MQ in our project, but recently I found a very tricky
 problem in the pub/sub pattern.

 the setup is like this:

 version: 3.2.3, c++ API.

 transport protocol: tcp

 one SUB socket, subscribe to everything, high water mark = 1
 one PUB socket, high water mark = 1

 there is application level heartbeat between pub and sub, so if SUB socket
 discover there is no heartbeat from PUB, it assumes PUB is gone and will
 reset itself.
 This means whenever PUB is shutdown, SUB will close the socket, create it
 again, reconnect, and resubscribe.
 (The heartbeat is not necessary on normal PUB shutdown, but is essential
 in situation like PUB server power down or cable cut.)

 What I found is, if I keep on restarting PUB server, ie. once every 5
 seconds, the SUB socket can fail to subscribe and never receive anything
 from PUB.
 This is very hard to reproduce and happens once in a few hundred times.

 Unfortunately we are not allowed to use tcpdump here, so all I can do is
 use strace on the io thread to monitor socket functions.
 When it's broken, the messages SUB exchange with PUB after reconnect is
 like this:

 recvfrom(53, \377\0\0\0\0\0\0\0\1\177, 12, 0, NULL, NULL) = 10
 | 0  ff 00 00 00 00 00 00 00  01 7f
 recvfrom(53, 0x7f881000b9db, 2, 0, 0, 0) = -1 EAGAIN (Resource temporarily
 unavailable)
 sendto(53, \377\0\0\0\0\0\0\0\1\177\1\2, 12, 0, NULL, 0) = 12
 | 0  ff 00 00 00 00 00 00 00  01 7f 01 02
 recvfrom(53, \1\1, 2, 0, NULL, NULL)  = 2
 | 0  01 01
 recvfrom(53, \0\0, 8192, 0, NULL, NULL) = 2
 | 0  00 00
 sendto(53, \0\0, 2, 0, NULL, 0)   = 2

 The last line is what broke it, when it's normal, the last line look like
 this:
 sendto(54, \0\0\0\1\1, 5, 0, NULL, 0) = 5

 After reading this ZMTP document:
 http://rfc.zeromq.org/spec:23
 I still can't crack the exact meanings of these messages, but I attempt to
 explain these messages like this:
 After connected, SUB tries to read 12 bytes from PUB, it got 10 initially,
 but received the last two eventually as 01 01.(I guess the 12 bytes are
 some signature)
 Then SUB sends 12 byts to PUB, with the first 10 bytes the same, but last
 two bytes as 01 02.
 Then SUB tries to read 8192 bytes, but only got two, which are 00 00
 Then SUB decides to send 00 00 back to PUB, and that's it, there will be
 nothing coming from this SUB socket.
 The TCP connection is still there, so there must be something wrong with
 the subscription.

 And in the normal case, SUB will send 00 00 00 11 11 instead of 00 00, I
 guess this must be the subscription message, but i don't understand why
 it's 5 bytes?

 So can someone please explain the exchange sequence on PUB/SUB, and maybe
 shed some lights on what is broken here?

 Thanks very much
 Shan


 The information contained in this email is strictly confidential and for
 the use of the addressee only, unless otherwise indicated. If you are not
 the intended recipient, please do not read, copy, use or disclose to others
 this message or any attachment. Please also notify the sender by replying
 to this email or by telephone (+44(020 7896 0011) and then delete the email
 and any copies of it. Opinions, conclusion (etc) that do not relate to the
 official business of this company shall be understood as neither given nor
 endorsed by it. IG is a trading name of IG Markets Limited (a company
 registered in England and Wales, company number 04008957) and IG Index
 Limited (a company registered in England and Wales, company number
 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
 London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG
 Index Limited (register number 114059) are authorised and regulated by the
 Financial Services Authority.
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] segfault after timeout

[KIU Shueng Chuan]

If you are using the s_recv() from zhelpers.h, the return value is either
NULL or a C string that you need to free().
On Aug 11, 2013 3:48 PM, Papp Áron aphex...@gmail.com wrote:

 Hi All,

 I just found the problem on the segmentation fault issue.

 After changing:
 std::string message = s_recv(socket);
 To:
 char* message = s_recv(socket);

 No fault occurs. Btw, until this point, i had no problem using std::string
 for reciving messages...

 Thank you mates for all your work!

 Best,
 Aron




 2013/8/10 Papp Áron aphex...@gmail.com

 Hi All,

 My environment:
 - Linux 3.2.0, 64 bit
 - ZMQ version 3.2.3
 - socket: ZMQ_REQ
 - connection with zmq_connect()
 - protocol: TCP
 - lang: C/C++

 The program is very simple, it connects to a zmq-tcp socket, sends a
 request, waits for an answer, then returns. If no answer comes, it should
 return after timeout.

 The problem is, if I set ZMQ_RCVTIMEO, and exits no replier, it stops
 s_recv() command after the timeout specified in RCVTIMEO, but the program
 exits with Segmentation fault.

 Here's the gdb output:
 Program received signal SIGSEGV, Segmentation fault.
 0x76f6d86f in ?? () from /lib/x86_64-linux-gnu/libc.so.6

 Thanks in advance!

 Best,
 Aron



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Is ZMTP3 backwards compatibility with ZMTP1.0 broken?

[KIU Shueng Chuan]

According to the Backwards Interoperability sections of RFC15 and RFC23,
bit 0 of the flags field is used to probe whether the peer is using
ZMTP/1.0.
So now it needs to be left as %x7F.



On Mon, Aug 12, 2013 at 7:13 AM, Pieter Hintjens p...@imatix.com wrote:

 On Sun, Aug 11, 2013 at 11:19 PM, Merijn Verstraaten
 mer...@inconsistent.nl wrote:

  RFC23 states that a backward compatibility detecting handshake starts as
 follows:
  Send a 10-octet pseudo-signature consisting of %xFF size %x7F where
 'size' is the number of octets in the sender's identity (0 or greater) plus
 1. The size SHALL be 8 octets in network byte order and occupies the
 padding field.
 
  However, RFC13 states that ZMTP1.0 long length messages follow the
 format %xFF size flags, where bit 0 of flags specifies whether there are
 more messages to come, which is wrong for an identity frame. Do existing
 ZMTP1.0 implementations simply ignore this flag on identity frames?

 Good catch. For sure ZMTP 1.0 implementations don't check this, but
 I'm wondering why we chose %x7F. That might be a mistake, based on the
 explanation of the flags field in RFC 13 (bit 0 is put before bits
 1-7). I suspect the intention was to create a valid frame, with the
 reserved bits all set to 1. So, %xFE.

 -Pieter
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] problems with installation under MinGW

[KIU Shueng Chuan]

The libzmq Makefile.mingw32 has been updated. You could update your git
folder and try again.
On Jul 12, 2013 4:48 PM, Laurent Alebarde l.aleba...@free.fr wrote:

  No, of course, I downloaded the Makefile from your link and substitute
 it to the one generated by configure. So, now, going into the
 builds/mingw32 directory and doing : mingw32-make -f Makefile.mingw32, a
 lot of things build, and then I have the following error :

 socket_base.o: In function `ZN3zmq13socket_base_t6createEiPNS_5ctx_tEji':
 C:\MinGW\msys\1.0\home\Administrateur\libzmq\builds\mingw32/../../src/socket_base.cpp:117:
 undefined reference to `zmq::stream_t::stream_t(zmq::ctx_t*, unsigned int,
 int)'
 stream_engine.o: In function `ZN3zmq15stream_engine_t15mechanism_readyEv':
 C:\MinGW\msys\1.0\home\Administrateur\libzmq\builds\mingw32/../../src/stream_engine.cpp:642:
 undefined reference to `zmq::mechanism_t::peer_identity(zmq::msg_t*)'
 stream_engine.o: In function `ZN3zmq15stream_engine_t9handshakeEv':
 C:\MinGW\msys\1.0\home\Administrateur\libzmq\builds\mingw32/../../src/stream_engine.cpp:525:
 undefined reference to
 `zmq::null_mechanism_t::null_mechanism_t(zmq::options_t const)'
 C:\MinGW\msys\1.0\home\Administrateur\libzmq\builds\mingw32/../../src/stream_engine.cpp:530:
 undefined reference to
 `zmq::plain_mechanism_t::plain_mechanism_t(zmq::session_base_t*,
 zmq::options_t const)'
 collect2: ld has returned 1 exec state code (a retourné 1 code d'état
 d'exécution)
 mingw32-make: *** [libzmq.dll] Error 1

 Le 12/07/2013 10:12, KIU Shueng Chuan a écrit :

 Did you run mingw32-make -f Makefile.mingw32 in the builds/mingw32
 directory?
 It is written to have the following rule:
 %.o: ../../src/%.cpp



 On Fri, Jul 12, 2013 at 4:05 PM, Laurent Alebarde l.aleba...@free.frwrote:

  Thank you Shueng Chuan. I have tested Makefile.mingw32. But make still
 fails :

 make: *** No rule to make target `address.o', needed by `libzmq.dll'.
 Stop.

 When I search for address in the original Makefile generated by
 configure, nothing is found.

 Le 12/07/2013 01:32, KIU Shueng Chuan a écrit :

  Hi Laurent,

  An alternative is to just use the Makefile in the following directory
 https://github.com/zeromq/zeromq3-x/tree/master/builds/mingw32

  If necessary, change the optimization flags.


 On Thu, Jul 11, 2013 at 4:56 PM, KIU Shueng Chuan nixch...@gmail.comwrote:

 Not all the tests have been ported to run under Windows.
 Check the Makefile, probably there is an entry there that you could
 invoke to only compile the library.



 On Thu, Jul 11, 2013 at 4:34 PM, Laurent Alebarde l.aleba...@free.frwrote:

  Thanks a lot Shueng Chuan. I did a :

 find libzmq/ -exec dos2unix.exe '{}' ';'

 and then ./autogen.sh did its job without error.

 Now, I have another problem when I run make :

 test_sub_forward.cpp: In function 'int main()':
 test_sub_forward.cpp:65:21: error: variable 'main()::timespec t' has
 initializer but incomplete type
 test_sub_forward.cpp:66:24: error: 'nanosleep' was not declared in this
 scope
 make[1]: *** [test_sub_forward.o] Error 1
 make: *** [all-recursive] Error 1

 It seems from

- http://www.opendebug.com/article/489046
- http://lists.gnu.org/archive/html/bug-gnulib/2010-04/msg00032.html

 that it is relative to the pthread implementation. I have to dig more,
 but if you have the solution, you are welcome of course.

 Cheers,


 Laurent.


 Le 10/07/2013 23:35, KIU Shueng Chuan a écrit :

 One of the autogen related files has DOS line endings instead of Unix
 line feeds. Run dos2unix on it. I think it's the m4 utility that is choking
 on the DOS carriage returns.
 On Jul 11, 2013 4:22 AM, Laurent Alebarde l.aleba...@free.fr wrote:

  Hi list,

 I need to test my app under several versions of Windows, at least XP
 32, XP 64, and 7, in the MinGW and MinGW-64 environments. I have Git 
 Cloned
 from GitHub, and I am following the INSTALL file, which starts with a
 ../autogen.sh which fails first :

 $ ./autogen.sh
 autogen.sh: error: could not find libtool. libtool is required to run
 autogen.sh

 I added libtool to the MinGW environment, then autogen runs, but fails
 :

 autoreconf-2.68: Entering directory `.'
 autoreconf-2.68: configure.ac: not using Gettext
 autoreconf-2.68: running: aclocal -I config --force -I config
 configure.ac:431: error: `\ ' is already registered with
 AC_CONFIG_FILES.
 /mingw/src/autoconf/26/autoconf2.5-2.68-1/src/autoconf-2.68/lib/autoconf/status.m4:290:
 AC_CONFIG_FILES is expanded from...
 configure.ac:431: the top level
 /bin/m4: cannot remove temporary directory /tmp/ar2800.3924/m4-kvQiAN:
 Directory not empty
 autom4te-2.68: /bin/m4 failed with exit status: 1
 aclocal-1.11: /mingw/bin/autom4te-2.68 failed with exit status: 1
 autoreconf-2.68: aclocal failed with exit status: 1
 autogen.sh: error: autoreconf exited with status 0

 Any idea please ? Or a link to a working procedure for 0MQ under MinGW
 ?

 I know there exist a rubby script, but only for 32 bits, so I have