Re: [zeromq-dev] Who said that?
Martin Sustrik said the following on 12/21/2010 2:08 AM: On 12/20/2010 09:21 PM, Oliver Smith wrote: Martin Sustrik said the following on 12/17/2010 5:15 AM: By bad data you mean malformed 0MQ frames? With 2.1 those are discarded silently and the connection is closed. We should also report the problem via sys://log. No, it's probably data getting reused before it is transmitted - i.e. my issue, but it's very hard to track down because I don't know which of the many nodes is sending it :) The only way I can track that down is to send everything to separate ports and collate at the receiver, until I'm done debugging, and then reunify all of them again. Ok. So what's needed is message tracking for debugging purposes. Fair enough. So what about, for example, writing a simple wrapper over zmq_send() that would attach a message part containing identity of the particular node to every message? Ah - I was arguing this particular incident as one of several reasons why it might be beneficial to have access to the prior-hop information above (i.e. in the application) the zmq_recv(). Another being WAN-transit cases where you do want to avoid exposing any form of internal network-state information from the sender into the packet beyond what is already deducible. That is: you don't actually care what the IP address and port of the last-sender/forwarder are, just that one is different than the other. [Image: An application calling recv() on a single zmq::socket which is abstracting multiple bsd::socket or inproc sockets connected to any number/type of sources] Another use case: cache efficiency. lastSrc = None cache = None while True: msg = zmqSock.recv() src = msg.socketMuxDistinguisher() if src != lastSrc: cache = cacheForSrc[lastSrc] doWork(msg, cache) if itsTimeForACacheCheck(): expireOldCaches() It could be made exceedingly clear that this is NOT any form of a source/return address by allowing the user to provide the local distinguisher. I'm not proposing, in any way or form, that this provide a value that the user ever pass to any ZMQ function. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Who said that?
Erh: cache = cacheForSrc[src] lastSrc = src doWork(msg, cache) ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Who said that?
On 12/16/2010 10:39 PM, Oliver Smith wrote: I'm not actually looking for the sender's address so much as who */I/* got the message from, IE the previous hop. While the address and port would be nice for diagnosing sources of bad data within a fabric (I spent 2 days this week tracking down what turned out to be a bad stick of ram in a machine corrupting outgoing data), it could be any thread-unique value that discretely identifies a given TCP-socket-pair connection to this zmq context. By bad data you mean malformed 0MQ frames? With 2.1 those are discarded silently and the connection is closed. We should also report the problem via sys://log. Martin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Who said that?
Hi Oliver, Trying to re-remember how to obtain the return address for a given ZMQ message, or the ZMQ representation thereof (a UUID or something). The message originator might be behind a NAT system, so the originating address could be useless (there are too many 192.168.0.1s in the world). What I want to know is, where does the ZMQ interface I got this message from think it came from? So that I can distinguish messages from different incoming streams behind the multiplexed socket I'm talking to. Consider the following pseudo code: msg = interface.receive() user = userLookup(msg.streamIdentifier) if user: user.handle(msg) else: loginOrGTFO(msg) I realize I could just slap some kind of GUID into the payload, but in these particular message streams, bits are at a premium. I recognize that the sender address/port in an IP packet should never be trusted any more than the words PRIORITY MAIL or YOU'VE WON $10,000,000! on a physical envelope, however in my particular use case - and for similar reasons - it makes an excellent starting point. There's no API to get sender's address. The reason is that in multi-hop scenarios is not even clear, who's the sender: The original sender or the device at the last hop? However, with XREQ/XREP sockets the identites of individual hops are aggregated in the request (as initial message parts). Check the guide for details. Martin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Who said that?
Martin Sustrik said the following on 12/16/2010 6:41 AM: There's no API to get sender's address. The reason is that in multi-hop scenarios is not even clear, who's the sender: The original sender or the device at the last hop? I'm not actually looking for the sender's address so much as who */I/* got the message from, IE the previous hop. While the address and port would be nice for diagnosing sources of bad data within a fabric (I spent 2 days this week tracking down what turned out to be a bad stick of ram in a machine corrupting outgoing data), it could be any thread-unique value that discretely identifies a given TCP-socket-pair connection to this zmq context. Consider: p = sock.recv() if p.previousHopID() in sessions: p = decrypt(p, sessions[p.previousHopID].encryptionStuff) p = process(p) sock.send(p) else: session = Session(p.previousHopID()) sessions[p.previousHopID()] = session sock.setsockopt(zmq.SENDMORE, True) sock.send(encrypt(salt, server.encryptionStuff)) p = process(p) sock.setsockopt(zmq.SENDMORE, False) sock.send(p) Note carefully: If the previousHopID() was the IP/port, you could easily get confused when a node behind NAT disconnects and another node subsequently connects with the same NATed ip/port. However, with XREQ/XREP sockets the identites of individual hops are aggregated in the request (as initial message parts). Check the guide for details. As I said - I can't put the ID information in the WAN packets. I just have to not use ZMQ for the WAN transit. I'm using a half-baked little ZMQ-TCP-TCP-ZMQ bridge for now. - Oliver ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Who said that?
Trying to re-remember how to obtain the return address for a given ZMQ message, or the ZMQ representation thereof (a UUID or something). The message originator might be behind a NAT system, so the originating address could be useless (there are too many 192.168.0.1s in the world). What I want to know is, where does the ZMQ interface I got this message from think it came from? So that I can distinguish messages from different incoming streams behind the multiplexed socket I'm talking to. Consider the following pseudo code: msg = interface.receive() user = userLookup(msg.streamIdentifier) if user: user.handle(msg) else: loginOrGTFO(msg) I realize I could just slap some kind of GUID into the payload, but in these particular message streams, bits are at a premium. I recognize that the sender address/port in an IP packet should never be trusted any more than the words PRIORITY MAIL or YOU'VE WON $10,000,000! on a physical envelope, however in my particular use case - and for similar reasons - it makes an excellent starting point. - Oliver ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
