Re: [zeromq-dev] Pull request to retire devices and replace with proxies

2012-09-09 Thread Brian Granger
I still prefer device. On Sun, Sep 9, 2012 at 3:47 AM, Pieter Hintjens p...@imatix.com wrote: On Sun, Sep 9, 2012 at 3:21 PM, Michael Haberler mai...@mah.priv.at wrote: fan-in/fan-out, inlet/outlet, Two halves don't make a whole. dispatcher, hub, feeder, Not bad. bushing, busbar,

Re: [zeromq-dev] Pull request to retire devices and replace with proxies

2012-09-07 Thread Brian Granger
I have not followed this very closely, but it is not clear to me that proxy is better than device. I can definitely imagine usage cases for devices that are not very proxy-ish. I probably also don't feel that the trouble of renaming is worth it. Cheers, Brian On Fri, Sep 7, 2012 at 12:53 PM,

Re: [zeromq-dev] [ANNOUNCE] ZeroMQ/2.2.0 stable released

2012-04-04 Thread Brian Granger
On Wed, Apr 4, 2012 at 7:21 AM, Santy, Michael michael.sa...@dynetics.com wrote: In theory bindings aim to work with all released versions of 0MQ, though it's up to each binding group to make this work. I've been meaning to bring up this topic for a while.  It seems to me that knowing that a

Re: [zeromq-dev] ZMQ Python with Pypy

2012-03-05 Thread Brian Granger
Justin, Hold on a second though. PyZMQ uses Cython, which, as far as I know, doesn't run on pypy. You will need to look at the ctypes based version here: https://github.com/svpcom/pyzmq-ctypes Cheers, Brian On Mon, Mar 5, 2012 at 1:05 AM, Justin Cook jhc...@gmail.com wrote: Sorry to keep

Re: [zeromq-dev] multi-part messages

2012-01-22 Thread Brian Granger
On Sun, Jan 22, 2012 at 7:48 AM, Pieter Hintjens p...@imatix.com wrote: On Sun, Jan 22, 2012 at 9:36 AM, john skaller skal...@users.sourceforge.net wrote: The problem is like this: if you use say an array, you can put the pieces into it in any temporal order. This allows you to write simple

Re: [zeromq-dev] multi-part messages

2012-01-22 Thread Brian Granger
multipart messages in the network protocols we were using. The idea and implementation are brilliant, and Martin definitely gets the credit for this (even if he may regret it...). Cheers, Brian On Sun, Jan 22, 2012 at 11:43 AM, Brian Granger elliso...@gmail.com wrote: On Sun, Jan 22, 2012 at 7:48 AM

Re: [zeromq-dev] Proposal for libzmq maintenance

2012-01-10 Thread Brian Granger
On Tue, Jan 10, 2012 at 4:17 AM, MinRK benjami...@gmail.com wrote: On Mon, Jan 9, 2012 at 23:46, Martin Lucina mar...@lucina.net wrote: On Mon, 9 Jan 2012 23:20:01 -0600 Pieter Hintjens p...@imatix.com wrote: [snip] It would be good to see pull request explicity defined as an e-mail

Re: [zeromq-dev] 0MQ/3.1

2011-11-08 Thread Brian Granger
I vote option 2 as well. On Mon, Nov 7, 2011 at 5:11 PM, MinRK benjami...@gmail.com wrote: On Mon, Nov 7, 2011 at 15:17, Pieter Hintjens p...@imatix.com wrote: On Tue, Nov 8, 2011 at 1:51 AM, Martin Sustrik sust...@250bpm.com wrote: It's up to Pieter whether he wants to maintain 3-0

Re: [zeromq-dev] XREP/ROUTER no longer receive messages atomically in 3.0/4.0

2011-09-21 Thread Brian Granger
On Sat, Sep 17, 2011 at 11:52 PM, Martin Sustrik sust...@250bpm.com wrote: On 09/06/2011 12:23 AM, Pieter Hintjens wrote: On Monday, September 5, 2011, MinRK wrote:     Okay, then I will put my official vote behind making RCVMORE as the     sole-indicator of a contiguous message.  This would

Re: [zeromq-dev] using websockets

2011-09-21 Thread Brian Granger
We are using Tornado + PyZMQ regularly for ZMQ/WS connections. It works extremely well. On Tue, Sep 20, 2011 at 8:58 AM, Noospheer Team t...@noospheer.org wrote: Hi, I want to able to point servers running 0mq at each other across the web using the standard websocket URI, ie. ws://example.com

Re: [zeromq-dev] multiplexer pattern

2011-09-20 Thread Brian Granger
Martin, On Sun, Sep 18, 2011 at 12:58 AM, Martin Sustrik sust...@250bpm.com wrote: Hi Benjamin, The request/reply is moving to what it really is: A way for clients to access a set of homogenous and stateless services, providing a way to load-balance and handle failure in distributed systems.

Re: [zeromq-dev] first message never received when SUB binds (0MQ 3)

2011-09-06 Thread Brian Granger
On Tue, Sep 6, 2011 at 10:12 AM, MinRK benjami...@gmail.com wrote: On Tue, Sep 6, 2011 at 01:02, Martin Sustrik sust...@250bpm.com wrote: On 09/04/2011 08:47 PM, MinRK wrote: When binding with SUB, and connecting with PUB, it seems impossible to receive the first message. This is caused

Re: [zeromq-dev] [ANNOUNCE] ZeroMQ/2.1.9 stable is now available

2011-08-29 Thread Brian Granger
Thanks Min! On Mon, Aug 29, 2011 at 5:36 PM, MinRK benjami...@gmail.com wrote: PyZMQ eggs, MSI installers have been updated to 2.1.9. homebrew package manager also updated to 2.1.9. -MinRK On Mon, Aug 29, 2011 at 14:37, Pieter Hintjens p...@imatix.com wrote: Oh, thanks for catching that.

Re: [zeromq-dev] ZMQ_IDENTITY revisited

2011-07-13 Thread Brian Granger
Martin, Given, that ZMQ_IDENTITY is a biggest source of complexity in the codebase (while at the same time being semantically unsound) I'll have to remove it from the codebase sooner or later be able to develop the product further. So, what will happen is that we'll have 0MQ/2.x (old API,

Re: [zeromq-dev] best way to integrate pyzmq pyqt event loops

2011-04-26 Thread Brian Granger
Hi, Here is example of how we are doing this in IPython: https://github.com/ipython/ipython/blob/master/IPython/zmq/ipkernel.py#L450 But, we are not using the IOLoop in this example. If you also want to use the Qt event loop for IO related things, you will probably need to integrate the file

Re: [zeromq-dev] Moving to 0MQ/3.0, yes or no?

2011-04-10 Thread Brian Granger
Martin (and others), In the recent month, the discussion about 0MQ/3.0 have shown that the radical parting with the past design flaws goal of the endeavour cannot be achieved. Every tweak has its user that will be hurt if its removed. That makes the 3.0 version more or less useless: The flaws

Re: [zeromq-dev] 0MQ conference program

2011-04-08 Thread Brian Granger
for 100-200 participants. Should be great. We have some folks near Berkeley, but really any where we can find facilities in the bay area would work. Brian Granger, who suggested the Bay area as location, has kindly agreed to join the organizing team. I'd like to ask for more volunteers

Re: [zeromq-dev] Zero-copy message API

2011-04-03 Thread Brian Granger
Pieter, In pyzmq, we exploit the non-copying message API in very subtle ways, so this is something we will have to watch very carefully. Getting non-copying messages working with a language like Python is very difficult and subtle and I strongly encourage the zeromq devs to look at how we handle

Re: [zeromq-dev] ZMQ and Gevent

2011-03-17 Thread Brian Granger
Hi, On Thu, Mar 17, 2011 at 1:45 AM, Linker linker.m@gmail.com wrote: Hi,All!     The GEvent is a very cool project for using coroutine in Python. My question is could we use ZeroMQ with GEvent in Python? Thanks! I think this is working (link provided by S. McCoy). It uses pyzmq and the

Re: [zeromq-dev] Discuss: 0MQ releases and stability

2011-02-16 Thread Brian Granger
Because of the scale of zeromq development, I would recommend that all new development go on in topic branches. There's a real conflict of interest here. Code stays on topic branches until it's 'ready' but without being tried by fairly large numbers of people it takes a long time to become

Re: [zeromq-dev] Discuss: 0MQ releases and stability

2011-02-15 Thread Brian Granger
Hi, Proposals: 1. Stop using topic branches for new development, use the master for this Because of the scale of zeromq development, I would recommend that all new development go on in topic branches. When a topic branch is ready for merging into master, open a pull request, do a review

Re: [zeromq-dev] MongoDB driver with pyzmq

2011-01-22 Thread Brian Granger
Dan, As Min mentioned, this is something we have thought about before. But, to do this right, you would need to modify the MongoDB code itself to talk zeromq, rather than http. Just bridging zeromq-http-mongodb is much less interesting (but still of some value). If you are intersted in going the

Re: [zeromq-dev] Tornado loops

2011-01-12 Thread Brian Granger
Daniel, This is my first post to this list since I only started working with it last week. So far I have to say it is mind-blowing! Thank you for this! Glad you like it! Currently I am using the Python binding and everything works quite nicely so far! When I read about Tornado's eventloop

Re: [zeromq-dev] Are IronPython bindings for ZeroMQ available?

2011-01-12 Thread Brian Granger
On Tue, Jan 11, 2011 at 12:43 PM, Brad Smith bnsm...@gmail.com wrote: I noticed a thread on this mailing list in which IronPython bindings were mentioned: http://lists.zeromq.org/pipermail/zeromq-dev/2010-April/003368.html More specifically, someone wrote this: But you can simply use the

Re: [zeromq-dev] Incompatibility between PyZMQ and uWSGI

2010-11-23 Thread Brian Granger
Jaime, I think the issue is that uWSGI likely uses an asynchronous event loop underneath with non-blocking sockets. When zeromq/pyzmq is used in that context, zeromq socket have to be integrated into the event loop at a very low level using: 1) zmq_poll (for zeromq 2.0.x and 2.1.x) 2)

Re: [zeromq-dev] Connecting to a different socket after connecting and sending messages to one socket?

2010-11-20 Thread Brian Granger
On Sat, Nov 20, 2010 at 8:01 AM, Praveen Baratam praveen.bara...@gmail.comwrote: Dear Chuck, Thanks for writing back. I am aware of the solution you suggested using an intermediate broker. But in the architecture we are planning, introducing an intermediate broker node will create a bottle

Re: [zeromq-dev] More security thoughts

2010-11-17 Thread Brian Granger
On Wed, Nov 17, 2010 at 1:23 AM, Diablo 666 thediablo...@hotmail.de wrote: Hi, (you don't really want CPU bound work in IO threads) I don't actually see why this would be a problem. But maybe a special encryption thread could solve this issue. This thread would encrypt the message and

Re: [zeromq-dev] Proposal for new build system maintainer: Mikko Koppanen

2010-11-17 Thread Brian Granger
On Wed, Nov 17, 2010 at 2:55 AM, Martin Sustrik sust...@250bpm.com wrote: Hi all, I would like to propose Mikko Koppanen to become an official co-maintainer of 0MQ build system together with Mato Lucina. Mikko is running 0MQ build environment (build.valokuva.org) and have valuably

Re: [zeromq-dev] PyZMQ 2.0.10

2010-11-16 Thread Brian Granger
Many thanks to Min for his excellent work on PyZMQ for this release!!! PyZMQ has really taken a huge step forward because of his efforts. Cheers, Brian On Tue, Nov 16, 2010 at 10:21 AM, MinRK benjami...@gmail.com wrote: Hello list, I cut pyzmq 2.0.10 last night, and pushed it to the

[zeromq-dev] More security thoughts

2010-11-16 Thread Brian Granger
Hi, We (the pyzmq devs) have been thinking and talking a lot about security lately and Min has been prototyping various ideas. I wanted to report back on our findings as I think they are relevant to the larger security discussion. For the record, there are 3 main security approaches that are

Re: [zeromq-dev] how to integrate zmq to normal epoll web server

2010-11-16 Thread Brian Granger
I know you may be tied to C++, but the Python bindings have a nice event loop that is integrates with the Tornado web server. Cheers, Brian On Tue, Nov 16, 2010 at 8:04 PM, technical issue techwe...@gmail.comwrote: Hello all, I am a newbie of ZMQ. I want to write a web server in C++ that

Re: [zeromq-dev] EncryptedSocket added to pyzmq in branch

2010-11-03 Thread Brian Granger
Hi, As original poster, I think I did a poor job of clarifying exactly what we are seeking feedback on. Let me attempt that now and summarize where we are at in our collective thinking on this issue 1. Is the EncryptedSocket interface that Min has prototyped in his branch a good plugin

Re: [zeromq-dev] EncryptedSocket added to pyzmq in branch

2010-11-03 Thread Brian Granger
I have changed no bindings. This is a *new* object that allows you to conveniently do encryptsend.  I did change send(..., encrypt=True) to send_encrypted(...), so it looks more like existing extensions in pyzmq. But then doesn't that mean that all of our helper methods like: send_multipart

Re: [zeromq-dev] EncryptedSocket added to pyzmq in branch

2010-11-02 Thread Brian Granger
On Tue, Nov 2, 2010 at 1:31 AM, Pieter Hintjens p...@imatix.com wrote: On Tue, Nov 2, 2010 at 8:41 AM, Martin Sustrik sust...@250bpm.com wrote: One problem I can see is that you can enable/disable encryption on per-message basis, which presumably means you have encryption bit stored somewhere

Re: [zeromq-dev] EncryptedSocket added to pyzmq in branch

2010-11-02 Thread Brian Granger
On Tue, Nov 2, 2010 at 3:53 AM, Burak Arslan burak.ars...@arskom.com.tr wrote:  On 11/02/10 10:31, Min RK wrote: There's no encrypted bit or anything. This isn't 'Encrypted ZMQ', it's just an object that encrypts messages prior to sending them. Think of it more as a wrapper than anything.  

Re: [zeromq-dev] EncryptedSocket added to pyzmq in branch

2010-11-02 Thread Brian Granger
Pieter, Thanks for the summary... 0MQ is not as such secure, any more than its underlying transports are secure.  That is, you don't expect TCP to do encryption, and neither does 0MQ.  However, 0MQ applications that carry data across the Internet _do_ need security and there's been quite a

Re: [zeromq-dev] PyZmq on Python 2.4

2010-11-01 Thread Brian Granger
Sylvain, PyZMQ does not support Python 2.4. It is difficult enough to support 2.5-3.1. Cheers, Brian On Mon, Nov 1, 2010 at 1:08 PM, Sylvain Demers sylvain.dem...@ubisoft.com wrote: Hello all, The next weeks, I need to evaluate PyZmq. I have to create the Python 2.4 bindings (python

[zeromq-dev] EncryptedSocket added to pyzmq in branch

2010-11-01 Thread Brian Granger
Hi, There has been a lot of discussion about adding security features like encryption to 0MQ. We have been talking/thinking about this some in the Python bindings (pyzmq) because we are at the point where we are deploying applications using 0MQ at the internet scale. Just tonight, Min posted

Re: [zeromq-dev] python binding, REQ/REP blocking, apparent deadlock

2010-10-16 Thread Brian Granger
I have actually not used zeromq or pyzmq in really long connections like you are describing, but I think some others here have reported issues with that. You might poke around the list archives for threads on this topic. Cheers, Brian On Thu, Oct 14, 2010 at 3:30 PM, George Coles

[zeromq-dev] Changes to dev build process of pyzmq

2010-10-04 Thread Brian Granger
Hi, This might affect a number of you who are using pyzmq from github. The .c files of pyzm are autogenerated by Cython. We used to include these autogenerated .c files in the github repo, but because they change often and in significant ways, the pyzmq repo was growing faster than we want.

Re: [zeromq-dev] odd zmqstream.py behavior

2010-10-04 Thread Brian Granger
Hi, Min and I looked at this today and this is a bug in pyzmq. When a callback in the eventloop raises, we are currently closing the socket, which we should not do. We are still figuring out the error handling symantics, but we should have a fix in place soon in pyzmq master. Cheers, Brian

Re: [zeromq-dev] Encryption (OpenSSL/TLS)

2010-09-30 Thread Brian Granger
Great topic... 1) VPN level encryption, handled UNDER zmq. 2) SSL level encryption, handled BY zmq 3) message level encryption, handled ON TOP OF zmq. #2 is not really possible, for the previously-mentioned zmq system features of intelligent message routing by intermediate nodes.  However,

[zeromq-dev] Eventlet, zeromq and websockets

2010-09-29 Thread Brian Granger
Hi, I just got a note about this from Ben Ford one of the developers of Eventlet. Eventlet is a coroutine-base networking framework for Python that has recently added zeromq support (through pyzmq). Ben just pointed me to this example of a zeromq-websocket bridge for a distributed chat system.

Re: [zeromq-dev] Assertion failed: fetched (rep.cpp:232) with XREQ + REP

2010-09-29 Thread Brian Granger
On Tue, Sep 28, 2010 at 11:37 PM, Martin Sustrik sust...@250bpm.com wrote: On 09/28/2010 11:17 PM, Brian Granger wrote: Thanks everyone for pointing our my mistake here.  But I then have a secondary question.  Currently it is possible to crash a REP socket by not adhering to the REQ/REP

Re: [zeromq-dev] pyzmq-static 2.0.8

2010-09-29 Thread Brian Granger
of these files will have its own .c and .so files. Static linking libzmq.a into all of those .so files individually would almost impossible. Cheers, Brian On Wed, Sep 29, 2010 at 3:47 PM, Brian Granger elliso...@gmail.com wrote: Yes, I saw that about a week ago.  Unfortunately it looks like they actually

Re: [zeromq-dev] synchronizing pub and sub

2010-09-28 Thread Brian Granger
On Sun, Sep 26, 2010 at 6:20 AM, Pieter Hintjens p...@imatix.com wrote: Here is a more robust model for pubsub synchronization: * Publisher opens PUB socket and starts sending Hello messages (not data). * Subscribers connect SUB socket and when they receive a Hello message they tell the

Re: [zeromq-dev] [Request] Custom load balancer/routers

2010-09-28 Thread Brian Granger
On Mon, Sep 27, 2010 at 11:29 PM, Thijs Terlouw thijsterl...@gmail.com wrote: Would it be possible to make the load-balancer for the 2.1 branch of ZeroMQ more easily customizable? +1 I see mentions to lb_t inside push and xreq but the lb_t class doesn't appear to be extensible because the

Re: [zeromq-dev] Assertion failed: fetched (rep.cpp:232) with XREQ + REP

2010-09-28 Thread Brian Granger
Thanks everyone for pointing our my mistake here. But I then have a secondary question. Currently it is possible to crash a REP socket by not adhering to the REQ/REP protocol. Don't we want to make it more difficult to crash a REP server than this? Cheers, Brian On Tue, Sep 28, 2010 at 3:47

Re: [zeromq-dev] ZeroMQ Organization ownership...

2010-09-28 Thread Brian Granger
of this, which is to bring leading members of the community into the 'owners' team of the ZeroMQ organization on github.  This is not a legal entity or anything, simply a user group that has rights to create projects within that structure, bring in collaborators, etc. I'm happy to announce that Brian

[zeromq-dev] Assertion failed: fetched (rep.cpp:232) with XREQ + REP

2010-09-24 Thread Brian Granger
Hi, This was a big from earlier this summer that I am still seeing. I am observing an assert error in line rep.cpp:232 when connecting a REP socket to bound XREQ socket. I guess my first question is this: Is this combination of sockets supported? That is, should this work? Here is a

Re: [zeromq-dev] [PATCH] Added Python code for weather update PUB/SUB example.

2010-09-23 Thread Brian Granger
Brian, It looks like this patch was not against pyzmq/master. It looks like you created this patch on top of a branch that already had the wuserver and wuclient files. Can you recreate a clean patch on top of pyzmq/master or just send me the full files? Thanks! Brian On Sun, Sep 19, 2010 at

Re: [zeromq-dev] Python Code Example: Synchronized Pub Sub

2010-09-23 Thread Brian Granger
Pieter, On Thu, Sep 9, 2010 at 1:33 AM, Pieter Hintjens p...@imatix.com wrote: Hunter, Thanks, the two examples are now added to the repository and should show up. What repo did you add these examples to? Would it make sense to have them in the pyzmq examples? Cheers, Brian -Pieter On

Re: [zeromq-dev] [PATCH] Added Python code for weather update PUB/SUB example.

2010-09-23 Thread Brian Granger
... Cheers, Brian - Pieter On 23 Sep 2010 19:58, Brian Granger elliso...@gmail.com wrote: Brian, It looks like this patch was not against pyzmq/master. It looks like you created this patch on top of a branch that already had the wuserver and wuclient files. Can you recreate a clean patch

Re: [zeromq-dev] Querying socket type

2010-09-23 Thread Brian Granger
On Thu, Sep 23, 2010 at 2:46 PM, gonzalo diethelm gdieth...@dcv.cl wrote: I believe there is no way to query what type a socket is; is this right? I could use such a function and can think of a couple of use cases. One of them would be adding many sockets (of possibly different types) to a

Re: [zeromq-dev] message_t reusing

2010-09-22 Thread Brian Granger
On Wed, Sep 22, 2010 at 1:16 AM, Ilja Golshtein ilej...@narod.ru wrote: Hello! Please, clarify if it is Ok == zmq::message_t msg; socket_a.recv(msg);l You need to do a copy of the msg before sending it the first time. This is because a send takes ownership of your msg and actually init's

Re: [zeromq-dev] Deadlock between Python GC/zmq.Socket.close() and zmq_free_fn () both acquiring GIL

2010-09-22 Thread Brian Granger
We just pushed a new MessageTracker object that let's you query when zeromq is done with a message: http://github.com/zeromq/pyzmq/blob/master/zmq/_zmq.pyx#L211 This may help in resolving this issue. Let us know if you have any questions. Cheers, Brian On Tue, Sep 21, 2010 at 10:20 AM, Brian

Re: [zeromq-dev] message_t reusing

2010-09-22 Thread Brian Granger
On Wed, Sep 22, 2010 at 2:47 PM, Pieter Hintjens p...@imatix.com wrote: On Wed, Sep 22, 2010 at 2:08 PM, Pieter Hintjens p...@imatix.com wrote: Strange.  I remember discussing this with Martin Sustrik and hearing, when a message is sent 0MQ nullifies it. OK, so the user guide is wrong on

Re: [zeromq-dev] message_t reusing

2010-09-22 Thread Brian Granger
. Cheers, Brian On Wed, Sep 22, 2010 at 3:24 PM, Brian Granger elliso...@gmail.com wrote: On Wed, Sep 22, 2010 at 2:47 PM, Pieter Hintjens p...@imatix.com wrote: On Wed, Sep 22, 2010 at 2:08 PM, Pieter Hintjens p...@imatix.com wrote: Strange.  I remember discussing this with Martin Sustrik

Re: [zeromq-dev] Deadlock between Python GC/zmq.Socket.close() and zmq_free_fn () both acquiring GIL

2010-09-21 Thread Brian Granger
Martin, We have been working on some things in a branch of pyzmq that may help solve this issue. Here is the idea: * If you allow non-copy sends of mutable data, you need a way to know when zeromq is completely done with the message. * We have create a MessageTracker object that send(msg,

[zeromq-dev] PyZMQ version 2.0.8 officially released.

2010-09-21 Thread Brian Granger
Hi, I am please to announce the 2.0.8 release of PyZMQ: http://github.com/zeromq/pyzmq/downloads http://pypi.python.org/pypi?:action=displayname=pyzmqversion=2.0.8 Not too much has changed in this release and it is designed to work with zeromq 2.0.8, but may also work with 2.0.9 and 2.0.7. As

Re: [zeromq-dev] [PATCH] Added Python code for weather update PUB/SUB example.

2010-09-20 Thread Brian Granger
Brian, Thanks, I will include this in the pyzmq examples... Brian On Sun, Sep 19, 2010 at 12:58 AM, Brian Dorsey briandor...@gmail.com wrote: I've attached Python code for the wuserver / wuclient examples from chapter 1 of the guide, MIT license. Take care, -Brian

Re: [zeromq-dev] ZeroConf on top of ZeroMQ

2010-09-15 Thread Brian Granger
On Wed, Sep 15, 2010 at 2:16 AM, Martin Sustrik sust...@250bpm.com wrote: On 09/15/2010 11:10 AM, Pieter Hintjens wrote: On Wed, Sep 15, 2010 at 10:54 AM, Gaspard Buchergasp...@teti.ch  wrote: Because in my environment (art installations, stage performances) nobody runs a DNS server... And

Re: [zeromq-dev] ruby rbzmq and ffi-rzmq gems

2010-09-15 Thread Brian Granger
On Wed, Sep 15, 2010 at 8:09 AM, Chuck Remes cremes.devl...@mac.com wrote: Right now the zmq and ffi-rzmq gems have slightly different APIs. In a perfect world, an implementation detail like C extension versus FFI would be completely hidden so users could switch back and forth without

Re: [zeromq-dev] Rogue clients and/or bad networks (issues 30, 65)

2010-09-13 Thread Brian Granger
On Tue, Sep 7, 2010 at 6:05 PM, Kelly Brock ke...@inocode.com wrote: Hi folks, Given that the 0MQ code has not bee through any kind of security audit (however formal/informal that may be) we should not be claiming that 0MQ 2.0.x is in any way suitable for deployment of Internet-connected

Re: [zeromq-dev] Ctrl+C problem fixed in master

2010-09-13 Thread Brian Granger
Thanks for fixing this Martin!!! Cheers, Brian On Wed, Sep 8, 2010 at 1:19 AM, Martin Sustrik sust...@250bpm.com wrote: Hi all, The problem with interpreted languages bindings being unresponsive to Ctrl+C was fixed in the master branch (0MQ/2.1 to be). Thanks to Brian Granger, Daisuke

Re: [zeromq-dev] Segmentation fault with 2.0.8 in zmq_msg_close(), possible PyZMQ issue (or not)

2010-09-13 Thread Brian Granger
On Wed, Sep 8, 2010 at 9:26 AM, Martin Lucina m...@kotelna.sk wrote: sust...@250bpm.com said: On 7/9/2010, Martin Lucina m...@kotelna.sk wrote: This would seem to indicate that message has already been freed, possibly by Python. However, what I'm wondering about is:  If it fails on line

[zeromq-dev] Possible Windows bug with XREQ/XREP

2010-09-13 Thread Brian Granger
Hi, I think we have found a subtle bug in 2.0.7 on Windows. It is a bit subtle and hard to reproduce, but here is an overview: * A client has an XREQ socket that connects to a servers XREP socket that is bound to a port. * The client is doing basic request/reply with a single server * While

Re: [zeromq-dev] Segmentation fault with 2.0.8 in zmq_msg_close(), possible PyZMQ issue (or not)

2010-09-13 Thread Brian Granger
On Mon, Sep 13, 2010 at 6:40 PM, Thijs Terlouw thijsterl...@gmail.com wrote: I'm seeing a segmentation fault in zmq_msg_close() called from zmq::pipe_t::~pipe_t(). This is on 2.0.8 with a Python application so the problem may be related to Python memory management issues, but maybe not. We

Re: [zeromq-dev] 2.0.8dev version of pyzmq

2010-09-13 Thread Brian Granger
On Mon, Sep 6, 2010 at 9:02 AM, Brian Granger elliso...@gmail.com wrote: On Mon, Sep 6, 2010 at 12:23 AM, Martin Lucina m...@kotelna.sk wrote: p...@imatix.com said: On Thu, Sep 2, 2010 at 6:03 PM, Martin Lucina m...@kotelna.sk wrote: I noticed that when updating to the 2.0.8 PUSH/PULL you

Re: [zeromq-dev] 2.0.8dev version of pyzmq

2010-09-06 Thread Brian Granger
On Mon, Sep 6, 2010 at 12:23 AM, Martin Lucina m...@kotelna.sk wrote: p...@imatix.com said: On Thu, Sep 2, 2010 at 6:03 PM, Martin Lucina m...@kotelna.sk wrote: I noticed that when updating to the 2.0.8 PUSH/PULL you also removed the DOWNSTREAM/UPSTREAM constants. Was that deliberate?

Re: [zeromq-dev] my Arm Story

2010-09-06 Thread Brian Granger
On Mon, Aug 30, 2010 at 6:37 AM, Moritz Rosenthal m...@m2mgermany.de wrote: Hi! at #zeromq they said it might be of interest to some receivers of this list, so I'm gonna tell my short story of try, error and success. How I built 0MQ for my embedded ARM: First I wanted to try it at my PC.

Re: [zeromq-dev] Trapping of EINTR in blocking recv and poll

2010-09-06 Thread Brian Granger
Martin, I totally respect your choices on this one. A few notes below... On Sun, Sep 5, 2010 at 12:26 AM, Martin Sustrik sust...@imatix.com wrote: Hi Brian, Another reason I would like to get this in to a 2.0.9 release is that 2.1 involves lots of new changes that are likely to make things

Re: [zeromq-dev] Twisted ZeroMQ Reactor

2010-09-06 Thread Brian Granger
Timothy, This is great! Let's keep in touch if you need anything done to pyzmq to support this. I have used Twisted a lot and it would be very nice to have zmq support in twisted. On Sat, Sep 4, 2010 at 4:10 PM, Timothy Fitz timothyf...@gmail.com wrote: For those of you looking for Twisted

Re: [zeromq-dev] Deadlock between Python GC/zmq.Socket.close() and zmq_free_fn () both acquiring GIL

2010-09-06 Thread Brian Granger
Mato, I am actually on vacation/holiday this week, could you file an issue on github for me to track this: http://github.com/zeromq/pyzmq/issues Next week when i return I am going to be doing some pyzmq work and should be able to look at this. Cheers, Brian On Fri, Sep 3, 2010 at 2:56 PM,

Re: [zeromq-dev] Device numbering in 0MQ/2.0.8

2010-09-04 Thread Brian Granger
On Sat, Sep 4, 2010 at 1:39 AM, Pieter Hintjens p...@imatix.com wrote: Hi Folks, 2.0.8 includes a special feature known as OMG what did Pieter do there? Basically I renumbered the device constants to make the order more logical but this means any code calling zmq_device(3) that was

Re: [zeromq-dev] Routing messages??

2010-09-03 Thread Brian Granger
Min, Why is the null-message delimiter needed? Brian On Fri, Sep 3, 2010 at 9:47 AM, MinRK benjami...@gmail.com wrote: Yes, I believe REQ should work with a null-message delimiter, if that fits your send pattern: xrep.send_multipart(['a', '', 'the message']) On Fri, Sep 3, 2010 at 02:55,

Re: [zeromq-dev] zmq_poll not waiting for timeout if a socket is coming up

2010-09-03 Thread Brian Granger
On Fri, Sep 3, 2010 at 4:50 AM, Martin Lucina m...@kotelna.sk wrote: sust...@250bpm.com said: On 09/03/2010 12:51 PM, Martin Lucina wrote: How would that get you better performance? You'd still have to deal with the case where ZMQ_FD would become ready and ZMQ_EVENTS would return nothing

Re: [zeromq-dev] zmq_poll not waiting for timeout if a socket is coming up

2010-09-03 Thread Brian Granger
On Fri, Sep 3, 2010 at 3:51 AM, Martin Lucina m...@kotelna.sk wrote: sust...@250bpm.com said: Is this a bug?  What should poll do with a socket that is still connecting? No, it's not a bug. The first events you see are actually some internal messages from I/O thread managing the connecting

Re: [zeromq-dev] Trapping of EINTR in blocking recv and poll

2010-09-03 Thread Brian Granger
Everyone, First, thanks everyone for responding while I slept... sustrik: thanks for implementing this - I will try your branch later today and get back to you. mato: thanks for summarizing the state of things. Peiter: thanks for being willing to flex the release policies... More notes inline

[zeromq-dev] Trapping of EINTR in blocking recv and poll

2010-09-02 Thread Brian Granger
Sustrik and mato, I just tested this idea of having the blocking recv and poll not trap and silence EINTR. It work exactly like we want, at least on OS X. I just put in a print statement that prints if EINTR is returned, and if my program is sitting in blocking recv and I send SIGINT, the print

[zeromq-dev] zmq_poll not waiting for timeout if a socket is coming up

2010-09-02 Thread Brian Granger
Hi, I have a simple event loop that is doing non-blocking IO using s.recv(NOBLOCK) and Poller.poll. I am seeing the following behavior that might be a bug: * I am using a REQ socket that connects to a REP socket. * I do a send right after calling connect. * Then I enter a loop to watch for

Re: [zeromq-dev] Vulnerability of devices to incoming messages

2010-09-01 Thread Brian Granger
Great, thanks! Brian On Sun, Aug 29, 2010 at 8:10 PM, Pieter Hintjens p...@imatix.com wrote: On Sun, Aug 29, 2010 at 10:06 PM, Brian Granger elliso...@gmail.com wrote: Did your fix get merged? Yes, it's in 2.0.8. - Pieter Hintjens iMatix - www.imatix.com -- Brian E. Granger, Ph.D

Re: [zeromq-dev] 2.0.8dev version of pyzmq

2010-08-29 Thread Brian Granger
On Sun, Aug 29, 2010 at 6:36 AM, Jon Dyte j...@totient.co.uk wrote: Brian Granger wrote: Over the next few days, we are going to be doing a lot of work on pyzmq itself.  Here are some of the highlights: * Additional device types and support. Excellent. Be really interesting to finally

Re: [zeromq-dev] Automagical zdevice/1.0 now available

2010-08-29 Thread Brian Granger
Peiter, This looks really nice. I like the idea of using JSON to configure a device. There are decent JSON libraries in Python, so we could even configure devices programmatically that way. But why have this code as a separate project? Wouldn't it make sense just to keep this code in zeromq

Re: [zeromq-dev] Load Balancing/Distributing/Queuing Algos: A Discussion

2010-08-29 Thread Brian Granger
Stefan, I have played around with other load balancing algorithms and I do agree, it would be nice to have an interface that allows users to pick which algorithm is used. The two other approaches I have looked at are: * Randomized choice of peer, * Randomly pick two peers and write to the least

Re: [zeromq-dev] Client Server Model in Python

2010-08-29 Thread Brian Granger
As others have mentioned, you *absolutely* want to run a centralized queue device that both the client and workers connect to. This will load balance the requests to the workers (which will be separate processes, not threads). The performance and scalability of this configuration should be

Re: [zeromq-dev] NEWS: Git 'master' is now 2.1.x development

2010-08-29 Thread Brian Granger
Thanks for the update, this sounds like some serious goodness! Brian On Wed, Aug 25, 2010 at 7:08 AM, Martin Lucina m...@kotelna.sk wrote: Hi all, with the release of 0MQ 2.0.8, the Git 'master' branch has been tentatively labeled as 0MQ 2.1.0. We will announce a more detailed roadmap for

Re: [zeromq-dev] Vulnerability of devices to incoming messages

2010-08-29 Thread Brian Granger
Pieter, Did your fix get merged? Cheers, Brian On Wed, Aug 11, 2010 at 11:41 AM, Pieter Hintjens p...@imatix.com wrote: On Wed, Aug 11, 2010 at 8:37 PM, Matt Weinstein matt_weinst...@yahoo.com wrote: Here's one argument against this: If you forward a malformed stream to an XREP you have a

Re: [zeromq-dev] tcp_connecter Assertion failed

2010-08-29 Thread Brian Granger
I just tested this with 2.0.8 and on my machine it looks fine. Thanks for fixing this! Brian On Wed, Jun 30, 2010 at 9:03 PM, rad jan jan@gmail.com wrote: Hi, folks, Thanks for your great work! We use zmq as the message system between modules on different servers. The servers are in an

Re: [zeromq-dev] building python bindings on Windows

2010-08-28 Thread Brian Granger
I have updated the README with this information: http://github.com/zeromq/pyzmq/commit/42a1472889ce25405b38919f0588ad3dbca0ab5f Please let me know if you find anything else on Windows. Cheers, Brian On Tue, Jun 29, 2010 at 7:41 AM, Brian Granger elliso...@gmail.com wrote: Chris, On Sun

Re: [zeromq-dev] [PATCH] Python bindings

2010-08-28 Thread Brian Granger
Hi, I am currently working on a 2.0.8 release for pyzmq. In 2.0.8, PUSH and PULL are there, but EMTHREAD has not yet been removed. The 2.0.8 release will have these changes, but I have already made them in the pyzmq master. Sorry I have been so far behind on pyzmq development. Cheers, Brian

[zeromq-dev] 2.0.8dev version of pyzmq

2010-08-28 Thread Brian Granger
Hi, I just pushed changes to PyZMQ that make the git master 2.0.8 compatible. This is not yet the official release, but it should be 100% complete and compatible with the 2.0.8 API. The main changes from 2.0.7 were PUSH/PULL and additional error types. Here is the code:

Re: [zeromq-dev] Vulnerability of devices to incoming messages

2010-08-11 Thread Brian Granger
On Wed, Aug 11, 2010 at 12:02 PM, Matt Weinstein matt_weinst...@yahoo.comwrote: On Aug 11, 2010, at 2:52 PM, MinRK wrote: On Wed, Aug 11, 2010 at 11:37, Matt Weinstein matt_weinst...@yahoo.comwrote: Here's one argument against this: If you forward a malformed stream to an XREP you have

Re: [zeromq-dev] Vulnerability of devices to incoming messages

2010-08-11 Thread Brian Granger
On Wed, Aug 11, 2010 at 12:12 PM, Jon Dyte j...@totient.co.uk wrote: whilst on the subject of devices and the good points made within this thread I do have some reservations about these more 'esoteric' uses and the existing devices. initially we had 1) queue which conventionally takes

Re: [zeromq-dev] Vulnerability of devices to incoming messages

2010-08-11 Thread Brian Granger
I think the core question that this bug brings up is the following: In what situations do we want a 0MQ process to crash? Here is my simple answer: never You could imagine giving less absolute answers such as: * Only when sockets are used in unsupported ways. * Only when a peer does something

Re: [zeromq-dev] Portability of 0MQ API

2010-08-09 Thread Brian Granger
Hi, Martin gives a good summary of what prompted the creation of zmq_errno. In the Python universe it is actually not uncommon to run into the CRT issues. We use zmq_errno exclusively in the Python bindings because of this issue. Cheers, Brian On Mon, Aug 9, 2010 at 12:51 AM, Martin Lucina

Re: [zeromq-dev] basic_istream/ostream

2010-08-01 Thread Brian Granger
On Sun, Aug 1, 2010 at 2:00 PM, Martin Sustrik sust...@250bpm.com wrote: Matt Weinstein wrote: As an aside, I just can't wait to write: socket msgs In theory, each could create a new message part. Then there could be a special endm (end of message operator) that would write an

Re: [zeromq-dev] Devices, and pipeline again

2010-07-29 Thread Brian Granger
On Thu, Jul 29, 2010 at 9:15 AM, Martin Lucina m...@kotelna.sk wrote: p...@imatix.com said: [snip part about devices] As you can see I'm using ZMQ_PUSH and ZMQ_PULL in place of ZMQ_DOWNSTREAM and ZMQ_UPSTREAM. It seems to work. A node pulls data, works on it, and pushes it on. So

Re: [zeromq-dev] zmq-socket name aliases

2010-07-29 Thread Brian Granger
On Wed, Jul 28, 2010 at 3:36 PM, Pieter Hintjens p...@imatix.com wrote: On Wed, Jul 28, 2010 at 4:30 PM, Serge Aleynikov se...@aleynikov.org wrote: You're certainly not the only one having trouble with the naming of UPSTREAM/DOWNSTREAM sockets. Every time I think this notion is settled

Re: [zeromq-dev] Devices, and pipeline again

2010-07-29 Thread Brian Granger
On Thu, Jul 29, 2010 at 8:46 AM, Pieter Hintjens p...@imatix.com wrote: Hi All, I'm writing a new generic zmq_device that will replace zmq_queue, zmq_forwarder and zmq_streamer. The code is here: http://dpaste.de/4DcP/ It looks like you are only working on the command line part. Is this

  1   2   3   >