Re: [zeromq-dev] zmqstream with req socket only sends the first message

2015-02-03 Thread MinRK 
On Tue, Feb 3, 2015 at 7:00 AM, drebs  wrote:

Thanks a lot for your answer, now it works like a charm.
>
> I couldn't find documentation about this, do you think it's a good idea to
> add
> a paragraph to zmqstream.send docstring (or anywhere else) about threaded
> use?
>
> If you think it's a good idea, I have a suggestion based on the info you
> gave
> me. It's in the patch below, but feel free to modify in the way you think
> it
> is more appropriate.
>
Since this is an issue generic to any use of tornado IOLoop, I would be a
bit more brief, and link to the tornado docs

.

-MinRK


> --
>
> From cf9418dc14b2dc2e1aca5be1b79a105982dcd877 Mon Sep 17 00:00:00 2001
> From: drebs 
> Date: Tue, 3 Feb 2015 12:50:59 -0200
> Subject: [PATCH] Add doc for threaded zmqstream.send.
>
> ---
>  zmq/eventloop/zmqstream.py |8 
>  1 file changed, 8 insertions(+)
>
> diff --git a/zmq/eventloop/zmqstream.py b/zmq/eventloop/zmqstream.py
> index 86a97e4..59149ec 100644
> --- a/zmq/eventloop/zmqstream.py
> +++ b/zmq/eventloop/zmqstream.py
> @@ -242,6 +242,14 @@ class ZMQStream(object):
>  def send(self, msg, flags=0, copy=True, track=False, callback=None):
>  """Send a message, optionally also register a new callback for
> sends.
>  See zmq.socket.send for details.
> +
> +When using threads, calling send on the zmqstream from another
> thread
> +doesn’t properly tell the IOLoop thread that there’s an event to
> +process. This could cause small delays if the IOLoop is already
> +processing lots of events, but it can also cause the message to
> never
> +be sent if the zmq socket is the only one it’s handling. In this
> case,
> +you should hand off the stream.send to the IOLoop’s thread via
> +IOLoop.add_callback.
>  """
>  return self.send_multipart([msg], flags=flags, copy=copy,
> track=track, callback=callback)
>
A


> --
> 1.7.10.4
>
>
>
> --
>
>
> MinRK codificou 9.4K bytes:
> >Calling send on the zmqstream from another thread doesn’t properly
> tell
> >the IOLoop thread that there’s an event to process. This could call
> small
> >delays if the IOLoop is already processing lots of events, but it can
> >cause the message to never send if the zmq socket is the only one it’s
> >handling.
> >
> >You want your ZmqREQConnection.send to hand off the stream.send to the
> >IOLoop’s thread via IOLoop.add_callback:
> >
> >  def send(self, *args, **kwargs):
> >  print("Sending message to backend: (%s, %s)" % (args, kwargs))
> >  self._ioloop.add_callback(lambda : self._stream.send(*args,
> **kwargs))
> >
> >-MinRK
> >
> >On Mon, Feb 2, 2015 at 12:58 PM, drebs [1]dr...@riseup.net wrote:
> >>
> >
> >  Hello, zmq community.
> >
> >  I’m trying to use a pyzmq’s zmqstream with a REQ socket to talk to a
> >  txzmq REP
> >  socket, but only the first message is ever sent and received back.
> >
> >  REQ pyzmq endpoint code: [2]http://paste.debian.net/143615/
> >  REP txzmq endpoint code: [3]http://paste.debian.net/143617/
> >
> >  I would expect that all the messages would be sent. Am I missing
> >  something
> >  here?
> >
> >  If I let some requests accumulate, and then start the server, all
> the
> >  accumulated requests are replied, but the new ones are not sent.
> >
> >  If I replace the REQ endpoint by a pure pyzmq version with no
> zmqstream
> >  or
> >  ioloop, it works fine ([4]http://paste.debian.net/143619/).
> >
> >  Anyone can give some light on this?
> >
> >  Thanks a lot!
> >  drebs.
> >
> >  —
> >
> >
> --
> >
> >  zeromq-dev mailing list
> >  [5]zeromq-dev@lists.zeromq.org
> >  [6]http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >​
> >
> > References
> >
> >Visible links
> >1. http://mailto:dr...@riseup.net/
> >2. http://paste.debian.net/143615/
> >3. http://paste.debian.net/143617/
> >4. http://paste.debian.net/143619/
> >5. mailto:zeromq-dev@lists.zeromq.org
> >6. http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> > ___
> > zeromq-dev mailing list
> > zeromq-dev@lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
​
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmqstream with req socket only sends the first message

2015-02-03 Thread drebs 
Thanks a lot for your answer, now it works like a charm.

I couldn't find documentation about this, do you think it's a good idea to add
a paragraph to zmqstream.send docstring (or anywhere else) about threaded use?

If you think it's a good idea, I have a suggestion based on the info you gave
me. It's in the patch below, but feel free to modify in the way you think it
is more appropriate.

--

From cf9418dc14b2dc2e1aca5be1b79a105982dcd877 Mon Sep 17 00:00:00 2001
From: drebs 
Date: Tue, 3 Feb 2015 12:50:59 -0200
Subject: [PATCH] Add doc for threaded zmqstream.send.

---
 zmq/eventloop/zmqstream.py |8 
 1 file changed, 8 insertions(+)

diff --git a/zmq/eventloop/zmqstream.py b/zmq/eventloop/zmqstream.py
index 86a97e4..59149ec 100644
--- a/zmq/eventloop/zmqstream.py
+++ b/zmq/eventloop/zmqstream.py
@@ -242,6 +242,14 @@ class ZMQStream(object):
 def send(self, msg, flags=0, copy=True, track=False, callback=None):
 """Send a message, optionally also register a new callback for sends.
 See zmq.socket.send for details.
+
+When using threads, calling send on the zmqstream from another thread
+doesn’t properly tell the IOLoop thread that there’s an event to
+process. This could cause small delays if the IOLoop is already
+processing lots of events, but it can also cause the message to never
+be sent if the zmq socket is the only one it’s handling. In this case,
+you should hand off the stream.send to the IOLoop’s thread via
+IOLoop.add_callback.
 """
 return self.send_multipart([msg], flags=flags, copy=copy, track=track, 
callback=callback)
 
-- 
1.7.10.4



-- 


MinRK codificou 9.4K bytes:
>Calling send on the zmqstream from another thread doesn’t properly tell
>the IOLoop thread that there’s an event to process. This could call small
>delays if the IOLoop is already processing lots of events, but it can
>cause the message to never send if the zmq socket is the only one it’s
>handling.
> 
>You want your ZmqREQConnection.send to hand off the stream.send to the
>IOLoop’s thread via IOLoop.add_callback:
> 
>  def send(self, *args, **kwargs):
>  print("Sending message to backend: (%s, %s)" % (args, kwargs))
>  self._ioloop.add_callback(lambda : self._stream.send(*args, **kwargs))
> 
>-MinRK
> 
>On Mon, Feb 2, 2015 at 12:58 PM, drebs [1]dr...@riseup.net wrote:
>>
> 
>  Hello, zmq community.
> 
>  I’m trying to use a pyzmq’s zmqstream with a REQ socket to talk to a
>  txzmq REP
>  socket, but only the first message is ever sent and received back.
> 
>  REQ pyzmq endpoint code: [2]http://paste.debian.net/143615/
>  REP txzmq endpoint code: [3]http://paste.debian.net/143617/
> 
>  I would expect that all the messages would be sent. Am I missing
>  something
>  here?
> 
>  If I let some requests accumulate, and then start the server, all the
>  accumulated requests are replied, but the new ones are not sent.
> 
>  If I replace the REQ endpoint by a pure pyzmq version with no zmqstream
>  or
>  ioloop, it works fine ([4]http://paste.debian.net/143619/).
> 
>  Anyone can give some light on this?
> 
>  Thanks a lot!
>  drebs.
> 
>  —
> 
>--
> 
>  zeromq-dev mailing list
>  [5]zeromq-dev@lists.zeromq.org
>  [6]http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> 
>​
> 
> References
> 
>Visible links
>1. http://mailto:dr...@riseup.net/
>2. http://paste.debian.net/143615/
>3. http://paste.debian.net/143617/
>4. http://paste.debian.net/143619/
>5. mailto:zeromq-dev@lists.zeromq.org
>6. http://lists.zeromq.org/mailman/listinfo/zeromq-dev

> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmqstream with req socket only sends the first message

2015-02-02 Thread MinRK 
Calling send on the zmqstream from another thread doesn’t properly tell the
IOLoop thread that there’s an event to process. This could call small
delays if the IOLoop is already processing lots of events, but it can cause
the message to never send if the zmq socket is the only one it’s handling.

You want your ZmqREQConnection.send to hand off the stream.send to the
IOLoop’s thread via IOLoop.add_callback:

def send(self, *args, **kwargs):
print("Sending message to backend: (%s, %s)" % (args, kwargs))
self._ioloop.add_callback(lambda : self._stream.send(*args, **kwargs))

-MinRK

On Mon, Feb 2, 2015 at 12:58 PM, drebs dr...@riseup.net
 wrote:
>

Hello, zmq community.

I’m trying to use a pyzmq’s zmqstream with a REQ socket to talk to a txzmq
REP
socket, but only the first message is ever sent and received back.

REQ pyzmq endpoint code: http://paste.debian.net/143615/
REP txzmq endpoint code: http://paste.debian.net/143617/

I would expect that all the messages would be sent. Am I missing something
here?

If I let some requests accumulate, and then start the server, all the
accumulated requests are replied, but the new ones are not sent.

If I replace the REQ endpoint by a pure pyzmq version with no zmqstream or
ioloop, it works fine (http://paste.debian.net/143619/).

Anyone can give some light on this?

Thanks a lot!
drebs.

—
--

zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

​
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] zmqstream with req socket only sends the first message

2015-02-02 Thread drebs 
Hello, zmq community.

I'm trying to use a pyzmq's zmqstream with a REQ socket to talk to a txzmq REP
socket, but only the first message is ever sent and received back.

REQ pyzmq endpoint code: http://paste.debian.net/143615/
REP txzmq endpoint code: http://paste.debian.net/143617/

I would expect that all the messages would be sent. Am I missing something
here?

If I let some requests accumulate, and then start the server, all the
accumulated requests are replied, but the new ones are not sent.

If I replace the REQ endpoint by a pure pyzmq version with no zmqstream or
ioloop, it works fine (http://paste.debian.net/143619/).

Anyone can give some light on this?

Thanks a lot!
drebs.

-- 


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev