Re: [zeromq-dev] HWM default

[Martin Sustrik]

On 05/09/2011 02:41 PM, Paul Colomiets wrote:

> Well, I have no practical experience, but a lot of alloc_assert's in the
> code says that it will be aborted. BTW, it will be so not only if
> overcommit_memory is set right, but also if ulimit is set for the
> application's memory. Which is awfully bad IMO.

What else would you do if you run out of memory? Any recovery mechanism 
in user space is likely to fail because there's no memory available :) 
Killing (and possibly restarting) the app seems like a reasonable option 
to me.

> We should distinguish deadlocks inside 0MQ (such as one introduced
> by the shutdown functionality) which should be considered 0MQ bugs
> and deadlocks in applications on top of 0MQ (the ones we are
> discussing now) which should be considered application bugs.
>
>
> Don't see why shutdown would introduce deadlocks inside 0mq

I referred to our old discussion.

> But thinking of it a bit more about default HWM, it seems that issues
> are same using TCP or unix domain sockets. And for future in-kernel
> implementation HWM should probably always be set. So it's probably ok to
> have reasonable default.

Yes. With kernel implementation the buffers *are* TCP buffers and HWM 
*is* the TCP buffer size. So there's no way to have it unlimited.

Martin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Just ran some tests with x64

[Christian Martinez]

Heh. Your guess is correct!

Followed the note on the site about how to build for x64 and just went with it. 
So far so good but was just wondering if anyone hit any snags.


--CM

From: zeromq-dev-boun...@lists.zeromq.org 
[mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Steven McCoy
Sent: Wednesday, May 11, 2011 6:26 PM
To: ZeroMQ development list
Subject: Re: [zeromq-dev] Just ran some tests with x64

On 12 May 2011 08:30, Christian Martinez 
mailto:ch...@microsoft.com>> wrote:
Lots of warnings in the compile about possible loss of data but the tests are 
running great.

Is this a tested scenario? Do I need to dig into each warning?


Depends on the compiler, the platform, and what you are building, from your 
email address I'm taking a wild guess at MSVC 2010 on Windows 7?

:-)

PGM compiles without warnings in MSVC as I've tested several times, ZeroMQ has 
been varied but not overly well maintained because the testing has been with 
MSVC Express which doesn't support x64.  There certainly have been a few posts 
recently discussing it though.

Compilers like ICC just emit too many warnings by default though, and GCC 
slowly gains more and more useless warnings enabled by default as noted 
recently by Linus on LKML.

--
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Just ran some tests with x64

[Steven McCoy]

On 12 May 2011 08:30, Christian Martinez  wrote:

>  Lots of warnings in the compile about possible loss of data but the tests
> are running great.
>
>
>
> Is this a tested scenario? Do I need to dig into each warning?
>
>
>
Depends on the compiler, the platform, and what you are building, from your
email address I'm taking a wild guess at MSVC 2010 on Windows 7?

:-)

PGM compiles without warnings in MSVC as I've tested several times, ZeroMQ
has been varied but not overly well maintained because the testing has been
with MSVC Express which doesn't support x64.  There certainly have been a
few posts recently discussing it though.

Compilers like ICC just emit too many warnings by default though, and GCC
slowly gains more and more useless warnings enabled by default as noted
recently by Linus on LKML.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Inconsistent behavior during tests

[Joshua Foster]

The basic problem is that the remote is producing messages faster than the 
consumer (local). This is a common problem in messaging systems and if left 
unchecked will kill the queue. ZeroMQ has a few ways to help with the problem. 
The first is the HWM (High Water Mark). Once the HWM is reached, it will start 
throwing messages away. This is helpful in that the application won't die, but 
you need to have your application be able to handle missing data. If the data 
just comes in bursts, but the consumer will eventually be able to handle it, 
you can set the swap option. 0MQ will then start swapping the messages to disk 
up once the HWM is reached up to the allotted size. Of course, once the swap 
space is filled - it starts throwing messages away.

If you need to process more messages than the consumers handle, you need to 
start looking into sharding the data. You could do this by creating a simple 
load balancer device. I would strongly recommend reading the guide as it 
mentions a lot of this and more (http://zguide.zeromq.org/page:all).

I mostly have experience using the Java bindings and have not seen any of the 
inconsistent behavior. Since you are just looking at a simple throughput test, 
you may want to modify the remote_thr and put some sleeps in after a batch of 
messages are added. This will allow the client to throttle a little, but you 
will probably need to play with the numbers if you want to see the max. You 
will also notice that what bindings are used can make a difference in the 
performance. The C and C++ bindings will be the fastest. Python and Ruby will 
be the slowest. I don't think anyone has done benchmarks between the different 
bindings.

Joshua

On May 11, 2011, at 5:20 PM, Christian Martinez wrote:

> Hmm.. Are you using clrzmq2?
> 
> Built libzmq 2.1.6 by opening sln and just doing rebuild all.. System info 
> below
>  
> OS Name Microsoft Windows 7 Enterprise
> Version6.1.7601 Service Pack 1 Build 7601
> System ManufacturerHewlett-Packard
> System Model   HP EliteBook 8540w
> System Type  x64-based PC
> ProcessorIntel(R) Core(TM) i7 CPU   Q 720  @ 1.60GHz, 1600 
> Mhz, 4 Core(s), 8 Logical Processor(s)
> Installed Physical Memory (RAM) 16.0 GB
> Total Physical Memory  15.9 GB
> Available Physical Memory  11.1 GB
> Total Virtual Memory 31.9 GB
> Available Virtual Memory 26.3 GB
> Page File Space 15.9 GB
> Page File  C:\pagefile.sys
>  
> --CM
>  
> From: zeromq-dev-boun...@lists.zeromq.org 
> [mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Joshua Foster
> Sent: Wednesday, May 11, 2011 2:06 PM
> To: ZeroMQ development list
> Subject: Re: [zeromq-dev] Inconsistent behavior during tests
>  
> I'm not sure what you are seeing here are my test runs...
> 
> E:\zmqwin32>remote_thr.exe tcp://127.0.0.1: 120 1000
> 
> E:\zmqwin32>remote_thr.exe tcp://127.0.0.1: 120 1000
> 
> E:\zmqwin32>local_thr.exe tcp://127.0.0.1: 120 1000
> message size: 120 [B]
> message count: 1000
> mean throughput: 752049 [msg/s]
> mean throughput: 721.967 [Mb/s]
> 
> E:\zmqwin32>local_thr.exe tcp://127.0.0.1: 120 1000
> message size: 120 [B]
> message count: 1000
> mean throughput: 748902 [msg/s]
> mean throughput: 718.946 [Mb/s]
> 
> I'm running on the local machine with Windows 7, Intel 2.8ghz quad core with 
> 4GB of memory. I have 0MQ 2.1.6 compiled with VS2010. 
> 
> Joshua
> 
> On 5/11/2011 4:11 PM, Christian Martinez wrote:
> Running tests back to back on same laptop back to back sending 1000 
> messages.
>  
> C:\software\zeromq-clrzmq2-f420936\remote_thr\bin\Release>remote_thr.exe 
> tcp://127.0.0.1: 120 1000
>  
> C:\software\zeromq-clrzmq2-f420936\remote_thr\bin\Release>remote_thr.exe 
> tcp://127.0.0.1: 120 1000
>  
> Unhandled Exception: ZMQ.Exception: Visual C++ CRT: Not enough memory to 
> complete call to strerror.
>at ZMQ.Socket.Send(Byte[] message, SendRecvOpt[] flags)
>   at remote_thr.Program.Main(String[] args)
>  
> Is this normal? If this condition is known to occur under load then how do I 
> best modify the test to throttle or do whatever is necessary prevent the 
> second result while still not killing throughput?
>  
> --CM
>  
>  
>  
>  
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>  
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Just ran some tests with x64

[Christian Martinez]

Lots of warnings in the compile about possible loss of data but the tests are 
running great.

Is this a tested scenario? Do I need to dig into each warning?

--CM
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZMTP/2.0 framing proposal

[Marko Mikulicic]

On 12 May 2011 00:01, Pieter Hintjens  wrote:

> Martin,
>
> Looking at the next generation of the 0MQ wire level protocol, I think
> the latest websocket framing spec would make a good fit for the base
> framing layer.  There are several advantages to tracking websockets on
> this, if it's possible.
>
>
> http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-07#section-4
> shows the websocket framing.
>

Interesting idea. Do you plan maintaining backward compat with the existing
proto, and upgrade to v2 after some handshake?
Or we could have two wire protocols using different "transports" (tcp:// vs
tcp-compat://)

Do you think being fully compatible or just taking it as basis/inspiration ?

Cheers,
Marko
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Brian Rossa]

So guys, I have some evidence for my claims. I can give it to you in any 
combination of the following ways:
1) Python code and a shell script that implements a PUB/SUB connection pattern 
similar to mine where intermediate nodes introduce realistic processing 
delays2) Data (CSV files) generated by running the Python code/shell scripts3) 
SciPy code that walks on the data and makes pretty graphs4) JPEGs of the pretty 
graphs.
AFAICT, the bottom line is that (!!!) despite HWM=1 (!!!) on both sides of my 
PUB/SUB sockets there can be significant end-to-end latency, and that latency 
tends to grow with time. This behavior is consistent with intermediate message 
queuing, something the HWM is -- at least in my understanding -- supposed to 
prevent.
This result is very preliminary. I concede that I may still be missing 
something dramatic. 
Please advise.
Cheers!~br
> From: p...@imatix.com
> Date: Wed, 11 May 2011 18:46:03 +0200
> To: zeromq-dev@lists.zeromq.org
> Subject: Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1
> 
> On Wed, May 11, 2011 at 6:36 PM, Dirkjan Ochtman  wrote:
> > On Wed, May 11, 2011 at 18:25, Pieter Hintjens  wrote:
> >> Did you set the HWM _before_ connecting and binding? If you set it
> >> afterwards, it has no effect. (Rather, it has effect only on future
> >> binds and connects).
> >
> > Heh. Can we make that assert in some next release?
> 
> That was discussed at yesterday's meeting. The answer is "no", because
> it's legal to set the HWM for future connects & binds. What's lacking,
> I think most will agree, is any way to inspect the queue sizes. Also a
> hot topic, but we have some ideas.
> 
> -Pieter
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
  ___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZMTP/2.0 framing proposal

[Michael Kogan]

+1 on that idea.
Getting the WS bytes directly to/from clients and channeling that into a 
message bus would  be great.

On May 11, 2011, at 3:01 PM, Pieter Hintjens wrote:

> Martin,
> 
> Looking at the next generation of the 0MQ wire level protocol, I think
> the latest websocket framing spec would make a good fit for the base
> framing layer.  There are several advantages to tracking websockets on
> this, if it's possible.
> 
> http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-07#section-4
> shows the websocket framing.
> 
> -Pieter
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] ZMTP/2.0 framing proposal

[Pieter Hintjens]

Martin,

Looking at the next generation of the 0MQ wire level protocol, I think
the latest websocket framing spec would make a good fit for the base
framing layer.  There are several advantages to tracking websockets on
this, if it's possible.

http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-07#section-4
shows the websocket framing.

-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Inconsistent behavior during tests

[Christian Martinez]

Hmm.. Are you using clrzmq2?
Built libzmq 2.1.6 by opening sln and just doing rebuild all.. System info below

OS Name Microsoft Windows 7 Enterprise
Version6.1.7601 Service Pack 1 Build 7601
System ManufacturerHewlett-Packard
System Model   HP EliteBook 8540w
System Type  x64-based PC
ProcessorIntel(R) Core(TM) i7 CPU   Q 720  @ 1.60GHz, 1600 Mhz, 
4 Core(s), 8 Logical Processor(s)
Installed Physical Memory (RAM) 16.0 GB
Total Physical Memory  15.9 GB
Available Physical Memory  11.1 GB
Total Virtual Memory 31.9 GB
Available Virtual Memory 26.3 GB
Page File Space 15.9 GB
Page File  C:\pagefile.sys

--CM

From: zeromq-dev-boun...@lists.zeromq.org 
[mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Joshua Foster
Sent: Wednesday, May 11, 2011 2:06 PM
To: ZeroMQ development list
Subject: Re: [zeromq-dev] Inconsistent behavior during tests

I'm not sure what you are seeing here are my test runs...

E:\zmqwin32>remote_thr.exe tcp://127.0.0.1: 120 1000

E:\zmqwin32>remote_thr.exe tcp://127.0.0.1: 120 1000

E:\zmqwin32>local_thr.exe tcp://127.0.0.1: 120 1000
message size: 120 [B]
message count: 1000
mean throughput: 752049 [msg/s]
mean throughput: 721.967 [Mb/s]

E:\zmqwin32>local_thr.exe tcp://127.0.0.1: 120 1000
message size: 120 [B]
message count: 1000
mean throughput: 748902 [msg/s]
mean throughput: 718.946 [Mb/s]

I'm running on the local machine with Windows 7, Intel 2.8ghz quad core with 
4GB of memory. I have 0MQ 2.1.6 compiled with VS2010.

Joshua

On 5/11/2011 4:11 PM, Christian Martinez wrote:
Running tests back to back on same laptop back to back sending 1000 
messages.

C:\software\zeromq-clrzmq2-f420936\remote_thr\bin\Release>remote_thr.exe 
tcp://127.0.0.1: 120 1000

C:\software\zeromq-clrzmq2-f420936\remote_thr\bin\Release>remote_thr.exe 
tcp://127.0.0.1: 120 1000

Unhandled Exception: ZMQ.Exception: Visual C++ CRT: Not enough memory to 
complete call to strerror.
   at ZMQ.Socket.Send(Byte[] message, SendRecvOpt[] flags)
  at remote_thr.Program.Main(String[] args)

Is this normal? If this condition is known to occur under load then how do I 
best modify the test to throttle or do whatever is necessary prevent the second 
result while still not killing throughput?

--CM







___

zeromq-dev mailing list

zeromq-dev@lists.zeromq.org

http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Inconsistent behavior during tests

[Joshua Foster]

I'm not sure what you are seeing here are my test runs...

E:\zmqwin32>remote_thr.exe tcp://127.0.0.1: 120 1000

E:\zmqwin32>remote_thr.exe tcp://127.0.0.1: 120 1000

E:\zmqwin32>local_thr.exe tcp://127.0.0.1: 120 1000
message size: 120 [B]
message count: 1000
mean throughput: 752049 [msg/s]
mean throughput: 721.967 [Mb/s]

E:\zmqwin32>local_thr.exe tcp://127.0.0.1: 120 1000
message size: 120 [B]
message count: 1000
mean throughput: 748902 [msg/s]
mean throughput: 718.946 [Mb/s]

I'm running on the local machine with Windows 7, Intel 2.8ghz quad core 
with 4GB of memory. I have 0MQ 2.1.6 compiled with VS2010.


Joshua

On 5/11/2011 4:11 PM, Christian Martinez wrote:


Running tests back to back on same laptop back to back sending 
1000 messages.


C:\software\zeromq-clrzmq2-f420936\remote_thr\bin\Release>remote_thr.exe 
tcp://127.0.0.1: 120 1000


C:\software\zeromq-clrzmq2-f420936\remote_thr\bin\Release>remote_thr.exe 
tcp://127.0.0.1: 120 1000


Unhandled Exception: ZMQ.Exception: Visual C++ CRT: Not enough memory 
to complete call to strerror.


   at ZMQ.Socket.Send(Byte[] message, SendRecvOpt[] flags)

  at remote_thr.Program.Main(String[] args)

Is this normal? If this condition is known to occur under load then 
how do I best modify the test to throttle or do whatever is necessary 
prevent the second result while still not killing throughput?


--CM


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Inconsistent behavior during tests

[Christian Martinez]

Running tests back to back on same laptop back to back sending 1000 
messages.

C:\software\zeromq-clrzmq2-f420936\remote_thr\bin\Release>remote_thr.exe 
tcp://127.0.0.1: 120 1000

C:\software\zeromq-clrzmq2-f420936\remote_thr\bin\Release>remote_thr.exe 
tcp://127.0.0.1: 120 1000

Unhandled Exception: ZMQ.Exception: Visual C++ CRT: Not enough memory to 
complete call to strerror.
   at ZMQ.Socket.Send(Byte[] message, SendRecvOpt[] flags)
  at remote_thr.Program.Main(String[] args)

Is this normal? If this condition is known to occur under load then how do I 
best modify the test to throttle or do whatever is necessary prevent the second 
result while still not killing throughput?

--CM


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Windows throughput perfomance

[Anton Yemelyanov]

Just out of curiosity: Which platform?  And are you building with
optimizations turned on (i.e. release)?

Anton

2011/5/11 Pieter Hintjens 

> 2011/5/11 Геннадий Казачёк :
>
> > I've started to learn zmq by compiling and running tests (remote_thr and
> > local_thr) to see amount of data I can utilize.
> > I've run applications (without any modifications) on 2 remote hosts in 1
> > Gbit network over tcp transport.
> > Results was about 10 Mbit/sec, no matter of message size I used (from 512
> > byte to 128Kb).
>
> Interesting. Can you run the same tests on the same machine? Also, how
> many messages are you sending?
>
> -Pieter
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Considering zeromq for project at Microsoft but have some questions on throughput behavior

[Christian Martinez]

Here's tests that take the range of time. Tried 3M but got an out of memory 
error on client side

C:\software\zeromq-clrzmq2-f420936\local_thr\bin\Release>local_thr.exe 
tcp://*: 120 200
Your average throughput is 75325 [msg/s]
Your average throughput is 72 [Mb/s]

C:\software\zeromq-clrzmq2-f420936\local_thr\bin\Release>local_thr.exe 
tcp://*: 120 200
Your average throughput is 48171 [msg/s]
Your average throughput is 46 [Mb/s]

--CM

-Original Message-
From: zeromq-dev-boun...@lists.zeromq.org 
[mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Pieter Hintjens
Sent: Wednesday, May 11, 2011 9:48 AM
To: ZeroMQ development list
Subject: Re: [zeromq-dev] Considering zeromq for project at Microsoft but have 
some questions on throughput behavior

On Wed, May 11, 2011 at 6:38 PM, Christian Martinez  wrote:

> While the latency tests using the c# bindings appear consistent on my 
> laptop (around 300 [us]) the throughput tests have a fairly large drop 
> off when # of messages are increased.

30K messages isn't meaningful. You should run a test that lasts at least 10-30 
seconds.

-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Considering zeromq for project at Microsoft but have some questions on throughput behavior

[Pieter Hintjens]

On Wed, May 11, 2011 at 6:38 PM, Christian Martinez  wrote:

> While the latency tests using the c# bindings appear consistent on my laptop
> (around 300 [us]) the throughput tests have a fairly large drop off when #
> of messages are increased.

30K messages isn't meaningful. You should run a test that lasts at
least 10-30 seconds.

-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Pieter Hintjens]

On Wed, May 11, 2011 at 6:36 PM, Dirkjan Ochtman  wrote:
> On Wed, May 11, 2011 at 18:25, Pieter Hintjens  wrote:
>> Did you set the HWM _before_ connecting and binding? If you set it
>> afterwards, it has no effect. (Rather, it has effect only on future
>> binds and connects).
>
> Heh. Can we make that assert in some next release?

That was discussed at yesterday's meeting. The answer is "no", because
it's legal to set the HWM for future connects & binds. What's lacking,
I think most will agree, is any way to inspect the queue sizes. Also a
hot topic, but we have some ideas.

-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Ian Barber]

On Wed, May 11, 2011 at 5:36 PM, Dirkjan Ochtman  wrote:

> Heh. Can we make that assert in some next release?
>
> Cheers,
>
> Dirkjan


I thought about chucking an error in that case on the PHP binding (even sent
Mikko a misguided patch), but there's no way of knowing whether someone will
later make another bind or connect after the HWM is set.

Ian
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Considering zeromq for project at Microsoft but have some questions on throughput behavior

[Christian Martinez]

While the latency tests using the c# bindings appear consistent on my laptop 
(around 300 [us]) the throughput tests have a fairly large drop off when # of 
messages are increased.

C:\software\zeromq-clrzmq2-f420936\local_thr\bin\Release>local_thr.exe 
tcp://*: 120 3
Your average throughput is 304081 [msg/s]
Your average throughput is 291 [Mb/s]

C:\software\zeromq-clrzmq2-f420936\local_thr\bin\Release>local_thr.exe 
tcp://*: 120 5
Your average throughput is 9654 [msg/s]
Your average throughput is 9 [Mb/s]

Are there switches or args I should pass to make this more consistent?

--CM


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Dirkjan Ochtman]

On Wed, May 11, 2011 at 18:25, Pieter Hintjens  wrote:
> Did you set the HWM _before_ connecting and binding? If you set it
> afterwards, it has no effect. (Rather, it has effect only on future
> binds and connects).

Heh. Can we make that assert in some next release?

Cheers,

Dirkjan
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_msg_init_data with zero deallocation function

[Pieter Hintjens]

2011/5/11 Ilja Golshtein :

> In zguide the zmq_msg_init_data syntax with zero ffn is used a couple of 
> times, though I failed to find if it is explained
> why it is wrong or at least dangerous.

I removed all these examples from the guide because they were very
misleading. The _init_data method should only be used when you really
need zero copy. The name is too similar to init_size and suggests it's
easy and safe to use. Really, it's quite tricky until you figure it
out.

In the guide examples it was being used with constant message bodies
("HELLO"). In reality you would always pass a ffn, and never use this
for stack-based data, since the stack can often unroll before the
message is actually sent.

I'd like to rename this method 'zmq_msg_init_zero_copy_for_experts_only' :)

-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Pieter Hintjens]

On Wed, May 11, 2011 at 4:46 PM, Brian Rossa  wrote:

> There are a number of design issues that I'd like help with in the medium
> term. For now, though, there's one that needs some urgent attention:
> *apparent* queuing of messages on PUB/SUB fan-in despite having HWM set to 1
> for both the senders and receivers.

Did you set the HWM _before_ connecting and binding? If you set it
afterwards, it has no effect. (Rather, it has effect only on future
binds and connects).

-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Windows throughput perfomance

[Pieter Hintjens]

2011/5/11 Геннадий Казачёк :

> I've started to learn zmq by compiling and running tests (remote_thr and
> local_thr) to see amount of data I can utilize.
> I've run applications (without any modifications) on 2 remote hosts in 1
> Gbit network over tcp transport.
> Results was about 10 Mbit/sec, no matter of message size I used (from 512
> byte to 128Kb).

Interesting. Can you run the same tests on the same machine? Also, how
many messages are you sending?

-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Brian Rossa]

> There is a buffer in IPC as well I believe - so it is possible that's coming 
> into play. Do you have HWM set on both sides or just on one? 

Both sides. 
Cheers! ~br   ___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Ian Barber]

On Wed, May 11, 2011 at 4:39 PM, Brian Rossa  wrote:

>  Ian,
>
> I was concerned that this might be the case but -- and these results are
> highly preliminary -- switching to ipc:// sockets doesn't seem to solve the
> problem.
>
> Still working on a demonstration.
>
> Cheers!
> ~br
>

There is a buffer in IPC as well I believe - so it is possible that's coming
into play. Do you have HWM set on both sides or just on one?

Ian
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Brian Rossa]

Ian,
I was concerned that this might be the case but -- and these results are highly 
preliminary -- switching to ipc:// sockets doesn't seem to solve the problem.
Still working on a demonstration.
Cheers!~br



Date: Wed, 11 May 2011 16:27:37 +0100
From: ian.bar...@gmail.com
To: zeromq-dev@lists.zeromq.org
Subject: Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

On Wed, May 11, 2011 at 3:46 PM, Brian Rossa  wrote:





My understanding of the HWM semantic is that, when the HWM is reached, new 
messages will not be queued. That means that the queued messages are not the 
newest messages. There are good reasons for why this has to be the case. Fine. 
Still, this growing lag for prolific senders doesn't make any sense to me. The 
message that I receive at the system endpoint should only be as old as the last 
zmq_recv from that node. Yet, I am experiencing situations in which one of the 
intermediate nodes will "take a break" -- eg, it won't send any messages for a 
while -- and the endpoint will continue to process old messages from it for a 
few more minutes. Shouldn't it process JUST ONE message from a node that's 
"taking a break" if HWM is 1 on both sides of that socket pair?


If the messages are reasonably small, this lag could be caused by TCP buffers - 
your fast node messages are stuck in a big TCP buffer, and ZeroMQ is fair 
queing from each of the sockets, so takes a little while to consume. The HWM 
only applies to stuff that has been pulled out of the TCP buffer and into 
zeromq, so for small messages your effective 'queue' could be a fair bit bigger 
than 1. 


Ian



___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev 
  ___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Ian Barber]

On Wed, May 11, 2011 at 3:46 PM, Brian Rossa  wrote:

> My understanding of the HWM semantic is that, when the HWM is reached, new
> messages will not be queued. That means that the queued messages are not the
> newest messages. There are good reasons for why this has to be the case.
> Fine. Still, this growing lag for prolific senders doesn't make any sense to
> me. The message that I receive at the system endpoint should only be as old
> as the last zmq_recv from that node. Yet, I am experiencing situations in
> which one of the intermediate nodes will "take a break" -- eg, it won't send
> any messages for a while -- and the endpoint will continue to process old
> messages from it for a few more minutes. Shouldn't it process JUST ONE
> message from a node that's "taking a break" if HWM is 1 on both sides of
> that socket pair?
>
>
If the messages are reasonably small, this lag could be caused by TCP
buffers - your fast node messages are stuck in a big TCP buffer, and ZeroMQ
is fair queing from each of the sockets, so takes a little while to consume.
The HWM only applies to stuff that has been pulled out of the TCP buffer and
into zeromq, so for small messages your effective 'queue' could be a fair
bit bigger than 1.

Ian
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_msg_init_data with zero deallocation function

[Ilja Golshtein]

Although the note about stack is technically correct, the behaviour is 
undefined from C programming language standpoint. 
I think it must be clearly stated the lifetime of the data must be controlled. 
In fact it must be kept until zeromq context closed, isn't it?

In zguide the zmq_msg_init_data syntax with zero ffn is used a couple of times, 
though I failed to find if it is explained
why it is wrong or at least dangerous.

Thanks.

11.05.2011, 19:03, "Ivan Pechorin" :
> 2011/5/11 Dirkjan Ochtman ;:
>
>>>  Is the code
>>>  ==
>>>  zmq_msg_t msg;
>>>  zmq_msg_init_data (&msg, (void *)"something", 9,  NULL, NULL);
>>>  ==
>>>  valid?
>>  I think in this case "something" is allocated on the stack.
>
> It's more likely that it will be kept in read-only data section, not on stack.
> Therefore, the code snippet cited above should work fine.
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-- 
Best regards,
Ilja Golshtein.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_msg_init_data with zero deallocation function

[Ivan Pechorin]

2011/5/11 Dirkjan Ochtman :
>> Is the code
>> ==
>> zmq_msg_t msg;
>> zmq_msg_init_data (&msg, (void *)"something", 9,  NULL, NULL);
>> ==
>> valid?
>
> I think in this case "something" is allocated on the stack.

It's more likely that it will be kept in read-only data section, not on stack.
Therefore, the code snippet cited above should work fine.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Mikko Koppanen]

Hi Brian,

I had problems with pub-sub and hwm as well. The behaviour I was seeing was 
that despite the hwm messages were queued to publisher side and with large 
message size the memory usage sky rocketed within seconds after subscriber died.

I tested this with simple cpp code and could not reproduce. Are you able to 
isolate a reproducable test case?

-- 
Mikko Koppanen

On 11 May 2011, at 15:46, Brian Rossa  wrote:

> Hi all,
> 
> Let me say first that I'm a newb, but that I *kinda* feel like I've grokked 
> ZeroMQ. I'm working on a real-time vision system and we're using ZeroMQ to 
> connect its major computational components. My review so far: ZeroMQ is 
> awesome!
> 
> There are a number of design issues that I'd like help with in the medium 
> term. For now, though, there's one that needs some urgent attention: 
> *apparent* queuing of messages on PUB/SUB fan-in despite having HWM set to 1 
> for both the senders and receivers. I say "apparent" here to acknowledge that 
> problem is most likely with my implementation, and not ZeroMQ. I can't 
> isolate it on my side, though, and am generally at a loss to explain where 
> it's coming from.
> 
> We use standard PUB/SUB-bind/connect socket pairs throughout the system, all 
> with HWM set to 1 on both sides. The alleged queuing is apparent at the 
> system endpoint, where messages from all the intermediate processing nodes 
> are collected and displayed as a "movie." (For the most part, these messages 
> are small in size; they're JSON representations of bounding boxes). What we 
> are finding, though, is that messages from the fastest intermediate nodes -- 
> also the nodes that are generating the most messages as a consequence of 
> being fast -- tend to lag farther and farther behind messages from slower 
> nodes.
> 
> My understanding of the HWM semantic is that, when the HWM is reached, new 
> messages will not be queued. That means that the queued messages are not the 
> newest messages. There are good reasons for why this has to be the case. 
> Fine. Still, this growing lag for prolific senders doesn't make any sense to 
> me. The message that I receive at the system endpoint should only be as old 
> as the last zmq_recv from that node. Yet, I am experiencing situations in 
> which one of the intermediate nodes will "take a break" -- eg, it won't send 
> any messages for a while -- and the endpoint will continue to process old 
> messages from it for a few more minutes. Shouldn't it process JUST ONE 
> message from a node that's "taking a break" if HWM is 1 on both sides of that 
> socket pair?
> 
> My secret concern here is that there is some "gotcha" in the fair-queuing 
> semantics of PUB/SUB pairs that's causing this to manifest itself at the 
> fan-in point. But again, IT'S PROBABLY ME. Any intuition that you guys might 
> be able to share would be greatly appreciated. In the meantime I'll try to 
> cook up a toy example that clearly demonstrates the problem.
> 
> Cheers! And great work on ZeroMQ!
> ~Brian
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Brian Rossa]

Hi all,
Let me say first that I'm a newb, but that I *kinda* feel like I've grokked 
ZeroMQ. I'm working on a real-time vision system and we're using ZeroMQ to 
connect its major computational components. My review so far: ZeroMQ is awesome!
There are a number of design issues that I'd like help with in the medium term. 
For now, though, there's one that needs some urgent attention: *apparent* 
queuing of messages on PUB/SUB fan-in despite having HWM set to 1 for both the 
senders and receivers. I say "apparent" here to acknowledge that problem is 
most likely with my implementation, and not ZeroMQ. I can't isolate it on my 
side, though, and am generally at a loss to explain where it's coming from.
We use standard PUB/SUB-bind/connect socket pairs throughout the system, all 
with HWM set to 1 on both sides. The alleged queuing is apparent at the system 
endpoint, where messages from all the intermediate processing nodes are 
collected and displayed as a "movie." (For the most part, these messages are 
small in size; they're JSON representations of bounding boxes). What we are 
finding, though, is that messages from the fastest intermediate nodes -- also 
the nodes that are generating the most messages as a consequence of being fast 
-- tend to lag farther and farther behind messages from slower nodes.
My understanding of the HWM semantic is that, when the HWM is reached, new 
messages will not be queued. That means that the queued messages are not the 
newest messages. There are good reasons for why this has to be the case. Fine. 
Still, this growing lag for prolific senders doesn't make any sense to me. The 
message that I receive at the system endpoint should only be as old as the last 
zmq_recv from that node. Yet, I am experiencing situations in which one of the 
intermediate nodes will "take a break" -- eg, it won't send any messages for a 
while -- and the endpoint will continue to process old messages from it for a 
few more minutes. Shouldn't it process JUST ONE message from a node that's 
"taking a break" if HWM is 1 on both sides of that socket pair?
My secret concern here is that there is some "gotcha" in the fair-queuing 
semantics of PUB/SUB pairs that's causing this to manifest itself at the fan-in 
point. But again, IT'S PROBABLY ME. Any intuition that you guys might be able 
to share would be greatly appreciated. In the meantime I'll try to cook up a 
toy example that clearly demonstrates the problem.
Cheers! And great work on ZeroMQ!~Brian 
  ___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_msg_init_data with zero deallocation function

[Dirkjan Ochtman]

On Wed, May 11, 2011 at 15:38, Ilja Golshtein  wrote:
> According to zmq_msg_init_data manual
> ==
>       int zmq_msg_init_data (zmq_msg_t *msg, void *data, size_t size, 
> zmq_free_fn *ffn, void *hint);
> ==
> ffn is optional ("If provided, the deallocation function ffn shall be called 
> once the data buffer is no longer required by 0MQ").
>
> How does the thing work if ffn is not provided?

If ffn is not provided, the object passed in will never be freed.

> Is the code
> ==
> zmq_msg_t msg;
> zmq_msg_init_data (&msg, (void *)"something", 9,  NULL, NULL);
> ==
> valid?

I think in this case "something" is allocated on the stack. If the
message stays in the queue for a little bit (which might happen at any
time), the stack might've gone away and 0MQ is trying to read memory
that might have been overwritten. In almost all of the cases, you
should just use zmq_msg_init_size() and use memcpy() to copy your data
(e.g. "something") into zmq_msg_data(&msg). The guide explains this
quite well.

Cheers,

Dirkjan
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] zmq_msg_init_data with zero deallocation function

[Ilja Golshtein]

Hello!

According to zmq_msg_init_data manual
==
   int zmq_msg_init_data (zmq_msg_t *msg, void *data, size_t size, 
zmq_free_fn *ffn, void *hint);
==
ffn is optional ("If provided, the deallocation function ffn shall be called 
once the data buffer is no longer required by 0MQ").

How does the thing work if ffn is not provided?

Is the code
==
zmq_msg_t msg;
zmq_msg_init_data (&msg, (void *)"something", 9,  NULL, NULL);
==
valid?

Thanks.


-- 
Best regards,
Ilja Golshtein.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Windows throughput perfomance

[Геннадий Казачёк]

Hi!
I've started to learn zmq by compiling and running tests (remote_thr and
local_thr) to see amount of data I can utilize.
I've run applications (without any modifications) on 2 remote hosts in 1
Gbit network over tcp transport.
Results was about 10 Mbit/sec, no matter of message size I used (from 512
byte to 128Kb).
Simple file transfer shown about 250Mbit/sec (limited by the hdd speed, I
guess), and I don't understand such a low performance shown by zmq.
So, what did I do wrong?
Thanks.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Invalid argument (mutex.hpp:98)

[Azamat Shakhimardanov]

Hi Steven,

thanks, it was indeed sth with my debian libc version. The same code runs fine 
on different machine with ubuntu.

Sincerely
Azamat

--- On Tue, 5/10/11, Steven McCoy  wrote:

From: Steven McCoy 
Subject: Re: [zeromq-dev] Invalid argument (mutex.hpp:98)
To: "ZeroMQ development list" 
Date: Tuesday, May 10, 2011, 8:41 AM

On 10 May 2011 23:25, Azamat Shakhimardanov  wrote:


==466==  Address 0x469251c is 212 bytes inside a block of size 352 free'd

==466==    at 0x4023B6A: free (vg_replace_malloc.c:366)

==466==    by 0x4236739: fclose@@GLIBC_2.1 (iofclose.c:88)

==466==    by 0x432B0D0: pgm_time_init (time.c:342)

==466==    by 0x434B0D0: pgm_init (engine.c:164)

==466==    by 0x4051832: zmq_init (zmq.cpp:247)

==466==    by 0x8048829: main (testPublisherExperiment.c:32)

==466==


This is some weird glibc thing, the code isn't amazing unless the proc file 
system is failing:


http://code.google.com/p/openpgm/source/browse/trunk/openpgm/pgm/time.c#330
-- 

Steve-o 

-Inline Attachment Follows-

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] 0MQ unconf, Brussels

[Ian Barber]

On Wed, May 11, 2011 at 8:36 AM, Pieter Hintjens  wrote:

> >
> > Thanks again to all who came, it was a long day yesterday but really
> satisfying.
> >
>

Yep, very enjoyable day, thanks for organising Peter, and to everyone that
came along. Had a good train back reading the John Day book Mike suggested
as well!

Ian
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] 0MQ unconf, Brussels

[Mikko Koppanen]

On Wed, May 11, 2011 at 8:36 AM, Pieter Hintjens  wrote:
>
> Thanks again to all who came, it was a long day yesterday but really 
> satisfying.
>

Hi,

thank you for organising the event. We couldn't have made it through
the day without the croissants you got from the bakery at 6am.
Regarding the Jira: I filled in the form to apply for Hosted Jira
Studio and should hear back from Atlassion within 14 days according to
their website.

Thanks,
Mikko
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] 0MQ unconf, Brussels

[Pieter Hintjens]

Hi all,

I'd like to thank all those who came to Brussels for the meeting
yesterday. It was a small, constructive, focussed group.

Despite the lack of a structured agenda we covered pretty much every
area. We'll put up a wiki page with a summary of what was discussed
and agreed.

The first visible change is that we will drop the github issue tracker
and switch to Jira, collecting all ZeroMQ projects that want it, in
one single issue tracker. Secondly, we'll use this tracker much more
systematically for tracking and discussing changes.

For the rest, we mostly agreed on how to continue making 0MQ a
successful product and community. Pretty much the only area where we
did not find agreement was on monitoring, but IMO this is partly just
down to no-one actually trying to add monitoring and seeing how it
works.

I'll let Martin, who took notes, continue this report on the wiki.

Thanks again to all who came, it was a long day yesterday but really satisfying.

-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev