[zeromq-dev] Debian package for 2.0.6 uploaded

[Dagurashibanipal]

From: Adrian von Bidder 
Date: Wed, 17 Mar 2010 22:14:49 +0100
Subject: Debian packaging for 2.0.6

Heyho!

Below the Debian packaging in 2.0.6, you may want to import it into git.

I think I'll continue to work in the hg repository I've got right now,
because I don't have the time right now to figure out a proper process
based on the git repository.  Note that sicne I work from the tarball,
zeromq as contained in the packaging repository is the result of running
autogen.sh, so packaging directly from this debian/ directory is not
possible per se in the git repository.

In my opinion, it would still make sense to ship the debian/ dir in the
tarball as it is now (maybe with a "git snapshot" entry added at the top
of debian/changelog.)  Especially now as the packaging is not
"incomplete" anymore since the language bindings were removed from the
release.

cheers
-- vbi


---
 debian/README.Debian|   10 +-
 debian/README.source|   37 +++
 debian/TODO.source  |   17 
 debian/changelog|   17 +++-
 debian/cl-zeromq.files  |7 --
 debian/cl-zeromq.install|6 -
 debian/cl-zeromq.links  |1 -
 debian/control  |  150 -
 debian/copyright|  105 ++--
 debian/dirs |5 -
 debian/docs |2 -
 debian/hgpkg|  203 +++
 debian/libzeromq-dev.files  |   37 ---
 debian/libzeromq-dev.install|   19 
 debian/libzeromq-python.files   |1 -
 debian/libzeromq-python.install |1 -
 debian/libzeromq-ruby.files |1 -
 debian/libzeromq-ruby.install   |1 -
 debian/libzeromq0.files |2 -
 debian/libzeromq0.install   |1 -
 debian/libzmq-dev.install   |5 +
 debian/libzmq-dev.manpages  |2 +
 debian/libzmq0.install  |1 +
 debian/libzmq0.manpages |5 +
 debian/rules|  122 
 debian/shlibs.local |1 -
 debian/source/format|1 +
 debian/source/options   |1 +
 debian/zeromq-examples.files|2 -
 debian/zeromq-examples.install  |2 -
 debian/zeromq-perf.files|   10 --
 debian/zeromq-perf.install  |   10 --
 debian/zeromq-utils.files   |6 -
 debian/zeromq-utils.install |9 +-
 debian/zeromq-utils.manpages|3 +
 35 files changed, 471 insertions(+), 332 deletions(-)
 create mode 100644 debian/README.source
 create mode 100644 debian/TODO.source
 delete mode 100644 debian/cl-zeromq.files
 delete mode 100644 debian/cl-zeromq.install
 delete mode 100644 debian/cl-zeromq.links
 delete mode 100644 debian/dirs
 delete mode 100644 debian/docs
 create mode 100755 debian/hgpkg
 delete mode 100644 debian/libzeromq-dev.files
 delete mode 100644 debian/libzeromq-dev.install
 delete mode 100644 debian/libzeromq-python.files
 delete mode 100644 debian/libzeromq-python.install
 delete mode 100644 debian/libzeromq-ruby.files
 delete mode 100644 debian/libzeromq-ruby.install
 delete mode 100644 debian/libzeromq0.files
 delete mode 100644 debian/libzeromq0.install
 create mode 100644 debian/libzmq-dev.install
 create mode 100644 debian/libzmq-dev.manpages
 create mode 100644 debian/libzmq0.install
 create mode 100644 debian/libzmq0.manpages
 delete mode 100644 debian/shlibs.local
 create mode 100644 debian/source/format
 create mode 100644 debian/source/options
 delete mode 100644 debian/zeromq-examples.files
 delete mode 100644 debian/zeromq-examples.install
 delete mode 100644 debian/zeromq-perf.files
 delete mode 100644 debian/zeromq-perf.install
 delete mode 100644 debian/zeromq-utils.files
 create mode 100644 debian/zeromq-utils.manpages

diff --git a/debian/README.Debian b/debian/README.Debian
index 4f447cc..cbbd2aa 100644
--- a/debian/README.Debian
+++ b/debian/README.Debian
@@ -1,6 +1,4 @@
-zeromq for Debian
--
-
-
-
- -- Peter Busser   Tue, 15 Dec 2009 16:11:38 +0100
+Currently, the zeromq Debian packages provide only the C and C++
+language API.  The other language bindings are work in progress and will
+become available as time permits; help welcome (just email me at
+avbid...@fortytwo.ch.)
diff --git a/debian/README.source b/debian/README.source
new file mode 100644
index 000..c0c7266
--- /dev/null
+++ b/debian/README.source
@@ -0,0 +1,37 @@
+The source of this package is managed by using a hgpkg managed
+mercurial repository at https://fortytwo.ch/hg/pkg-zeromq.  (hgpkg is
+currently included in the debian/ directoriy here and not separately packaged,
+since it's not widely used yet.)
+
+Typically, you'd start by:
+
+ $ wget TODO get upstream tarball
+ $ hg clone https://fortytwo.ch/hg/pkg-zeromq zeromq
+ $ debian/hgpkg build
+
+Description: the repository contains an "upstream" and a "default" branch;
+"upstream" is directly imported from upstream's svn, and

Re: [zeromq-dev] ZMQ 2.0.6 connect SUB socket error

[Matthew Giedt]

I traced the application to the point where it calls getaddrinfo which is
failing on my linux machine. It's returning 1, with the error:

Name or service not known

I boiled down everything to: (copied straight from ip.cpp)

int main( int argc, char *argv[] )
{
addrinfo req;
memset (&req, 0, sizeof (req));

req.ai_family = AF_INET;

req.ai_socktype = SOCK_STREAM;
req.ai_flags = AI_NUMERICSERV | AI_ADDRCONFIG;
addrinfo *res;
int rc = getaddrinfo ( "127.0.0.1", "", &req, &res);

if (rc) {
cout << gai_strerror( rc ) << endl; // returns --> Name or service
not known
}

return 0;
}

I've tried ifconfig down / up, rebooting, to no avail. Would very much
appreciate any further advice, although it's obviously not a zmq issue.

Regards,
-Matt

On Wed, Mar 17, 2010 at 1:13 PM, Martin Sustrik  wrote:

> Matthew,
>
> The settings on your computer look OK.
>
> Your test program doesn't induce the problem on my box.
>
> Can you run it under debugger and step through zmq::resolve_ip_hostname
> function to see what's happening?
>
> Thanks.
> Martin
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Load balancing REQ/REP sockets

[Brian Granger]

Martin,

> The whole XREQ/XREP business is still messy. If you absolutely need it at
> the moment you can use XREQ to connect and load-balance messages to mutliple
> REPs. That way you'll get what you need without a need to do any special
> processing on the REP side.

I have tried the XREQ -> multiple REP socket topology.  The requests
are load balanced to the REP sockets and there can be multiple
requests in the air at a time.  BUT, the notion of load balancing is a
bit odd.  The load balancing logic is here:

http://github.com/sustrik/zeromq2/blob/master/src/lb.cpp#L90

The idea is that the next send will happen on the current+1 active
pipe.  Basically, this round-robins the requests to the different REP
sockets.  But, in my mind, this strategy is not a completely useful
load balancing.  Here is why.  Some requests may take longer/shorter
for the REP socket to process.  With the current approach, a REP
socket that handles a very short request, will have to sit there doing
nothing until its turn comes again.  Likewise, a REP socket that gets
a very time consuming request will continue to get new requests,
though it is busy.

A more useful and dynamic load balancing approach would be would be
the following:

* Look at the current + 1 pipe.  If it is not servicing a request, use it.
* If the current + 1 pipe is busy, see if the current + 2 pipe is
servicing a request.  If not use it.
* Continue through the active pipes.  If all are busy, use the current
+ 1 anyways.

Is there a way of telling if a pipe is servicing a request?  Would it
make sense to implement this?  This more dynamic load balancing would
be incredibly useful.

Cheers,

Brian

> However, it's experimental functionality, so no warranty is given :)
>
> Martin
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgran...@calpoly.edu
elliso...@gmail.com
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] P2P updates or another method?

[Jeff Dik]

Hi,

I've been wanting to use 0MQ at work, but can't quite get the behavior
I want.  I think I want P2P type connections, but run into the lack of
auto-connect code[1].

What I'm trying to get is for an application to send lots of messages
and have those stored on a queue until a client connects and receives
the messages.  Is that handled best with P2P, or am I overlooking
something else?

Thanks,
Jeff

[1]: http://www.mail-archive.com/zeromq-dev@lists.zeromq.org/msg00378.html
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] C# interface for Zmq 2.x

[Jeff Dik]

Hi,

I made some C# bindings a few weeks ago, but just committed it and
pushed it: http://github.com/s450r1/zeromq2/blob/master/bindings/clr/Zmq.cs

However, it's only been tested to work with the 0MQ from a few weeks
ago, before the language bindings were kicked out of the core 0MQ
repository :-)  I've been meaning to update it with the latest changes
to 0MQ, but haven't had a chance yet (as I have 6 month old twin
girls).  Feel free to use that as a starting point if you wish.
There's a few examples in
http://github.com/s450r1/zeromq2/tree/master/bindings/clr.  See the
Rakefile for how to build (or use rake[1] to build).

[1]: http://rake.rubyforge.org/

Thanks,
Jeff

On Wed, Mar 17, 2010 at 6:41 AM, Martin Sustrik  wrote:
> Hi Alex,
>
>> Of coz I would like to tried it.
>> But I know nth about c++ so tried to use SWIG to generate the C#
>> interface last night and failed last night.
>>
>> Could you send me the binding for 1.0 so that I can port it to 2.0?
>
> Here's the code for C# binding in 0MQ/1.0:
>
> http://github.com/sustrik/zeromq1/blob/master/libclrzmq/zmq.cs
>
> As you can see it's pretty trivial code that does nothing but forward
> each call to underlying C library.
>
> Here's the new 2.0 interface:
>
> http://github.com/sustrik/zeromq2/blob/master/include/zmq.h
>
> What has to be done:
>
> 1. Constants in zmq.h should be made avaiable in zmq.cs
> 2. Functions from zmq.h should be declared as static extenal functions
> in zmq.cs
> 3. Simple C# wrapper should be written to provide a nice OO interface.
>
> Basically all the code can be found in the old binding, however, if you
> run into problems, feel free to discuss it on the list.
>
> Martin
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZMQ 2.0.6 connect SUB socket error

[Mike Perik]

I may be missing something obvious since I more of a lurker on this list then a 
user, but the 'subscribing' line below states that it is subscribing on port 
 not 5551 which the original post did when 'localhost' was used.



mgi...@ubuntu:~$ ./Tester tcp://127.0.0.1:5551 tcp://127.0.0.1:5551 test.xml
publishing 
on: tcp://127.0.0.1:5551 success.
subscribing on: tcp://127.0.0.1:
terminate called after throwing an instance of 'zmq::error_t'
  what():  
Invalid argument
Aborted



  
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZMQ 2.0.6 connect SUB socket error

[Martin Sustrik]

Matthew,

The settings on your computer look OK.

Your test program doesn't induce the problem on my box.

Can you run it under debugger and step through zmq::resolve_ip_hostname 
function to see what's happening?

Thanks.
Martin

Matthew Giedt wrote:
> mgi...@ubuntu:~$ ./Tester tcp://127.0.0.1:5551  
> tcp://127.0.0.1:5551  test.xml
> publishing on: tcp://127.0.0.1:5551  success.
> subscribing on: tcp://127.0.0.1: 
> terminate called after throwing an instance of 'zmq::error_t'
>   what():  Invalid argument
> Aborted
> 
> 
> mgi...@ubuntu:~$ ifconfig lo
> loLink encap:Local Loopback 
>   inet addr:127.0.0.1  Mask:255.0.0.0
>   inet6 addr: ::1/128 Scope:Host
>   UP LOOPBACK RUNNING  MTU:16436  Metric:1
>   RX packets:4 errors:0 dropped:0 overruns:0 frame:0
>   TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
>   collisions:0 txqueuelen:0
>   RX bytes:240 (240.0 B)  TX bytes:240 (240.0 B)
> 
> 
> mgi...@ubuntu:~$ ping -c 1 localhost
> PING localhost (127.0.0.1) 56(84) bytes of data.
> 64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.048 ms
> 
> --- localhost ping statistics ---
> 1 packets transmitted, 1 received, 0% packet loss, time 0ms
> rtt min/avg/max/mdev = 0.048/0.048/0.048/0.000 ms
> mgi...@ubuntu:~/bdsquant/trunk/source/mktdata_engine/cpp$
> 
> On Wed, Mar 17, 2010 at 11:27 AM, Martin Sustrik  > wrote:
> 
> Hello,
> 
> 
> Matthew Giedt wrote:
> 
> Hello --
> 
> I just upgraded from 2.0b2 to 2.0.6 -- (downloaded, ./configure
> --> make --> make install, verified new libraries and include)
> 
> When running the following program with:
> 
> ./test tcp://lo:5551 tcp://localhost:5551 data.txt
> 
>  I get:
> 
> publishing on: tcp://lo:5551 success.
> subscribing on: tcp://localhost:5551
> terminate called after throwing instance of zmq::error_t
>  what():  Invalid argument
> Aborted
> 
> Are you guys seeing the same thing? Is it an upgrade from
> previous beta issue?
> 
> 
> What system are you running on? It looks like a problem with
> resolving 'localhost', can you check if it works with
> "tcp://127.0.0.1:5551 " for both bind() and
> connect()?
> 
> Further, what does "ifconfig lo" and "ping -c 1 localhost" report on
> your system?
> 
> Martin
> 
> 
> 

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZMQ 2.0.6 connect SUB socket error

[Matthew Giedt]

mgi...@ubuntu:~$ ./Tester tcp://127.0.0.1:5551 tcp://127.0.0.1:5551 test.xml
publishing on: tcp://127.0.0.1:5551 success.
subscribing on: tcp://127.0.0.1:
terminate called after throwing an instance of 'zmq::error_t'
  what():  Invalid argument
Aborted


mgi...@ubuntu:~$ ifconfig lo
loLink encap:Local Loopback
  inet addr:127.0.0.1  Mask:255.0.0.0
  inet6 addr: ::1/128 Scope:Host
  UP LOOPBACK RUNNING  MTU:16436  Metric:1
  RX packets:4 errors:0 dropped:0 overruns:0 frame:0
  TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:240 (240.0 B)  TX bytes:240 (240.0 B)


mgi...@ubuntu:~$ ping -c 1 localhost
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.048 ms

--- localhost ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.048/0.048/0.048/0.000 ms
mgi...@ubuntu:~/bdsquant/trunk/source/mktdata_engine/cpp$

On Wed, Mar 17, 2010 at 11:27 AM, Martin Sustrik  wrote:

> Hello,
>
>
> Matthew Giedt wrote:
>
>> Hello --
>>
>> I just upgraded from 2.0b2 to 2.0.6 -- (downloaded, ./configure --> make
>> --> make install, verified new libraries and include)
>>
>> When running the following program with:
>>
>> ./test tcp://lo:5551 tcp://localhost:5551 data.txt
>>
>>  I get:
>>
>> publishing on: tcp://lo:5551 success.
>> subscribing on: tcp://localhost:5551
>> terminate called after throwing instance of zmq::error_t
>>  what():  Invalid argument
>> Aborted
>>
>> Are you guys seeing the same thing? Is it an upgrade from previous beta
>> issue?
>>
>
> What system are you running on? It looks like a problem with resolving
> 'localhost', can you check if it works with "tcp://127.0.0.1:5551" for
> both bind() and connect()?
>
> Further, what does "ifconfig lo" and "ping -c 1 localhost" report on your
> system?
>
> Martin
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZMQ 2.0.6 connect SUB socket error

[Martin Sustrik]

Hello,

Matthew Giedt wrote:
> Hello --
> 
> I just upgraded from 2.0b2 to 2.0.6 -- (downloaded, ./configure --> make 
> --> make install, verified new libraries and include)
> 
> When running the following program with:
> 
> ./test tcp://lo:5551 tcp://localhost:5551 data.txt
> 
>  I get:
> 
> publishing on: tcp://lo:5551 success.
> subscribing on: tcp://localhost:5551
> terminate called after throwing instance of zmq::error_t
>   what():  Invalid argument
> Aborted
> 
> Are you guys seeing the same thing? Is it an upgrade from previous beta 
> issue?

What system are you running on? It looks like a problem with resolving 
'localhost', can you check if it works with "tcp://127.0.0.1:5551" for 
both bind() and connect()?

Further, what does "ifconfig lo" and "ping -c 1 localhost" report on 
your system?

Martin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Socket connect status

[Martin Sustrik]

Diablo 666 wrote:
> 
> Hi,
> 
>  > The problem is that each end of the queue lives in a different thread.
>  > Writer thread cannot access read end of the queue and vice versa.
> 
> This should not be a problem, I guess. The calling thread can see the
> current write position, so the positions differ after a write.
> Changes in the read positions may be seen with a certain delay, which
> means, that the queue may be empty, but the calling thread sees
> different write and read positions. It can not happen, that the queue
> helds data but the calling thread sees no difference. So this approach
> should be save.

Read position will be sent eventually to the writer thread, but only 
after N messages are passed. So, if you send just 1 message you may 
never get read position.

My impression is that you are taking the hard way. Implementing a sync 
API is matter of few hours. Making existing code work in a sync manner 
is large and tedious task.

Martin

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] ZMQ 2.0.6 connect SUB socket error

[Matthew Giedt]

Hello --

I just upgraded from 2.0b2 to 2.0.6 -- (downloaded, ./configure --> make -->
make install, verified new libraries and include)

When running the following program with:

./test tcp://lo:5551 tcp://localhost:5551 data.txt

 I get:

publishing on: tcp://lo:5551 success.
subscribing on: tcp://localhost:5551
terminate called after throwing instance of zmq::error_t
  what():  Invalid argument
Aborted

Are you guys seeing the same thing? Is it an upgrade from previous beta
issue?

With thanks,
-Matt



#include 
#include 
#include 

#include 

using namespace std;

int main( int argc, char *argv[] )
{
string line;
zmq::message_t msg;
zmq::context_t ctx ( 1, 1 );
zmq::socket_t pub ( ctx, ZMQ_PUB );
zmq::socket_t sub ( ctx, ZMQ_SUB );

cout << "publishing on: " << argv[ 1 ];
pub.bind( argv[ 1 ] );
cout << " success." << endl;

cout << "subscribing on: " << argv[ 2 ];
sub.connect( argv[ 2 ] );
cout << " success." << endl;

cout << "opening file: " << argv[ 3 ] << endl;
ifstream xmlfile ( argv[ 3 ] );

if( xmlfile.is_open() )
{
while(! xmlfile.eof() )
{
getline( xmlfile, line );

cout << " <-- " << line << endl;

const char* xml = line.c_str();
const int len = strlen( xml ) + 1;

msg.rebuild ( len );
memcpy( msg.data(), xml, len );

pub.send( msg );
sub.recv( &msg );

cout << " --> " << (char *)msg.data() << endl;

sleep( 1 );
}
}

xmlfile.close();
return 0;
}
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Socket connect status

[Brian Granger]

Martin,

>> I have thought about implementing something like this in the Python
>> bindings using Python's builtin socket library, but having it in 0MQ
>> itself would be nice.  Having a pure Python version would enable
>> people to use 0MQ in situations where they can't or don't want to
>> install libzmq.  For some Python users, it is critical for them to
>> have pure python (no compiled code).
>
> Interesting. What's the reason? Security?

No, some people in the Python community like to have everything "pure
Python" for installation simplicity.  This would allow them to install
and use pyzmq without a C++ compiler.  Some deployment settings don't
allow compiled code - for example Google App Engine.  You can upload a
pure python module to GAE, but you can't compile code or upload
compiiled code.

> As for the sync API. Yes, it would be nice to have it. At the same time it's
> pretty low hanging fruit so if somebody wants to give it a try...

I have focused on other stuff right now but maybe eventually.

Cheers,

Brian


-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgran...@calpoly.edu
elliso...@gmail.com
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Socket connect status

[Diablo 666]

Hi,

> The problem is that each end of the queue lives in a different thread. 
> Writer thread cannot access read end of the queue and vice versa.

This should not be a problem, I guess. The calling thread can see the
current write position, so the positions differ after a write.
Changes in the read positions may be seen with a certain delay, which
means, that the queue may be empty, but the calling thread sees
different write and read positions. It can not happen, that the queue
helds data but the calling thread sees no difference. So this approach
should be save.

Andreas
  
_
http://redirect.gimas.net/?n=M1003xHotmail2
Alles in einem Postfach  –  Ich will Hotmail!___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Socket connect status

[Martin Sustrik]

Diablo 666 wrote:
> Hi,
> 
>  > Ok. Fair enough. I believe more people are struggling with the same
>  > problem. What about writing a simple sync API implementation?
> 
> for my use case, it would be sufficient to check if the underlying queue
> is empty. This should be very easy to implement by simply checking if
> begin_pos == end_pos in yqueue_t and calling this method via the
> socket. This would prevent any use of synchronizing. Do you see any
> drawback within this approach?

The problem is that each end of the queue lives in a different thread. 
Writer thread cannot access read end of the queue and vice versa. Note 
that there's no mutex that would lock the whole queue and let you check 
whether it's empty. It's a lock-free queue (synchronisation between the 
threads is done using CPU's atomic operations and memory barriers).

Martin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Socket connect status

[Diablo 666]

Hi,

> Ok. Fair enough. I believe more people are struggling with the same 
> problem. What about writing a simple sync API implementation?

for my use case, it would be sufficient to check if the underlying queue
is empty. This should be very easy to implement by simply checking if
begin_pos == end_pos in yqueue_t and calling this method via the
socket. This would prevent any use of synchronizing. Do you see any
drawback within this approach?

Andreas
  
_
http://redirect.gimas.net/?n=M1003xHM5Adressen2
Ein Postfach für Alles –  bei Hotmail 5 E-Mail-Adressen online verwalten!___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] C# interface for Zmq 2.x

[Martin Sustrik]

Hi Alex,

> Of coz I would like to tried it. 
> But I know nth about c++ so tried to use SWIG to generate the C#
> interface last night and failed last night.
> 
> Could you send me the binding for 1.0 so that I can port it to 2.0?

Here's the code for C# binding in 0MQ/1.0:

http://github.com/sustrik/zeromq1/blob/master/libclrzmq/zmq.cs

As you can see it's pretty trivial code that does nothing but forward 
each call to underlying C library.

Here's the new 2.0 interface:

http://github.com/sustrik/zeromq2/blob/master/include/zmq.h

What has to be done:

1. Constants in zmq.h should be made avaiable in zmq.cs
2. Functions from zmq.h should be declared as static extenal functions 
in zmq.cs
3. Simple C# wrapper should be written to provide a nice OO interface.

Basically all the code can be found in the old binding, however, if you 
run into problems, feel free to discuss it on the list.

Martin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] C# interface for Zmq 2.x

[Yu, Alex (Equity Technology)]

Thanks Martin,

Of coz I would like to tried it. 
But I know nth about c++ so tried to use SWIG to generate the C#
interface last night and failed last night.

Could you send me the binding for 1.0 so that I can port it to 2.0?

Kindly Regards,
Alex

-Original Message-
From: Martin Sustrik [mailto:sust...@250bpm.com] 
Sent: Wednesday, March 17, 2010 6:12 PM
To: Yu, Alex (Equity Technology)
Cc: zeromq-...@mail.imatix.com; alexc...@gmail.com
Subject: Re: [zeromq-dev] C# interface for Zmq 2.x

Hi Alex,

> I am a new user for zmq and want to make use of the zmq 2.X in my C# 
> project.
> 
> I found that there is not c# wrapping for zmq 2.x. Is there any C# 
> wrapping for 2.x just like that in 1.x?

No, the C# binding wasn't yet ported to 0MQ/2.0. However, it's a couple 
of hours' work. Would you like to give it a try?

Martin

--
This message w/attachments (message) may be privileged, confidential or 
proprietary, and if you are not an intended recipient, please notify the 
sender, do not use or share it and delete it. The information contained in this 
e-mail was obtained from sources believed to be reliable; however, the accuracy 
or completeness of this information is not guaranteed. Unless specifically 
indicated, this message is not an offer to sell or a solicitation of any 
investment products or other financial product or service, an official 
confirmation of any transaction, or an official statement of Merrill Lynch. 
Subject to applicable law, Merrill Lynch may monitor, review and retain 
e-communications (EC) traveling through its networks/systems. The laws of the 
country of each sender/recipient may impact the handling of EC, and EC may be 
archived, supervised and produced in countries other than the country in which 
you are located. This message cannot be guaranteed to be secure or error-free. 
References to "Merrill Lynch" are references to any company in the Merrill 
Lynch & Co., Inc. group of companies, which are wholly-owned by Bank of America 
Corporation. Securities and Insurance Products: * Are Not FDIC Insured * Are 
Not Bank Guaranteed * May Lose Value * Are Not a Bank Deposit * Are Not a 
Condition to Any Banking Service or Activity * Are Not Insured by Any Federal 
Government Agency. Past performance is no guarantee of future results. 
Attachments that are part of this E-communication may have additional important 
disclosures and disclaimers, which you should read. This message is subject to 
terms available at the following link: 
http://www.ml.com/e-communications_terms/. By messaging with Merrill Lynch you 
consent to the foregoing.
--
 
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Load balancing REQ/REP sockets

[Martin Sustrik]

Jon Dyte wrote:
> Brian Granger wrote:
>> Hi,
>>
>> REQ/REP sockets have a load balancing feature.  If you have 1 master
>> with a REQ socket that binds:
>>
>> ctx = zmq.Context()
>> s = ctx.socket(zmq.REQ)
>> s.bind('tcp://127.0.0.1:')
>>
>> And multiple workers that connect REP sockets to the REQ:
>>
>> # Each worker does this.
>> ctx = zmq.Context()
>> s = ctx.socket(zmq.REP)
>> s.connect('tcp://127.0.0.1:')
>>
>> When the master does a send, the requests will be load balanced to all
>> connected workers.  That works fine.
>>
>> But, a REQ socket cannot do another send until it gets back a reply
>> (by doing a recv).  So here is my question:
>>
>> How can you actually get load balancing in practice if the REQ socket
>> in the master cannot issue another send
>> (to a different worker) until first worker has replied.  Another way
>> of putting this is I don't see how to get the multiple
>> workers handling requests *simultaneously*.  Am I missing something or
>> is this by design.
>>
>> I can see how to accomplish this pattern with P2P sockets, but it
>> would require a much more complicated application logic to implement.
>>
>> thoughts?
>>
>> Cheers,
>>
>> Brian
>>
>>   
> you can get load balancing from multiple REQ sockets, but each 
> individual REQ socket
> effectively serialises each req and reply as far as I understand.
> 
> to do this intra-process you do something like see attached zmqserver.cpp
> which uses Xrep/xreq with a client like
> 
> // zmq
> #include 
> #include 
> #include 
> #include 
> 
> 
> int main(int argc , char*argv[])
> {
>  zmq::context_t ctx (1, 1);
> 
>  zmq::socket_t s (ctx, ZMQ_REQ);
> 
>  s.connect ("tcp://127.0.0.1:43210");
>  for (unsigned char i = 'A' ; i < 'Z' ; ++i)
>  { 
>zmq::message_t request(1);
>  *((char*)request.data()) =  i;
>  printf("send %x\n", (int) *((char*)request.data()));
>  s.send(request);
>zmq::message_t response;
>  s.recv(&response);
>  printf("recv %x\n", (int) *((char*)response.data()));
>  sleep(::getpid()%3);
>  }
>  return 0;
> 
> }
> 
> 
> you can also use the zmq_queue device which does the same thing but 
> doesnt use the inprocess endpoints.
> 
> thus you can have
> 
> REQ*<->SHARED_QUEUE(XREP,XREQ) <->REP*
> 
> * multiple of these, 1 shared queue.
> 
> caveat at the moment you have to do some special processing in the REP 
> socket.
> 
> Normally i have the shared queue do the binds for the xrep,xreq sockets 
> and then
> the workers(REP) and clients(REQ) connect to that.
> 
> There is a working zmq_queue device in the latest release.
> 
> does this make sense?

The whole XREQ/XREP business is still messy. If you absolutely need it 
at the moment you can use XREQ to connect and load-balance messages to 
mutliple REPs. That way you'll get what you need without a need to do 
any special processing on the REP side.

However, it's experimental functionality, so no warranty is given :)

Martin

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] C# interface for Zmq 2.x

[Martin Sustrik]

Hi Alex,

> I am a new user for zmq and want to make use of the zmq 2.X in my C# 
> project.
> 
> I found that there is not c# wrapping for zmq 2.x. Is there any C# 
> wrapping for 2.x just like that in 1.x?

No, the C# binding wasn't yet ported to 0MQ/2.0. However, it's a couple 
of hours' work. Would you like to give it a try?

Martin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] 0MQ version 2.0.6 (beta) released

[Martin Sustrik]

Hi Michael,

> You mentioned earlier that there was a number of fixes to zero-copy
> committed after 0MQ 2.0b2.  Would you like me to re-run the Infiniband SDP
> tests with 0MQ 2.0.6?  If so, would it be helpful if I did anything
> differently?

I don't think anything changed performance-wise, however, if you have 
few minutes to spend, it would be helpful if you'd run throughput tests 
on top of InfiniBand (perf/local_thr & perf/remote_thr) for different 
message sizes (1B, 2B, ..., 256MB).

I've already sent throughput estimates based on your latency results, 
but there are only implied, thus the real numbers would be helpful.

Martin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Subtle zmq_poll bug #1

[Martin Sustrik]

Brian Granger wrote:

>> What I wanted to see was whether the offending file descriptor is completely
>> bogus or whether it was closed beforehand by 0MQ...
> 
> I am not sure how to tell that from dtrace.  But I can send the output
> if you want.

Yes, please. The worse that can happen is that I won't be able to figure 
out anything from the trace.

Martin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Socket connect status

[Martin Sustrik]

Brian,

>> Ok. Fair enough. I believe more people are struggling with the same
>> problem. What about writing a simple sync API implementation?
>>
>> s = zmq_sync_socket (ZMQ_SUB, "tcp://localhost:");
>> zmq_sync_send (s, &msg);
>> zmq_sync_recv (s, &msg);
>>
>> The implementation would be pretty trivial. The only thing needed is to
>> prepend frame header to each message sent and remove the frame header
>> from each message received. For desctiption of the wire format, have a
>> look here:
>>
>> http://api.zeromq.org/zmq_tcp.html
> 
> I have thought about implementing something like this in the Python
> bindings using Python's builtin socket library, but having it in 0MQ
> itself would be nice.  Having a pure Python version would enable
> people to use 0MQ in situations where they can't or don't want to
> install libzmq.  For some Python users, it is critical for them to
> have pure python (no compiled code).

Interesting. What's the reason? Security?

As for the sync API. Yes, it would be nice to have it. At the same time 
it's pretty low hanging fruit so if somebody wants to give it a try...

Martin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Load balancing REQ/REP sockets

[Jon Dyte]

Brian Granger wrote:

Hi,

REQ/REP sockets have a load balancing feature.  If you have 1 master
with a REQ socket that binds:

ctx = zmq.Context()
s = ctx.socket(zmq.REQ)
s.bind('tcp://127.0.0.1:')

And multiple workers that connect REP sockets to the REQ:

# Each worker does this.
ctx = zmq.Context()
s = ctx.socket(zmq.REP)
s.connect('tcp://127.0.0.1:')

When the master does a send, the requests will be load balanced to all
connected workers.  That works fine.

But, a REQ socket cannot do another send until it gets back a reply
(by doing a recv).  So here is my question:

How can you actually get load balancing in practice if the REQ socket
in the master cannot issue another send
(to a different worker) until first worker has replied.  Another way
of putting this is I don't see how to get the multiple
workers handling requests *simultaneously*.  Am I missing something or
is this by design.

I can see how to accomplish this pattern with P2P sockets, but it
would require a much more complicated application logic to implement.

thoughts?

Cheers,

Brian

  
you can get load balancing from multiple REQ sockets, but each 
individual REQ socket

effectively serialises each req and reply as far as I understand.

to do this intra-process you do something like see attached zmqserver.cpp
which uses Xrep/xreq with a client like

// zmq
#include 
#include 
#include 
#include 


int main(int argc , char*argv[])
{
 zmq::context_t ctx (1, 1);

 zmq::socket_t s (ctx, ZMQ_REQ);

 s.connect ("tcp://127.0.0.1:43210");
 for (unsigned char i = 'A' ; i < 'Z' ; ++i)
 {  


   zmq::message_t request(1);
 *((char*)request.data()) =  i;
 printf("send %x\n", (int) *((char*)request.data()));
 s.send(request);
  
 zmq::message_t response;

 s.recv(&response);
 printf("recv %x\n", (int) *((char*)response.data()));
 sleep(::getpid()%3);
 }
 return 0;

}


you can also use the zmq_queue device which does the same thing but 
doesnt use the inprocess endpoints.


thus you can have

REQ*<->SHARED_QUEUE(XREP,XREQ) <->REP*

* multiple of these, 1 shared queue.

caveat at the moment you have to do some special processing in the REP 
socket.


Normally i have the shared queue do the binds for the xrep,xreq sockets 
and then

the workers(REP) and clients(REQ) connect to that.

There is a working zmq_queue device in the latest release.

does this make sense?


Jon

//  zmq
#include 
#include 
#include 
#include 

struct args
{
  zmq::context_t* _ctx;
  std::string _address;
};


void * worker(void * arg)
{

  args * a = (args*) arg;

  zmq::socket_t rep(*(a->_ctx), ZMQ_REP);
  
  rep.connect(a->_address.c_str());



  while (true)
{
  zmq::message_t request;
  rep.recv(&request);

  unsigned char l = *((char*)request.data());
  char* msg = (char*)request.data();

  printf("%d %c\n", (int)pthread_self(), *(msg + l +1));

  zmq::message_t response(l+2);

  ::memcpy(response.data(), request.data(), l+2);
  //*((char*)response.data()) = '?';
  rep.send(response);
}
  
  return NULL;
}





class queue
{
public:
  queue (zmq::socket_t& reply, zmq::socket_t& request)
: xrep(reply), 
  xreq(request)
  {

items[0].socket = reply; // socket_t has operator void*
items[0].fd = 0;
items[0].events = ZMQ_POLLIN ;//| ZMQ_POLLOUT;
items[0].revents = 0;

items[1].socket = request; // socket_t has operator void*
items[1].fd = 0;
items[1].events = ZMQ_POLLIN;
items[1].revents = 0;

m_next_request_method = &queue::get_request;
m_next_response_method = &queue::get_response;

  }


private:
  zmq::socket_t & xrep;
  zmq::socket_t & xreq;
  zmq_pollitem_t items[2];
  zmq::message_t request_msg;
  zmq::message_t response_msg;


  typedef void (queue::*next_method)();

  next_method m_next_request_method;
  next_method m_next_response_method;

public:
  void next_request()
  {
(this->*m_next_request_method)();
  }

  void next_response()
  {
(this->*m_next_response_method)();
  }


  void get_request()
  {
if ( items[0].revents & ZMQ_POLLIN )
  {
	int rc = xrep.recv(&request_msg, ZMQ_NOBLOCK);
	if (!rc) return;
	items[0].events &= ~ZMQ_POLLIN;
	items[1].events |= ZMQ_POLLOUT;
	m_next_request_method = &queue::send_request;
  }
  }
  void send_request()
  {
if ( items[1].revents & ZMQ_POLLOUT) 
  {
	int rc = xreq.send(request_msg, ZMQ_NOBLOCK);
	if (!rc) return;
	items[1].events &= ~ZMQ_POLLOUT;
	items[0].events |= ZMQ_POLLIN;
	m_next_request_method = &queue::get_request;
  }

  }

  void get_response()
  {
if ( items[1].revents & ZMQ_POLLIN )
  {
	int rc = xreq.recv(&response_msg, ZMQ_NOBLOCK);
	if (!rc) return;
	items[1].events &= ~ZMQ_POLLIN;
	items[0].events |= ZMQ_POLLOUT;
	m_next_response_method = &queue::send_response;
  }
  }
  void send_response()
  {
if ( items[0].revents & ZMQ_POLLOUT) 
  {
	int r