Re: [zeromq-dev] ZeroMQ for Android: Problem in building the Android APK

[Sharon Ben-Asher]

Hi,

Today, I tried to work with v2.2.0:
followed the instructions as close as I can, the build was successful, as far 
as I could see.
However, when running in the device, encountered this problem:
Cannot load library: reloc_library[1289]:   157 cannot locate 
'_ZNSt9basic_iosIcSt11char_traitsIcEE4initEPSt15basic_streambufIcS1_E'...

Thanks,
Sharon.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] syncpub/syncsub example losing messages (ZMQ 3.2.0-rc1, Solaris 11)

[Alex Keahan]

Alex Keahan  gbtradingllc.com> writes:

> $ ./syncpub &
> [1] 6611
> $ Waiting for subscribers
> 
> $ ./syncsub A &
> [2] 6612
> $ ./syncsub B &
> [3] 6613
> $ Broadcasting messages
> Message mismatch: received 'Msg #1266' expected 'Msg #1001'
> A: received 1000 updates
> Message mismatch: received 'Msg #1266' expected 'Msg #1001'
> B: received 1000 updates
> 


Whoops.   The high water mark for outbound messages was changed from 0 (no
limit) in 2.2 to 1000 in 3.2.

Not a bug, please disregard.


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] 0mq at TheStangeLoop, 23-25 Sept 2012

[Benjamin Henrion]

0mq at TheStangeLoop, 23-25 Sept 2012:

https://thestrangeloop.com/sessions/software-architecture-using-zeromq

-- 
Benjamin Henrion 
FFII Brussels - +32-484-566109 - +32-2-3500762
"In July 2005, after several failed attempts to legalise software
patents in Europe, the patent establishment changed its strategy.
Instead of explicitly seeking to sanction the patentability of
software, they are now seeking to create a central European patent
court, which would establish and enforce patentability rules in their
favor, without any possibility of correction by competing courts or
democratically elected legislators."
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] A query about implementing aborts in request reply broker

[Ian Barber]

On Mon, Jul 16, 2012 at 11:17 AM, aalok sood  wrote:

> Thanks for your suggestion Ian.
>
> I have a similar model in place, but the issue is how do I find out how
> many responses are going to come in on my pull socket.
>
> In essence, can I somehow find out how many backend workers are connected
> to my broker.
>
>
There is no easy way to do that - the idea with pub sub is that you
generally want to send a message to everyone and no worry precisely about
who the subscribers are. In your case, it might be better to have a REQ-REP
model - clients make a REQ when they're ready to do some work, your server
has a ROUTER and you track the ids, and you send out a message to each one
when you have something to do, then you can track the state internally. If
you wanted to keep the pub/pull model, you could have a worker send a
message to the pull saying that they are starting some work, and use that
to track the number you expect. Of course, you have to consider the
possibility of worker failure - so if a worker starts a job then dies, what
is you action going to be around that, both in the regular and cancellation
cases.

Ian
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] what are the best practices to send complex objects?

[Andrzej Dworak]

Hi Ron,

Sure, but, as I said before - I liked both, and both are great. I liked them 
far more than anything else. So, I don't claim one is better than another. 
What, in my case, made me chose MP over PB in the end is:

1.   I need to send bool, int8,16,32,64, float, double, string, and arrays 
of these (various lengths from 10 up to even a few millions)

2.   I need to encode types of the data sent in each message.

3.   I've prototyped simple tests for ser/deser in the way it will be used 
using both libs

4.   Running the tests on supported platforms, on average, MP was 2x faster 
than PB

Probably, in a normal situation (without extra encoding of the types) PB would 
do closer to MP.

MP is pretty stable and it gains in popularity but AFAIK there are no big 
players behind. PB is supported and used by Google, so not much worries about 
stability there :)

Cheers,
Andrzej


From: zeromq-dev-boun...@lists.zeromq.org 
[mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Ron Elliott
Sent: 17 July 2012 23:23
To: ZeroMQ development list
Subject: Re: [zeromq-dev] what are the best practices to send complex objects?


I'm not the OP however would it be possible for you to explain why you chose 
MessagePack over ProtoBufs? I currently use ProtoBufs myself and don't have any 
issues with it however I think this would be interesting to know
On Jul 17, 2012 2:18 PM, "Andrzej Dworak" 
mailto:andrzej.dwo...@cern.ch>> wrote:
Hi,
I did some extensive testing of serialization libraries (tested C++ and Java 
implementations) more or less 3 months ago. Two products that I liked the most 
were MessagePack and Google Protocol Buffers. For my case, where I need to have 
self-describing serialization, I've chosen MessagePack. The requirements were 
quite high as this is going to be used by one of a core systems at CERN to 
operate LHC and other accelerators. So far no problems with MP, it's a great 
product, well, the same as ZeroMQ :)
Cheers,
Andrzej



From: 
zeromq-dev-boun...@lists.zeromq.org 
[zeromq-dev-boun...@lists.zeromq.org]
 on behalf of aalok sood [aa...@ideadevice.com]
Sent: 17 July 2012 12:01
To: ZeroMQ development list
Subject: Re: [zeromq-dev] what are the best practices to send complex objects?
If you are **really** concerned about performance, I suggest you have your own 
protocol that both sides understand.
This way you send data as it is, the remote side would know how to interpret it.


On Tue, Jul 17, 2012 at 3:27 PM, Cem Karan 
mailto:cfkar...@gmail.com>> wrote:
I've been playing with MessagePack (http://msgpack.org/) for serialization and 
BLOSC (http://blosc.pytables.org/trac) for compression.  They seem to work 
pretty well together.  How fast is fast?  BLOSC is designed to move data around 
in memory faster than memcpy() is able to do it, and I haven't had any 
complaints about how fast MessagePack works (OTOH, I'm I/O bound, so I tend not 
to hit any limits).

Thanks,
Cem Karan

On Jul 17, 2012, at 3:37 AM, anti_tenzor wrote:

> Hello, all,
>
> I'd like to use ZMQ to send complex objects between applications (using
> CLRZMQ c# wrapper). Since it is possible to send byte[] arrays only,
> I am forced to implement some serialization-deserialization mechanism.
>
> But the hole idea is to be as fast as possible! So it looks like this
> algo should be:
> 1. Extremely fast
> 2. Produce very compact byte arrays
>
> Could you please suggest some best practices or libraries (.net), that
> meets these requirements?
>
> How do you send complex ojects via ZMQ in real-life applications?
>
> Thank you in advance!
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZeroMQ for Android: Problem in building the Android APK

[Victor Perron]

I thought, not use 'assign'. But the issue has been reported and discussed
in the Zeromq dev list.
I changed the Wiki today; you should stick to 2.2 for now, because a proper
fix is out of quick reach with 3.x .

It's either a lot of changes in the instructions in order to make zeromq
use stlport library, or get rid of the STL functions almost entirely.
Even using the STL has very little chances to succeed and will need heavy
patches to zeromq, would it only be for the headers inclusion; but I'm also
unsure that everything zeromq uses is defined by stlport.
Moreover, stlport is beta, so it's quite dangerous to rely on it.

I'd appreciate if somebody really involved into libzmq 3.x core could
comment about this...

On Wed, Jul 18, 2012 at 2:55 PM, Sharon Ben-Asher
wrote:

> Forget to mention that I didn't know what you meant by "fix those lines"
> How do I fix them?
>
> -Original Message-
> From: Victor Perron 
> Subject: Re: [zeromq-dev] zeromq-dev Digest, Vol 55, Issue 14
> To: ZeroMQ development list 
> Message-ID:
> <
> caf9fejhk-9lorwhhmrinkeaiq5akwacxijvv9k6x45j5d__...@mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi,
>
> In that case it seems that you discovered something serious.
> I took a look at it, and indeed the assign() symbol from std::string class
> seems undefined.
> That would mean, Android does _not_ define such a function in its SDK.
>
> Now, the proper way to get around this is a patch to Zeromq 3.x.
> You'll have to fix those lines (my guess):
>
> ./src/ipc_listener.cpp:151:filename.assign(addr_);
> ./src/socket_base.cpp:331:options.last_endpoint.assign (addr_);
> ./src/socket_base.cpp:480:options.last_endpoint.assign (addr_);
> ./src/tcp_address.cpp:503:addr_str.assign (name_, delimiter -
> name_);
> ./src/tcp_address.cpp:504:mask_str.assign (delimiter + 1);
> ./src/tcp_address.cpp:511:addr_str.assign (name_);
>
> ... and try to compile again. If there are no other undefined symbols,
> that should work.
> That issue did not exist in ZeroMQ 2.2 though, that one didn't make an
> extensive use of std::string.
>
> Keep me updated, we'll see for a pull request when I'll be less busy ;)
>
> Best regards,
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>



-- 
Victor
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZeroMQ for Android: Problem in building the Android APK

[Sharon Ben-Asher]

Forget to mention that I didn't know what you meant by "fix those lines"
How do I fix them?

-Original Message-
From: Victor Perron 
Subject: Re: [zeromq-dev] zeromq-dev Digest, Vol 55, Issue 14
To: ZeroMQ development list 
Message-ID:

Content-Type: text/plain; charset="iso-8859-1"

Hi,

In that case it seems that you discovered something serious.
I took a look at it, and indeed the assign() symbol from std::string class 
seems undefined.
That would mean, Android does _not_ define such a function in its SDK.

Now, the proper way to get around this is a patch to Zeromq 3.x.
You'll have to fix those lines (my guess):

./src/ipc_listener.cpp:151:filename.assign(addr_);
./src/socket_base.cpp:331:options.last_endpoint.assign (addr_);
./src/socket_base.cpp:480:options.last_endpoint.assign (addr_);
./src/tcp_address.cpp:503:addr_str.assign (name_, delimiter -
name_);
./src/tcp_address.cpp:504:mask_str.assign (delimiter + 1);
./src/tcp_address.cpp:511:addr_str.assign (name_);

... and try to compile again. If there are no other undefined symbols, that 
should work.
That issue did not exist in ZeroMQ 2.2 though, that one didn't make an 
extensive use of std::string.

Keep me updated, we'll see for a pull request when I'll be less busy ;)

Best regards,

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZeroMQ for Android: Problem in building the Android APK

[Sharon Ben-Asher]

Hi,

Well, after realizing its an issue with STL, I spent half the day on the net 
researching Android NDK and C++ STL.
Indeed, the default implementation is "minimal" according to the docs.
The issue is really an issue since we are using the Android NDK as standalone 
compiler.  As a full build env, you can specify in the Application.mk (their 
version of makefile) that you want to use STL.  They even support several 
versions of the library. 
So perhaps someone needs to repackage the whole directory tree into an Android 
project.  I think I lack the knowledge of how to do it. 

However, I did find that the standalone toolchain does come with gunstl_shared 
lib.  And I even managed to specify the proper argument to configure and to 
build new shared object files. 
However, now when I try to run the APK, I get an error about not finding 
libgnustl_shared.so.  tried to put it together with libjzmq.so, but to no avail.
I am running out of time with this experiment, they want to move to an all-java 
solution. So if time permits, I will try to build the 2.2 version.

Sharon.

-Original Message-
From: Victor Perron 
Subject: Re: [zeromq-dev] zeromq-dev Digest, Vol 55, Issue 14
To: ZeroMQ development list 
Message-ID:

Content-Type: text/plain; charset="iso-8859-1"

Hi,

In that case it seems that you discovered something serious.
I took a look at it, and indeed the assign() symbol from std::string class 
seems undefined.
That would mean, Android does _not_ define such a function in its SDK.

Now, the proper way to get around this is a patch to Zeromq 3.x.
You'll have to fix those lines (my guess):

./src/ipc_listener.cpp:151:filename.assign(addr_);
./src/socket_base.cpp:331:options.last_endpoint.assign (addr_);
./src/socket_base.cpp:480:options.last_endpoint.assign (addr_);
./src/tcp_address.cpp:503:addr_str.assign (name_, delimiter -
name_);
./src/tcp_address.cpp:504:mask_str.assign (delimiter + 1);
./src/tcp_address.cpp:511:addr_str.assign (name_);

... and try to compile again. If there are no other undefined symbols, that 
should work.
That issue did not exist in ZeroMQ 2.2 though, that one didn't make an 
extensive use of std::string.

Keep me updated, we'll see for a pull request when I'll be less busy ;)

Best regards,

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Socket use from several threads

[Chuck Remes]

On Jul 18, 2012, at 2:49 AM, moteus wrote:

> Chuck Remes  chuckremes.com> writes:
> 
> 
>> Oh, so the threads are short-lived then. In that case, you should create a
> pool of them and have each thread
>> "check out" the socket from the pool when it needs to use it and check it 
>> back
> in again when the thread exits.
>> If the pool is protected by a mechanism (like a mutex) that employs a full
> memory barrier, then it will be safe.
> 
> Thanks.
> And one more question. 
> Can I start recv in one thread but get result in other?
> For example. I use DEALER socket instead REQ and get timeout on recive.
> I put socket back to pool. Another thread get this socket from pool and send 
> it's own request. This thread can recognize its response and ignore other
> responses. 
> Or better just to recreate socket like described in guide?

I don't know. Try it out and let us know how it works.

cr
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] (no subject)

[Victor Perron]

OK.

The libstdc++ shared object from the NDK defines very few symbols, around
twenty, mostly for what google claims to support: exceptions and such.
In order to use the STL, one must use their "stlport" beta library which I
believe is included since NDKr5.
The shared object is present on my SGSIII, but I think that maybe it's
better to statically include the needed symbols in libzmq to prevent future
issues (and as long as it's in beta and that older devices mat use it)

The other way to go would be to remove all those STL calls from libzmq (as
2.2 was doing) but... ;)

Thing is, I'm afraid that those functions you reported not found are not
even in Android's stlport. And I'm not happy yo use another, third-party C
library.

I'll update the wiki about this issue, and I think you should stick to
zeromq2.2 before that; that one has been properly tested as working on
Android.

Regards,

On Wed, Jul 18, 2012 at 1:31 AM, Eric  wrote:

> Thanks,
>
> I replaced assign by clear/append. But it still not works...something make
> me feel that I don't have any STD lib available :
> making a call to arm-linux-androideabi-ld libjzmq.so give me this :
>
> /home/xxx/libjzmq.so: undefined reference to `std::basic_ios std::char_traits >::init(std::basic_streambuf std::char_traits >*)'
> /home/xxx/libjzmq.so: undefined reference to
> `__gnu_cxx::__exchange_and_add(int volatile*, int)'
> /home/xxx/libjzmq.so: undefined reference to
> `std::_Rb_tree_decrement(std::_Rb_tree_node_base*)'
> /home/xxx/libjzmq.so: undefined reference to `std::basic_string std::char_traits, std::allocator >::basic_string(std::string
> const&)'
> /home/xxx/libjzmq.so: undefined reference to `std::string::compare(char
> const*) const'
> /home/xxx/libjzmq.so: undefined reference to `std::locale::locale()'
> /home/xxx/libjzmq.so: undefined reference to `__aeabi_uldivmod'
> /home/xxx/libjzmq.so: undefined reference to `std::string::find(char
> const*, unsigned int, unsigned int) const'
> /home/xxx/libjzmq.so: undefined reference to `std::string::append(char
> const*)'
> /home/xxx/libjzmq.so: undefined reference to `std::__throw_bad_alloc()'
> /home/xxx/libjzmq.so: undefined reference to
> `std::string::assign(std::string const&)'
> /home/xxx/libjzmq.so: undefined reference to `vtable for
> std::basic_stringstream, std::allocator
> >'
> /home/xxx/libjzmq.so: undefined reference to `std::basic_string std::char_traits, std::allocator >::~basic_string()'
> /home/xxx/libjzmq.so: undefined reference to
> `std::basic_stringstream, std::allocator
> >::~basic_stringstream()'
> /home/xxx/libjzmq.so: undefined reference to
> `std::__throw_length_error(char const*)'
> /home/xxx/libjzmq.so: undefined reference to
> `std::_Rb_tree_increment(std::_Rb_tree_node_base const*)'
> /home/xxx/libjzmq.so: undefined reference to `VTT for
> std::basic_stringstream, std::allocator
> >'
> /home/xxx/libjzmq.so: undefined reference to `__gnu_cxx::__atomic_add(int
> volatile*, int)'
> /home/xxx/libjzmq.so: undefined reference to
> `std::string::_Rep::_S_empty_rep_storage'
> /home/xxx/libjzmq.so: undefined reference to `std::ostream&
> std::ostream::_M_insert(unsigned long)'
> /home/xxx/libjzmq.so: undefined reference to
> `std::ostream::operator<<(int)'
> /home/xxx/libjzmq.so: undefined reference to `std::ios_base::ios_base()'
> /home/xxx/libjzmq.so: undefined reference to `std::string::_M_leak()'
> /home/xxx/libjzmq.so: undefined reference to `std::basic_ostream std::char_traits >& std::__ostream_insert std::char_traits >(std::basic_ostream
> >&, char const*, int)'
> /home/xxx/libjzmq.so: undefined reference to
> `std::string::_M_mutate(unsigned int, unsigned int, unsigned int)'
> /home/xxx/libjzmq.so: undefined reference to `std::locale::~locale()'
> /home/xxx/libjzmq.so: undefined reference to `std::string::clear()'
> /home/xxx/libjzmq.so: undefined reference to
> `std::__throw_logic_error(char const*)'
> /home/xxx/libjzmq.so: undefined reference to
> `std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*,
> std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)'
> /home/xxx/libjzmq.so: undefined reference to `std::basic_string std::char_traits, std::allocator >::basic_string(char const*,
> std::allocator const&)'
> /home/xxx/libjzmq.so: undefined reference to
> `std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*,
> std::_Rb_tree_node_base&)'
> /home/xxx/libjzmq.so: undefined reference to `std::string::_M_leak_hard()'
> /home/xxx/libjzmq.so: undefined reference to `vtable for
> __cxxabiv1::__vmi_class_type_info'
> /home/xxx/libjzmq.so: undefined reference to `std::basic_string std::char_traits, std::allocator >::basic_string(std::string
> const&, unsigned int, unsigned int)'
> /home/xxx/libjzmq.so: undefined reference to `vtable for
> std::basic_stringbuf, std::allocator >'
> /home/xxx/libjzmq.so: undefined reference to `vtable for
> std::basic_streambuf >'
> /home/xxx/libjzmq.so: undefined reference to `std::basic_string std::char_traits, std::allocator >::basic

Re: [zeromq-dev] Socket use from several threads

[moteus]

Chuck Remes  chuckremes.com> writes:


> Oh, so the threads are short-lived then. In that case, you should create a
pool of them and have each thread
> "check out" the socket from the pool when it needs to use it and check it back
in again when the thread exits.
> If the pool is protected by a mechanism (like a mutex) that employs a full
memory barrier, then it will be safe.

Thanks.
And one more question. 
Can I start recv in one thread but get result in other?
For example. I use DEALER socket instead REQ and get timeout on recive.
I put socket back to pool. Another thread get this socket from pool and send 
it's own request. This thread can recognize its response and ignore other
responses. 
Or better just to recreate socket like described in guide?



___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Zeromq 3.2 pub sub in a large production environment

[Susan Tharakan]

Hi,

I am planning to use zeromq 3.2 pub sub in our production systems and
wanted to check with the community. How stable is zeromq 3.2 for production
use? Has anybody used it in production systems for pub sub model.

For our application the following two requirements are critical

1. Minimize latency
2. Avoid data loss


Currently I'm using a slightly modified version of zeromq 2.0.8 in
production for a 400 machine cluster with reasonable success. The major
issue we've had with zeromq 2.x are

1. High bandwidth utilization during high traffic scenarios because of the
lack of subscriber filtering, possibly resulting in buffer over runs at
producer and drops[data loss].


2. Loosing messages when the subscribers cant keep pace with the producers.
Using the SWAP option in zeromq 2.0 has saved the day in some specific
scenarios as it provided some cushion with  spurty traffic.

With zeromq 3.x the SWAP option seems to be discontinued, which is a big
concern for our usecase, but I'm guessing the buffered queue lengths will
also go down as only required msgs are kept in queue now. Any work arounds
for this problem?

The other important requirement we have is to be able to monitor what's
happening inside of zeromq. We'd done it by writing one of our custom apis
for 2.0 which provided some insight into how much data was buffered at each
end. How easy is it to do this for zeromq 3.x / any plans for it in the zmq
roadmap?

Thanks for your time
Susan
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev