Re: [zeromq-dev] TLS (openssl) for ZeroMQ

[Steven McCoy]

It’s how Thomson Reuters uses OpenSSL and they’re the biggest law firm in
the world.  Here's the code, now open source, but not matching license, for
reference:

https://github.com/thomsonreuters/Elektron-SDK/blob/master/Cpp-C/Eta/Impl/Transport/ripcsslutils.c

On Tue, Dec 25, 2018 at 15:16 Luca Boccassi  wrote:

> On Tue, 2018-12-25 at 11:47 -0500, Steven McCoy wrote:
> > A common workaround is to dynamically pull in OpenSSL at
> > runtime.  Have a
> > context option that specifies the OpenSSL “.so” or “.dll” path and
> > cache
> > the required API inside a struct or similar.
>
> Sorry but that's not a legal solution, but rather a way to try and be
> "clever" and waltz around the precise wording of the licenses. A lawyer
> could very well convince a judge that that still counts as a single
> unit of compilation and as distributions. We cannot take chances like
> that - we don't have any lawyers "working" here, it's a community
> driven project.
>
> > On Tue, Dec 25, 2018 at 6:30 Luca Boccassi 
> > wrote:
> >
> > > On Tue, 2018-12-25 at 00:53 +0100, 林宝龙 wrote:
> > > > The problem of first option we met is that OpenSSL provides a lot
> > > > configurable things, for example, trust group, external
> > > > verification
> > > > callback, etc. We must add more options to sockopt to have such
> > > > things
> > > > configurable. For the callback functions, if we continue using
> > > > setsockopt,
> > > > we need to cast function pointer to void pointer and vice versa,
> > > > looks not
> > > > good.
> > >
> > > As mentioned, there is really no alternative to continue supporting
> > > bindings. Also, exposing a third party API/ABI again would mean
> > > that
> > > the users would need to start worrying about OpenSSL's API/ABI
> > > changes,
> > > and keep them in sync with the internal usage of the library. That
> > > would not be maintainable.
> > >
> > > So it looks like there are both legal and implementation problems.
> > > So
> > > let's take a step back: why is the current
> > > encryption/authentication
> > > support via CURVE and GSSAPI not sufficient? What is lacking that
> > > you
> > > need in your application?
> > >
> > > > About the licence issue, I'm not familiar with those licenses,
> > > > and I
> > > > have
> > > > asked someone inside my company, got the answer that I can use
> > > > OpenSSL in
> > > > libzmq with an exception, I don't know how. He said that we will
> > > > share the
> > > > code out in the end, but can't contribute back to libzmq
> > > > directly.
> > > > Does it
> > > > same as what you concern? Do you have more information that we
> > > > must
> > > > stop
> > > > using OpenSSL inside libzmq?
> > >
> > > Yes an exception is needed as I said, but not just from you: from
> > > every
> > > single copyright holder of libzmq, of which there are many. That's
> > > because adding an exception to the license is a change in license,
> > > and
> > > cannot legally be done unilaterally.
> > >
> > > Note that this is not only a problem for contributing code back,
> > > but
> > > also for your application. You cannot distribute those changes to
> > > anybody without a license change, which means you cannot give your
> > > application to anybody without breaching the terms of the license,
> > > and
> > > thus copyright law.
> > >
> > > > On Mon, 24 Dec 2018, 23:42 Luca Boccassi  > > > wrote:
> > > >
> > > > > On Mon, 24 Dec 2018, 23:03 林宝龙  > > > >
> > > > > > Hi,
> > > > > >
> > > > > > We are adding TLS support for ZeroMQ(based on 4.2.5). Product
> > > > > > reason, we
> > > > > > choosed OpenSSL as TLS library.
> > > > > >
> > > > > > Ask community for suggestions, which solution below is
> > > > > > better?
> > > > > > 1. Use TLS public certification, private key, etc as socket
> > > > > > option (set
> > > > > > through setsockopt), ZeroMQ manages the OpenSSL context,
> > > > > > one  OpenSSL
> > > > > > context per socket_base_t object.
> > > > > > 2. Use OpenSSL context as socket option(set through
&

Re: [zeromq-dev] [External] Re: A PGM/EPGM question

[Steven McCoy]

Maybe have to use zmq_poll with both in and out events?  Ultimately in_event()
needs to fire on the pgm_sender which calls process_upstream() that
processes a NAK.

On 23 March 2018 at 13:43, Montero, Antonio UTC CCS <
antonio.mont...@fs.utc.com> wrote:

> Understood however that is not the behavior I am seeing. Although that is
> likely to be the case for EPGM since those are UDP packets although from my
> understanding regardless whether incoming data is multicast or unicast, PGM
> is binding to any address and specific port. The kernel will pass all data
> received on an interface to any listening socket as long as the destination
> port patches that of the socket binding.
>
>
>
> Now let’s put aside UDP for a sec, what about when using pgm transport?
> These are raw sockets and any unicast NAK are actually sent from remote SUB
> to the PUB unicast address and source port (which is randomly selected at
> the time of creating the raw PGM PUB socket). At that point the PUB socket
> should be the only one listening on its own unicast address and source
> port. Correct?
>
>
>
> *This is a snapshot of what my netstat –ln looks like at the moment. This
> is with both ( PUB and SUB created and running on the same host ).*
>
>
>
> Proto Recv-Q Send-Q Local Address 
>Foreign
> Address  State
>
> *Sockets associated with PUB:*
>
> raw   164672  0 2001:db8::2b0:19ff:fe73:d890%2147479552:113
> ::%622984:*113
>
> raw   164672  02001:db8::2b0:19ff:fe73:d890%2147479552:113
> ::%623304:* 113
>
> raw00::%2147479552:113
>  ::%622984:*
>  113
>
> *Sockets associated with SUB:*
>
> raw   164672  02001:db8::2b0:19ff:fe73:d890%2147479552:113
> ::%622984:* 113
>
> raw   164672  02001:db8::2b0:19ff:fe73:d890%2147479552:113
> ::%623304:* 113
>
> raw00::%2147479552:113
>::%622984:*
> 113
>
>
>
> You would notice how the Recv-Q is full on both PUB and SUB related
> send/router alert send sockets.
>
> These are my thoughts as to why they are full and not because of the same
> reason:
>
>
>
> For the case of the SUB associated sockets the 2001 address ones basically
> used to send NAKs to the remote PUB:
>
> These get full as soon as a remote PUB starts sending multicast data. I
> think the SUB send socket is connecting with the destination port used to
> send multicast traffic. I can see whenever a SUB sends NAK to the PUB that
> the source port on unicast packet matches that of the destination port of
> the multicast group. However this is not really an issue since the SUB
> socket is configured in PGM as receive only therefore any ODATA/SPM data
> received on its send socket is not processed. The SUB socket however is
> also getting the multicast data via the local binding: ::%2147479552:113
> which as seen is emptying out its queue fine and I could verify the node is
> receiving data at the application level.
>
>
>
> For the case of the PUB associated sockets the 2001 address ones basically
> used to send ODATA/SPM/RDATA/NCF to remote SUB:
>
> Even though its local binding: ::%2147479552:113 is also receiving the
> multicast data sent by remote PUB it is thrown out since the PUB socket is
> configured as send only at PGM level and so ODATA/SPM data received is
> thrown out.
>
> However, its send associated sockets do receive unicast NAKs from remote
> SUB and as seen above they are being put on the socket’s Recv-Q however the
> queue is full because NAKs are not being processed by the PUB socket.
>
>
>
> Note: The exact same behavior is seen with EPGM the only difference is
> that none of the sockets Recv-Q get full because they are being emptied out
> at the UDP layer upon arrival however I suspect that once forwarded to the
> PGM layer the PGM socket buffers would show the same thing as netstat –ln
> above.
>
>
>
> Even though I think is redundant and probably not a good idea to run the
> same code when creating either a ZMQ PUB and/or SUB socket since
> essentially those socket types are restricted to do specific things like
> send/receive only, that does not appear to be the cause of the issue here.
> I have read in some of the openpgm doc that it is necessary for the
> application to frequently call pgm_recv as that somehow moves the pgm state
> machine to do things, *however my issue here is how to accomplish that
> from the ZMQ API layer*, that is the whole point of using ZMQ in my case
> in the first place.

Re: [zeromq-dev] [External] Re: A PGM/EPGM question

[Steven McCoy]

The problem is that the kernel will not multicast UDP unicast packets to
each socket listening so it is probable the wrong socket is hearing the NAK.

On Fri, Mar 23, 2018 at 12:07 Montero, Antonio UTC CCS <
antonio.mont...@fs.utc.com> wrote:

> ZMQ’s implementation of PUB socket type does not allow for receive calls
> to be made (zmq_recv is disabled), hence why I am trying to figure out how
> does one trigger ZMQ to call “pgm_recv” on the PUB socket in order to get
> the PUB socket to processes received NAKs from a remote SUB socket?
>
> I have tried querying the PUB socket state via ZMQ_EVENTS to triggering
> the processing of any commands available for the socket however that does
> not seem to move the PGM state machine in terms of processing NAKs.
>
>
>
> I am running both a PUB and SUB on the same application on the same host
> and although I see the same set of sockets being created at the PGM level
> for both PUB and SUB ZMQ sockets which includes multiple sockets binding to
> the same port, this does not appear to cause any issues in terms of my SUB
> socket able to receive multicast messages from a remote PUB and respond
> with unicast NAKs when data loss is detected.
>
>
>
> Any ideas as to how a user should get ZMQ lib to trigger NAKs processing
> for a PUB socket using either pgm/epgm transports?
>
>
>
> Thanks,
>
> Antonio Montero.
>
> *From:* zeromq-dev [mailto:zeromq-dev-boun...@lists.zeromq.org] *On
> Behalf Of *Steven McCoy
> *Sent:* Friday, March 23, 2018 9:55 AM
> *To:* ZeroMQ development list
> *Subject:* [External] Re: [zeromq-dev] A PGM/EPGM question
>
>
>
> You should check the PUB socket has a loop that is processing the incoming
> NAK requests, this is usually recv call based.  The symptoms indicate that
> the protocol is operating TX-only.
>
>
>
> —
>
> Steve-o
>
>
>
> On Wed, Mar 21, 2018 at 19:50 Montero, Antonio UTC CCS <
> antonio.mont...@fs.utc.com> wrote:
>
> Hello,
>
> I am having a bit of a hard time getting a ZMQ PUB socket reacting to PGM
> NAKs which means at this point I am not able to recover lost packets
>
> I have tried with both protocols: (pgm and epgm). Still getting the same
> result.
>
>
>
> I have a setup where I create both a PUB and SUB sockets in that order in
> the same ZMQ context running on the same host and connected to the same
> IPv6 multicast address and port.
>
> I have N nodes and each node has a PUB and SUB. All N nodes send messages
> asynchronously and all N nodes receive all messages. My multicast network
> is working fine whether I use pgm or epgm and all N nodes communicate with
> each other over IPv6 multicast.
>
> The issue I am having is when a packet loss occurs, a remote SUB sends a
> unicast NAK back to the source PUB however I am not seeing any NCF or RDATA
> being sent by the source PUB. I have verified that the packets in question
> are in fact still in the Tx Window as reported by the SPMs being sent by
> the source PUB. I have ongoing traffic on a periodic basis which triggers a
> send and receive respectably on the PUB and SUB sockets and I am clearing
> out the ZMQ_EVENTS after every send and/or receive. I also have a polling
> thread running every 150ms to check for ZMQ_EVENTS on both PUB and SUB.
>
>
>
> Nothing seems to work in terms of triggering the PUB to react and process
> the NAKs received from remote SUB. Looking at the code a bit I see this
> function zmq::pgm_socket_t::process_upstream but
>
> can’t tell if and how it is being triggered. It does not appear to be from
> my perspective.
>
>
>
> Any help or direction would be appreciated. Thanks.
>
>
>
> --
>
> Antonio
>
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.zeromq.org_mailman_listinfo_zeromq-2Ddev=DwMFaQ=ilBQI1lupc9Y65XwNblLtw=KE-_zI6ApNLT6qvQ1tC8zssu327OLN9lWlhIigWhJA4=AYMnN2d160L4oOUMYUzTsb0enU6l7vTnRPY_52rLMy0=LcJtAkEY4h2bzvGKaxr7OMpdGRbSSgLTF12pJkc7N70=>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] A PGM/EPGM question

[Steven McCoy]

You should check the PUB socket has a loop that is processing the incoming
NAK requests, this is usually recv call based.  The symptoms indicate that
the protocol is operating TX-only.

—
Steve-o

On Wed, Mar 21, 2018 at 19:50 Montero, Antonio UTC CCS <
antonio.mont...@fs.utc.com> wrote:

> Hello,
>
> I am having a bit of a hard time getting a ZMQ PUB socket reacting to PGM
> NAKs which means at this point I am not able to recover lost packets
>
> I have tried with both protocols: (pgm and epgm). Still getting the same
> result.
>
>
>
> I have a setup where I create both a PUB and SUB sockets in that order in
> the same ZMQ context running on the same host and connected to the same
> IPv6 multicast address and port.
>
> I have N nodes and each node has a PUB and SUB. All N nodes send messages
> asynchronously and all N nodes receive all messages. My multicast network
> is working fine whether I use pgm or epgm and all N nodes communicate with
> each other over IPv6 multicast.
>
> The issue I am having is when a packet loss occurs, a remote SUB sends a
> unicast NAK back to the source PUB however I am not seeing any NCF or RDATA
> being sent by the source PUB. I have verified that the packets in question
> are in fact still in the Tx Window as reported by the SPMs being sent by
> the source PUB. I have ongoing traffic on a periodic basis which triggers a
> send and receive respectably on the PUB and SUB sockets and I am clearing
> out the ZMQ_EVENTS after every send and/or receive. I also have a polling
> thread running every 150ms to check for ZMQ_EVENTS on both PUB and SUB.
>
>
>
> Nothing seems to work in terms of triggering the PUB to react and process
> the NAKs received from remote SUB. Looking at the code a bit I see this
> function zmq::pgm_socket_t::process_upstream but
>
> can’t tell if and how it is being triggered. It does not appear to be from
> my perspective.
>
>
>
> Any help or direction would be appreciated. Thanks.
>
>
>
> --
>
> Antonio
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] AF_UNIX on Windows 10

[Steven McCoy]

I guess time to update the build scripts,


> *Unix style sockets (AF_UNIX) are available on Windows*AF_UNIX is an
> address family which has enabled inter-process communication on Unix for
> countless years. The windows equivalent is named pipes, which offers
> similar facilities. Based on user feedback, we heard that you’d like an
> easier path when porting tools relying on AF_UNIX to Windows. As a result,
> two Win32 processes can now use the AF_UNIX address family over Winsock API
> to communicate with each other.


https://blogs.msdn.microsoft.com/commandline/2017/12/19/af_unix-comes-to-windows/

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] openpgm

[Steven McCoy]

Flipping the direction and failing suggests that one of the hosts may have
not correctly determined the local network interfaces.  You can always
explicitly set the local interfaces to bind to when creating the socket.
Later versions in GitHub have corrected some known interface issues with
newer RHEL releases and special configurations.  However because of lack of
testing no new official release has been pushed out yet.

-- 
Steve-o

On 11 April 2017 at 12:07, Marlborough, Rick 
wrote:

> Designation: Non-Export Controlled Content
>
> Folks;
>
> We are attempting to use openpgm with zero mq epgm
> transport. We have a multi node system here. It looks like the send and
> receive mechanism will work in one direction. We have a test sender on node
> 1  and a test receiver on node 2. The messages flow and are received by the
> receiver. If we flip the scenario around and run the sender on node 2 and
> the receiver on node 1, there is no message flow. The same scenario is also
> a problem for pgm. tcp transport works fine in both directions.
>
>
>
> We are using ZeroMQ 4.2.2 and OpenPGM 5.0.0.122 on Red Hat
> 7 64 bit.
>
>
>
> Thanx
>
> Rick
>
>
>
> *From:* zeromq-dev [mailto:zeromq-dev-boun...@lists.zeromq.org] *On
> Behalf Of *Doron Somech
> *Sent:* Tuesday, April 11, 2017 11:34 AM
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] openpgm
>
>
>
> Try...
>
>
>
> On Apr 11, 2017 17:54, "Marlborough, Rick" 
> wrote:
>
> Designation: Non-Export Controlled Content
>
> Folks;
>
> Having an issue with openpgm as used through Zero MQ. Is
> this the correct forum to ask about this?
>
>
>
> Thanx
>
> Rick
>
> 3.1.1001
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> 
>
> 3.1.1001
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] FYI: RSSL/Java now open source

[Steven McCoy]

Reuters proprietary protocol suite with TCP and a couple of multicast
flavours has been opened up fro a while now, albeit Java-only so far:

https://github.com/thomsonreuters/Elektron-SDK/tree/master/Java/Eta/Source/impl/com/thomsonreuters/upa/transport

This may be an interesting project for integrating as new protocol inside
ZeroMQ or at least adding another set of UDP flavours.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] On hooking memory allocations

[Steven McCoy]

On 2 December 2016 at 04:50, Auer, Jens  wrote:

> thanks for the suggestion. I will definitely gie it a try. I tried
> tcmalloc and jemalloc once, but the improvement was not so big. If I
> understood tcmalloc correctly they use a per-thread cache for allocations.
> I didn't think of gc.
>

These are redundant on FreeBSD where jemalloc is already the default
allocator and Linux where ptmalloc3 also implements a per-thread cache
(SLAB).  On Windows you can use the Concurrency CRT API for similar
features.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] BDFL literally

[Steven McCoy]

Is zero.mq owned by someone else?

On 6 October 2016 at 17:39, Ewen McNeill  wrote:

> On 6/10/16 20:18, Trevor Bernard wrote:
>
>> Don't worry about sponsorship. My company will pay to renew them for
>> the next while (within reason). Just send me the details and I'll pay
>> the the bill.
>>
>
> Replying to the list, since a couple of people have asked.
>
> I think Doron would much appreciate someone paying for the ZeroMQ related
> domains (first list below) for a while.
>
> For the others, it's probably only worth paying for them if (a) they point
> at something useful that is difficult to find another way (or linked to a
> lot) or (b) it'd be problematic if a domain squatter got hold of the name.
> Pieter actively took most of these off auto-renew, expecting them to expire
> -- I talked with him about it earlier in the year, and he clearly intended
> to let many of them expire.  So it doesn't seem helpful to renew the ones
> that Pieter planned to let go without a good reason.
>
>
> ZeroMQ-related Domains (now held by Doron; I'm sure he'd welcome someone
> paying their renewal fees -- around EUR 12-13/domain/year I think):
>
> curvezmq.org
> filemq.org
> zeromq.com
> zeromq.net
> zeromq.org
> zmtp.org
> zyre.com
> zyre.org
>
>
> Book-related domains (not sure if all of these got set up/used, but
> clearly reserved for books Pieter was working on; dates are the current
> expiry date):
>
> codeconnected.org (2016-12-14)
> cultureandempire.com (2017-08-05)
> psychopathcode.com (2017-09-28)
> thepsychopathcode.com (2017-09-28)
> scalablec.org (2017-01-08)
> scalable-c.com (2017-01-31)
> scalablec.com (2017-01-31)
>
> (only cultureandempire.com is marked to auto-renew, and I'm not sure if
> that was intended or just overlooked when turning off auto-renew; the ones
> that point at something mostly seem to point at the relevant gitbook page.)
>
>
> Personal:
>
> hintjens.com (2017-10-24)
>
> (Transferred to a family member earlier this year, on the condition that
> hintjens.com itself stayed pointing at Pieter's blog and they kept paying
> to renew it; I'm sure they'd be pleasantly surprised if someone were to pay
> to renew it for a couple more years.)
>
>
> Software projects (past/present/future):
>
> changeflow.com (2017-03-07)
> openamq.org (2017-08-21)
> restms.org (2020-10-30)
>
> (as noted in my earlier message someone just paid to renew restms.org for
> several years -- thanks; the other two are old iMatix projects, and AFAICT
> the changeflow.com one doesn't point at anything)
>
>
> Activism related:
>
> digistan.org (2017-09-24)
> eupaco.org (2017-10-25)
>
> (These two are on auto-renew from the remaining prepaid balance of
> Pieter's account, but might be good candidates for transfer if there was
> someone clearly community supported to take custody of them as an archive)
>
>
> Other (unknown/not pointing at anything):
>
> xpoc.org (2016-10-21)
> formiq.io (2017-01-07)
> swsi.info (2016-10-10)
> smooth.af (2017-02-01; seems to be a Rick Roll, not sure why)
> freeandopenwar.com (2017-02-14)
> ipocracy.org (2016-11-16; redirect to hintjens.com)
> smoothscript.com (2017-02-15)
> brusselsvalley.com (2017-02-28)
>
> I think all of those "other" ones might as well expire unless someone
> recognises an in-progress project they were supposed to go with.
>
> Plus three "iMatix" domains which I or one of the ex-iMatix developers
> will pay for (as noted in my earlier message the main one, imatix.com,
> just got renewed for several years by kind donation).  And two "Xitami"
> (early iMatix webserver) related ones which might as well just expire in a
> couple of years -- Xitami hasn't been an active project in about 10 years.
>
>
> Ewen
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZeroMQ Reliable Multicast just halts and never continues

[Steven McCoy]

It is probably worth checking whether NAK sending and processing is
actually working.  Ethernet networks are pretty reliable these days and the
outage may be linked with the only packet loss occurring.

-- 
Steve-o

On 15 June 2016 at 10:23, Paul Krauss  wrote:

> Hi Folks – I am using 0MQ and CLRZMQ wrapper for C# on Windows 7.
>
> I have a client app running on a Windows box that receives reliable
> multicast from 6 different server processes.
>
> The server processes are all sending reliable multicast using 0MQ. Each
> server uses different ports and groups.
>
>
>
> About once a week, one of my clients will just stop receiving the reliable
> multicast from ALL sources and it will not receive data again until we
> restart the client app.
>
>
>
> I know the data is being sent from the servers correctly because all of my
> other clients receive all the data from the servers during the same
> time-frame.
>
>
>
> It’s very frustrating and I’m not sure what causes it or what to do.
>
> Any ideas, or thoughts?
>
> Thank you - Paul
> This e-mail message (together with any attached documents) is strictly
> confidential and intended solely for the addressee (including the
> addressee's employing organization, assigns and affiliates). It may contain
> information that is proprietary to Geneva Trading USA LLC, its affiliates
> and assigns. It is controlled by law, or is covered by legal, professional
> or other privilege. If you are not the intended addressee nor associated
> with the intended addressee’s organization, you must not use, disclose or
> copy this transmission, and are asked to notify the sender of its receipt.
> Please be further advised that the unauthorized interception or retrieval
> of e-mail may be a criminal violation of the Electronic Communications
> Privacy Act.
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Question about PGM Pub/Sub

[Steven McCoy]

On 11 April 2016 at 05:42, Luedicke, Jens  wrote:

>
>
> The sockets are connected to “epgm://eth1;239.255.255.255.0:”.
>
>
>
> If I start the processes on different systems in my LAN, everything works
> fine. However,
>
> once I set a packet loss on one network device, there a quite a few NAKs,
> but no repair process
>
> in started and eventually the ping/pong process stops.
>
>
>

This typically implies the sender may not be listening on the correct IP
address, I would suggest replacing "eth1" with the IP address you wish to
use.

I have embarked on a new campaign to improve the feedback the API provides
when setting up connectivity as this continues to be an ongoing common
problem and entirely non-obvious on new platforms like OS X El Capitan.
Today there are multiple levels of logging but the interesting pieces
require compile time configuration.

Enable trace logging to see which interfaces are in use with the
environment variable PGM_MIN_LOG_LEVEL=TRACE

...
Trace: Binding receive socket to INADDR_ANY
Trace: Join multicast group 239.192.0.1 on interface index 4
Trace: Multicast send interface set to 10.208.0.98 index 4
...

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] select and Multiple Provider Restrictions on Windows

[Steven McCoy]

On 27 January 2016 at 09:07, Ilya Kulakov  wrote:

> According to
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms739177(v=vs.85).aspx
> ,
> implementation of POSIX's select on Windows restricts fd_set to contain
> descriptors from only one provider.
> This recently became a problem with my VMCI implementation, because VMCI
> sockets are of a different family
> (I assume provider = family in Window's terminology).
>
>
Provider is more like the technology stack, i.e. TCP/IP, IPX, Bluetooth,
ATM, etc.  The loopback adapter is using vanilla IP and thus resides in the
same provider as other physical and virtual adapters.

https://msdn.microsoft.com/en-us/library/windows/desktop/ms737522(v=vs.85).aspx

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Potential bug in select

[Steven McCoy]

On 27 January 2016 at 10:15, Ilya Kulakov  wrote:

> Hi,
>
> I found the following 2 chunks of code suspecting:
>
> - https://github.com/zeromq/libzmq/blob/master/src/select.cpp#L110-L113
> - https://github.com/zeromq/libzmq/blob/master/src/select.cpp#L170-L173
>
> Why do we need to clear descriptos, if readfds/writefds/exceptfds are
> initialized with contents of source_set_* at each iteration of the loop?
> It may lead to an error if set is modified _while_ there is an active
> select in another thread (I don't know if an instance of select_t can be
> used from multiple threads).
> Other than that this code looks pointless.
>
> What do you think?
>

Standard fair is that one event can cause the API to remove the socket from
the set before one iteration of the loop has finished.

https://github.com/zeromq/libzmq/blob/master/src/select.cpp#L203-L212

Of note there is actually a double check with L209-210.

If you can guarantee the file descriptor is moved to retired_fd then you
can remove the removal from the descriptor set.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] random missed messages with epgm pub/sub

[Steven McCoy]

On 12 January 2016 at 08:27, Bart van Wissen  wrote:

> Our observations:
> - For random topics, random messages are not received by the subscriber
>
>
This is an inherit problem if the senders are short lived, the only
solution is to add a broker that can maintain a longer lifetime to ensure
delivery.  If you add brokers you may not need multicast at all.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] PGM and sending interface

[Steven McCoy]

On 6 January 2016 at 04:47, Jim Hague  wrote:

> I've not read RFC3678 before, but after a quick look it seems to me that
> the
> RFC isn't concerned with the socket binding, and hence the source address
> used.
>
>
Yes, the source of a multicast is specified usually by the multicast
address and interface, however with IP aliasing that interface
specification is effectively broken.  Note that an address is insufficient
as IPv6 local link addresses are scoped to the interface, thus it must be
interface index + address.


> As a minimal disturbance to the existing API I'm trying adding an alias
> index
> to t pgm_interface_req_t. That in turn means that the alias index must be
> available from pgm_getaddrinfo(), so I'm changing the group_source_req
> items
> in pgm_addrinfo_t to a pgm_group_source_req which adds the alias index. I
> guess an alternative would be to change pgm_bind3() (or add a pgm_bind4())
> where recv and send would be specified via an address. I guess that would
> mean
> adding receive and send addresses to the result from pgm_getaddrinfo().
>
>
The direction appears to take the address from pgm_getaddrinfo() and place
into a new member variable of pgm_interface_req_t:

struct pgm_interface_req_t {
  uint32_t ir_interface; struct sockaddr_storage ir_address;  uint32_t
ir_scope_id;
}; Although now with the address the explicit scope identifier is not
required, thus it simply becomes: struct pgm_interface_req_t {
  uint32_t ir_interface; struct sockaddr_storage ir_address;}; There should
be some form of precedence for this, I usually refer to mDNSResponder code
for platform peculiarities. Using the alias index is going to be subject to
race conditions when interfaces change and I'd rather try to avoid that. --
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] PGM and sending interface

[Steven McCoy]

On 5 January 2016 at 09:41, Jim Hague  wrote:

> I've read the network parameters docs, and I don't think it helps in my
> case.
> ZMQ pgm_socket.cpp calls pgm_getaddrinfo() to parse the transport, and then
> uses the interface number when setting up the call to pgm_bind3(). Inside
> pgm_bind3(), pgm_if_indextoaddr() is used to get the address used in the
> call
> to bind(). The socket is therefore always bound to the first IP address
> associated with the interface, and packets sent out have that address as
> their
> origin. And I need the packet origin address to be the second IP address.
> Am I
> missing something?
>

I think I remember this is due to IPv6 addressing requiring a bind on the
local link instead of the global link for addressing.  I'm thus not sure
how one would distinguish between IPv6 and AIX interface logic.  What
happens with IPv6 on AIX?  With AIX Linux compatibility how do the
APIs getifaddrs()
and  if_indextoaddr()  work?  See the code comment:

/* interfaces indexes refer to the link layer, we want to find the internet
layer address.
 * the big problem is that multiple IPv6 addresses can be bound to one link
- called scopes.
 * we can just pick the first scope and let IP routing handle the rest.
 */

https://github.com/steve-o/openpgm/blob/master/openpgm/pgm/indextoaddr.c#L34

So outside of the routing table how can one resolve the interface correctly
here?

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] PGM and sending interface

[Steven McCoy]

On 5 January 2016 at 13:34, Jim Hague  wrote:

> I think AIX is a bit of a red herring here. I can produce exactly the same
> scenario on Linux using iproute2


Interesting, thus this invalidates most of RFC 3678:

/* section 5.1 of RFC 3678: basic (delta-based) protocol-independent
multicast
 * source filter APIs.
 *
 * required for OSX 10.6 and earlier.
 */
struct group_req
{
  uint32_t gr_interface; /* interface index */
  struct sockaddr_storage gr_group; /* group address */
};

And it looks like it pretty much breaks group_source_req and group_filter
too.   Unless you really don't care about multicast coming from any of the
addresses on the interface which appears a little foolish?

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] PGM and sending interface

[Steven McCoy]

On 22 December 2015 at 05:31, Jim Hague  wrote:

> Unfortunately, I have some configurations that have more than one IP
> address
> assigned to an interface. To be clear, I don't mean a Linux aliased
> interface
> like eth0:1. I mean
>
> $ ip addr
> ...
> 2: eth0:  mtu 1500 qdisc pfifo_fast state
> UP
> group default qlen 1000
> link/ether 00:30:67:9f:51:49 brd ff:ff:ff:ff:ff:ff
> inet 192.168.1.2/24 brd 192.168.1.255 scope global dynamic eth0
>valid_lft 33011sec preferred_lft 33011sec
> inet 10.169.10.2/24 scope global eth0
>valid_lft forever preferred_lft forever
>
>
I would recommend using a network name (/etc/networks) or network address
if a unique adapter name is not available.  This page introduces the
different means available in OpenPGM:

https://code.google.com/p/openpgm/wiki/OpenPgmConceptsTransport#Network_Parameters



> The OpenPGM page at code.google.com seems closed for business. Where does
> OpenPGM reside these days? As you may have inferred, I have some OpenPGM
> patches to add AIX support too...
>

Google has closed Google Code, but you can still use the interface for the
docs.  The repository has moved over to GitHub, there was a link on the
Google site but it appears to have vanished:

https://github.com/steve-o/openpgm

I think it has the latest patch set from contributions, a lot of minor bug
fixes just like this.

 --
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] No rate limitation using JZMQ

[Steven McCoy]

On 18 December 2015 at 12:44, Benjamin Errouane  wrote:

> A new parameter was added in PGM that can set the individual limits for
>> Original Data (ODATA) and Repair Data (RDATA) packets thus reserving
>> bandwidth for repairs and ancillary data.
>
>
> Are these parameters somehow accessible in JZMQ? I would have thought the
> setRate-function would use these limits in some way. But if I'm
> understanding your statements correctly the throttling is achieved in the
> zmq-Framework rather than in the OpenPGM-framework, right?
>

ZeroMQ has a small subset of the socket options OpenPGM makes available.
This is driven by demand, thus the C++ API and the Java API would both have
to be modified.



> The general recommendation however for ZeroMQ users has been to implement
>> a level of application rate limiting before feeding into ZeroMQ, however it
>> can be coarse limiting as the underlying framework implements fine grained
>> throttling.
>
>
> I thought about limiting the rate at application level but this is not
> really the preferred solution, because the network should be adaptable to
> changes in the network topology. What kind of throttling does Zmq
> implement? Right now I'm not experiencing any throttling, but this may be
> due to the application currently pushing messages at maximum rate.
>

There is currently a fixed leaky bucket in OpenPGM, there is nothing
fundamentally stopping anyone changing this to a token bucket or to an
adaptive scheme.


One more question regarding the currently implemented rate limitation. Do I
> understand it correctly that the rate is limited for a single buffer, so if
> I send one huge message, the rate is limited. But the rate is not limited
> between different messages, so sending a lot of tiny messages is not
> rate-limited the same way?


The rate limit is for everything sent on the wire including protocol
messages.

Generally you will find better throughput using less large packets,
jumbograms preferred, especially on Windows platforms.  However the library
has really only been designed at low latency high packet rate scenarios.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Is there any retransmission mechanism in ZMQ?

[Steven McCoy]

On 20 November 2015 at 01:33, Noh, Jangho  wrote:

> I expected subscribers send NAK and publishers send NCF and RDATA.
> However, I could not find those packets using wireshark.
>
If you do not see a NAK on the wire it is possible the network
specification are insufficient for the library to select the correct
network adapter, please try specifying the NIC as well as the multicast
address, e.g. 192.168.1.1;239.192.0.1.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZMQ, multicast and MTU

[Steven McCoy]

On 20 November 2015 at 16:27, Jim Hague  wrote:

> I guess I'd need to set the PGM socket option PGM_MTU to achieve this, but
> that's not exposed in ZMQ. Indeed, it seems to be set in ZMQ, but it's
> nailed
> to 1500 (pgm_max_tpdu).
>
> I'm thinking of creating a patch to add a ZMQ socket option to set PGM
> TPDU,
> but before I do, am I missing something obvious? How do people normally
> handle
> this scenario?
>

PGM options have been exposed on demand, thus quite a few are missing in
ZeroMQ.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] why PGM is not a complete multicast protocol?

[Steven McCoy]

On 4 November 2015 at 18:53, Noh, Jangho  wrote:

> Question#1.
>
> According to Juniper Networks, PGM is not a complete multicast protocol.
> Why is that?
>
> How PGM makes distribution path???
>
>
>
> PGM adds reliability to multicast traffic streams. *It is not a complete
> multicast protocol* like the Distance Vector Routing Multicast Protocol
> (DVMRP) or Protocol Independent Multicast (PIM). Adding PGM to a router
> does not enable the router to perform multicast functions. Instead, a PGM
> router with multicast capabilities and a preconfigured multicast protocol
> such as PIM can offer more reliable multicast services to PGM sources and
> receivers. PGM is not an alternative to multicast routing protocols, but an
> enhancement of the multicast capabilities already present and configured on
> the router.
>
>
> http://www.juniper.net/documentation/en_US/junos13.2/topics/topic-map/mcast-pgm.html
>
>
This refers to protocols that assist a routing element do its main job of
routing.  PGM functionality in a router improves the overall performance of
the protocol.



> Question#2.
>
> How PGM receivers handle out of order packets?? (related to sliding window)
>
> Suppose a PGM receiver expects to get packet#1 but it received packet#3.
> And later on, the receiver got packet#1 and packet#2.
>
> Is there any specified document for this scenario?
>
>
>
PGM receivers have a receive window that performs packet re-ordering and
initiates delayed backchannel recovery procedures.  Generally switched
networks re-order upto two packets and the PGM protocol by default delays
sending negative acknowledgements upstream till after two packets have been
missed.

The PGM protocol document is the best source of information:
https://tools.ietf.org/html/rfc3208

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZeroMQ License

[Steven McCoy]

On 15 October 2015 at 16:57, Peter Ritter  wrote:

> I got a quick question about the ZeroMQ license. I was under the
> impression that ZeroMQ is distributed under the LGPL license (GNU Lesser
> General Public License). However, I just used the Windows installer (64Bit)
> and it pretty much looked like I had to agree to the GPL (GNU General
> Public License). I ran the installer twice just to be sure. I didn't see
> anything about LGPL. Maybe I'm imagining things, but I thought I'd ask
> nonetheless.
>
>
Distributing under the LGPL also requires providing the license document
for GPL:

http://grokbase.com/t/zeromq/zeromq-dev/112ftr1kqz/licensing-question

The Windows installer simply presents both GPL and LGPL license files
together in order to follow the requirements.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Request for comments on Windows packing

[Steven McCoy]

I'm looking to grab a Windows 10 KN VM and start building new Windows
packages again.  Interested in feedback on what to support and how.  In
looking at 64-bit supported Windows that could limit the platforms to MSVC
2012/v110, MSVC 2013/v120, and MSVC 2015/v140.  I don't see much point for
32-bit or Windows XP anymore outside of paid support.

There does not appear to be any alternatives for the Boost library naming
schema?

The CMake configuration has taken many patches that have severely hindered
multi-compiler package creation.  The options I see are maintaining
separate environments for end-user single-build CMake and another for
multiple-build CMake, alternatively I have been playing with WiX and can
look at creating packaging with that, with cleaner x64 support than the
Nullsoft/NSIS various forks.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Problems with ZeroMQ 4.1.2 and multicast transports on Windows

[Steven McCoy]

On 23 July 2015 at 15:23, Andrew Novikov as.a...@gmail.com wrote:

 If ZMQ_RATE of the multicast socket is lower than about 8 megabits per
 second, packets are sent to the network at the specified rate. But if
 the rate is set higher than that number, messages begin to get queued
 and newer transmitted faster than about 9-10 megabits per second. In
 fact, it seems that congestions start to appear and the average data
 transmission rate drops even lower than 8 megabits per second.
 Moreover, if I set the number of I/O threads in the ZMQ context to a
 value higher than 1, strange transmission gaps begin to appear: the
 zmqsend may stop transmitting data for several dozen seconds and then
 suddenly resume only to stop transmitting soon (messages continue to
 be generated and queued during the gaps).


Windows itself is not helping here, ZeroMQ and PGM is presently tuned for
larger numbers of smaller messages.  To work on Windows one needs to use
smaller quantities (significantly less than 10,000 per second) and larger
sizes, preferably at jumbogram sizes with packet fragmentation.  The cause
is that the WinSock interface has just too much overhead compared with the
performance of the stack itself.  Pushing too high rate into the stack also
causes problems due to inefficiencies of buffer management.  The stack
itself is incredibly good and can better Linux on the same hardware so it
is quite a complicated situation.

Ideal packet sizes for throughput are 64-128KB, I'll have comparable
configurations for other reliable multicast stacks.

So it is a research project and may require modification to the ZeroMQ
interface to open up options to make it easier to work with.  However also
note that pushing high data rates and
no-congestion-control-trivial-network-saturation protocols are not a good
mix.

For interesting alternative projects which I keep following please check
out these:

http://uftp-multicast.sourceforge.net/

https://github.com/real-logic/Aeron

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] pgm over ipv6

[Steven McCoy]

On 2 June 2015 at 19:04, Sergey Zinov s.zi...@emw.hs-anhalt.de wrote:

 it seems that pgm/epgm transport over ipv6 does not work as supposed.
 When a loss(even single) occurs, subscriber pauses for several seconds,
 then resumes skipping the lost packets.


This implies recovery is not functioning possibly due to using the wrong
interface and scope.  Turn on PGM debugging for verbose logging when the
socket is created to capture what it is trying to do.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Casting bug in atomic_counter.hpp - probably only on non x86 or ARM CPUs.

[Steven McCoy]

On 2 June 2015 at 07:51, Dr. David Kirkby (Kirkby Microwave Ltd) 
drkir...@kirkbymicrowave.co.uk wrote:

 atomic_counter.hpp:87:71: error: invalid conversion from 'volatile
 integer_t* {aka volatile unsigned int*}' to 'uint32_t* {aka unsigned
 int*}' [-fpermissive]

 This is line 87 which the error refers to.

 integer_t new_value = atomic_add_32_nv (value, increment_);


This is correct for Oracle Studio,
https://docs.oracle.com/cd/E19253-01/816-5180/6mbbf01oj/index.html

This looks like a defective GCC, I don't think Oracle GCC 3 has these
problems.

You can try using the GCC intrinsics in preference for this particular
compiler version.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Preferred Transport to Handle Lossy Networks?

[Steven McCoy]

On 1 June 2015 at 12:33, Peter Krey k...@ripple.com wrote:

 Yes use the PGM or EPGM transport. If you want to use an additional layer
 at the application level, you can implement your own resend logic with
 message sequence numbers if your network is truely massively losing
 messages.


I think on wireless PGM is not a great idea unless you can implement a
congestion control method that works, with tweaks for radio interference
patterns.  PGM is 1-to-many, for 1-to-1 there are a plethora of options, my
first choice excepted it not being implemented in 0MQ would be QUIC from
Google, for long links it would be UDT.  For real lossy environments you
may need UDP-Lite and possibly a one-way broadcast channel.

I'm the developer of the PGM implementation, the project is currently on
hiatus until I have a new use case for it.  Presently I only have need to
1-to-1 channels.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Website For Windows Installers (miru.hk) Is Down

[Steven McCoy]

Oops, an nginx misconfiguration for IPv6 apologies if you are using that.

Also, IPv4 it requires HTTP 1.1 virtual hosts support, i.e. stick
97.107.136.71 miru.hk in the hosts file and operate as normal.
Alternatively use a command line tool such as cURL and bypass DNS like this:

curl -O http://miru.hk/archive/ZeroMQ-4.0.4~miru1.0-x86.exe --resolve
97.107.136.71

--
Steve-o
On 26 March 2015 at 12:49, Peter Rosario peter.j.rosa...@rrd.com wrote:

 Could you give me the IP address of the server?  With that I can bypass
 the miru.hk DNS name entirely.

 I tried

 http://64.22.103.234/archive/ZeroMQ-4.0.4~miru1.0-x86.exe

 and

 http://97.107.136.71/archive/ZeroMQ-4.0.4~miru1.0-x86.exe

 Neither one worked.

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Website For Windows Installers (miru.hk) Is Down

[Steven McCoy]

Oh ok, you have a DNS server that is ignoring the upstream TTL and
presenting stale values (64.22.103.234).

;; ANSWER SECTION:

miru.hk.300 IN  A   97.107.136.71

-- 
Steve-o

On 26 March 2015 at 12:21, Peter Rosario peter.j.rosa...@rrd.com wrote:

 I just checked it and it is still down.  This link doesn't work:

 http://miru.hk/archive/ZeroMQ-4.0.4~miru1.0-x86.exe

 On Thu, Mar 26, 2015 at 8:40 AM, Steven McCoy steven.mc...@miru.hk
 wrote:

 The site was under maintenance due to moving data centers, it is back up
 now but the latest version available is 4.0.4

 --
 Steve-o

 On 25 March 2015 at 18:16, Peter Rosario peter.j.rosa...@rrd.com wrote:

 Website for windows installers (miru.hk) is down.  I am evaluating
 ZeroMQ to see if I should use it in my product.

 TIA

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Website For Windows Installers (miru.hk) Is Down

[Steven McCoy]

The site was under maintenance due to moving data centers, it is back up
now but the latest version available is 4.0.4

-- 
Steve-o

On 25 March 2015 at 18:16, Peter Rosario peter.j.rosa...@rrd.com wrote:

 Website for windows installers (miru.hk) is down.  I am evaluating ZeroMQ
 to see if I should use it in my product.

 TIA

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Website For Windows Installers (miru.hk) Is Down

[Steven McCoy]

It is 97.107.136.71 for IPv4 and 2600:3c03:0:0:f03c:91ff:fe89:7488 for
IPv6, however your understanding of HTTP is limited to version 1.0 of the
protocol for which each site must have a unique IP address.  HTTP 1.1
introduced the concept of name-based virtual hosting
http://en.wikipedia.org/wiki/Virtual_hosting so that multiple sites can
be hosted on a single IP address and enable the vast swath of shared
hosting services now available on the Internet.  A HTTP 1.1 request looks
like this ( for the URL http://www.example.com/ ):

GET / HTTP/1.1
Host: www.example.com


Compared with the previous version which only provided the path:

GET / HTTP/1.0


Thus it is necessary to provide a hostname in the request otherwise you may
receive a response for a different website.  In the case of
http://97.107.136.71/ you will see the stock files for the nginx webserver
http://en.wikipedia.org/wiki/Nginx.

-- 
Steve-o

On 26 March 2015 at 16:04, Peter Rosario peter.j.rosa...@rrd.com wrote:

 I asked you to give me the IP address of the server.  You haven't told me
 what it is.  Am I to infer that it is 97.107.136.71?

 Also, it is not necessary to modify the hosts file or use the --resolve
 switch.  I merely have to replace the host name with the host IP address.
 That is what I have done and it doesn't work.  Have you clicked on the link

 http://97.107.136.71/archive/ZeroMQ-4.0.4~miru1.0-x86.exe

 to see for yourself?


 On Thu, Mar 26, 2015 at 12:36 PM, Steven McCoy steven.mc...@miru.hk
 wrote:

 Oops, an nginx misconfiguration for IPv6 apologies if you are using that.

 Also, IPv4 it requires HTTP 1.1 virtual hosts support, i.e. stick
 97.107.136.71 miru.hk in the hosts file and operate as normal.
 Alternatively use a command line tool such as cURL and bypass DNS like this:

 curl -O http://miru.hk/archive/ZeroMQ-4.0.4~miru1.0-x86.exe --resolve
 97.107.136.71

 --
 Steve-o
 On 26 March 2015 at 12:49, Peter Rosario peter.j.rosa...@rrd.com wrote:

 Could you give me the IP address of the server?  With that I can bypass
 the miru.hk DNS name entirely.

 I tried

 http://64.22.103.234/archive/ZeroMQ-4.0.4~miru1.0-x86.exe

 and

 http://97.107.136.71/archive/ZeroMQ-4.0.4~miru1.0-x86.exe

 Neither one worked.

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Website For Windows Installers (miru.hk) Is Down

[Steven McCoy]

Can you visit the page http://miru.hk/zeromq/ ?  You may be really unlucky
and your anti-virus is blocking something.

:(

-- 
Steve-o

On 26 March 2015 at 16:34, Peter Rosario peter.j.rosa...@rrd.com wrote:

 I stand corrected.  You are right (I haven't done virtual hosting in a
 long while)​.

 I ran the following command and got the following results:

 $ nslookup miru.hk

 Server: 208.67.222.222

 Address: 208.67.222.222#53


 Non-authoritative answer:

 Name: miru.hk

 Address: 97.107.136.71

 I don't think my issue is DNS related.

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] set ZMQ_RATE not working with OpenPGM?

[Steven McCoy]

On 7 January 2015 at 04:05, Meng Zhuo mengzhuo1...@gmail.com wrote:

 After I set rate with 31415, log shows zeromq had accepted it.
 but the OpenPGM still shows Setting ODATA rate regulation to 12500 bytes
 per second.


Try setting the Rate before issuing the Connect() on the socket.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Building ZeroMQ on Windows

[Steven McCoy]

On 7 December 2014 at 15:08, Kirby Ross kirby.ros...@gmail.com wrote:

 ZeroMQ-2.20~miru1.0-win32
 czmq-2.2.0
 cppzmq-master

 When I bring up the czmq project in VS 2008 and try and build it. I
 continue to get this error:
 fatal error C1083: Cannot open include file: 'zmq.h': No such file or
 directory


The problem is that Visual Studio does not know where the ZeroMQ headers
and subsequently libraries are located, you will have to update the project
file to point to the installed locations.  The ZeroMQ library name in the
installer package also changes per version, compiler, and architecture.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Aeron messaging

[Steven McCoy]

On 19 November 2014 04:58, Pieter Hintjens p...@imatix.com wrote:

 Not sure about this. It looks like we can do ZeroMQ better, and yet
 they stay away from mentioning ZeroMQ as a competitor (as did Martin
 in his Strange Loop talk). I mean, how can you say open source high
 performance messaging and then use AMQP and JMS as your comparison
 points?


It is mentioned in the blog and other places, such that Aeron can sit
side-by-side with PGM underneath ZeroMQ.  There are a lot of links at the
end of the HighScalability.com article, Todd Montgomery chimes in a lot
with the references.

But I do agree with their design that the core architectural piece is a
reliable log, but I am perplexed how Aeron supports congestion control but
they are only going to implement back pressure.  The major problem with PGM
is lack of a congestion control mechanism that scales above 10k/s, the
reliability can certainly be improved alongside the memory usage with the
log design from Aeron but I am still left wanting more.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Aeron messaging

[Steven McCoy]

From a developer from 29 West (LBM et al) and LMAX (Disruptor) previously
creating Simple Binary Encoding (SBE) message encoding we now have the
Aeron transport.  I think they just need to add someone from Lockless Inc.

 *
http://highscalability.com/blog/2014/11/17/aeron-do-we-really-need-another-messaging-system.html

 * https://github.com/real-logic/Aeron

Not sure how it shapes up compared with QUIC, I presume better suited to
smaller messages.

I wonder if it would be better to have three independent threads running a
non-super optimized transport than throw three at a single super speedy one.

Looking forward to testing the C++ multicast implementation \:D/

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] PyZMQ with Libsodium on Windows?

[Steven McCoy]

I think it is easier now as they have more support than just MSVC2013 (C99
compat) when crypto was added to ZeroMQ.

On 9 October 2014 17:47, MinRK benjami...@gmail.com wrote:

 It's not bundled simply because I couldn't build it on Windows. If you can
 come up with a simple fix for building bundled libsodium, then I would
 bundle libsodium on Windows.

 -MinRK

 On Thu, Oct 9, 2014 at 1:48 PM, André Caron andre.l.ca...@gmail.com
 wrote:

 Hi there!

 I'm trying to secure some PyZMQ communications with the curve security
 features.  The code seems straightforward, but when I call
 `zmq.auth.create_certificates()` to generate some keys, I get the following
 exception:

 zmq.error.ZMQError: Not supported

 This seems to be due to the absence of libsodium.

 In the release notes for PyZMQ 14.1.0, I can see that libsodium is not
 bundled with PyZMQ on Windows.

 First, I'm curious to know why it's not bundled on Windows.  If it's
 simply a matter of finding a volunteer, maybe I can pitch in?

 Second, I'd like to know if there's any known procedure to get it running
 on Windows?

 Thanks!

 André

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] libzmq v4.0.5

[Steven McCoy]

Fair warning I am not in a position to push 4.0.5 Windows packages when the
release is official.  I'm currently on an OSX macbook without any Windows
or MSVC toolsets \:D/

-- 
Steve-o

On 21 September 2014 18:13, Pieter Hintjens p...@imatix.com wrote:

 Hi all,

 This is a pre-release announcement, to give time for backporting. If
 there are any fixes on libzmq master that you'd like to see in 4.0.5,
 please let us know. For info, this is what's currently in that
 release:

 * Fixed #1191; CURVE mechanism does not verify short term nonces.
 * Fixed #1190; stream_engine is vulnerable to downgrade attacks.
 * Fixed #1088; assertion failure for WSAENOTSOCK on Windows.
 * Fixed #1015; race condition while connecting inproc sockets.
 * Fixed #994; bump so library number to 4.0.0
 * Fixed #939, assertion failed: !more (fq.cpp:99) after many ZAP requests.
 * Fixed #872; lost first part of message over inproc://.
 * Fixed #797, keep-alive on Windows.

 -Pieter
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Trouble compiling czmq under Visual Studio

[Steven McCoy]

On 12 September 2014 11:12, Tom Quarendon tom.quaren...@teamwpc.co.uk
wrote:

  Since Visual Studio doesn’t support C99, the C source files are compiled
 as C++. This appears to have thrown up a type issue.



MSVC 2013 supports most of C99, its not so important any more.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Send and receive using link-local IPv6 address using zmq 3.2.3 and czmq 1.4.1

[Steven McCoy]

On 10 September 2014 01:49, Gerrit Hendrikus van Doorn 
g.h.vando...@gmail.com wrote:

  And the same address?  Those are not the most helpful examples to
 provide.
  Do the non-CZMQ versions work?  Do you have ip6tables running?

 I mentioned that I run the applications on the same machine (to simplify
 the setup). I get the same results with ip6tables enabled or disabled.
 What would be a better example?

 eth0  Link encap:Ethernet  HWaddr A0:D3:C1:04:9F:60
   inet addr:134.63.132.24  Bcast:134.63.133.255  Mask:255.255.254.0
   inet6 addr: fe80::a2d3:c1ff:fe04:9f60/64 Scope:Link

 So, running them on the *same* machine with:
 ./zmq_server tcp://[::1]:54321
 ./zmq_client tcp://[fe80::a2d3:c1ff:fe04:9f60%%eth0]:54321

 Results in the server binding correctly to [::1]:54321 but the client
 never receiving anything (block on recv). When I run the client using:
 ./zmq_client tcp://[::1]:54321
 I do receive data. However, I don't want to use the loopback address.


This is how IP networking works, you are specifying completely different
networks and they correctly do not communicate with each other.  Are you
looking for the IPv6 wild card address? [::].

  ./zmq_server tcp://[::]:1234

  ./zmq_client tcp://[::1]:1234
  ./zmq_client tcp://[fe80::a2d3:c1ff:fe04:9f60%%eth0]:1234


-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Send and receive using link-local IPv6 address using zmq 3.2.3 and czmq 1.4.1

[Steven McCoy]

On 9 September 2014 18:51, Gerrit Hendrikus van Doorn 
g.h.vando...@gmail.com wrote:

 Correction: I do use the same ports when testing (so both 5 or 54321)


And the same address?  Those are not the most helpful examples to provide.
 Do the non-CZMQ versions work?  Do you have ip6tables running?

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Lock-free algorithms

[Steven McCoy]

On 4 September 2014 06:32, Michi Henning mi...@triodia.com wrote:

 https://code.google.com/p/thread-sanitizer/issues/detail?id=69

 In the context of lock-free techniques in zmq, one of the commenters
 mentioned:

  Just a drive-by comment: TSan should not produce false positives on
 lock-free algorithms, if they use atomic operations correctly. If they
 don't then these algorithms are broken.


Are not such statements based around usage of new standard constructs for
atomic operations in C++11 and C99?  Hence the note about bespoke TSan
instrumentation later in the article.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] How to use the libzmq binaries for Windows?

[Steven McCoy]

The libraries should match the CRT version they are built and linked
against.  As mentioned previously Microsoft are attempting to stop
releasing further incompatible CRTs from MSVC2013 onwards.

-- 
Steve-o


On 1 August 2014 09:49, Full Name knockoutu...@myway.com wrote:

 Hello everyone,

 Have this question related to the Windows distribution of ZeroMQ.

 In the http://zeromq.org/distro:microsoft-windows I see tables of libzmq*
 binaries listed. What are they for?

 I'm trying to build the Java Binding for ZeroMQ for Win7 x64 and following
 this guide: http://zeromq.org/bindings:java. The Windows 64bit Build
 Instructions works up to Linking CXX shared library lib\jzmq.dll where
 it complains about not finding libzmq.lib.

 I'm using the Windows install ZeroMQ-3.2.4~miru1.0-x64.exe with WinSDK 7.1
 (I'm compiling from command line since I don't have Visual Studio) and saw
 those files in the install page. So I renamed libzmq-v100-mt-3_2_4.lib to
 libzmq.lib, tried nmake again and it worked. I have a dll (21KB) and a jar
 (43KB) as output.

 I went with a hunch on that one and happened to work. Is that correct? Is
 that what the listed libzmq* binaries are for or?

 This is posted on the the project github page also:
 https://github.com/zeromq/libzmq/issues/1128

 Regards,
 koo

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] How to use the libzmq binaries for Windows?

[Steven McCoy]

Most of the additional libraries that build above ZeroMQ are using
libraries built from the MSVC project or solutions files and not the
prepackaged binaries.  Windows has particular requirements for CRT
compatibility which changes with every compiler release, in addition to
regular version and architecture tags.  It has recently announced that the
CRT is supposed to stabilize around CRT 2013.

One ZeroMQ user is performing CRT magic to make the libraries compatible
with multiple compilers, you will have to check the archives for further
details.

-- 
Steve-o


On 14 July 2014 08:37, Full Name knockoutu...@myway.com wrote:

 Hello everyone,

 Have this question related to the Windows distribution of ZeroMQ.

 In the http://zeromq.org/distro:microsoft-windows I see tables of libzmq*
 binaries listed. What are they for?

 I'm trying to build the Java Binding for ZeroMQ for Win7 x64 and following
 this guide: http://zeromq.org/bindings:java. The Windows 64bit Build
 Instructions works up to Linking CXX shared library lib\jzmq.dll where
 it complains about not finding libzmq.lib.

 I'm using the Windows install ZeroMQ-3.2.4~miru1.0-x64.exe with WinSDK 7.1
 (I'm compiling from command line since I don't have Visual Studio) and saw
 those files in the install page. So I renamed libzmq-v100-mt-3_2_4.lib to
 libzmq.lib, tried nmake again and it worked. I have a dll (21KB) and a jar
 (43KB) as output.

 I went with a hunch on that one and happened to work. Is that correct? Is
 that what the listed libzmq* binaries are for or?

 This is posted on the the project github page also:
 https://github.com/zeromq/libzmq/issues/1128

 Regards,
 koo
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] FlatBuffers from Google

[Steven McCoy]

Competition with SBE,


http://google-opensource.blogspot.com/2014/06/flatbuffers-memory-efficient.html

A comparison of these new zero-copy formats:


https://kentonv.github.io/capnproto/news/2014-06-17-capnproto-flatbuffers-sbe.html

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Dissecting Message Queues by Tyler Treat

[Steven McCoy]

I don't think I saw this posted yet: noted on highscalability.com today,

http://www.bravenewgeek.com/dissecting-message-queues/

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] JZMQ stable/production ready build

[Steven McCoy]

On 26 June 2014 05:54, Benjamin Cordes benjamin.l.cor...@gmail.com wrote:

 With regards to multicast: is PGM planned for JeroMQ? Would this use the
 OpenPGM Java implementation? OpenPGM points to
 https://github.com/steve-o/javapgm , which was last updated a year ago.


It's a wish list item.  Quite a few people have asked, no one has stepped
up to complete the work.  I'm out of resources for developing it further
for a while, it has been more of a toy project to learn Java and Maven as I
have no target use case for it yet.  If one comes along I would finish it
off.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] libzmq ABI compatibility across msvc versions

[Steven McCoy]

On 13 May 2014 04:10, Daniel Krikun dkrik...@gmail.com wrote:

 There are binary installers for windows (
 http://zeromq.org/distro:microsoft-windows) which are very convenient to
 use. The installer for legacy version 2.2.0  only contains binaries built
 with vc100 toolkit.

If you download the source package you can build your own library with the
provided project and solution files.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq performance over epgm

[Steven McCoy]

On 9 May 2014 07:37, space3...@mail.bg wrote:

 I am trying to send 50 bytes of data via multicast using epgm protocol
 and the zmq framework. It takes around 45 seconds for the whole data to
 arrive at the receiver. HWM is set to 6500, ZMQ_RATE is 100 - for
 some reason with higher rate more time is needed. Send and receive BUF are
 2000. When the data is being sent 100% of cpu is used by the
 publisher. If I am interpreting the data given by Wireshark right, there is
 no dropping and not too many retransmits.

Surprising but here are the general points:

   - TCP is accelerated and UDP is not thus there are significant CPU
   overheads in transmission from segmentation to checksumming.
   - Higher bandwidth rates can be counterproductive when the underlying
   system cannot achieve a stable throughput at that rate, this occurs
   predominantly with Windows.
   - The CPU may end up spinning with insufficiently stable timers but high
   resolution PGM parameters.

Generally multicast is preferred for larger scale out whilst TCP is pretty
good these days for moderate usage.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] FYI: Simple Binary Encoding

[Steven McCoy]

Move over Captain Proto and Protocol Buffers, at 16-25 times faster
throughput:

http://mechanical-sympathy.blogspot.com/2014/05/simple-binary-encoding.html

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Martin Fowler MicroServices

[Steven McCoy]

On 23 April 2014 03:33, techbird techb...@birdsoft.co.uk wrote:

  http://martinfowler.com/articles/microservices.html

 In this excellent publication Martin Fowler references 0MQ as a possible
 light weight bus to communicate events/messages between micro-services.


This is pretty much the standard TIB model from TIBCO since the 90's.
 Treat software like a hardware bus and then you can plug and play
services, upgrade, failover at will.  It makes testing and monitoring
services incredibly nice and easy.

That is a long article.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Strange problem with Openpgm. Help us!

[Steven McCoy]

On 18 April 2014 10:25, Michel Pelletier pelletier.mic...@gmail.com wrote:

 I'm afraid I can't help you with you specific pgm problem, but if you
 don't mind me playing devil's advocate for a second, it seems like you're
 doing a lot of engineering work to distribute a file to 20 servers.  Have
 you considered using an existing multicast tool like uftp or udpcast to
 distribute the file?


Yes, I see this a lot.  Yes multicast is ideal for fast file distribution
but congestion control and reliability are not a given.  One of the first
PGM implementations was created and predominantly used for wide file
distribution, but they conveniently gave a file transfer protocol above
that and everything was designed for satellite style high bandwidth, high
latency, low packet rates.  These days peer-to-peer distribution above TCP
overlay networks is significantly cheaper to deploy and only costs
additional latency through multi-hop traversal.

OpenPGM was created to be flexible but only applied to high packet rate,
low latency applications, and ZeroMQ has incorporated this model.  There is
a congestion control protocol taken from an earlier SmartPGM implementation
but it has not aged well at all and so it is disabled by default and not
accessible at all through the ZeroMQ interface.  NORM would be a better
choice of protocol here, if only because of it being stable and proven with
additional features, but at the cost of some level of performance.  This is
the challenge, no stable scalable high performance congestion protocol has
been invented yet suitable for 10GigE+ multicast.

The new link for UFTP is here:

http://uftp-multicast.sourceforge.net/

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Yahoo shows big performance improvement by switching Storm from ZeroMQ to Netty

[Steven McCoy]

On 7 April 2014 04:10, Asquith, Edward edward.asqu...@baesystems.comwrote:

  Obviously Netty supports a narrower set of use cases, but being pure
 Java this certainly caught my attention; better performance and a
 simplified deployment model is a compelling combo.



I think JeroMQ would have done the same and more, JNI is really slow.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_connect invalid argument specifying nic interface

[Steven McCoy]

On 19 March 2014 13:26, Tim Chen che...@gmail.com wrote:


 I'm getting an invalid argument when I try using pgm w zmq 3.2.4 when my
 URL looks like:

 epgm://bond0;239.1.2.3:45678

 If I remove bond0 it works fine.
 Any idea why or how I can get more information on the error?


Ideally zmq_strerror http://api.zeromq.org/3-2:zmq-strerror would report
this but you may need to enable PGM trace
roleshttps://code.google.com/p/openpgm/wiki/OpenPgm5CReferenceErrorHandlingto
see something more helpful, e.g.

PGM_MIN_LOG_LEVEL=trace ./my-program

There are some odd errors in the latest Linux kernels returning unexpected
error codes when performing name lookups, these have only been resolved in
trunk.  The workaround is to specify the adapter by IP address to to add an
alias in /etc/networks.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] encoder::has_data() method deprecated?

[Steven McCoy]

On 14 March 2014 18:43, Brian Adamson brian.adam...@nrl.navy.mil wrote:

 Also, I have a short overview/write-up of what I've done with ZeroMQ/NORM
 that I can post to the list if there's interest?


 I'd like to see this.  I'd like to see what other features of NORM you can
fit in and their value.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Increasing transparently ZMQ performance up to 2400%

[Steven McCoy]

The obvious question is what is the relationship between Torus UFS,
Solarflare OpenOnLoad, and Mellanox VMA?


On 28 February 2014 12:37, Guillermo Lopez Taboada 
guillermo.lopez.tabo...@gmail.com wrote:


 Hi Pieter,

 Our sockets have intra-server latencies (RTT/2 from process to process) as
 low as 230 nanoseconds and inter-servers as low as 1.1 microsecond. This
 makes any socket-based communication-intensive run very very fast. The
 response time and the scalability can improve significantly so we can
 alleviate virtually any communication bottleneck.

 BTW, some people suggested we should include IPC in our tests. We have not
 included IPC as the performance compared to TCP loopback is quite similar,
 but, if you do not agree with that and see benefits of using IPC, then
 congratulations, UFS is for you!

 Results of a test. For local_lat  remote_lat with 128 bytes we have seen

 TPC loopback:  average latency: 14.295 [us]

 IPC: average latency: 12.628 [us]

 UFS smdev: average latency: 4.570 [us]

 Regards,

 Guillermo


 On Thu, Feb 27, 2014 at 3:21 PM, Pieter Hintjens p...@imatix.com wrote:

 Guillermo,

 It's a very nice technical demo. Are there other problems such as the
 infamous 10 million connections that you also solve?

 -Pieter

 On Thu, Feb 27, 2014 at 2:14 AM, Guillermo Lopez Taboada
 guillermo.lopez.tabo...@gmail.com wrote:
 
  Dear all,
 
  Just let me complete my last message with regards to some questions that
  came out of the list:
 
  1) We are accelerating ZeroMQ by using a high performance socket
 library in
  user land, UFS, instead of relying on the OS for TCP/IP. UFS is not a
  replacement of ZeroMQ, is an add-on, a plug-in for any socket-based
  application. The way it works is as a preloadable library.
 
  2) The usage is pretty simple, running a performance test on a server:
 
  # ZeroMQ over TCP loopback:
  $  ./local_lat tcp://127.0.0.1: 1 10  
  $  ./remote_lat tcp://127.0.0.1: 1 10
  message size: 1 [B]
  roundtrip count: 10
  average latency: 14.041 [us]
 
 
  # ZeroMQ over UFS (application transparent):
  $ LD_LIBRARY_PATH=$UFS_HOME/lib64 LD_PRELOAD=$UFS_HOME/lib64/libufs.so
  ./local_lat tcp://127.0.0.1: 1 10 
  $ LD_LIBRARY_PATH=$UFS_HOME/lib64 LD_PRELOAD=$UFS_HOME/lib64/libufs.so
  ./remote_lat tcp://127.0.0.1: 1 10
  message size: 1 [B]
  roundtrip count: 10
  average latency: 4.830 [us]
 
  3) Free demos of the software are available by request at
 i...@torusware.com
 
  Best regards,
 
  Guillermo
 
 
  On Tue, Feb 25, 2014 at 11:06 PM, Guillermo Lopez Taboada
  guillermo.lopez.tabo...@gmail.com wrote:
 
 
  Dear all,
 
  Up to 24x performance increase is the outcome of our latest PoC for a
  major investment bank.
 
  The details are here:
  http://torusware.com/increase-zeromq-performance-by-up-to-2400/
 
  Let us know if you want to put your servers at full speed.
 
 
  --
  Best regards,
 
  Guillermo
  ---
  Guillermo Lopez Taboada
  Torus Software Solutions, CEO
  Email:  guillermo.lo...@torusware.com
  Phone: +34-657-662-998
  WWW: http://www.torusware.com
 
  DISCLAIMER:
  This message is intended exclusively for its addressee and may contain
  information that is CONFIDENTIAL and protected by professional
 privilege. If
  you are not the intended recipient you are hereby notified that any
  dissemination, copy or disclosure of this communication is strictly
  prohibited by law. If this message has been received in error, please
  immediately notify us via e-mail and delete it. Thank you.
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev




 --
 Best regards,

 Guillermo
 ---
 Guillermo Lopez Taboada
 Torus Software Solutions, CEO
 Email:  guillermo.lo...@torusware.com
 Phone: +34-657-662-998
 WWW: http://www.torusware.com


 *DISCLAIMER:This message is intended exclusively for its addressee and may
 contain information that is CONFIDENTIAL and protected by professional
 privilege. If you are not the intended recipient you are hereby notified
 that any dissemination, copy or disclosure of this communication is
 strictly prohibited by law. If this message has been received in error,
 please immediately notify us via e-mail and delete it. Thank you.*

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Sandstorm: 55 Gbps at 72% load

[Steven McCoy]

This might be of interest, if rather hilarious as an implementation of
anything.  Basically a static content webserver that stores content as
a tcpdump
and just replays the packets bypassing any serious CPU effort.

http://highscalability.com/blog/2014/2/12/paper-network-stack-specialization-for-performance.html

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] 0MQ on unicast UDP

[Steven McCoy]

On 29 January 2014 09:59, Конюхов Павел kos...@forsys.ru wrote:

 Can I adapt pgm/epgm to unicast UDP or perhaps there is some other way?


The quest has been started here:
https://github.com/zeromq/libzmq/issues/807

PGM is theoretically agnostic to multicast channel and thus unicast but it
really is not so convenient to other protocols with explicit point-to-point
support (such as TRDP).

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] [zmqpp] Indentation style?

[Steven McCoy]

OS X and Linux uses the same compilers: GCC and Clang, but different
linkers.


On 24 January 2014 15:08, Lindley French lindl...@gmail.com wrote:

 In fact, OSX seems to require a much different build. -soname doesn't seem
 to be supported, -f isn't a valid flag to ar, etc.


 On Fri, Jan 24, 2014 at 3:04 PM, Lindley French lindl...@gmail.comwrote:

 Update: I don't know why it seemed to be working before, but I've
 confirmed that OSX and linux handle variadic templates the same way. The
 key is there can't be any conversion if you want the non-template overload
 to be called. Therefore I'm solving the problem by adding reinterpret_casts
 to the operators.

 In the process of investigating this, I've discovered that OSX does not
 support the -rdynamic gcc option (or the equivalent -Wl,-export-dynamic).
 The zmqpp Makefile will need to be updated to avoid passing this option on
 OSX.


 On Fri, Jan 24, 2014 at 1:57 PM, Lindley French lindl...@gmail.comwrote:

 What is the defined indentation style for zmqpp? I see a lot of it uses
 tabs, but a few places it uses four spaces. I want to make sure things stay
 consistent. My editor is configured for 4 space indents; I prefer that
 since it tends to give a more uniform look across editors.

 Also, my variadic template pull request was tested on linux, but now
 what I try to build on OSX I'm getting compile errors. I think it's because
 the variadic add() function is being preferred to the (void*,int) overload
 used by the insertion operators for some reason. I'm not sure why this
 would be different between platforms.




 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Best 0MQ usable with gcc-2.95?

[Steven McCoy]

On 14 January 2014 12:21, Dmitry Antipov anti...@dev.rtsoft.ru wrote:

 I need to develop 0MQ-based application for an old (RedHat 7.x-based)
 system with gcc-2.95. Unfortunately upgrading is practically impossible.


Do you have to build on the target system?  Use static linking of libc 
libstdc++ from a host with a newer GCC or Clang.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Linking of libzmq (git HEAD) fails under Mac OS X 10.9 Mavericks, ld: unknown option: --version-script=libzmq.vers

[Steven McCoy]

I'm not seeing a version number actually being used in this use case, would
it not be more pertinent to just use --export-symbols which would be OSX
friendly?

https://www.gnu.org/software/gnulib/manual/html_node/LD-Version-Scripts.html

Or just using the attributes flags together with Windows based DLL imports
and exports:

https://www.gnu.org/software/gnulib/manual/html_node/Exported-Symbols-of-Shared-Libraries.html#Exported-Symbols-of-Shared-Libraries



On 13 January 2014 12:33, Bruno D. Rodrigues bruno.rodrig...@litux.orgwrote:

 https://github.com/zeromq/libzmq/pull/821


 On Jan 13, 2014, at 16:53, Mathias Hablützel h...@zhaw.ch wrote:

  Hi everyone,
 
  Hit this little problem:
 
  $ make all
  Making all in src
  /Applications/Xcode.app/Contents/Developer/usr/bin/make  all-am
   CXX  libzmq_la-address.lo
   CXX  libzmq_la-clock.lo
 
  [snip]
 
   CXX  libzmq_la-tipc_listener.lo
   CXX  libzmq_la-tipc_connecter.lo
   CXXLDlibzmq.la
  ld: unknown option: --version-script=libzmq.vers
  clang: error: linker command failed with exit code 1 (use -v to see
 invocation)
  make[2]: *** [libzmq.la] Error 1
  make[1]: *** [all] Error 2
  make: *** [all-recursive] Error 1
 
  Then I added in the Makefile manually the -v switch for LDFLAGS and ran
  it again:
 
  $ make all
  Making all in src
  /Applications/Xcode.app/Contents/Developer/usr/bin/make  all-am
   CXXLDlibzmq.la
  ld: unknown option: --version-script=libzmq.vers
  clang: error: linker command failed with exit code 1 (use -v to see
 invocation)
  make[2]: *** [libzmq.la] Error 1
  make[1]: *** [all] Error 2
  make: *** [all-recursive] Error 1
 
  So I double checked if my linker is kind of outdated:
 
  $
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld
 -v
  @(#)PROGRAM:ld  PROJECT:ld64-224.1
  configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64
 armv6m armv7m armv7em
  LTO support using: LLVM version 3.3svn, from Apple Clang 5.0 (build
 500.2.79)
  $ ld -v
  @(#)PROGRAM:ld  PROJECT:ld64-136
  configured to support archs: i386 x86_64 armv7 armv7s
  LTO support using: LLVM version 3.3
 
  Since I doubt that between a less new and a newer patch-release a
  option is lost I can only assume that I overlooked something else. And
  yes, I was able to build libzmq beforehand (IIRC it was the 4.0.1
  version).
 
  Any pointer where to search further?
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] vs110 via msbuild vs120 x64 a no go

[Steven McCoy]

In case anyone tries this, a recent update from Microsoft has some how
blocked this combination and causes a new unique error message:

C:\Program Files
(x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.Cpp.Platform.targets(58,5):
error MSB8032: The Platform or PlatformToolset is not available from a
64bit environment.  Consider building from 32bit environment instead.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Building on Windows (32 and 64 bit)

[Steven McCoy]

On 8 January 2014 10:43, Robin Newton robin.new...@linguamatics.com wrote:

 The easy thing to do is to use the Windows installers
 (http://zeromq.org/distro:microsoft-windows) to get the various files I
 need onto a machine, and then copy them. However:-

 (a) The latest installers are for release 4.0.1, rather than for 4.0.3.


4.0.3 is up now, along with all the supported compiler versions.

I'm looking at libsodium 0.4.5 now as support has been added for older
compilers together with pre-built binaries for more platforms.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] msvc12 project with libsodium

[Steven McCoy]

On 3 January 2014 16:10, Balázs Varga bb.va...@gmail.com wrote:

 Since there's no libzmq with libsodium in the windows installer
 (v4.0.1).


I stopped at 4.0.0.rc1:

http://miru.hk/archive/ZeroMQ-4.0.0rc1~nacl~miru1.0-x86.exehttp://miru.hk/archive/ZeroMQ-4.0.0rc1~miru1.0-x86.exe
http://miru.hk/archive/ZeroMQ-4.0.0rc1~nacl~miru1.0-x64.exehttp://miru.hk/archive/ZeroMQ-4.0.0rc1~miru1.0-x64.exe


It's only possible to target MSVC2013 and was rather tedious at that.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] When unreliability is desired

[Steven McCoy]

On 17 December 2013 17:30, Lindley French lindl...@gmail.com wrote:

 It would be really neat if we could integrate something like
 http://feclib.sourceforge.net/documentation.html into this. Not sure how
 that possibility would work license-wise, though.


Google's QUIC includes XOR FEC as their studies have shown little
requirement for full on Reed Solomon encoding.

OpenPGM includes Reed Solomon FEC as studies have shown it dramatically
improves performance for large number of receivers.  However Microsoft went
ahead and implemented an incompatible scheme to everyone else so nothing
works out-of-the-box.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] DSCP/Diffserv

[Steven McCoy]

On 24 November 2013 06:20, Pieter Hintjens p...@imatix.com wrote:

 It doesn't, but it's easy enough to add, presumably.


For Unix land it is, note that none of this applies to Vista+ which has a
completely separate and rather unwieldy set of APIs which they of course
rewrote completely for IPv6.  Thus any code using QoS should handle failure
on Windows.

/* IP_TOS only works on WinSock2 with system override, listed support
Windows 2000-only:
 * http://support.microsoft.com/kb/248611
 * Recommended APIs: GQoS (IPv4 only), qWAVE QOS (Vista+)

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] When unreliability is desired

[Steven McCoy]

On 7 December 2013 00:34, lindl...@gmail.com wrote:

 I'll take a look at tipc. I've been using the pgm classes as a reference
 so far. They appear to use v1_encoder while tcp uses v2_encoder.


If you can get a nice clean framework setup independent of the TCP/Unix
socket setup I think many others and myself included would like to
experiment adding many different types of protocol to 0mq.  I have many on
my wish list.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] brain foggy -- need help

[Steven McCoy]

On 29 October 2013 20:34, Andrew Hume and...@research.att.com wrote:

 the race condition is, of course, that even when sending significant
 amounts (MB) of data,
 the sync message is received and returned before any data socket msgs are
 processed.


Send an inline flag message in the push socket and have the client wait on
a pull message for status from server.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] error: ISO C++ 1998 does not support 'long long'

[Steven McCoy]

On 23 October 2013 05:23, Peter Kleiweg pklei...@xs4all.nl wrote:

 ... sodium/randombytes.h:29: error: ISO C++ 1998 does not support
 'long long'


It is valid C++11, it looks like libsodium doesn't care too much about the
older standards.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] zmq_poll stops expiring timeout in virtualized Windows 7

[Steven McCoy]

On 16 October 2013 06:11, Bjorn Reese
bre...@mail1.stofanet.dkjavascript:_e({}, 'cvml',
'bre...@mail1.stofanet.dk');
 wrote:

 Alternatively, we could upgrade to C++11 and get a portable solution
 with std::chrono::steady_clock

http://en.cppreference.com/w/cpp/chrono/steady_clock


You can actually just look at the implementation for the clock and be
surprised at what function they are using.  Similarly with Boost::chrono, I
filed a ticket with that one but to not much avail for monotonic
correctness.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_poll stops expiring timeout in virtualized Windows 7

[Steven McCoy]

On 16 October 2013 06:11, Bjorn Reese bre...@mail1.stofanet.dk wrote:

 Alternatively, we could upgrade to C++11 and get a portable solution
  with std::chrono::steady_clock


Ok, here you go: *a steady clock is not necessarily monotonic*.

https://svn.boost.org/trac/boost/ticket/7719

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_poll stops expiring timeout in virtualized Windows 7

[Steven McCoy]

On 16 October 2013 10:25, Laurent Alebarde l.aleba...@free.fr wrote:

  A simplier approach is to encapsulate the standard clock of your choice
 and save in a static member the value of the last get. Then if the standard
 clock goes back in time, you return the saved value + 1.


For clock drift this is fine, one doesn't even have to bother with the
increment for many cases.  However there can be major problems with NTP,
daylight savings, and suspend/hibernation states.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Automatic generation of project files (CZMQ, CMake, ...)

[Steven McCoy]

On 15 October 2013 11:15, Pieter Hintjens p...@imatix.com wrote:

 For what it's worth I'm moving CZMQ to automatically generate its
 project files from a single minimal data file. We're generating CMake,
 autoconf, MSVC, Android, and MingW32 files.


A make maker maker, I'm sure that is gong to be fun to maintain.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq_poll stops expiring timeout in virtualized Windows 7

[Steven McCoy]

On 15 October 2013 13:48, Felipe Farinon felipe.fari...@powersyslab.comwrote:

 I have found that
 VirtualBox's QueryPerformanceCounter doesn't guarantee monotonicity
 https://www.virtualbox.org/ticket/11951. They argue that even Windows
 QueryPerformanceCounter doesn't guarantee it.


This is true, the performance counters are highly unreliable.  With OpenPGM
I followed the SQL Server teams approach and move towards Windows
multimedia timers they offer greater stability and lower cost.  Copy 
pasting:

/* Multi-media like timer API, returns 100-nanoseconds intervals.
 *
 * Faster than QueryPerformanceCounter and hence deployed for SQL Server.
 *
http://blogs.msdn.com/b/psssql/archive/2008/12/16/how-it-works-sql-server-no-longer-uses-rdtsc-for-timings-in-sql-2008-and-sql-2005-service-pack-3-sp3.aspx
 */
static
pgm_time_t
pgm_mmtime_update (void)
{
FILETIME ft;
int64_t aligned_ft;

GetSystemTimeAsFileTime (ft);
memcpy (aligned_ft, ft, sizeof (int64_t));
return (pgm_time_t)( aligned_ft / 10 );
}

The caveat being you are still affected by NTP adjustments, but this is
available on platform where *GetTickCount64* is not.

GHC Haskell had similar discussions:

http://ghc.haskell.org/trac/ghc/ticket/5865


Chromiums discussion on topic:

https://chromium.googlesource.com/chromium/src/+/acca91d2f89419fe27e275e55fac228713497da4/base/time_win.cc


-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] EPGM network devices support question

[Steven McCoy]

On 10 October 2013 20:46, Daniele Stanzani daniele.stanz...@4ward.itwrote:

  Does EPGM break network devices PGM support?


Yes, for the benefit of convenience and that not many users have PGM
network devices.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Windows installer for 4.x

[Steven McCoy]

On 10 October 2013 04:30, Pieter Hintjens piet...@gmail.com wrote:

 Not yet.


:-)

Maybe Friday, bit busy.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] tcp source port

[Steven McCoy]

On 7 October 2013 13:05, Frank Rehberger frank.rehber...@web.de wrote:

 I googled at bit found this page http://en.wikipedia.org/wiki/URI_scheme

 Transfering the schema from SSH and SFTP to 0MQ-protocol-URI how
 about using something like the following syntax to encode the source
 port into the user-info of the URI,  using the ;  to delimit the
 key-value parameters from each other?

 tcp://sourceport=1200;sourceip=192.168.1.2@host:port/my/path

 or:

 tcp://username;sourceport=1200;sourceip=192.168.*.*@host:port/my/path


Nominally the form would be:

tcp://host:port/?sourcePort=1200sourceIp=192.168.1.2


But the article has (probably deprecated) precedence for

tcp://host:port/?;sourcePort=1200;sourceIp=192.168.1.2


The most interesting is a mix of SFTP and MPEG-TS UDP streaming:

tcp://sourceaddress@host:port/


Which one may think would be valid to extend thus:

tcp://sourceaddress:sourceport@host:port/


-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] tcp source port

[Steven McCoy]

On 5 October 2013 05:44, Sébastien Rautureau sebrautur...@gmail.com wrote:

 There may be several options :
 - extend the following tcp uri format by adding an optionnal local port =
 tcp://serveradress:serverport:localport


This is clearly breaking the format, so not valid.


 - add a new socket option


The only practical option for the API, but architecture wise it would be
the non-sensible approach.  Simply configure your firewall or add in
HAProxy (or one of the many alternatives) and fix the ports in that instead.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Why I might want to use JeroMQ if I'm completely fine with JZMQ?

[Steven McCoy]

On 1 October 2013 17:00, Pieter Hintjens p...@imatix.com wrote:

 - JeroMQ does not support pgm:// or epgm:// since OpenPGM has no Java
 version


Technically there is now, just not complete and requires JDK 7.

https://github.com/steve-o/javapgm
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Version 4.0 release fork

[Steven McCoy]

On 25 September 2013 11:15, KIU Shueng Chuan nixch...@gmail.com wrote:

 You could generate the import library from the provided libsodium-4.def
 file per the instructions at http://www.mingw.org/wiki/MSVC_and_MinGW_DLLs
 .


Importantly there is no 64-bit library and I don't want to bring in all the
GCC and MinGW dependencies (libgcc.a libmingwex.a).  I've gone through this
with OpenPGM 2 before and the preferred direction is a native build using
native tools.

I have MSVC2013 all building correctly, all it is now is about how feasible
it is to bring in the older compilers.  MSVC2013 has completely dropped XP
support for example.  Considering the C99  C++11 support that is provided
I'm not so concerned for shrink wrapped deployment.

Developers are free to build their own versions with GCC mixed libraries.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] RE : How to send/receive a custom C++ object via zeromq sockets?

[Steven McCoy]

On 18 September 2013 07:47, Michi Henning mi...@triodia.com wrote:

 I've been working with Cap'n Proto the past two weeks, and I have to say
 I'm impressed.


I've only just stumbled along this, Cap'n Proto is written by the developer
of Google Protobufs 2 after having departed Google:

http://kentonv.github.io/capnproto/index.html

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Version 4.0 release fork

[Steven McCoy]

Bonus:  rc1 plus NaCl for v120 only, curve_keygen.exe uses non-packaged
static libraries surprisingly well (16KB):

http://miru.hk/archive/ZeroMQ-4.0.0rc1~nacl~miru1.0-x86.exehttp://miru.hk/archive/ZeroMQ-4.0.0rc1~miru1.0-x86.exe
http://miru.hk/archive/ZeroMQ-4.0.0rc1~nacl~miru1.0-x64.exehttp://miru.hk/archive/ZeroMQ-4.0.0rc1~miru1.0-x64.exe


Bug fix: curve_keygen.c needs assert.h.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Version 4.0 release fork

[Steven McCoy]

On 23 September 2013 23:34, Steven McCoy steven.mc...@miru.hk wrote:

 09/23/2013  11:26 PM 1,423,600 libsodium-v120-mt-sgd-0_4_3.lib
 09/23/2013  11:26 PM 2,064,212 libsodium-v120-mt-s-0_4_3.lib


This is only building on v120/MSVC2013 due to C99 usage.  There is some new
function that is linking that breaks all the older toolkits.  Does someone
want to start a C89 libsodium fork?

It definitely does not build out of the box, there is DJB's laziness with
floating point literals and various instances of Unix only headers, some
odd C99 incompatibilities in inline functions, but no major code changes
required.

About 5 hours to get v3.2.4 and v4.0.0 ready and built in supported
configurations.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Version 4.0 release fork

[Steven McCoy]

On 24 September 2013 09:36, Pieter Hintjens p...@imatix.com wrote:

 Given that CURVE is still experimental, we can build 4.0.0 without it,
 on Windows. My hope is that using it in libzmq will push progress on
 Window packaging for it, but we don't need to have a critical
 dependency on that. All the CURVE code compiles conditionally, if
 HAVE_LIBSODIUM is not defined.


Ok, I'm thinking of releasing the standard shared/dynamic library packages
free and keeping the static library packages and 2013-only NaCl build for
commercial interests.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Version 4.0 release fork

[Steven McCoy]

Ok, I got this far:

C:\git\libsodium-0.4.3*build-windows.cmd*
Cleaning build area ...
Starting build ...
Building targets for x86 ...
targetting-v120
09/23/2013  11:25 PM 1,376,716 libsodium-v120-mt-sgd-0_4_3.lib
09/23/2013  11:25 PM 1,992,294 libsodium-v120-mt-s-0_4_3.lib
Building targets for x64 ...
targetting-v120
File Not Found
09/23/2013  11:26 PM 1,423,600 libsodium-v120-mt-sgd-0_4_3.lib
09/23/2013  11:26 PM 2,064,212 libsodium-v120-mt-s-0_4_3.lib
Build finished.

C:\git\zeromq-4.0.0*build-windows-installers.cmd*
Cleaning build area ...
Starting build ...
Building targets for x86 ...
targetting-v120
09/23/2013  11:26 PM 1,662,464 libzmq-v120-mt-gd-4_0_0.dll
09/23/2013  11:26 PM12,124 libzmq-v120-mt-gd-4_0_0.lib
09/23/2013  11:27 PM16,012,616 libzmq-v120-mt-sgd-4_0_0.lib
09/23/2013  11:27 PM   481,280 libzmq-v120-mt-4_0_0.dll
09/23/2013  11:27 PM11,946 libzmq-v120-mt-4_0_0.lib
09/23/2013  11:28 PM74,459,760 libzmq-v120-mt-s-4_0_0.lib
File Not Found
Building targets for x64 ...
targetting-v120
File Not Found
09/23/2013  11:29 PM 2,084,864 libzmq-v120-mt-gd-4_0_0.dll
09/23/2013  11:29 PM11,888 libzmq-v120-mt-gd-4_0_0.lib
09/23/2013  11:29 PM21,605,978 libzmq-v120-mt-sgd-4_0_0.lib
09/23/2013  11:31 PM   577,024 libzmq-v120-mt-4_0_0.dll
09/23/2013  11:31 PM11,698 libzmq-v120-mt-4_0_0.lib
09/23/2013  11:31 PM74,656,794 libzmq-v120-mt-s-4_0_0.lib
Build finished.


That's ZeroMQ 4.0.0 + OpenPGM 5.2.122 + libsodium 0.4.3 in dynamic and
static library forms.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] libzmq/3.2.4 stable now available

[Steven McCoy]

New Windows packages posted, now with MSVC2012 U3 and MSVC2013 RC whether
that makes any difference.

Still no idea how to handle the 4.0.0 tree ...
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Version 4.0 release fork

[Steven McCoy]

On 20 September 2013 04:08, Pieter Hintjens p...@imatix.com wrote:

 I've forked off the libzmq master to
 https://github.com/zeromq/zeromq4-x, in preparation for release 4.0.0.


MSVC2008 is broken due to inttypes.h dependency.

I have installers for Windows built sans *libsodium *but mushrooming to
30MB to include static libraries.  Investigating *libsodium*, looks more
tedium as the developers are not conducive to anything other than
*Autoconf* although
there is a *CMake *pull request that was rejected and that may be utilised.

https://github.com/jedisct1/libsodium/pull/74/files

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] C#: recent .NET2.0 compatible versions?

[Steven McCoy]

On 11 September 2013 16:01, Igor 'Lo' (И.L.) bombsiteunres...@gmail.comwrote:

 On one of the project, I **have** to deliver .NET2.0 targeted signed
 assemblies. Is there any recent version of clrzmq this is suitable to this
 requirement? I see the trunk uses Linq extensively, which is making it
 impossible to build on 2.0..


You probably need to start a new fork and use tools discussed here:

*http://stackoverflow.com/q/3462419/175849*
*
*
-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] czmq compilation failure (-Werror)

[Steven McCoy]

On 10 September 2013 16:20, Patrick Noffke patrick.nof...@gmail.com wrote:

 I am attempting to compile czmq with a cross-compiler (gcc 4.8.0), and I
 am getting a bunch of errors like this:

 zbeacon.c: In function 'zbeacon_publish':
 zbeacon.c:176:5: error: ISO C90 forbids mixed declarations and code
 [-Werror=pedantic]
 zmsg_t *msg = zmsg_new ();



As per the error message and this RFC:

http://rfc.zeromq.org/spec:21


Use a C99 compiler for CZMQ.  With GCC this is usually with a flag -std=gnu99
or -std=c99.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Jenkins CI build for windows

[Steven McCoy]

On 2 September 2013 05:01, richard_new...@waters.com wrote:

 Since Travis CI does seem to support windows (though sounds like they will
 do real soon now), I set up a Jenkins CI build for libzmq on windows at *
 http://libzmq.cloudapp.net/* http://libzmq.cloudapp.net/ if anyone is
 interested.


Nice, kind of funny to see the CMake builds being picked up more now in
contrast to the bundled project and solution files.

Toolset support must be added to the libzmq repo and it will cause
collateral damage on a lot of the work already done adding multiple
platforms and compilers.  This is mainly due to CMake's very young support
for it and harsh conflicts with compiler detection.  I would like to see
suggestions for ways forward on this as the maintenance is quite arduous
with multiple people pushing updates in different directions.

Possibly the installer may be tidied up with moving to WiX as native
support is now in CMake, but I haven't done any further research on it to
date.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Storm rejecting ZeroMQ

[Steven McCoy]

On 26 August 2013 11:22, Andy Pook andy.p...@gmail.com wrote:

 Nothing above 2.1.7 works … It just a weird, very unstable piece of
 software. It could have been a really awesome library. But it just has all
 these weird problems with it.


Didn't Zed Shaw also have similar comments, presumably the people behind
Nitro too?

I think a lot of these were resolved and Pieter introduced the new workflow
to ensure things don't break similarly in future.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] nitro

[Steven McCoy]

On 23 August 2013 13:20, Michael Haberler mai...@mah.priv.at wrote:

 http://gonitro.io/


Copy  pasting:

Q: How is this different than ZeroMQ?

Nitro is in many ways a reaction to ZeroMQ (ZMQ), and is heavily inspired
by it (especially czmq). The team at bu.mp (that wrote Nitro) uses ZeroMQ
extensively, including shipping it on tens of millions of smartphones, and
there's a lot to love about ZMQ.
So much, in fact, that the deficiencies of ZMQ we encountered motivated
Nitro--because the basic communication model that ZMQ pioneered is mostly
wonderful to use and develop against.

However ZeroMQ was designed for relatively low-concurrency, very high
throughput (especially multicast PGM) communication on private, trusted,
low-latency networks... not for large scale public Internet services with
high connection counts, fickle clients and wobbly links.

These are some of the design decisions Nitro made that differ from ZeroMQ:


   - Nitro provides more transparency about the socket/queue state (client
   X is connected, queue count is Y) for monitoring reasons and because
   clients quite often never come back in public networks, so state needs to
   be cleared, etc.
   - Nitro does not commit messages to a particular socket at send() time,
   but does send() on a general queue and lets peers work steal stripes of
   frames as soon as they have room on their TCP buffer. This makes for a lot
   more transparency about the true high-water mark for a socket, it
   constrains the total number of messages that may be lost due to a client
   disconnect, and it can minimize mean latency of receipt of any general
   message (vs. naive round robin).
   - Nitro clears private queues for dead peer sockets instead of leaving
   them around indefintely in case they return. This fixes one of the biggest
   issues with doing high-concurrency work in ZeroMQ: an unavoidable memory
   leak in ROUTER sockets when there is pending data for clients who will
   never return.
   - ZeroMQ (esp 2.X) had some more or less hard coded peer limits (1024),
   which is far less than a good C daemon on epoll can handle; nitro has no
   such restrictions.
   - ZeroMQ does not have any crypto story, so we had to roll our own
   awkardly using NaCl. With Nitro, we built NaCl/libsodium in, including a
   key exchange step, so you don't need to ship keys with every frame.
   - ZeroMQ's heritage of private networks has bit us and others with
   things that are assert()s instead of closes-and-logs. On the public
   internet, sometimes people with your socket with a random HTTP request. It
   is also not clear how much attention ZeroMQ has paid to invalid data in the
   form of attacks on inbound TCP streams, integer overflows, etc. Nitro tries
   to be paranoid and it shuts down bad peers instead of crashing.
   - ZeroMQ ROUTER sockets also have some O(n) algorithms, where n is the
   number of connected peers on a socket; nitro is all O(1). This doesn't
   matter much when you have 5 or 10 or 50 big server systems pushing loads to
   each other on a private network, but it sucks when you have 50,000
   mostly-idle clients on high-latency Internet links.
   - In practice we found the typed socket paradigm (REQ/REP/PUSH) more
   of a hindrance than a help. We often ended up with hybrid schemes, like
   REQ/maybe REP, or REQ/multi REP. Also, if you want REQ/REP with
   multiple clients where you do some processing to produce the REP result,
   you'll need to chain together ROUTER/DEALER. REQ/REP stacks and make sure
   you carefully track the address frames. Nitro lets you create any topology
   you want, and the routing is relatively abstracted from the application
   developer--you don't need to worry how deep or shallow you are, for example.
   - We found having the ZMQ's routing information in special MORE frames
   that have implict rules that differ on the basis of socket types (DEALER
   will generate it, REQ will not) cumbersome.
   - ZMQ Socket options have documented rules about when they take effect
   and when not, but these rules are not enforced by the API so they can bite
   you. Nitro separates things that must be decided at construction time from
   those you can modify on the fly (_sub and _unsub, etc).
   - Pub/sub based on message body was limiting for us in practice.
   Oftentimes we wanted a separation of the channel name and the message
   body.
   - ZMQ sockets are not thread safe. So the way to make a multicore
   exploiting RPC service is to chain tcp/ROUTER frontends to an array of
   inproc backends, each running in a separate pthread. This is a layer of
   complexity nitro removes by just having sockets be thread afe.
   - ZMQ_FD is edge triggered. It's much harder to integrate an
   edge-triggered interface into other event loops. Though it has a
   theoretical performance benefit, Nitro uses a level-triggered activity fd
   to make integration easier for 3rd party binding developers.
   - Nitro is written in C. We 

Re: [zeromq-dev] nitro

[Steven McCoy]

On 23 August 2013 14:37, Steven McCoy steven.mc...@miru.hk wrote:


- On very small (40 byte) TCP messages and inproc messages, ZeroMQ is
faster (30-40%) than Nitro; Nitro's use of mutex/cond on socket queues
probably costs it there.

 I'm seeing atypical pthread mutex usage in the code-base, it is trivial to
optimise with an adaptive mutex and further improvements are usually
available with members of the spin lock family.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] ZMQ_FLAT_PULL feature discussion

[Steven McCoy]

On 13 August 2013 11:29, Daniel Krikun krikun.dan...@gmail.com wrote:

 -- it is activated using ZMQ_CONFLATE socket option at the receiver side


I'm not a fan of using the word *conflate *here as it is clearly not
combining any data just dropping.  As an alternative I would propose *unary*
.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] non-destructive queue iterator?

[Steven McCoy]

On 9 August 2013 13:43, Michael Haberler mai...@mah.priv.at wrote:

 it's unclear how I would replicate the following queue usage:

 A queue of pending commands is checked periodically for limit violations;
 however, the queue entries are not consumed


You may be able to simulate functionality with a broker, when you need to
evaluate the inbound pending messages, simply consume every message from
the broker and republish.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] 0MQ, Boost.Asio, and Windows

[Steven McCoy]

On 10 August 2013 16:40, Damien Kick dki...@mac.com wrote:

 Well, the reason that I want to integrate with Boost.Asio is more for
 getting its
 implementation of the reactor pattern for free, including timers, etc.


I don't know if it is the best pattern, but I appreciate the split made by
the Chromium developers: have separate main loops for different purposes:
networking, UI, etc.  Thus you could have a Boost.Asio main loop with timer
events and simply send a timer message to the 0mq main loop.

Its also relatively easy to replicate Windows' thread pool timers with
C++11 Chrono to perform such a function and thus very portable.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] zmq nodejs epgm can't work on mac osx 10.8.3

[Steven McCoy]

On 29 July 2013 10:51, yang tao yangtao.com...@gmail.com wrote:

 apple@appletekiMacBook-Air:/sources/zeromq$ node pub_pgm.js

 epgm://192.168.1.100;239.192.1.1:

 apple@appletekiMacBook-Air:/sources/zeromq$ node sub_pgm.js

 epgm://192.168.1.100;239.192.1.1:



Sender and subscriber must be on separate hosts.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Help: epgm pub/sub does not work

[Steven McCoy]

On 29 July 2013 11:01, Wang, Xin (WorldQuant) xin.w...@worldquant.comwrote:


 int rc = zsocket_connect(subscriber, epgm://10.88.150.227:);

 int rc = zsocket_connect(subscriber, epgm://eth0;10.88.150.227:);***
 *

 **


Try using the IP of the interface, e.g.

 int rc = zsocket_connect(subscriber, epgm://1.2.3.4;10.88.150.227:);

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Unit tests failing on win 8

[Steven McCoy]

On Sunday, July 28, 2013, James Gatannah wrote:

 t may very well fall into the realm of Some tests haven't been ported for
 windows yet or Just use cmake and don't worry about the tests.


Any break is worth investigating if only to raise issues with the toolkit.
 On Windows MSVC is the primary compilation environment, CMake is for
building the release installers.  I'm not aware if any packages have been
made for Cygwin or Mingw, but the platforms are certainly at varying levels
of stability for 64-bit.

Can you try different combinations of architecture and compiler and report
the status?  Mingw in particular funky as you can cross build in Linux and
see different results.

The stretch platform is wine-gcc, but executables from the other toolsets
should run fine there too.

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev