It's still quite hard to see there could be a limit on subscriptions, in systems I work on we have 100k + on single publisher.
I would dig deeper and make sure 1. subscriptions make it through to the publisher, this is where the filtering happens (apart from pgm/mcast transport) 2. Check the subscriptions that arrives on the publisher match the subject you are publishing To do this you can change your publisher socket to ZMQ_XPUB then poll that socket for incoming messages. The first byte will be 1 (subscribe) followed by the subject pattern, On Wed, May 15, 2019 at 10:58 AM Bekritsky, Benjamin < benjamin.bekrit...@zebra.com> wrote: > If I have another process that continually cycles through 0-1024 and > anything greater than 1000 doesn’t come through… the issue is not a simply > missing the original messages, it is that the subscriptions don’t seem to > go through… > > > > *From:* James Harvey <jamesdillonhar...@gmail.com> > *Sent:* Wednesday, May 15, 2019 12:54 PM > *To:* ZeroMQ development list <zeromq-dev@lists.zeromq.org> > *Cc:* Bekritsky, Benjamin <benjamin.bekrit...@zebra.com> > *Subject:* Re: [zeromq-dev] max PUB/SUB topics > > > > [External Email] > > And regarding the c example where you see the issue if you start > publishing before subscribing. > > *" However, if instead of alternating pub/sub, I do the pubs first and > then the subs, I occasionally hit the 1000 limit. " * > > > > That is never going to be reliable as the publisher will throw away the > data if there are no current subscriptions > > > > Search up "late joiner" for possible solutions if that is a feature you > need to support. > > > > On Wed, May 15, 2019 at 10:39 AM James Harvey <jamesdillonhar...@gmail.com> > wrote: > > Hi Benjy, > > > > From your original example you are mixing binary for subscription subjects > and strings for publish subjects. Use the same for both. > > > > The issue is not with the number of subscriptions, its the fact the > subjects you send do not match the subjects you subscribed to. > > > > You std::string insert() is expecting a signed char, not unsigned > > > > incoming_string.insert(0,1,i%256); > > > > after 127 its going to turn negative and the std string is going to fail > to print a unicode character, spitting out unprintable data > > > > James > > > > > > > > On Wed, May 15, 2019 at 9:53 AM Bekritsky, Benjamin via zeromq-dev < > zeromq-dev@lists.zeromq.org> wrote: > > Justin, > > > > I rewrote your code in c and get the same results as you showed for > Python. I’m on Ubuntu 16.04 and using the “latest_release” branch (4.3.1). > > > > https://gist.github.com/bekritsk/c73af9c3080fb976328b053e0c454166 > <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_bekritsk_c73af9c3080fb976328b053e0c454166&d=DwMFaQ&c=Qwsh1H-X9ypOoLLEcAIltQ&r=wDgxT19weImS5vjsvnN8A59V8HMU1jmK8Tt64xQOZwA&m=EeNomsL8Of6KU6kxFDGILaAiG5URWw2vnonkRJA-39c&s=ZRtffhCaa7lqlH3Tc1MHsRtDiu0ISOVKdmY_wnrxQq4&e=> > > > > However, if instead of alternating pub/sub, I do the pubs first and then > the subs, I occasionally hit the 1000 limit. > > > > I tried doing the same thing in Python, but Python seems to work whether > the pub/subs are alternating or all the pubs are first followed by all of > the subs. > > > > Thanks, > > Benjy > > > > *From:* zeromq-dev <zeromq-dev-boun...@lists.zeromq.org> *On Behalf Of *Justin > Karneges > *Sent:* Tuesday, May 14, 2019 6:01 PM > *To:* Jamie Stewart via zeromq-dev <zeromq-dev@lists.zeromq.org> > *Subject:* Re: [zeromq-dev] max PUB/SUB topics > > > > [External Email] > > Works for me with 4.2.5 on Ubuntu 18.04. > > > > https://gist.github.com/jkarneges/0b8f6e6d70c23807de0ad02c8ed72fcb > <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_jkarneges_0b8f6e6d70c23807de0ad02c8ed72fcb&d=DwMFaQ&c=Qwsh1H-X9ypOoLLEcAIltQ&r=wDgxT19weImS5vjsvnN8A59V8HMU1jmK8Tt64xQOZwA&m=if5ZcgfMqCPKYZ553AP14LU19id8HIg6pMiaE2ez4Yw&s=9owZiXJwjq6awqaVthdwfEAbzrKv2cMkcA2YEHTGbZA&e=> > > > > On Tue, May 14, 2019, at 12:59 AM, Bekritsky, Benjamin via zeromq-dev > wrote: > > Justin, > > > > Thanks for the guidance, but I set the HWMs to 0 and neither application > crashed. I also played with changing the HWMs to 2048 and am still > encountering the same issue. > > > > I then subscribed to “all” and received the messages from all the topics > (0-1024); > > > > It seems as though it is limiting the number of subscriptions to 1000. If > I subscribed from 20 -1024, I get topics 20 – 1019. > > > > I am using lib 4.2.3 and the publisher and subscriber on the same Ubuntu > box. > > > > Thanks, > > Benjy > > *From:* zeromq-dev <zeromq-dev-boun...@lists.zeromq.org> *On Behalf Of *Justin > Karneges > > *Sent:* Tuesday, May 14, 2019 7:10 AM > > *To:* Jamie Stewart via zeromq-dev <zeromq-dev@lists.zeromq.org> > > *Subject:* Re: [zeromq-dev] max PUB/SUB topics > > > > [External Email] > > Hi, > > > > Subscribing sends a message to the publisher. I believe you're hitting the > default high water mark, and subscription messages are getting dropped. > > > > If I'm expecting lots of subscriptions, my rule of thumb is to set > SNDHWM=0 on the SUB socket and RCVHWM=0 on the PUB socket. This way the > application will sooner die than lose a subscription message. > > > > Justin > > > > On Mon, May 13, 2019, at 10:06 AM, Bekritsky, Benjamin via zeromq-dev > wrote: > > I am using libzmq and setting up topics using binary data. In other words, > my topic is the first 2 bytes of my message. > > > > I subscribe by setting a and b to the values I’m interested in: > > for (uint16_t i = a; i <= b; i++) > > { > > d = zmq_setsockopt(sub_socket, ZMQ_SUBSCRIBE, &i, 2); > > if (d) > > { > > printf ("E: subscription failed: %s\n", strerror > (errno)); > > return -1; > > } > > } > > > > My publisher publishes like this: > > string incoming_string = " this message"; > > incoming_string.insert(0,1,i>>8); > > incoming_string.insert(0,1,i%256); > > > > int rc = zmq_send(pub_socket, incoming_string.c_str(), > incoming_string.size(), 0); > > > > I can’t seem to subscribe to more than 1000 topics (values 0x0000 – > 0x03e7). From what I understand, thousands of topics should be available. > > > > Any guidance would be helpful. > > > > Thanks, > > Benjy > > > > Benjamin Bekritsky > > c/o Zebra Technologies > > 2 Negev St. (Motorola Building) > > 4th Floor > > Airport City 7019900 > > Israel > > > > > > > ------------------------------ > > > > - CONFIDENTIAL- > > > > This email and any files transmitted with it are confidential, and may > also be legally privileged. If you are not the intended recipient, you may > not review, use, copy, or distribute this message. If you receive this > email in error, please notify the sender immediately by reply email and > then delete this email. > > > ------------------------------ > > > > - CONFIDENTIAL- > > This email and any files transmitted with it are confidential, and may > also be legally privileged. If you are not the intended recipient, you may > not review, use, copy, or distribute this message. If you receive this > email in error, please notify the sender immediately by reply email and > then delete this email. > > _______________________________________________ > > zeromq-dev mailing list > > zeromq-dev@lists.zeromq.org > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.zeromq.org_mailman_listinfo_zeromq-2Ddev&d=DwMFaQ&c=Qwsh1H-X9ypOoLLEcAIltQ&r=wDgxT19weImS5vjsvnN8A59V8HMU1jmK8Tt64xQOZwA&m=if5ZcgfMqCPKYZ553AP14LU19id8HIg6pMiaE2ez4Yw&s=MFBykfTCfPngIL34eSeJ2V9NMDX85SYfQxo28ITv7RM&e=> > > > > > > _______________________________________________ > > zeromq-dev mailing list > > zeromq-dev@lists.zeromq.org > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.zeromq.org_mailman_listinfo_zeromq-2Ddev&d=DwMFaQ&c=Qwsh1H-X9ypOoLLEcAIltQ&r=wDgxT19weImS5vjsvnN8A59V8HMU1jmK8Tt64xQOZwA&m=EeNomsL8Of6KU6kxFDGILaAiG5URWw2vnonkRJA-39c&s=ml270hTRrev__Pw8FDFclDZaU-1DVTGAcU4KzR5bA_c&e=> > > > > > > _______________________________________________ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.zeromq.org_mailman_listinfo_zeromq-2Ddev&d=DwMFaQ&c=Qwsh1H-X9ypOoLLEcAIltQ&r=wDgxT19weImS5vjsvnN8A59V8HMU1jmK8Tt64xQOZwA&m=EeNomsL8Of6KU6kxFDGILaAiG5URWw2vnonkRJA-39c&s=ml270hTRrev__Pw8FDFclDZaU-1DVTGAcU4KzR5bA_c&e=> > >
_______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
