Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-27 Thread Ritesh Adval
Hi Min, This is great, thanks for looking into it. I am looking forward to trying out the fix. -Ritesh Sent from my iPhone. On Jan 27, 2013, at 9:16 AM, Yu Dongmin wrote: > Hi, > > I've found a culprit which caused the data loss. > > When ZMQ send a large message, the stream_engine sends d

Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-27 Thread Yu Dongmin
Hi, I've found a culprit which caused the data loss. When ZMQ send a large message, the stream_engine sends data through multiple out_event calls. The ZMQ linger option only guarantees messages are delivered to the peer pipe. By the speculative write out_event is called at least once but larg

Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-26 Thread Yu Dongmin
Hi, My guess was it might have an issue on libzmq (zeromq c library) when large messages were heavily sent. Thanks Min On Jan 26, 2013, at 4:01 PM, Ritesh Adval wrote: > Hi Min, > > Thanks for the update.Just to confirm, > Are you saying that this issue is on zeromq c library or jzmq c wrap

Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-25 Thread Ritesh Adval
Hi Min, Thanks for the update.Just to confirm, Are you saying that this issue is on zeromq c library or jzmq c wrapper? Just an update that when I replaced DEALER socket which connects to ROUTER socket of broker with REQ socket and replaced DEALER socket which connects to DEALER socket of brok

Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-25 Thread Min
I was able to reproduce the issue on jzmq even on zeromq 3.2.2. What I discovered is about last 30K bytes of 45K message was not sometimes delivered to in-router on raw close. I didn't build equivalent C code, as jzmq is a thin wrapper of native C library it could have the same problem. But I did

Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-23 Thread Ritesh Adval
Hello, I have created a bug for this issue with instructions and java test case. Its at https://zeromq.jira.com/browse/LIBZMQ-497 Thanks Ritesh On Tue, Jan 22, 2013 at 6:30 PM, Ritesh Adval wrote: > Thanks Min, > > I will create a bug with instruction and unit test. I was also > experimenting

Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-22 Thread Ritesh Adval
Thanks Min, I will create a bug with instruction and unit test. I was also experimenting with Java only version of zeromq ( https://github.com/zeromq/jeromq). When running same test it does not drop message but has some other issue. -Ritesh On Mon, Jan 21, 2013 at 11:53 PM, Min wrote: > Rites

Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-21 Thread Min
Ritesh, If you can reproduce the problem, Java code should be fine. Community could look into it. Thanks Min 2013년 1월 17일 목요일에 Ritesh Adval님이 작성: > Hi Charles, > > I have test program in JAVA, I am not a C programmer so i will probably > take me time to reproduce this in C. Can someone first t

Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-16 Thread Ritesh Adval
Hi Charles, I have test program in JAVA, I am not a C programmer so i will probably take me time to reproduce this in C. Can someone first take a look at my JAVA program to see if I am not doing anything stupid. Should I create bug and attach Java maven project? Its very easy to run it, all you n

Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-16 Thread Charles Remes
On Jan 16, 2013, at 4:08 PM, Ritesh Adval wrote: > Hi Charles, > > Yes I close the socket in my thread after sending 100 messages, and I expect > that LINGER will make sure messages are sent to the other end, I expected > that context termination will block and make sure any pending messages a

Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-16 Thread Ritesh Adval
Hi Charles, Yes I close the socket in my thread after sending 100 messages, and I expect that LINGER will make sure messages are sent to the other end, I expected that context termination will block and make sure any pending messages are sent, but thats not happening. context termination returns q

Re: [zeromq-dev] Zeromq messages getting dropped

2013-01-16 Thread Charles Remes
On Jan 16, 2013, at 3:50 PM, Ritesh Adval wrote: > I found that people have reported this error and one suggestion is to put > sleep after sending message. If I put sleep after sending message using a > socket it does work but not always, specially under large message size and > heavy load. >

[zeromq-dev] Zeromq messages getting dropped

2013-01-16 Thread Ritesh Adval
Hi, I have been using zeromq (2.2.0) with java binding which is used by storm ( https://github.com/zeromq/jzmq). Though I am using jzmq binding from storm, I think this is a zeromq bug. I have a JAVA junit test which reproduce this error. I can send it to someone who can help look into this issue.