Looks fine to me.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
: 1.4.1
> libSDP: 1.1.99
>
> Regards,
> Mike
>
>
>
> *From:* Martin Sustrik [mailto:sust...@250bpm.com]
> *Sent:* Thu 3/11/2010 3:42 AM
> *To:* Santy, Michael
> *Cc:* zeromq-dev@lists.zeromq.or
eromq-dev@lists.zeromq.org
Subject: Re: [zeromq-dev] Non-contiguous message thoughts
Hi Michael,
> Attached is a spreadsheet of performance for the copy/non-copy latency
> test from 4K up to 256MB. Its no surprise that memory copying makes a
> difference on large messages when us
Hi Michael,
> Attached is a spreadsheet of performance for the copy/non-copy latency
> test from 4K up to 256MB. Its no surprise that memory copying makes a
> difference on large messages when using infiniband. I'll let you slice
> and dice the data the way you see fit. Let me know if you ne
On 11 March 2010 05:58, Santy, Michael wrote:
> Attached is a spreadsheet of performance for the copy/non-copy latency
> test from 4K up to 256MB. Its no surprise that memory copying makes a
> difference on large messages when using infiniband. I'll let you slice and
> dice the data the way yo
Martin,
Attached is a spreadsheet of performance for the copy/non-copy latency test
from 4K up to 256MB. Its no surprise that memory copying makes a difference on
large messages when using infiniband. I'll let you slice and dice the data the
way you see fit. Let me know if you need any clari
Michael Santy wrote:
> Sounds great. Let me know how I can help.
If you want to help, I've just committed a patch to allow 0MQ queues to
rollback any unflushed messages (message parts) written to the queue.
It'll be necessary for implementing multi-part message atomicity later
on. The code con
Sounds great. Let me know how I can help.
On 3/8/10 2:01 PM, "Martin Sustrik" wrote:
> Brian, Michael,
>
> Thanks for suggestions!
>
> Let do it this way:
>
> 1. I'll implement the functionality.
> 2. During the implementation I may find a good reason to prefer one way
> of modifying C API
Martin,
> Let do it this way:
>
> 1. I'll implement the functionality.
> 2. During the implementation I may find a good reason to prefer one way of
> modifying C API to another. If so, we'll discuss it then.
> 3. With C API in place it would be easier to reason about providing the
> functionality
Brian, Michael,
Thanks for suggestions!
Let do it this way:
1. I'll implement the functionality.
2. During the implementation I may find a good reason to prefer one way
of modifying C API to another. If so, we'll discuss it then.
3. With C API in place it would be easier to reason about providi
Martin,
>> sock.send([msg1, msg2, msg3])
>>
>> On the other side:
>>
>> messages = sock.recv()
>
> One thing to be considered is keeping the case of single-part message as
> simple as possible (i.e. as simple as it is now) given that this will be the
> case in 99 times of 100.
>
> message = sock.r
On 3/8/10 1:26 PM, "Martin Sustrik" wrote:
> Brian Granger wrote:
>> Martin,
>>
>>> Yes. The field has to be there anyway.
>>>
>>> We just have to find some suitable name for the accessor function...
>>>
>>> bool zmq_msg_end (zmq_msg_t *msg);
>>
>> While this may not be possible in C/C++,
Brian Granger wrote:
> Martin,
>
>> Yes. The field has to be there anyway.
>>
>> We just have to find some suitable name for the accessor function...
>>
>> bool zmq_msg_end (zmq_msg_t *msg);
>
> While this may not be possible in C/C++, in Python, I would be tempted
> to hide the message group
> s
Martin,
> Yes. The field has to be there anyway.
>
> We just have to find some suitable name for the accessor function...
>
> bool zmq_msg_end (zmq_msg_t *msg);
While this may not be possible in C/C++, in Python, I would be tempted
to hide the message group
stuff in send/recv:
sock.send([msg1, m
Michael,
>> Example of an application receiving messages consisting of two pieces:
>>
>> while (true) {
>> zmq_recv (s, &msg1, 0);
>> zmq_recv (s, &msg2, 0);
>> process_it (msg1, msg2);
>> }
>
> So am I correct in assuming the application would have to know whether the
> message is
On 3/8/10 1:15 AM, "Martin Sustrik" wrote:
>
> Example of an application receiving messages consisting of two pieces:
>
> while (true) {
> zmq_recv (s, &msg1, 0);
> zmq_recv (s, &msg2, 0);
> process_it (msg1, msg2);
> }
So am I correct in assuming the application would have t
Martin,
>> OK, this was the other aspect part I was wondering. So will you be
>> guaranteed that if 1 message from the group arrives that all will
>> arrive?
>
> Exactly. The group is meant to have exactly the same sematics as multi-part
> message would do.
>
> It may be worth calling it "multi-p
Brian Granger wrote:
>> Also, you don't want 0MQ to deliver 1st part of the message while remaining
>> parts are not yet available. That kind of thing can possibly cause
>> deadlocks.
>
> OK, this was the other aspect part I was wondering. So will you be
> guaranteed that if 1 message from the g
Martin,
>> I haven't been able to follow this thread very closely, but I do have
>> one question:
>>
>> You can already do non-copying sends of non-contiguous memory using
>> 0MQ, you just have to send multiple messages. What additional benefit
>> would message groups give beyond that?
>
> It's a
Jon,
> by which you mean a continuation_indicator would have to be in the app
> message and the
> api thread could loop creating zmq_msg and then calling zmq_recv?
Example of an application receiving messages consisting of two pieces:
while (true) {
zmq_recv (s, &msg1, 0);
zmq_recv (s
Hi Brian,
> I haven't been able to follow this thread very closely, but I do have
> one question:
>
> You can already do non-copying sends of non-contiguous memory using
> 0MQ, you just have to send multiple messages. What additional benefit
> would message groups give beyond that?
It's atomici
I haven't been able to follow this thread very closely, but I do have
one question:
You can already do non-copying sends of non-contiguous memory using
0MQ, you just have to send multiple messages. What additional benefit
would message groups give beyond that?
Cheers,
Brian
On Sun, Mar 7, 201
Martin Sustrik wrote:
> Michael,
>
>
>>> It's less trivial to implement (at very least it would require tweaking
>>> the wire protocol), however, the added value is much greater than in the
>>> simple iovec case.
>>>
>> Agreed, it certainly seems non-trivial, especially over some of the s
Chuck,
> Any issues with non-aligned access? I realize the format already
> causes that for the large message case (> 254 bytes).
>
> I only ask because most processors these days are optimized for
> access aligned along the 4-byte boundary (or perhaps it doesn't
> matter anymore).
I don't beli
On Mar 7, 2010, at 10:41 AM, Martin Sustrik wrote:
> Unfortunately we cannot avoid tweaking current wire format. For the
> description of current format have a look here:
>
> http://api.zeromq.org/zmq_tcp.7.html
>
> My proposal for tweaked format is: Insert a single byte between
> frame-length
Guys,
Now I realised the non-contiguous message design would solve the problem
I've been struggling with for several months already!
Specifically, it's the problem of when 0MQ itself needs to add some info
to the message. For example, in multi-hop request/reply scenario each
message should hol
Dhammika,
> I like msg groups idea, but it sounds more like an application feature.
>
> I was thinking about application level framing, adding a msg header,
> body and checksum. If we can setup different buffers for each msg
> segment, then sender won't have to create a contiguous block and
> rec
I like msg groups idea, but it sounds more like an application feature.
I was thinking about application level framing, adding a msg header,
body and checksum. If we can setup different buffers for each msg
segment, then sender won't have to create a contiguous block and
receiver can avoid having
Michael,
>> It's less trivial to implement (at very least it would require tweaking
>> the wire protocol), however, the added value is much greater than in the
>> simple iovec case.
>
> Agreed, it certainly seems non-trivial, especially over some of the socket
> types. What did you have in mind
>
> I've been following different train of thought: When sending data like
> those in Michael's scenario, the splits are not random. Rather they
> mirror business logic of the application. For example, the structure of
> the message may look like this (matrixA, matrixB, matrixC).
Precisely -- we
Dhammika,
> We can add scatter/gather IO API to zmq_msg. Proper readv/writev on
> sockets will need some changes in encoder/decoder, but we can use
> current encoder/decoder and read/write each iov entry separately.
>
> typedef struct
> {
> void *content;
> unsigned char shared;
> uns
Hi Martin,
We can add scatter/gather IO API to zmq_msg. Proper readv/writev on
sockets will need some changes in encoder/decoder, but we can use
current encoder/decoder and read/write each iov entry separately.
typedef struct
{
void *content;
unsigned char shared;
unsigned char vsm_si
Carlos A. Rocha wrote:
> Pardon my ignorance, what is IB and SDP? Google is failing on me (or
> vice-versa...)
;)
http://en.wikipedia.org/wiki/InfiniBand
http://en.wikipedia.org/wiki/Sockets_Direct_Protocol
Martin
___
zeromq-dev mailing list
zeromq-d
On Fri, Mar 5, 2010 at 10:41 AM, Michael Santy
wrote:
>> This is espacially interesting as you are running on IB... is that SDP
>> or what?
>
> Yep, its sdp.
Pardon my ignorance, what is IB and SDP? Google is failing on me (or
vice-versa...)
Thanks,
--
Carlos
___
Copy version of the test will follow shorty.
Here it is!
Martin
/*
Copyright (c) 2007-2010 iMatix Corporation
This file is part of 0MQ.
0MQ is free software; you can redistribute it and/or modify it under
the terms of the Lesser GNU General Public License as published by
Michael,
Or better yet, just send me the source code of the test and I'll run it on
our IB network.
Here you go! The first test is attached. It does zero-copy transfer. Run
it like this:
./local_lat tcp://192.168.0.111: 100 100
&
./remote_lat tcp://192.168.0.111: 100 100
Or better yet, just send me the source code of the test and I'll run it on
our IB network.
On 3/5/10 9:12 AM, "Carlos A. Rocha" wrote:
> Hi Martin,
>
> On Fri, Mar 5, 2010 at 9:20 AM, Martin Sustrik wrote:
>>> I am actually setting up a related test to be run over the weekend.
>>> Anyone has
On 3/5/10 8:18 AM, "Martin Sustrik" wrote:
>
> What would be really valueable would be seeing impact of copy on _large_
> messages. We've run the tests for messages up to 1MB long and concluded
> that there's no real difference. Thus it would be interesting to run the
> test with/without copy
Hi Martin,
On Fri, Mar 5, 2010 at 9:20 AM, Martin Sustrik wrote:
>> I am actually setting up a related test to be run over the weekend.
>> Anyone has some insight or suggestion of what I data should I collect
>> to measure memory performance?
>
> Oops. Just sent an email to Michael with descripti
Carlos,
> On Fri, Mar 5, 2010 at 8:58 AM, Martin Sustrik wrote:
>> However, it would at least allow you to test what the performance would
>> be if copying is avoided.
>>
>> If you run such a test, please, post the results. It would be extremely
>> valuable.
>
> I am actually setting up a relate
Michael Santy wrote:
>
>> If you run such a test, please, post the results. It would be extremely
>> valuable.
>
> Is there any particular aspect to the problem that you think would be
> valuable to the 0MQ developers? If so, I'll try to design my test to show
> that aspect.
What would be reall
On Fri, Mar 5, 2010 at 8:58 AM, Martin Sustrik wrote:
> However, it would at least allow you to test what the performance would
> be if copying is avoided.
>
> If you run such a test, please, post the results. It would be extremely
> valuable.
I am actually setting up a related test to be run ove
>
> If you run such a test, please, post the results. It would be extremely
> valuable.
Is there any particular aspect to the problem that you think would be
valuable to the 0MQ developers? If so, I'll try to design my test to show
that aspect.
___
>
> Btw, cheap and nasty solution would be to send you data as a sequence of
> small messages (one message per allocated block).
>
> This would of course break atomicity of message and mess with load
> balancing (half of a message going to A, other half going to B).
>
> However, it would at leas
Michael Santy wrote:
>
>> * You don't want to mess with lock that manage data across the IO/app
>> thread boundary.
>> * memcpy is not that slow, and by using it, you enable 0MQ to
>> deallocate the msg ASAP.
>>
>
> Thanks a bunch for the in-depth response. Those are all good thoughts worth
>
>
> * You don't want to mess with lock that manage data across the IO/app
> thread boundary.
> * memcpy is not that slow, and by using it, you enable 0MQ to
> deallocate the msg ASAP.
>
Thanks a bunch for the in-depth response. Those are all good thoughts worth
considering before moving any
Michael Santy wrote:
>
>> At Fri, 05 Mar 2010 09:10:19 +0100, Martin Sustrik wrote:
>>
>> This is possible, if language runtime supports callbacks from foreign
>> thread (to call language code from C). However, handwritten memory
>> management wrapper kills performance. Unless you have to deal wit
> At Fri, 05 Mar 2010 09:10:19 +0100, Martin Sustrik wrote:
>
> This is possible, if language runtime supports callbacks from foreign
> thread (to call language code from C). However, handwritten memory
> management wrapper kills performance. Unless you have to deal with
> very large messages, i
> In general, using zero-copy techniques from garbage collected languages
> is either impossible (Java?) or problematic (python).
>
> However, if what you are interested in is C/C++, the idea described
> above is doable.
I am coming at it from a C/C++ perspective, but I can see how garbage
colle
At Fri, 05 Mar 2010 09:10:19 +0100, Martin Sustrik wrote:
> In general, using zero-copy techniques from garbage collected languages
> is either impossible (Java?) or problematic (python).
This is possible, if language runtime supports callbacks from foreign
thread (to call language code from C).
Hi Michael,
> I'm relatively new to 0MQ and like what I see so far. I believe in my
> initial attempt to apply 0MQ to clean up some legacy socket code, I may
> have uncovered a use case that 0MQ does not yet address that may be
> generally useful for high-performance messaging.
>
> The code i
Mike,
I too am using 0MQ in an HPC context and am worrying about sending
large objects
> The code in question takes a few separate data structures (matrices and
> associated metadata) and sends it to another process over a socket. The
> most straightforward way for me to accomplish this with
Hello All,
I'm relatively new to 0MQ and like what I see so far. I believe in my initial
attempt to apply 0MQ to clean up some legacy socket code, I may have uncovered
a use case that 0MQ does not yet address that may be generally useful for
high-performance messaging.
The code in question ta
53 matches
Mail list logo
