On Thu, Nov 16, 2017 at 2:01 PM, Clebert Suconic <clebert.suco...@gmail.com> wrote:
> On Thu, Nov 16, 2017 at 6:10 AM, Martyn Taylor <mtay...@redhat.com> wrote: > > Clebert, > > > > We need to distinguish between streamed messages and large messages. For > > the basic large message case, i.e. a single large message sent to the > > broker. I agree with what you have here. > > > > Streamed large messages (i.e. messages that are received in chunks) > allows > > us to store the message without having to keep the whole thing in broker > > memory. This get's complicated with cross protocol as: > > > > 1. Not all protocols support streamed messages. > Exactly.. right now in Stomp.. we read the whole body in memory before > sending. Just like what is in master now for AMQP. > > > 2. Cross protocol may require message conversions > Just like in Stomp... we read the whole message in memory > > > > > Both 1. and 2. would likely mean that we'd likely need to read the whole > > message into memory and I'm not sure we really want to do this? It > defeats > > one of the main purposes of streamed messages. (The others being saving > > client memory and reducing the amount of data to resend on connection > drop.) > > Most times the user will have no control over this. Say you sent a > 100K + 1 byte over the configured limit, String messages in Core. the > client will stream it as it being large This is why I think we need to distinguish between broker large messages and streamed messages. This configuration option is really for streaming messages. In our docs we talk about the ability to send messages that are larger than the broker memory using streaming. What happens if I do this and try to consume it via STOMP or another protocol that doesn't support streaming? > . > > > > > > For 1. I wonder whether or not we want to support this at all, or if we > do, > > make it configurable. > > For 2. Is it possible to transform the message via a stream? If not then > > perhaps we treat this as a raw bytes message? > > It is possible as I said.. I think it's a separate task. I may try to > do it within this scope. > OK sounds good.
