This is a bit more complex.
LargeServerMessafe is too tight with core. Will be a bit challenging changing it. I will make large Message a property of the message so it can be resumed in AMQP. It will still be compatible change. Won’t need to change it to 3.0. I will update the design here once I do some more work. On Thu, Nov 16, 2017 at 9:46 AM Clebert Suconic <[email protected]> wrote: > I will try to make a conversion. I would rather treat than make another > parameter. > > On Thu, Nov 16, 2017 at 9:31 AM Clebert Suconic <[email protected]> > wrote: > >> There is no distinction. >> >> It will stream if the packet is more then min Large message size on >> core. Or max frame size in AMQP. >> >> I don’t know how distinguish really large ones from the ones that are >> just above the limit. >> >> On Thu, Nov 16, 2017 at 9:11 AM Martyn Taylor <[email protected]> wrote: >> >>> On Thu, Nov 16, 2017 at 2:01 PM, Clebert Suconic < >>> [email protected]> >>> wrote: >>> >>> > On Thu, Nov 16, 2017 at 6:10 AM, Martyn Taylor <[email protected]> >>> wrote: >>> > > Clebert, >>> > > >>> > > We need to distinguish between streamed messages and large >>> messages. For >>> > > the basic large message case, i.e. a single large message sent to the >>> > > broker. I agree with what you have here. >>> > > >>> > > Streamed large messages (i.e. messages that are received in chunks) >>> > allows >>> > > us to store the message without having to keep the whole thing in >>> broker >>> > > memory. This get's complicated with cross protocol as: >>> > > >>> > > 1. Not all protocols support streamed messages. >>> > Exactly.. right now in Stomp.. we read the whole body in memory before >>> > sending. Just like what is in master now for AMQP. >>> > >>> > > 2. Cross protocol may require message conversions >>> > Just like in Stomp... we read the whole message in memory >>> > >>> > > >>> > > Both 1. and 2. would likely mean that we'd likely need to read the >>> whole >>> > > message into memory and I'm not sure we really want to do this? It >>> > defeats >>> > > one of the main purposes of streamed messages. (The others being >>> saving >>> > > client memory and reducing the amount of data to resend on connection >>> > drop.) >>> > >>> > Most times the user will have no control over this. Say you sent a >>> > 100K + 1 byte over the configured limit, String messages in Core. the >>> > client will stream it as it being large >>> >>> This is why I think we need to distinguish between broker large messages >>> and streamed messages. This configuration option is really for streaming >>> messages. In our docs we talk about the ability to send messages that >>> are >>> larger than the broker memory using streaming. What happens if I do this >>> and try to consume it via STOMP or another protocol that doesn't support >>> streaming? >>> >>> > . >>> > >>> > >>> > > >>> > > For 1. I wonder whether or not we want to support this at all, or if >>> we >>> > do, >>> > > make it configurable. >>> > > For 2. Is it possible to transform the message via a stream? If not >>> then >>> > > perhaps we treat this as a raw bytes message? >>> > >>> > It is possible as I said.. I think it's a separate task. I may try to >>> > do it within this scope. >>> > >>> OK sounds good. >>> >> -- >> Clebert Suconic >> > -- > Clebert Suconic > -- Clebert Suconic
