Nikita, Sounds like a good plan. Please share the design details prior getting down to the implementation.
— Denis > On Nov 23, 2017, at 4:38 AM, Nikita Amelchev <nsamelc...@gmail.com> wrote: > > Hi Igniters! > > I’m working on the similar feature for my own project. > I would like to suggest use in-line compression and write encoded bytes in > network channel by bytes array buffer. It allows us avoiding expensive > memory allocation. > The described design may be implemented in TcpCommunicationSpi level. We > can introduce pluggable compressor on TCP level where we will be able to > describe our compression strategy, for example, exclude some small messages > and many other. > > If the community doesn't mind I will file the ticket and will start > implementing it. > Any thoughts? > > 2017-11-23 12:06 GMT+03:00 Vladimir Ozerov <voze...@gridgain.com>: > >> Denis, >> >> Regarding zipped marshaller - this would be inefficient, because >> compression rate will be lower. >> >> On Thu, Nov 23, 2017 at 1:01 AM, Denis Magda <dma...@apache.org> wrote: >> >>> Nikita, >>> >>> Your solution sounds reasonable from the first glance. However, the >>> communication layer processes a dozen of small system messages that >> should >>> be excluded from the compression. Guess, that we will spend more time on >>> compressing/decompressing them thus diminishing the positive effect of >> the >>> compression. >>> >>> Alexey K., Vladimir O., >>> >>> What if we create Zip version of the binary marshaller the same way we >>> implemented GridClientZipOptimizedMarshaller? >>> >>> — >>> Denis >>> >>>> On Nov 22, 2017, at 5:36 AM, Alexey Kuznetsov <akuznet...@apache.org> >>> wrote: >>>> >>>> I think it is very useful feature. >>>> I also have experience when server nodes connected via fast network. >>>> But client nodes via very slow network. >>>> >>>> I implemeted GridClientZipOptimizedMarshaller and that solved my >> issue. >>>> But this marshaller works only with old >>>> and org.apache.ignite.internal.client.GridClient and has a lot of >>>> limitations. >>>> But compression was about 6-20x times. >>>> >>>> We need a solution for Ignite 2.x and client nodes. >>>> >>>> >>>> On Wed, Nov 22, 2017 at 7:48 PM, Nikita Amelchev <nsamelc...@gmail.com >>> >>>> wrote: >>>> >>>>> Hello, Igniters! >>>>> >>>>> I think it is a useful feature. I suggest to implement it to >>> communication >>>>> SPI like SSL encryption implemented. I have experience with this >> feature >>>>> and I can try to develop it. >>>>> >>>>> 2017-11-22 12:01 GMT+03:00 Alexey Kukushkin < >> kukushkinale...@gmail.com >>>> : >>>>> >>>>>> Forwarding to DEV list: Ignite developers, could you please share >> your >>>>>> thoughts on how hard it is to extend Ignite to compress data on the >>>>>> network. >>>>>> >>>>>> On Wed, Nov 22, 2017 at 10:04 AM, Gordon Reid (Nine Mile) < >>>>>> gordon.r...@ninemilefinancial.com> wrote: >>>>>> >>>>>>> Hi Igniters, >>>>>>> >>>>>>> >>>>>>> >>>>>>> I see there is a lot of discussion in certain threads about >>>>> compression. >>>>>>> This seems to have diverged into conversations about object versus >>>>> field >>>>>>> compression, and even throwing encryption into the mix. For my use >>>>> case, >>>>>> I >>>>>>> am not interested in compressing the cache stored in memory, I have >>>>>> plenty >>>>>>> of memory for my application. What I don’t have is a good network. I >>>>>> have a >>>>>>> high latency, low bandwidth network between my C# ignite client and >> my >>>>>> Java >>>>>>> ignite server. I only want to compress data when it is sent over the >>>>>>> network to remote nodes. It should be stored in the local memory >>>>>>> uncompressed. How can we achive this? Can the TcpCommunicationSpi >>>>> support >>>>>>> compression? >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Gordon. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> This email and any attachments are proprietary & confidential and >> are >>>>>>> intended solely for the use of the individuals to whom it is >>> addressed. >>>>>> Any >>>>>>> views or opinions expressed are solely for those of the author and >> do >>>>> not >>>>>>> necessarily reflect those of Nine Mile Financial Pty. Limited. If >> you >>>>>> have >>>>>>> received this email in error, please let us know immediately by >> reply >>>>>> email >>>>>>> and delete from your system. Nine Mile Financial Pty. Limited. ABN: >>> 346 >>>>>>> 1349 0252 >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> Alexey >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best wishes, >>>>> Amelchev Nikita >>>>> >>>> >>>> >>>> >>>> -- >>>> Alexey Kuznetsov >>> >>> >> > > > > -- > Best wishes, > Amelchev Nikita
