Disabling compression when reading large values >1MB
I am writing an internal compression routine that will compress large values with a Huffman table in memcached [version 1.5.10]. However I am finding that after reading large values and printing them inside memcached, I am getting garbled output. For example, I am reading one large value csv file in csv format where the value length is 1981182 [~2MB]. When I print the value inside the assoc_insert function I am getting output such as ▒L$ or ▒▒ . It only happens for large values and I have tried a telnet client as well as pymemcache [https://pymemcache.readthedocs.io/en/latest/index.html]. Are the large values getting compressed in memcached [or the client for that matter]? If so, is there a way to disable this compression? -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Disabling compression when reading large values >1MB
Not entirely clear on what you're doing. Are you modifying memcached itself to do internal compression? Some clients autocompress after threshholds. there's no internal compression. On Sun, 4 Nov 2018, Pradeep Sivakumar wrote: > I am writing an internal compression routine that will compress large values > with a Huffman table in memcached [version 1.5.10]. However I am finding that > after reading large values and printing them inside memcached, I am getting > garbled output. For example, I am reading one large value csv file in csv > format where the value length is 1981182 [~2MB]. When I print the value > inside the assoc_insert function I am getting output such as ▒L$ or ▒▒ . It > only > happens for large values and I have tried a telnet client as well as > pymemcache [https://pymemcache.readthedocs.io/en/latest/index.html]. Are the > large > values getting compressed in memcached [or the client for that matter]? If > so, is there a way to disable this compression? > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Disabling compression when reading large values >1MB
Thanks for the quick response. I have unlined my answers below: > On Nov 4, 2018, at 11:50 AM, dormando wrote: > > Not entirely clear on what you're doing. Are you modifying memcached > itself to do internal compression? Correct. I am doing internal compression inside memcached. > > Some clients autocompress after threshholds. there's no internal > compression. Which tells me that the compression happens in the telnet client or pymemcache. Is there a way to disable compression in telnet? > >> On Sun, 4 Nov 2018, Pradeep Sivakumar wrote: >> >> I am writing an internal compression routine that will compress large values >> with a Huffman table in memcached [version 1.5.10]. However I am finding that >> after reading large values and printing them inside memcached, I am getting >> garbled output. For example, I am reading one large value csv file in csv >> format where the value length is 1981182 [~2MB]. When I print the value >> inside the assoc_insert function I am getting output such as ▒L$ or ▒▒ . It >> only >> happens for large values and I have tried a telnet client as well as >> pymemcache [https://pymemcache.readthedocs.io/en/latest/index.html]. Are the >> large >> values getting compressed in memcached [or the client for that matter]? If >> so, is there a way to disable this compression? >> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "memcached" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to memcached+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> >> > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Disabling compression when reading large values >1MB
telnet has no concept of compression. if you're intending to modify the server code, you should study it a bit more. Sorry :( Since clients have compression built in, you shouldn't have to do this. Compressing/decompressing in the server won't be easy to accomplish and it will take a lot more bandwidth to ship uncompressed data to clients. On Sun, 4 Nov 2018, Pradeep Sivakumar wrote: > Thanks for the quick response. I have unlined my answers below: > > > On Nov 4, 2018, at 11:50 AM, dormando wrote: > > > > Not entirely clear on what you're doing. Are you modifying memcached > > itself to do internal compression? > > > Correct. I am doing internal compression inside memcached. > > > > Some clients autocompress after threshholds. there's no internal > > compression. > > > Which tells me that the compression happens in the telnet client or > pymemcache. Is there a way to disable compression in telnet? > > > >> On Sun, 4 Nov 2018, Pradeep Sivakumar wrote: > >> > >> I am writing an internal compression routine that will compress large > >> values with a Huffman table in memcached [version 1.5.10]. However I am > >> finding that > >> after reading large values and printing them inside memcached, I am > >> getting garbled output. For example, I am reading one large value csv file > >> in csv > >> format where the value length is 1981182 [~2MB]. When I print the value > >> inside the assoc_insert function I am getting output such as ▒L$ or ▒▒ . > >> It only > >> happens for large values and I have tried a telnet client as well as > >> pymemcache [https://pymemcache.readthedocs.io/en/latest/index.html]. Are > >> the large > >> values getting compressed in memcached [or the client for that matter]? If > >> so, is there a way to disable this compression? > >> > >> -- > >> > >> --- > >> You received this message because you are subscribed to the Google Groups > >> "memcached" group. > >> To unsubscribe from this group and stop receiving emails from it, send an > >> email to memcached+unsubscr...@googlegroups.com. > >> For more options, visit https://groups.google.com/d/optout. > >> > >> > > > > -- > > > > --- > > You received this message because you are subscribed to the Google Groups > > "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to memcached+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
