Re: [HACKERS] Custom compression methods

On Sun, Feb 7, 2021 at 5:15 PM Dilip Kumar wrote: > > On Fri, Feb 5, 2021 at 8:11 PM Dilip Kumar wrote: > > > > On Wed, Feb 3, 2021 at 2:07 AM Robert Haas wrote: > > > > > > Even more review comments, still looking mostly at 0001: > > > > > > If there's a reason why parallel_schedule is

Re: [HACKERS] Custom compression methods

This fails make-check world for me, and CFBOT will say the same. Be sure to compile with --enable-tap-tests cd . && TESTDIR='/home/pryzbyj/src/postgres/src/bin/pg_dump' PATH="/home/pryzbyj/src/postgres/tmp_install/usr/local/pgsql/bin:$PATH"

Re: [HACKERS] Custom compression methods

On Fri, Feb 5, 2021 at 11:07 AM Robert Haas wrote: > Personally, my preference is to just update the test outputs. It's not > important whether many people look closely to verify the differences; > we just need to look them over on a one-time basis to see if they seem > OK. After that it's 0

Re: [HACKERS] Custom compression methods

On Fri, Feb 5, 2021 at 10:56 AM Justin Pryzby wrote: > Could you comment on the patch I sent on Jan 30 ? I think it would be > squished > into 0001. I don't see why we have to do that. Seems fine to have it as a separate patch. > Also, what about the idea to add HIDE_COMPRESSAM ? Right now,

Re: [HACKERS] Custom compression methods

+* If the called has passed an invalid slot then create a new slot. *caller +* Slot for storing the modified tuple, incase the

Re: [HACKERS] Custom compression methods

On Tue, Feb 2, 2021 at 2:45 AM Robert Haas wrote: > > Some more review comments: > > 'git am' barfs on v0001 because it's got a whitespace error. Fixed > VARFLAGS_4B_C() doesn't seem to be used in any of the patches. I'm OK > with keeping it even if it's not used just because maybe someone will

Re: [HACKERS] Custom compression methods

On Fri, Feb 5, 2021 at 3:51 AM Robert Haas wrote: > > On Thu, Feb 4, 2021 at 11:39 AM Dilip Kumar wrote: > > Yeah, actually, I thought I would avoid calling slot_getallattrs if > > none of the attributes got decompress. I agree if we call this before > > we can avoid calling slot_getattr but

Re: [HACKERS] Custom compression methods

On Thu, Feb 4, 2021 at 11:39 AM Dilip Kumar wrote: > Yeah, actually, I thought I would avoid calling slot_getallattrs if > none of the attributes got decompress. I agree if we call this before > we can avoid calling slot_getattr but slot_getattr > is only called for the attribute which has

Re: [HACKERS] Custom compression methods

On Wed, Feb 3, 2021 at 2:07 AM Robert Haas wrote: While going through your comments, I need some suggestions about the one except this all other comments looks fine to me. > > It puzzles me that CompareCompressionMethodAndDecompress() calls > slot_getallattrs() just before clearing the slot. It

Re: [HACKERS] Custom compression methods

On Wed, Feb 3, 2021 at 2:07 AM Robert Haas wrote: > > Even more review comments, still looking mostly at 0001: > > If there's a reason why parallel_schedule is arranging to run the > compression test in parallel with nothing else, the comment in that > file should explain the reason. If there

Re: [HACKERS] Custom compression methods

Even more review comments, still looking mostly at 0001: If there's a reason why parallel_schedule is arranging to run the compression test in parallel with nothing else, the comment in that file should explain the reason. If there isn't, it should be added to a parallel group that doesn't have

Re: [HACKERS] Custom compression methods

Some more review comments: 'git am' barfs on v0001 because it's got a whitespace error. VARFLAGS_4B_C() doesn't seem to be used in any of the patches. I'm OK with keeping it even if it's not used just because maybe someone will need it later but, uh, don't we need to use it someplace? To avoid

Re: [HACKERS] Custom compression methods

On Fri, Jan 29, 2021 at 9:47 AM Neha Sharma wrote: > > Hi, > > I have been testing the patches for a while , below is the code coverage > observed on the v19 patches. > > Sr NoFile nameCode Coverage > BeforeAfter > Line %Function %Line %Function % >

Re: [HACKERS] Custom compression methods

Hi, I have been testing the patches for a while , below is the code coverage observed on the v19 patches. Sr No File name Code Coverage Before After Line % Function % Line % Function % 1 src/backend/access/brin/brin_tuple.c 96.7 100 96.7 100 2 src/backend/access/common/detoast.c 88 100 88.6 100

Re: [HACKERS] Custom compression methods

On Wed, Jan 20, 2021 at 12:37 AM Justin Pryzby wrote: > > Thanks for updating the patch. Thanks for the review > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > The most recent patch doesn't compile --without-lz4: > On Tue, Jan 05, 2021 at 11:19:33AM +0530, Dilip Kumar wrote: > > On

Re: [HACKERS] Custom compression methods

Thanks for updating the patch. On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > The most recent patch doesn't compile --without-lz4: On Tue, Jan 05, 2021 at 11:19:33AM +0530, Dilip Kumar wrote: > On Mon, Jan 4, 2021 at 10:08 PM Justin Pryzby wrote: > > I think I first saw it on cfbot and I

Re: [HACKERS] Custom compression methods

On Mon, Jan 11, 2021 at 12:21 PM Justin Pryzby wrote: > > On Mon, Jan 11, 2021 at 12:11:54PM +0530, Dilip Kumar wrote: > > On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar wrote: > > > On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby > > > wrote: > > > > > > > > On Mon, Jan 04, 2021 at 04:57:16PM

Re: [HACKERS] Custom compression methods

On Mon, Jan 11, 2021 at 12:11:54PM +0530, Dilip Kumar wrote: > On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar wrote: > > On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby wrote: > > > > > > On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote: > > > > On Mon, Jan 4, 2021 at 6:52 AM Justin

Re: [HACKERS] Custom compression methods

On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar wrote: > > On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby wrote: > > > > On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote: > > > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > > > And fails pg_upgrade check, apparently losing

Re: [HACKERS] Custom compression methods

On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby wrote: > > On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote: > > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > > And fails pg_upgrade check, apparently losing track of the compression (?) > > > > > > CREATE TABLE public.cmdata2

Re: [HACKERS] Custom compression methods

On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote: > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > And fails pg_upgrade check, apparently losing track of the compression (?) > > > > CREATE TABLE public.cmdata2 ( > > -f1 text COMPRESSION lz4 > > +f1 text > > ); > > I

Re: [HACKERS] Custom compression methods

On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > The most recent patch doesn't compile --without-lz4: > > compress_lz4.c:191:17: error: ‘lz4_cmcheck’ undeclared here (not in a > function) > .datum_check = lz4_cmcheck, > ... > > And fails pg_upgrade check, apparently losing track of the

Re: [HACKERS] Custom compression methods

The most recent patch doesn't compile --without-lz4: compress_lz4.c:191:17: error: ‘lz4_cmcheck’ undeclared here (not in a function) .datum_check = lz4_cmcheck, ... And fails pg_upgrade check, apparently losing track of the compression (?) CREATE TABLE public.cmdata2 ( -f1 text

Re: [HACKERS] Custom compression methods

> 28 дек. 2020 г., в 11:14, Andrey Borodin написал(а): > >> Thanks for the patch. Maybe we can allow setting custom compression >> methods for wal compression as well. > > No, unfortunately, we can't use truly custom methods. Custom compression > handlers are WAL-logged. So we can use only

Re: [HACKERS] Custom compression methods

> 28 дек. 2020 г., в 10:20, Dilip Kumar написал(а): > > On Sun, Dec 27, 2020 at 12:40 PM Andrey Borodin wrote: >> >> >> >>> 25 дек. 2020 г., в 14:34, Dilip Kumar написал(а): >>> >>> >>> >>> >>> >>> >>> >> >> Maybe add Lz4\Zlib WAL FPI compression on top of this patchset? I'm

Re: [HACKERS] Custom compression methods

On Sun, Dec 27, 2020 at 12:40 PM Andrey Borodin wrote: > > > > > 25 дек. 2020 г., в 14:34, Dilip Kumar написал(а): > > > > > > > > > > > > > > > > Maybe add Lz4\Zlib WAL FPI compression on top of this patchset? I'm not > insisting on anything, it just would be so cool to have it... >

Re: [HACKERS] Custom compression methods

> 25 дек. 2020 г., в 14:34, Dilip Kumar написал(а): > > > > > > > Maybe add Lz4\Zlib WAL FPI compression on top of this patchset? I'm not insisting on anything, it just would be so cool to have it... BTW currently there are Oid collisions in original patchset. Best regards,

Re: [HACKERS] Custom compression methods

On Wed, Dec 9, 2020 at 5:37 PM Dilip Kumar wrote: > > On Sat, Nov 21, 2020 at 3:50 AM Robert Haas wrote: > > > > On Wed, Nov 11, 2020 at 9:39 AM Dilip Kumar wrote: > > > There were a few problems in this rebased version, basically, the > > > compression options were not passed while compressing

Re: [HACKERS] Custom compression methods

On Tue, Dec 1, 2020 at 9:15 PM Dilip Kumar wrote: > > On Tue, Dec 1, 2020 at 4:50 PM Dilip Kumar wrote: > > > > On Sat, Nov 21, 2020 at 3:50 AM Robert Haas wrote: > > > > While working on this comment I have doubts. > > > > > I wonder in passing about TOAST tables and materialized views, which

Re: [HACKERS] Custom compression methods

On Tue, Dec 1, 2020 at 4:50 PM Dilip Kumar wrote: > > On Sat, Nov 21, 2020 at 3:50 AM Robert Haas wrote: > > While working on this comment I have doubts. > > > I wonder in passing about TOAST tables and materialized views, which > > are the other things that have storage. What gets stored for >

Re: [HACKERS] Custom compression methods

On Sat, Nov 21, 2020 at 3:50 AM Robert Haas wrote: While working on this comment I have doubts. > I wonder in passing about TOAST tables and materialized views, which > are the other things that have storage. What gets stored for > attcompression? For a TOAST table it probably doesn't matter

Re: [HACKERS] Custom compression methods

On Wed, Nov 25, 2020 at 12:50 AM Alvaro Herrera wrote: > > On 2020-Nov-24, Tom Lane wrote: > > > Robert Haas writes: > > > > Oh, I thought it had been suggested in previous discussions that these > > > should be treated as access methods rather than inventing a whole new > > > concept just for

Re: [HACKERS] Custom compression methods

On 2020-Nov-24, Tom Lane wrote: > Robert Haas writes: > > Oh, I thought it had been suggested in previous discussions that these > > should be treated as access methods rather than inventing a whole new > > concept just for this, and it seemed like a good idea to me. I guess I > > missed the

Re: [HACKERS] Custom compression methods

On Tue, Nov 24, 2020 at 1:21 PM Tom Lane wrote: > FWIW, I kind of agree with Robert's take on this. Heap and index AMs > are pretty fundamentally different animals, yet we don't have a problem > sticking them in the same catalog. I think anything that is related to > storage access could

Re: [HACKERS] Custom compression methods

Robert Haas writes: > On Tue, Nov 24, 2020 at 10:47 AM Dilip Kumar wrote: >> For the compression routine name, I did not include "Am" because >> currently, we are storing the compression method in the new catalog >> "pg_compression" not in the pg_am. So are you suggesting that we >> should

Re: [HACKERS] Custom compression methods

On Tue, Nov 24, 2020 at 10:47 AM Dilip Kumar wrote: > For the compression routine name, I did not include "Am" because > currently, we are storing the compression method in the new catalog > "pg_compression" not in the pg_am. So are you suggesting that we > should store the compression methods

Re: [HACKERS] Custom compression methods

On Tue, Nov 24, 2020 at 7:14 PM Robert Haas wrote: > > On Tue, Nov 24, 2020 at 7:11 AM Dilip Kumar wrote: > > About (4), one option is that we directly call the correct handler > > function for the built-in type directly from > > toast_(de)compress(_slice) functions but in that case, we are > >

Re: [HACKERS] Custom compression methods

On Tue, Nov 24, 2020 at 7:11 AM Dilip Kumar wrote: > About (4), one option is that we directly call the correct handler > function for the built-in type directly from > toast_(de)compress(_slice) functions but in that case, we are > duplicating the code, another option is that we call the >

Re: [HACKERS] Custom compression methods

On Sat, Nov 21, 2020 at 3:50 AM Robert Haas wrote: Most of the comments looks fine to me but I have a slightly different opinion for one of the comment so replying only for that. > I'm worried about how expensive this might be, and I think we could > make it cheaper. The reason why I think this

Re: [HACKERS] Custom compression methods

On Sat, Nov 21, 2020 at 3:50 AM Robert Haas wrote: > > On Wed, Nov 11, 2020 at 9:39 AM Dilip Kumar wrote: > > There were a few problems in this rebased version, basically, the > > compression options were not passed while compressing values from the > > brin_form_tuple, so I have fixed this. > >

Re: [HACKERS] Custom compression methods

On Wed, Nov 11, 2020 at 9:39 AM Dilip Kumar wrote: > There were a few problems in this rebased version, basically, the > compression options were not passed while compressing values from the > brin_form_tuple, so I have fixed this. Since the authorship history of this patch is complicated, it

Re: [HACKERS] Custom compression methods

On Wed, Oct 28, 2020 at 01:16:31PM +0530, Dilip Kumar wrote: ... I have added the next patch to support the compression options. I am storing the compression options only for the latest compression method. Basically, based on this design we would be able to support the options which are used

Re: [HACKERS] Custom compression methods

On Wed, Oct 21, 2020 at 8:51 PM Robert Haas wrote: > > On Thu, Oct 8, 2020 at 5:54 PM Tomas Vondra > wrote: > > And is the oidvector actually needed? If we have the extra catalog, > > can't we track this simply using the regular dependencies? So we'd have > > the attcompression OID of the

Re: [HACKERS] Custom compression methods

On Thu, Oct 22, 2020 at 2:11 AM Tomas Vondra wrote: > > On Wed, Oct 21, 2020 at 01:59:50PM +0530, Dilip Kumar wrote: > >On Sat, Oct 17, 2020 at 11:34 AM Dilip Kumar wrote: > >> > >> On Tue, Oct 13, 2020 at 10:30 AM Dilip Kumar wrote: > >> > > >> > On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra >

Re: [HACKERS] Custom compression methods

On Wed, Oct 21, 2020 at 01:59:50PM +0530, Dilip Kumar wrote: On Sat, Oct 17, 2020 at 11:34 AM Dilip Kumar wrote: On Tue, Oct 13, 2020 at 10:30 AM Dilip Kumar wrote: > > On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra > wrote: > > > > On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote:

Re: [HACKERS] Custom compression methods

On Thu, Oct 8, 2020 at 5:54 PM Tomas Vondra wrote: > And is the oidvector actually needed? If we have the extra catalog, > can't we track this simply using the regular dependencies? So we'd have > the attcompression OID of the current compression method, and the > preserved values would be

Re: [HACKERS] Custom compression methods

On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra wrote: > > On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote: > > > >> ... > > > >I have worked on this patch, so as discussed now I am maintaining the > >preserved compression methods using dependency. Still PRESERVE ALL > >syntax is not

Re: [HACKERS] Custom compression methods

On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote: ... I have worked on this patch, so as discussed now I am maintaining the preserved compression methods using dependency. Still PRESERVE ALL syntax is not supported, I will work on that part. Cool, I'll take a look. What's your

Re: [HACKERS] Custom compression methods

On Fri, Oct 9, 2020 at 3:24 AM Tomas Vondra wrote: > > On Thu, Oct 08, 2020 at 02:38:27PM +0530, Dilip Kumar wrote: > >On Wed, Oct 7, 2020 at 5:00 PM Dilip Kumar wrote: > >> > >> On Wed, Oct 7, 2020 at 10:26 AM Dilip Kumar wrote: > >> > > >> > On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra > >> >

Re: [HACKERS] Custom compression methods

On Thu, Oct 08, 2020 at 02:38:27PM +0530, Dilip Kumar wrote: On Wed, Oct 7, 2020 at 5:00 PM Dilip Kumar wrote: On Wed, Oct 7, 2020 at 10:26 AM Dilip Kumar wrote: > > On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra > wrote: > > > > On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote: >

Re: [HACKERS] Custom compression methods

On Wed, Oct 7, 2020 at 5:00 PM Dilip Kumar wrote: > > On Wed, Oct 7, 2020 at 10:26 AM Dilip Kumar wrote: > > > > On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra > > wrote: > > > > > > On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote: > > > >On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra >

Re: [HACKERS] Custom compression methods

On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra wrote: > > On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote: > >On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra > > wrote: > >> > >> On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote: > >> >On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra >

Re: [HACKERS] Custom compression methods

On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote: On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra wrote: On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote: >On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra > wrote: >> >> On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:

Re: [HACKERS] Custom compression methods

On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra wrote: > > On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote: > >On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra > > wrote: > >> > >> On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote: > >> >On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra >

Re: [HACKERS] Custom compression methods

On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote: On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra wrote: On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote: >On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra > wrote: > >Thanks, Tomas for your feedback. > >> 9) attcompression ... >>

Re: [HACKERS] Custom compression methods

On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra wrote: > > On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote: > >On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra > > wrote: > > > >Thanks, Tomas for your feedback. > > > >> 9) attcompression ... > >> > >> The main issue I see is what the patch

Re: [HACKERS] Custom compression methods

On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote: On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra wrote: Thanks, Tomas for your feedback. 9) attcompression ... The main issue I see is what the patch does with attcompression. Instead of just using it to store a the compression method,

Re: [HACKERS] Custom compression methods

On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra wrote: Thanks, Tomas for your feedback. > 9) attcompression ... > > The main issue I see is what the patch does with attcompression. Instead > of just using it to store a the compression method, it's also used to > store the preserved compression

Re: [HACKERS] Custom compression methods

Hi, I took a look at this patch after a long time, and done a bit of a review+testing. I haven't re-read the whole thread since 2017 so some of the following comments might be mistaken - sorry about that :-( 1) The "cmapi.h" naming seems unnecessarily short. I'd suggest using simply

Re: Re: [HACKERS] Custom compression methods

On Mon, Aug 31, 2020 at 10:45 AM Amit Khandekar wrote: > > On Thu, 13 Aug 2020 at 17:18, Dilip Kumar wrote: > > I have rebased the patch on the latest head and currently, broken into 3 > > parts. > > > > v1-0001: As suggested by Robert, it provides the syntax support for > > setting the

Re: [HACKERS] Custom compression methods

On Wed, Sep 2, 2020 at 4:57 AM Mark Dilger wrote: > > > > > On Aug 13, 2020, at 4:48 AM, Dilip Kumar wrote: > > > > v1-0001: As suggested by Robert, it provides the syntax support for > > setting the compression method for a column while creating a table and > > adding columns. However, we

Re: [HACKERS] Custom compression methods

> On Aug 13, 2020, at 4:48 AM, Dilip Kumar wrote: > > v1-0001: As suggested by Robert, it provides the syntax support for > setting the compression method for a column while creating a table and > adding columns. However, we don't support changing the compression > method for the existing

Re: Re: [HACKERS] Custom compression methods

On Thu, 13 Aug 2020 at 17:18, Dilip Kumar wrote: > I have rebased the patch on the latest head and currently, broken into 3 > parts. > > v1-0001: As suggested by Robert, it provides the syntax support for > setting the compression method for a column while creating a table and > adding columns.

Re: Re: [HACKERS] Custom compression methods

On Mon, Aug 24, 2020 at 2:12 AM Dilip Kumar wrote: > IIUC, the main reason for using this flag is for taking the decision > whether we need any detoasting for this tuple. For example, if we are > rewriting the table because the compression method is changed then if > HEAP_HASCUSTOMCOMPRESSED

Re: Re: [HACKERS] Custom compression methods

On Thu, Aug 13, 2020 at 5:18 PM Dilip Kumar wrote: > There was some question which Robert has asked in this mail, please find my answer inline. Also, I have a few questions regarding further splitting up this patch. > On Fri, Jun 19, 2020 at 10:33 PM Robert Haas wrote: > > > > > > - One

Re: [HACKERS] Custom compression methods

On Mon, Jun 29, 2020 at 12:31 PM Andres Freund wrote: > > This "custom compression methods" thread - vintage 2017 - Original > > code by Nikita Glukhov, later work by Ildus Kurbangaliev > > The "pluggable compression support" thread - vintage 2013 - Andres Freund > > The "plgz performance" thread

Re: [HACKERS] Custom compression methods

Hi, On 2020-06-24 07:59:47 -0400, Robert Haas wrote: > On Tue, Jun 23, 2020 at 4:00 PM Andres Freund wrote: > > https://postgr.es/m/20130621000900.GA12425%40alap2.anarazel.de is a > > thread with more information / patches further along. > > > > I confused this patch with the approach in > >

Re: [HACKERS] Custom compression methods

On Wed, Jun 24, 2020 at 5:30 PM Robert Haas wrote: > > On Tue, Jun 23, 2020 at 4:00 PM Andres Freund wrote: > > https://postgr.es/m/20130621000900.GA12425%40alap2.anarazel.de is a > > thread with more information / patches further along. > > > > I confused this patch with the approach in > >

Re: [HACKERS] Custom compression methods

On Tue, Jun 23, 2020 at 4:00 PM Andres Freund wrote: > https://postgr.es/m/20130621000900.GA12425%40alap2.anarazel.de is a > thread with more information / patches further along. > > I confused this patch with the approach in >

Re: [HACKERS] Custom compression methods

Hi, On 2020-06-23 14:27:47 -0400, Robert Haas wrote: > On Mon, Jun 22, 2020 at 4:53 PM Andres Freund wrote: > > > Or maybe we add 1 or 2 "privileged" built-in compressors that get > > > dedicated bit-patterns in the upper 2 bits of the size field, with the > > > last bit pattern being reserved

Re: [HACKERS] Custom compression methods

On Mon, Jun 22, 2020 at 4:53 PM Andres Freund wrote: > > Or maybe we add 1 or 2 "privileged" built-in compressors that get > > dedicated bit-patterns in the upper 2 bits of the size field, with the > > last bit pattern being reserved for future algorithms. (e.g. 0x00 = > > pglz, 0x01 = lz4, 0x10

Re: [HACKERS] Custom compression methods

Hi, On 2020-06-19 13:03:02 -0400, Robert Haas wrote: > - I can see three possible ways of breaking our dependence on 'pglz' > for TOAST compression. Option #1 is to pick one new algorithm which we > think is better than 'pglz' in all relevant ways and use it as the > default for all new

Re: Re: [HACKERS] Custom compression methods

On Thu, Mar 7, 2019 at 2:51 AM Alexander Korotkov wrote: > Yes. I took a look at code of this patch. I think it's in pretty good > shape. But high level review/discussion is required. I agree that the code of this patch is in pretty good shape, although there is a lot of rebasing needed at

Re: [HACKERS] Custom compression methods

The compile of this one has been broken for a long time. Is there a rebase happening? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Custom compression methods

On Mon, 1 Jul 2019 at 17:28, Alexander Korotkov wrote: > On Mon, Jul 1, 2019 at 5:51 PM Alvaro Herrera > wrote: > > On 2019-Jul-01, Alexander Korotkov wrote: > > > > > As I get we're currently need to make high-level decision of whether > > > we need this [1]. I was going to bring this topic

Re: [HACKERS] Custom compression methods

On Mon, Jul 1, 2019 at 5:51 PM Alvaro Herrera wrote: > On 2019-Jul-01, Alexander Korotkov wrote: > > > As I get we're currently need to make high-level decision of whether > > we need this [1]. I was going to bring this topic up at last PGCon, > > but I didn't manage to attend. Does it worth

Re: [HACKERS] Custom compression methods

On 2019-Jul-01, Alexander Korotkov wrote: > As I get we're currently need to make high-level decision of whether > we need this [1]. I was going to bring this topic up at last PGCon, > but I didn't manage to attend. Does it worth bothering Ildus with > continuous rebasing assuming we don't have

Re: [HACKERS] Custom compression methods

Hi, Thomas! On Mon, Jul 1, 2019 at 1:22 PM Thomas Munro wrote: > > On Sat, Mar 16, 2019 at 12:52 AM Ildus Kurbangaliev > wrote: > > in my opinion this patch is usually skipped not because it is not > > needed, but because of its size. It is not hard to maintain it until > > commiters will have

Re: [HACKERS] Custom compression methods

On Sat, Mar 16, 2019 at 12:52 AM Ildus Kurbangaliev wrote: > in my opinion this patch is usually skipped not because it is not > needed, but because of its size. It is not hard to maintain it until > commiters will have time for it or I will get actual response that > nobody is going to commit

Re: [HACKERS] Custom compression methods

On 3/19/19 4:44 PM, Chris Travers wrote: > > > On Tue, Mar 19, 2019 at 12:19 PM Tomas Vondra > mailto:tomas.von...@2ndquadrant.com>> wrote: > > > On 3/19/19 10:59 AM, Chris Travers wrote: > > > > > > Not discussing whether any particular committer should pick this > up

Re: [HACKERS] Custom compression methods

On Tue, Mar 19, 2019 at 12:19 PM Tomas Vondra wrote: > > On 3/19/19 10:59 AM, Chris Travers wrote: > > > > > > Not discussing whether any particular committer should pick this up but > > I want to discuss an important use case we have at Adjust for this sort > > of patch. > > > > The PostgreSQL

Re: [HACKERS] Custom compression methods

On 3/19/19 10:59 AM, Chris Travers wrote: > > > On Mon, Mar 18, 2019 at 11:09 PM Tomas Vondra > mailto:tomas.von...@2ndquadrant.com>> wrote: > > > > On 3/15/19 12:52 PM, Ildus Kurbangaliev wrote: > > On Fri, 15 Mar 2019 14:07:14 +0400 > > David Steele

Re: [HACKERS] Custom compression methods

On Mon, Mar 18, 2019 at 11:09 PM Tomas Vondra wrote: > > > On 3/15/19 12:52 PM, Ildus Kurbangaliev wrote: > > On Fri, 15 Mar 2019 14:07:14 +0400 > > David Steele wrote: > > > >> On 3/7/19 11:50 AM, Alexander Korotkov wrote: > >>> On Thu, Mar 7, 2019 at 10:43 AM David Steele >>>

Re: [HACKERS] Custom compression methods

On 3/15/19 12:52 PM, Ildus Kurbangaliev wrote: > On Fri, 15 Mar 2019 14:07:14 +0400 > David Steele wrote: > >> On 3/7/19 11:50 AM, Alexander Korotkov wrote: >>> On Thu, Mar 7, 2019 at 10:43 AM David Steele >> > wrote: >>> >>> On 2/28/19 5:44 PM, Ildus

Re: Re: Re: [HACKERS] Custom compression methods

On Fri, Mar 15, 2019 at 6:07 PM David Steele wrote: > On 3/7/19 11:50 AM, Alexander Korotkov wrote: > > On Thu, Mar 7, 2019 at 10:43 AM David Steele > > wrote: > > > > On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote: > > > > > there are another set of patches.

Re: Re: Re: [HACKERS] Custom compression methods

On 3/7/19 11:50 AM, Alexander Korotkov wrote: On Thu, Mar 7, 2019 at 10:43 AM David Steele > wrote: On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote: > there are another set of patches. > Only rebased to current master. > > Also I will change

Re: Re: [HACKERS] Custom compression methods

On Thu, Mar 7, 2019 at 10:43 AM David Steele wrote: > On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote: > > > there are another set of patches. > > Only rebased to current master. > > > > Also I will change status on commitfest to 'Needs review'. > > This patch has seen periodic rebases but no code

Re: Re: [HACKERS] Custom compression methods

On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote: there are another set of patches. Only rebased to current master. Also I will change status on commitfest to 'Needs review'. This patch has seen periodic rebases but no code review that I can see since last January 2018. As Andres noted in

Re: [HACKERS] Custom compression methods

On Mon, Dec 03, 2018 at 03:43:32PM +0300, Ildus Kurbangaliev wrote: > Hi, here is a rebased version. I hope it will get some review :) This patch set is failing to apply, so moved to next CF, waiting for author. -- Michael signature.asc Description: PGP signature

Re: [HACKERS] Custom compression methods

> On Thu, Sep 6, 2018 at 5:27 PM Ildus Kurbangaliev > wrote: > > Hi, attached latest set of patches. Rebased and fixed pg_upgrade errors > related with zlib support. Thank you for working on this patch, I believe the ideas mentioned in this thread are quite important for Postgres improvement.

Re: [HACKERS] Custom compression methods

Hi! On Mon, Jul 2, 2018 at 3:56 PM Ildus Kurbangaliev wrote: > On Mon, 18 Jun 2018 17:30:45 +0300 > Ildus Kurbangaliev wrote: > > > On Tue, 24 Apr 2018 14:05:20 +0300 > > Alexander Korotkov wrote: > > > > > > > > Yes, this patch definitely lacks of good usage example. That may > > > lead to

Re: [HACKERS] Custom compression methods

On Mon, Apr 23, 2018 at 12:34 PM, Konstantin Knizhnik wrote: > May be. But in any cases, there are several direction where compression can > be used: > - custom compression algorithms > - libpq compression > - page level compression > ... > > and them should be somehow finally "married" with

Re: [HACKERS] Custom compression methods

On Mon, Apr 23, 2018 at 7:34 PM, Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > IMHO end-user do not have skills and time to create their own compression > algorithms. And without knowledge of specific of particular data set, > it is very hard to implement something more efficient than

Re: [HACKERS] Custom compression methods

On Mon, 23 Apr 2018 19:34:38 +0300 Konstantin Knizhnik wrote: > > > Sorry, I really looking at this patch under the different angle. > And this is why I have some doubts about general idea. > Postgres allows to defined custom types, access methods,... > But do you

Re: [HACKERS] Custom compression methods

On 23.04.2018 18:32, Alexander Korotkov wrote: But that the main goal of this patch: let somebody implement own compression algorithm which best fit for particular dataset. Hmmm...Frankly speaking I don't believe in this "somebody". From my point of view the main value of this patch

Re: [HACKERS] Custom compression methods

On Mon, Apr 23, 2018 at 12:40 PM, Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > On 22.04.2018 16:21, Alexander Korotkov wrote: > > On Fri, Apr 20, 2018 at 7:45 PM, Konstantin Knizhnik < > k.knizh...@postgrespro.ru> wrote: > >> On 30.03.2018 19:50, Ildus Kurbangaliev wrote: >> >>> On

Re: [HACKERS] Custom compression methods

On 22.04.2018 16:21, Alexander Korotkov wrote: On Fri, Apr 20, 2018 at 7:45 PM, Konstantin Knizhnik > wrote: On 30.03.2018 19:50, Ildus Kurbangaliev wrote: On Mon, 26 Mar 2018 20:38:25 +0300 Ildus Kurbangaliev

Re: [HACKERS] Custom compression methods

On Sun, 22 Apr 2018 16:21:31 +0300 Alexander Korotkov wrote: > On Fri, Apr 20, 2018 at 7:45 PM, Konstantin Knizhnik < > k.knizh...@postgrespro.ru> wrote: > > > On 30.03.2018 19:50, Ildus Kurbangaliev wrote: > > > >> On Mon, 26 Mar 2018 20:38:25 +0300 > >> Ildus

Re: [HACKERS] Custom compression methods

On Fri, Apr 20, 2018 at 7:45 PM, Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > On 30.03.2018 19:50, Ildus Kurbangaliev wrote: > >> On Mon, 26 Mar 2018 20:38:25 +0300 >> Ildus Kurbangaliev wrote: >> >> Attached rebased version of the patch. Fixed

Re: [HACKERS] Custom compression methods

On 30.03.2018 19:50, Ildus Kurbangaliev wrote: On Mon, 26 Mar 2018 20:38:25 +0300 Ildus Kurbangaliev wrote: Attached rebased version of the patch. Fixed conflicts in pg_class.h. New rebased version due to conflicts in master. Also fixed few errors and

<    1   2   3   4   >