Re: Pluggable toaster

2023-06-16 Thread Aleksander Alekseev
Hi, > Pretty sure that reopening an already rejected patch that is competing > with compression dictionaries (which the rest of us are currently > focusing on) will achieve anything. Ooops, I didn't mean to be sarcastic: s/will achieve/will not achieve/ My apologies. > Consider joining the

Re: Pluggable toaster

2023-06-16 Thread Aleksander Alekseev
Hi Nikita, > We already have a lot of changes in Pluggable TOAST that were not committed > to the main GIT branch of this thread, so it seems that I have to merge them > and > reopen it. Pretty sure that reopening an already rejected patch that is competing with compression dictionaries (which

Re: Pluggable toaster

2023-06-16 Thread Nikita Malakhov
Hi, Seems that I missed the thread mentioned above. I strongly disagree with such statement, Pluggable TOAST could not be a part or Compression Dictionaries thread because the TOAST improvement is a more general subject, it involves much deeper and tricky changes in the core. And also is much

Re: Pluggable toaster

2023-06-14 Thread Aleksander Alekseev
Hi, > > I have a question for the community - why was this patch silently put to a > > "rejected" status? > > Should there no be some kind of explanation? I wouldn't say that it happened "silently" nor that the reason is so mysterious [1]. [1]:

Re: Pluggable toaster

2023-06-14 Thread Pavel Borisov
On Wed, 14 Jun 2023 at 14:05, Nikita Malakhov wrote: > > Hi! > > I have a question for the community - why was this patch silently put to a > "rejected" status? > Should there no be some kind of explanation? > > During this discussion I got the impression that for some reason some members > of

Re: Pluggable toaster

2023-06-14 Thread Nikita Malakhov
Hi! I have a question for the community - why was this patch silently put to a "rejected" status? Should there no be some kind of explanation? During this discussion I got the impression that for some reason some members of the community do not want the TOAST functionality, which has such

Re: Pluggable toaster

2023-02-07 Thread Pavel Borisov
Hi, hackers! Maybe I've read the thread too superficially, but for me, it seems like more of a discussion on what TOAST should NOT be. Maybe someone more in the topic could explain what is the consensus on what we require and what we like to to have in a new TOAST? For me, a good toast should be

Re: Pluggable toaster

2023-02-07 Thread Aleksander Alekseev
Hi, > I believe that is a misrepresentation of the situation. ZSON had > (has?) several systemic issues and could not be accepted to /contrib/ > in the way it was implemented, and it was commented that it would make > sense that the feature of compression assisted by dictionaries would > be

Re: Pluggable toaster

2023-02-06 Thread Nikita Malakhov
Hi, It'll be great to see more opinions on the approach as a whole. >The problem that I see being raised here, is that there was little >discussion and no observed community consensus about the design of >this complex feature *before* this patch with high complexity was >provided. >The next

Re: Pluggable toaster

2023-02-06 Thread Matthias van de Meent
On Mon, 6 Feb 2023 at 15:38, Aleksander Alekseev wrote: > I would like to point out however that there were several other pieces > of feedback that could have been missed: > > * No one wants to see this as an extension. This was my original > proposal (adding ZSON to /contrib/) and it was

Re: Pluggable toaster

2023-02-06 Thread Matthias van de Meent
On Mon, 6 Feb 2023 at 20:24, Andres Freund wrote: > > Hi, > > On 2023-02-06 16:38:01 +0300, Nikita Malakhov wrote: > > So we decided to reduce changes in the core to the minimum necessary > > to make it available through the hooks, because the hooks part is very > > lightweight and simple to keep

Re: Pluggable toaster

2023-02-06 Thread Andres Freund
Hi, On 2023-02-06 16:38:01 +0300, Nikita Malakhov wrote: > So we decided to reduce changes in the core to the minimum necessary > to make it available through the hooks, because the hooks part is very > lightweight and simple to keep rebasing onto the vanilla core. At least I don't think we

Re: Pluggable toaster

2023-02-06 Thread Aleksander Alekseev
Hi, > The main reason behind this decision is that keeping the first implementation > on the side of the vanilla (I mean rebasing it) over time is very difficult > due > to the very invasive nature of this solution. > > So we decided to reduce changes in the core to the minimum necessary > to

Re: Pluggable toaster

2023-02-06 Thread Nikita Malakhov
Hi! Existing TOAST has several very painful drawbacks - lack of UPDATE operation, bloating of TOAST tables, and limits that are implicitly implied on base tables by their TOAST tables, so it is seems not fair to say that Pluggable TOAST does not solve any problems but just introduces new ones.

Re: Pluggable toaster

2023-02-06 Thread Alvaro Herrera
On 2023-Feb-06, Nikita Malakhov wrote: > Currently we're busy revising the whole Pluggable TOAST API to make it > available as an extension and based on hooks to minimize changes in > the core. It will be available soon. Hmm, I'm not sure why would PGDG want to accept such a thing. I read

Re: Pluggable toaster

2023-02-06 Thread Aleksander Alekseev
Hi, > > So we're really quite surprised that it has got so little feedback. We've > > got > > some opinions on approach but there is no any general one on the approach > > itself except doubts about the TOAST mechanism needs revision at all. > > The problem for me is that what you've been posting

Re: Pluggable toaster

2023-02-05 Thread Andres Freund
Hi, On 2023-02-06 00:10:50 +0300, Nikita Malakhov wrote: > The problem, actually, is that the default TOAST is often not good for > modern loads and amounts of data.Pluggable TOAST is based not only > on pure enthusiasm, but on demands and tickets from production > databases. > The main demand

Re: Pluggable toaster

2023-02-05 Thread Nikita Malakhov
Hi hackers! In response to opinion in thread on Compresson dictionaries for JSONb [1] >The approaches are completely different, >but they seem to be trying to fix the same problem: the fact that the >default TOAST stuff isn't good enough for JSONB. The problem, actually, is that the default

Re: Pluggable toaster

2023-02-03 Thread Alvaro Herrera
On 2023-Jan-14, Nikita Malakhov wrote: > Hi! > Fails due to recent changes. Working on it. Please see my response here https://postgr.es/m/20230203095540.zutul5vmsbmantbm@alvherre.pgsql -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "Oh, great altar of passive

Re: Pluggable toaster

2023-01-14 Thread Nikita Malakhov
Hi! Fails due to recent changes. Working on it. On Sat, Jan 14, 2023 at 9:56 AM vignesh C wrote: > On Sun, 8 Jan 2023 at 01:40, Nikita Malakhov wrote: > > > > Hi! > > > > Thank you for your attention. > > I've rebased the patchset onto the latest master (from 07.01), but the > second part is

Re: Pluggable toaster

2023-01-13 Thread vignesh C
On Sun, 8 Jan 2023 at 01:40, Nikita Malakhov wrote: > > Hi! > > Thank you for your attention. > I've rebased the patchset onto the latest master (from 07.01), but the second > part is still > in pre-patch shape - it is working, but tests are failing due to changes in > TOAST relations > logic -

Re: Pluggable toaster

2023-01-04 Thread vignesh C
On Tue, 27 Dec 2022 at 02:32, Nikita Malakhov wrote: > > Hi hackers! > > Pluggable TOAST API with catalog control table PG_TOASTREL - pre-patch. > > Pluggable TOAST - TOAST API rework - introduce PG_TOASTREL catalog > relation containing TOAST dependencies. NOTE: here is a pre-patch,

Re: Pluggable toaster

2022-12-15 Thread Nikita Malakhov
Hi hackers! I want to bump this thread and remind the community about Pluggable TOAST. Overall Pluggable TOAST was revised to work without altering PG_ATTRIBUTE table - no atttoaster column, allow to drop unused Toasters and some other improvements. Sorry for the big delay, but it was a big piece

Re: Pluggable toaster

2022-11-07 Thread Aleksander Alekseev
Hi Nikita, > Pluggable TOAST is provided as an interface to allow developers to plug > in custom TOAST mechanics. It does not determines would it be universal > Toaster or one data type, but syntax for universal Toaster is out of scope > for this patchset. If I understand correctly, what is

Re: Pluggable toaster

2022-11-06 Thread Nikita Malakhov
Hi, Pluggable TOAST is provided as an interface to allow developers to plug in custom TOAST mechanics. It does not determines would it be universal Toaster or one data type, but syntax for universal Toaster is out of scope for this patchset. >True, but besides Jsonb Toaster one can implement a

Re: Pluggable toaster

2022-11-04 Thread Aleksander Alekseev
Hi again, > [1]: > https://www.postgresql.org/message-id/flat/CAJ7c6TOtAB0z1UrksvGTStNE-herK-43bj22=5xvbg7s4vr...@mail.gmail.com I added a link but forgot to use it, d'oh! Please note that if the goal is to merely implement "type-aware sub-TOASTers" then compression dictionaries [1] arguably

Re: Pluggable toaster

2022-11-04 Thread Aleksander Alekseev
Hi Nikita, > Setting TOAST for table and database is a subject for discussion. There is > already default > Toaster. Also, there is not much sense in setting Jsonb Toaster as default > even for table, do > not say database, because table could contain other TOASTable columns not of > Json

Re: Pluggable toaster

2022-11-03 Thread Nikita Malakhov
Hi, Setting TOAST for table and database is a subject for discussion. There is already default Toaster. Also, there is not much sense in setting Jsonb Toaster as default even for table, do not say database, because table could contain other TOASTable columns not of Json type. To be able to set

Re: Pluggable toaster

2022-11-03 Thread Aleksander Alekseev
Hi Nikita, Please, avoid top-posting [1]. > Toaster is set for the table column. Each TOASTable column could have > a different Toaster, so column option is the most obvious place to add it. This is a major limitation. IMO the user should be able to set a custom TOASTer for the entire table as

Re: Pluggable toaster

2022-11-03 Thread Nikita Malakhov
Hi! Thank you very much for the feedback. Answering accordingly to questions/remarks order: >I'm in the process of testing and reviewing the v23 patchset. So far I >just wanted to report that there seems to be certain issues with the >formatting and/or trailing whitespaces in the patches:

Re: Pluggable toaster

2022-11-03 Thread Aleksander Alekseev
Hi Nikita, > I'm going to submit a more detailed code review soon. As promised, here is my feedback. ``` + Toaster could be assigned to toastable column with expression + STORAGE external TOASTER toaster_name ``` 1.1. Could you please clarify what is the motivation behind such a

Re: Pluggable toaster

2022-11-01 Thread Aleksander Alekseev
Hi Nikita, > Please don't forget to run `pgindent` before formatting the patches > with `git format-patch` next time. There are also some compiler warnings, please see the attachment. > I'm going to submit a more detailed code review soon. -- Best regards, Aleksander Alekseev

Re: Pluggable toaster

2022-11-01 Thread Aleksander Alekseev
Hi Nikita, > Aleksander, thanks for the discussion! It seems to me that I have to add some > parts > of it to API documentation, to clarify the details on API purpose and > use-cases. I'm in the process of testing and reviewing the v23 patchset. So far I just wanted to report that there seems

Re: Pluggable toaster

2022-10-24 Thread Nikita Malakhov
Hi, Aleksander, thanks for the discussion! It seems to me that I have to add some parts of it to API documentation, to clarify the details on API purpose and use-cases. On Mon, Oct 24, 2022 at 6:37 PM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi Nikita, > > > > > TOAST

Re: Pluggable toaster

2022-10-24 Thread Aleksander Alekseev
Hi Nikita, > > > TOAST implementation is not necessary for Table AM. > > >What other use cases for TOAST do you have in mind? > > The main use case is the same as for the TOAST mechanism - storing and > retrieving > oversized data. But we expanded this case with some details - > - update TOASTed

Re: Pluggable toaster

2022-10-24 Thread Nikita Malakhov
Hi! >From personal experience with the project I have serious doubts this >is going to happen. Before such invasive changes are going to be >accepted there should be a clear understanding of how exactly TOASTers >are supposed to be used. This should be part of the documentation in >the patchset.

Re: Pluggable toaster

2022-10-24 Thread Aleksander Alekseev
Hi Nikita, > Using Table AM Routine and routing AM methods calls via it is a topic for > further discussion, > if Pluggable TOAST will be committed. [...] And even then it would be an open > issue. >From personal experience with the project I have serious doubts this is going to happen. Before

Re: Pluggable toaster

2022-10-24 Thread Nikita Malakhov
Hi, >This is exactly the point. In order to not to create a silver bullet, >TOASTers should be limited to a single TableAM. The one we know uses >pages of a known fixed size, the one that actually requires TOAST >because pages are relatively small, etc. Currently all our TOAST implementations

Re: Pluggable toaster

2022-10-24 Thread Aleksander Alekseev
Hi Nikita, > >I don't argue with most of what you say. I am just pointing out the > >reason why the chosen approach "N TOASTers x M TableAMs" will not > >work: > > We assume that TAM used in custom Toaster works as it is should work, > and leave TAM internals to this TAM developer - say, we do

Re: Pluggable toaster

2022-10-24 Thread Nikita Malakhov
Hi! >I don't argue with most of what you say. I am just pointing out the >reason why the chosen approach "N TOASTers x M TableAMs" will not >work: We assume that TAM used in custom Toaster works as it is should work, and leave TAM internals to this TAM developer - say, we do not want to change

Re: Pluggable toaster

2022-10-24 Thread Aleksander Alekseev
Hi Nikita, I don't argue with most of what you say. I am just pointing out the reason why the chosen approach "N TOASTers x M TableAMs" will not work: > Don't you think that this is an arguable design decision? Basically > all we know about the underlying TableAM is that it stores tuples >

Re: Pluggable toaster

2022-10-23 Thread Nikita Malakhov
Hi! Aleksander, >Don't you think that this is an arguable design decision? Basically >all we know about the underlying TableAM is that it stores tuples >_somehow_ and that tuples have TIDs [1]. That's it. We don't know if >it even has any sort of pages, whether they are fixed in size or not,

Re: Pluggable toaster

2022-10-23 Thread Aleksander Alekseev
Hi Nikita, > Pluggable TOAST API was designed with storage flexibility in mind, and Custom > TOAST mechanics is > free to use any storage methods Don't you think that this is an arguable design decision? Basically all we know about the underlying TableAM is that it stores tuples _somehow_ and

Re: Pluggable toaster

2022-10-22 Thread Nikita Malakhov
Hi! Aleksander, this is a good question. If I understood you correctly, you mean that the alternative TOAST mechanism B is using a specific Table AM A? Pluggable TOAST API was designed with storage flexibility in mind, and Custom TOAST mechanics is free to use any storage methods - we've tested

Re: Pluggable toaster

2022-10-22 Thread Aleksander Alekseev
Hi Nikita, > Aleksander, we have had this in mind while developing this feature, and have > checked it. Just a slight modification is needed > to make it work with Pluggable Storage (Access Methods) API. Could you please clarify this a little from the architectural point of view? Let's say

Re: Pluggable toaster

2022-10-21 Thread Nikita Malakhov
Hi! Aleksander, we have had this in mind while developing this feature, and have checked it. Just a slight modification is needed to make it work with Pluggable Storage (Access Methods) API. On Fri, Oct 21, 2022 at 4:01 PM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi Nikita, > >

Re: Pluggable toaster

2022-10-21 Thread Aleksander Alekseev
Hi Nikita, > Here's rebased patch: > v23-0001-toaster-interface.patch - Pluggable TOAST API interface with > reference (original) TOAST mechanics - new API is introduced but > reference TOAST is still left unchanged; > v23-0002-toaster-default.patch - Default TOAST mechanics is re-implemented >

Re: Pluggable toaster

2022-10-13 Thread Nikita Malakhov
Hi hackers, Fixed warning that failed build in previous patchset. Here's rebased patch: v23-0001-toaster-interface.patch - Pluggable TOAST API interface with reference (original) TOAST mechanics - new API is introduced but reference TOAST is still left unchanged; v23-0002-toaster-default.patch -

Re: Pluggable toaster

2022-10-12 Thread Nikita Malakhov
Hi hackers! Due to recent changes in postgres.h cfbot is failing again. Here's rebased patch: v22-0001-toaster-interface.patch - Pluggable TOAST API interface with reference (original) TOAST mechanics - new API is introduced but reference TOAST is still left unchanged;

Re: Pluggable toaster

2022-10-04 Thread Nikita Malakhov
Hi hackers! cfbot is unhappy again, with documentation package.Here's corrected patchset. Patchset consists of: v21-0001-toaster-interface.patch - Pluggable TOAST API interface along with reference TOAST mechanics - new API is introduced but reference TOAST is still unchanged;

Re: Pluggable toaster

2022-10-04 Thread Nikita Malakhov
Hi hackers! Now cfbot is happy, but there were warnings due to recent changes in PointerGetDatum function, so here's corrected patchset. Sorry, forgot to attach patch files. My fault. Patchset consists of: v20-0001-toaster-interface.patch - Pluggable TOAST API interface along with reference TOAST

Re: Pluggable toaster

2022-10-04 Thread Nikita Malakhov
Hi hackers! Now cfbot is happy, but there were warnings due to recent changes in PointerGetDatum function, so here's corrected patchset. Patchset consists of: v20-0001-toaster-interface.patch - Pluggable TOAST API interface along with reference TOAST mechanics - new API is introduced but

Re: Pluggable toaster

2022-10-03 Thread Nikita Malakhov
Hi hackers! Cfbot failed in meson build with previous patchsets, so I've rebased them onto the latest master and added necessary meson build info. Patchset consists of: v19-0001-toaster-interface.patch - Pluggable TOAST API interface along with reference TOAST mechanics - new API is introduced

Re: Pluggable toaster

2022-09-26 Thread Nikita Malakhov
Hi, Meson build for the patchset failed, meson build files attached and README/Doc package reworked with more detailed explanation of virtual function table along with other corrections. On Sun, Sep 25, 2022 at 1:41 AM Nikita Malakhov wrote: > Hi hackers! > Last patchset has an invalid patch

Re: Pluggable toaster

2022-09-24 Thread Nikita Malakhov
Hi hackers! Last patchset has an invalid patch file - v16-0003-toaster-docs.patch. Here's corrected patchset, sorry for the noise. On Sat, Sep 24, 2022 at 3:50 PM Nikita Malakhov wrote: > Hi hackers! > > Cfbot is still not happy with the patchset, so I'm attaching a rebased > one, rebased onto

Re: Pluggable toaster

2022-09-24 Thread Nikita Malakhov
Hi hackers! Cfbot is still not happy with the patchset, so I'm attaching a rebased one, rebased onto the current master (from today). The third patch contains documentation package, and the second one contains large README.toastapi file providing additional in-depth docs for developers. Comments

Re: Pluggable toaster

2022-09-23 Thread Nikita Malakhov
Hi hackers! Cfbot is not happy with previous patchset, so I'm attaching new one, rebased onto current master (15b4). Also providing patch with documentation package, and the second one contains large README.toastapi file providing additional in-depth docs for developers. Comments would be

Re: Pluggable toaster

2022-09-13 Thread Jacob Champion
On Mon, Sep 12, 2022 at 11:45 PM Nikita Malakhov wrote: > It would be more clear for complex data types like JSONB, where developers > could > need some additional functionality to work with internal representation of > data type, > and its full potential is revealed in our JSONB toaster

Re: Pluggable toaster

2022-09-13 Thread Nikita Malakhov
Hi hackers! Jacob, I agree that the bytea toaster makes a bad example due to core modification, and actually is not a good example of an extension. The vtable concept is intended for less invasive additional functionality - like providing detoast iterators in addition to standard detoast

Re: Pluggable toaster

2022-09-12 Thread Jacob Champion
On Wed, Aug 24, 2022 at 2:59 AM Nikita Malakhov wrote: > I've rebased actual branch onto the latest master and re-created patches. > Checked with git am, > all applied correctly. Please check the attached patches. > Rebased branch resides here: >

Re: Pluggable toaster

2022-08-24 Thread Nikita Malakhov
Hi hackers! I've rebased actual branch onto the latest master and re-created patches. Checked with git am, all applied correctly. Please check the attached patches. Rebased branch resides here: https://github.com/postgrespro/postgres/tree/toasterapi_clean Just to remind - I've decided to leave

Re: Pluggable toaster

2022-08-23 Thread Aleksander Alekseev
Hi Nikita, > I've decided to leave only the first 2 patches for review and send less > significant > changes after the main patch will be straightened out. > So, here is > v13-0001-toaster-interface.patch - main TOAST API patch, with reference TOAST > mechanics left as-is. >

Re: Pluggable toaster

2022-08-04 Thread Nikita Malakhov
Hi hackers! I've made a rebase according to Andres and Aleksander comments. Rebased branch resides here: https://github.com/postgrespro/postgres/tree/toasterapi_clean I've decided to leave only the first 2 patches for review and send less significant changes after the main patch will be

Re: Pluggable toaster

2022-08-02 Thread Andres Freund
Hi, On 2022-08-02 09:15:12 +0300, Nikita Malakhov wrote: > Attach includes: > v11-0002-toaster-interface.patch - contains TOAST API with default Toaster > left as-is (reference implementation) and Dummy toaster as an example > (will be removed later as a part of refactoring?). > >

Re: Pluggable toaster

2022-08-02 Thread Nikita Malakhov
Hi hackers! Sorry for the delay. I've made changes according to Aleksander comments: >This will produce a patchset with a consistent naming like: >... >Also cfbot [1] will know in which order to apply them. Done. >Unfortunately the three patches in question from this branch don't >pass `make

Re: Pluggable toaster

2022-07-29 Thread Nikita Malakhov
Hi hackers! Aleksander, thanks for the remark, seems that we've missed recent change - the pubication test does not have the new column 'Toaster'. Will send a corrected patch tomorrow. Also, thanks for the patch name note, I've changed it as you suggested. I'm on vacation, so I read emails not

Re: Pluggable toaster

2022-07-26 Thread Aleksander Alekseev
Hi Nikita, Thanks for an update! > 0002_toaster_interface_v10 contains TOAST API with Dummy toaster as an > example (but I would, > as recommended, remove Dummy toaster and provide it as an extension), and > default Toaster > was left as-is (reference implementation). > >

Re: Pluggable toaster

2022-07-25 Thread Nikita Malakhov
Hi hackers! Here is the patch set rebased to master from 22.07. I've got some trouble rebasing it due to conflicts, so it took some time. I've made some corrections according to Matthias and Aleksander comments, though not all of them, because some require refactoring of very old code and would

Re: Pluggable toaster

2022-07-23 Thread Nikita Malakhov
Hi hackers! Matthias, thank you very much for your feedback! Sorry, I forgot to attach files. Attaching here, but they are for the commit tagged "15beta2", I am currently rebasing this branch onto the actual master and will provide rebased version, with some corrections according to your

Re: Pluggable toaster

2022-07-22 Thread Matthias van de Meent
On Wed, 20 Jul 2022 at 11:16, Nikita Malakhov wrote: > > Hi hackers! Hi, Please don't top-post here. See https://wiki.postgresql.org/wiki/Mailing_Lists#Email_etiquette_mechanics. > We really need your feedback on the last patchset update! This is feedback on the latest version that was

Re: Pluggable toaster

2022-07-22 Thread Aleksander Alekseev
Hi Nikita, > I've reworked the patch set according to recommendations of Aleksander > Alekseev, Robert Haas > and Matthias van de Meent, and decided, as it was recommended earlier, > include only the most > important part in the first set. Also, I've added a large README on Pluggable > TOAST

Re: Pluggable toaster

2022-07-22 Thread Nikita Malakhov
Hi hackers! I've reworked the patch set according to recommendations of Aleksander Alekseev, Robert Haas and Matthias van de Meent, and decided, as it was recommended earlier, include only the most important part in the first set. Also, I've added a large README on Pluggable TOAST to sources,

Re: Pluggable toaster

2022-07-20 Thread Nikita Malakhov
Hi hackers! We really need your feedback on the last patchset update! On a previous question about TOAST API overhead - please check script in attach, we tested INSERT, UPDATE and SELECT operations, and ran it on vanilla master and on patched master (vanilla with untouched TOAST implementation

Re: Pluggable toaster

2022-07-11 Thread Nikita Malakhov
Hi! We have branch with incremental commits worm where patches were generated with format-patch - https://github.com/postgrespro/postgres/tree/toasterapi_clean I'll clean up commits from garbage files asap, sorry, haven't noticed them while moving changes. Best regards, Nikita Malakhov On Fri,

Re: Pluggable toaster

2022-07-01 Thread Matthias van de Meent
On Thu, 30 Jun 2022 at 22:26, Nikita Malakhov wrote: > > Hi hackers! > Here is the patch set rebased onto current master (15 rel beta 2 with commit > from 29.06). Thanks! > Just to remind: > With this patch set you will be able to develop and plug in your own TOAST > mechanics for table

Re: Pluggable toaster

2022-07-01 Thread Nikita Malakhov
Hi, Alexander, thank you for your feedback! I'd explain our thoughts: We thought that refactoring default TOAST mechanics via TOAST API in p. 0002 would be too much because the API itself already introduced a lot of changes, so we kept Default Toasters re-implementation for later patch. 0002

Re: Pluggable toaster

2022-07-01 Thread Aleksander Alekseev
Hi Nikita, > Here is the patch set rebased onto current master (15 rel beta 2 with commit > from 29.06). Thanks for the rebased patchset. This is a huge effort though. I suggest splitting it into several CF entries as we previously did with other patches (64-bit XIDs to name one, which BTW is

Re: Pluggable toaster

2022-06-30 Thread Nikita Malakhov
Rebased onto 15 REL BETA 2

Re: Pluggable toaster

2022-06-23 Thread Nikita Malakhov
Hi, Alexander, thank you for your feedback and willingness to help. You can send a suggested fixup in this thread, I'll check the issue you've mentioned. Best regards, Nikita Malakhov On Thu, Jun 23, 2022 at 4:38 PM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi Nikita, > > > We're

Re: Pluggable toaster

2022-06-23 Thread Aleksander Alekseev
Hi Nikita, > We're currently working on rebase along other TOAST improvements, hope to do > it in time for July CF. > Thank you for your patience. Just to clarify, does it include the dependent "CREATE TABLE ( .. STORAGE .. )" patch [1]? I was considering changing the patch according to the

Re: Pluggable toaster

2022-06-23 Thread Nikita Malakhov
i hackers, > > > For a pluggable toaster - in previous patch set part 7 patch file > contains invalid string. > > Fixup (v2 file should used instead of previous) patch: > > 7) 0007_fix_alignment_of_custom_toast_pointers.patch - fixes custom > toast pointer's > > alignment required

Re: Pluggable toaster

2022-06-17 Thread Aleksander Alekseev
Hi hackers, > For a pluggable toaster - in previous patch set part 7 patch file contains > invalid string. > Fixup (v2 file should used instead of previous) patch: > 7) 0007_fix_alignment_of_custom_toast_pointers.patch - fixes custom toast > pointer's > alignment require

Re: Pluggable toaster

2022-04-13 Thread Nikita Malakhov
Hi, For a pluggable toaster - in previous patch set part 7 patch file contains invalid string. Fixup (v2 file should used instead of previous) patch: 7) 0007_fix_alignment_of_custom_toast_pointers.patch - fixes custom toast pointer's alignment required by bytea toaster by Nikita Glukhov; On Wed

Re: Pluggable toaster

2022-04-05 Thread Nikita Malakhov
Hi, Thanks for advices. We have 4 branches, for each patch provided, you can check them out - (come copy-paste from the very fist email, where the patches were proposed) 1) 1_toaster_interface https://github.com/postgrespro/postgres/tree/toaster_interface Introduces syntax for storage and formal

Re: Pluggable toaster

2022-04-04 Thread Robert Haas
On Mon, Apr 4, 2022 at 4:05 PM Nikita Malakhov wrote: > - Is 'git apply' not a valid way to apply such patches? I have found that it never works. This case is no exception: [rhaas pgsql]$ git apply ~/Downloads/1_toaster_interface_v4.patch /Users/rhaas/Downloads/1_toaster_interface_v4.patch:253:

Re: Pluggable toaster

2022-04-04 Thread Nikita Malakhov
Hi, Thank you very much for your review, I'd like to get it much earlier. I'm currently working on cleaning up code, and would try to comment as much as possible. This patch set is really a large set of functionality, it was divided into 4 logically complete parts, but anyway these parts contain a

Re: Pluggable toaster

2022-04-04 Thread Greg Stark
Thanks for the review Robert. I think that gives some pretty actionable advice on how to improve the patch and it doesn't seem likely to get much more in this cycle. I'll mark the patch Returned with Feedback. Hope to see it come back with improvements in the next release.

Re: Pluggable toaster

2022-04-04 Thread Robert Haas
On Mon, Apr 4, 2022 at 12:13 PM Nikita Malakhov wrote: > I'm really sorry. Messed up some code while merging rebased branches with > previous (v1) > patches issued in December and haven't noticed that seg fault because of > corrupted code > while running check-world. > I've fixed messed code in

Re: Pluggable toaster

2022-04-03 Thread Nikita Malakhov
Hi, I'm checking. It seems that I've missed something while rebasing, we have had all tests clean before. On Sun, Apr 3, 2022 at 5:06 AM Andres Freund wrote: > Hi, > > On April 2, 2022 6:20:36 PM PDT, Greg Stark wrote: > >Hm. It compiles but it's failing regression tests: > > > >diff -U3 >

Re: Pluggable toaster

2022-04-02 Thread Andres Freund
Hi, On April 2, 2022 6:20:36 PM PDT, Greg Stark wrote: >Hm. It compiles but it's failing regression tests: > >diff -U3 /tmp/cirrus-ci-build/contrib/dummy_toaster/expected/dummy_toaster.out >/tmp/cirrus-ci-build/contrib/dummy_toaster/results/dummy_toaster.out >---

Re: Pluggable toaster

2022-04-02 Thread Greg Stark
Hm. It compiles but it's failing regression tests: diff -U3 /tmp/cirrus-ci-build/contrib/dummy_toaster/expected/dummy_toaster.out /tmp/cirrus-ci-build/contrib/dummy_toaster/results/dummy_toaster.out --- /tmp/cirrus-ci-build/contrib/dummy_toaster/expected/dummy_toaster.out 2022-04-02

Re: Pluggable toaster

2022-03-31 Thread Greg Stark
It looks like it's still not actually building on some compilers. I see a bunch of warnings as well as an error: [03:53:24.660] dummy_toaster.c:97:2: error: void function 'dummyDelete' should not return a value [-Wreturn-type] Also the "publication" regression test needs to be adjusted as it

Re: Pluggable toaster

2022-03-21 Thread Andres Freund
Hi, On 2022-03-22 02:31:21 +0300, Nikita Malakhov wrote: > Hi Hackers, > Because of 3 months have passed since Pluggable Toaster presentation and a > lot of > commits were pushed into v15 master - we would like to re-introduce > this patch > rebased onto actual master. Last

Re: Pluggable toaster

2022-03-10 Thread Nikita Malakhov
Hi Hackers, In addition to original patch set for Pluggable Toaster, we have two more patches (actually, small, but important fixes), authored by Nikita Glukhov: 1) 0001-Fix-toast_tuple_externalize.patch This patch fixes freeing memory in case of new toasted value is the same as old one

Re: Pluggable toaster

2022-02-01 Thread Teodor Sigaev
I agree ... but I'm also worried about what happens when we have multiple table AMs. One can imagine a new table AM that is specifically optimized for TOAST which can be used with an existing heap table. One can imagine a new table AM for the main table that wants to use something different for

Re: Pluggable toaster

2022-01-20 Thread Nikita Malakhov
Hi, The patch provides assigning toaster to a data column separately. Assigning to a data type is considered worthy but is also a topic for further discussion and is not included in patch. We've been thinking how to integrate AMs and custom toasters, so any thoughts are welcome. Regards, On

Re: Pluggable toaster

2022-01-20 Thread Robert Haas
On Thu, Dec 30, 2021 at 11:40 AM Teodor Sigaev wrote: > We are working on custom toaster for JSONB [1], because current TOAST is > universal for any data type and because of that it has some disadvantages: > - "one toast fits all" may be not the best solution for particular > type

Re: Pluggable toaster

2022-01-19 Thread Nikita Malakhov
Hi, >I'm not convinced that's universally true. Yes, I'm sure certain TOAST >implementations would benefit from tighter control over compression, but >does that imply compression and toast are redundant? I doubt that, >because we compress non-toasted types too, for example. And layering has >a

Re: Pluggable toaster

2022-01-18 Thread Tomas Vondra
On 1/18/22 15:56, Teodor Sigaev wrote: > Hi! > >> Maybe doing that kind of compression in TOAST is somehow simpler, but >> I don't see it. > Seems, in ideal world, compression should be inside toaster. > I'm not convinced that's universally true. Yes, I'm sure certain TOAST implementations

Re: Pluggable toaster

2022-01-18 Thread Teodor Sigaev
Hi! Maybe doing that kind of compression in TOAST is somehow simpler, but I don't see it. Seems, in ideal world, compression should be inside toaster. 2 Current toast storage stores chunks in heap accesses method and to provide fast access by toast id it makes an index. Ideas:    - store

  1   2   >