Re: Compressed pluggable storage experiments

hould ignore this problem, or >> you should fix it. I am working on an environment similar to the above extension(pg_cryogen which experiments pluggable storage api's) but don't have much knowledge on pg's logical replication.. Please suggest some approaches to support pg&#x

Re: Compressed pluggable storage experiments

On Fri, Oct 18, 2019 at 03:25:05AM -0700, Andres Freund wrote: Hi, On 2019-10-17 12:47:47 -0300, Alvaro Herrera wrote: On 2019-Oct-10, Ildar Musin wrote: > 1. Unlike FDW API, in pluggable storage API there are no routines like > "begin modify table" and "end modify t

Re: Compressed pluggable storage experiments

Hi, On 2019-10-17 12:47:47 -0300, Alvaro Herrera wrote: > On 2019-Oct-10, Ildar Musin wrote: > > > 1. Unlike FDW API, in pluggable storage API there are no routines like > > "begin modify table" and "end modify table" and there is no shared > > state

Re: Compressed pluggable storage experiments

On 2019-Oct-10, Ildar Musin wrote: > 1. Unlike FDW API, in pluggable storage API there are no routines like > "begin modify table" and "end modify table" and there is no shared > state between insert/update/delete calls. Hmm. I think adding a begin/end to modifytabl

Compressed pluggable storage experiments

Hi hackers, I've been experimenting with pluggable storage API recently and just feel like I can share my first experience. First of all it's great to have this API and that now community has the opportunity to implement alternative storage engines. There are a few applications that co

Re: Pluggable Storage - Andres's take

On 2019-Jun-06, Heikki Linnakangas wrote: > Pushed the first patch now. Andres already fixed the second issue in commit > b8b94ea129. Please don't omit the "Discussion:" tag in commit messages. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, R

Re: Pluggable Storage - Andres's take

On 18/05/2019 01:19, Ashwin Agrawal wrote: On Tue, Apr 9, 2019 at 6:17 AM Heikki Linnakangas > wrote: On 08/04/2019 20:37, Andres Freund wrote: > On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: >> There's a little bug in index-only scan executor node

Re: Pluggable Storage - Andres's take

Hi, On 2019-05-17 14:49:19 -0700, Ashwin Agrawal wrote: > On Fri, May 17, 2019 at 12:54 PM Andres Freund wrote: > > > Hi, > > > > On 2019-05-15 23:00:38 -0700, Ashwin Agrawal wrote: > > > Highlevel this looks good to me. Will look into full details tomorrow. > > > > Ping? > > > > I'll push the f

Re: Pluggable Storage - Andres's take

Hi, On 2019-05-17 16:56:04 -0700, Ashwin Agrawal wrote: > Question on the patch, if not too late > Why call table_beginscan() in TidNext() and not in ExecInitTidScan() ? > Seems cleaner to have it in ExecInitTidScan(). Largely because it's symmetrical to where most other scans are started ( c.f.

Re: Pluggable Storage - Andres's take

On Wed, May 15, 2019 at 11:54 AM Andres Freund wrote: > Attached is a prototype of a variation of this. I added a > table_tuple_tid_valid(TableScanDesc sscan, ItemPointer tid) > callback / wrapper. Currently it just takes a "plain" scan, but we could > add a separate table_beginscan variant too.

Re: Pluggable Storage - Andres's take

On Tue, Apr 9, 2019 at 6:17 AM Heikki Linnakangas wrote: > On 08/04/2019 20:37, Andres Freund wrote: > > On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: > >> There's a little bug in index-only scan executor node, where it mixes > up the > >> slots to hold a tuple from the index, and from

Re: Pluggable Storage - Andres's take

On Fri, May 17, 2019 at 12:54 PM Andres Freund wrote: > Hi, > > On 2019-05-15 23:00:38 -0700, Ashwin Agrawal wrote: > > Highlevel this looks good to me. Will look into full details tomorrow. > > Ping? > > I'll push the first of the patches soon, and unless you'll comment on > the second soon, I'l

Re: Pluggable Storage - Andres's take

Hi, On 2019-05-15 23:00:38 -0700, Ashwin Agrawal wrote: > Highlevel this looks good to me. Will look into full details tomorrow. Ping? I'll push the first of the patches soon, and unless you'll comment on the second soon, I'll also push ahead. There's a beta upcoming... - Andres

Re: Pluggable Storage - Andres's take

On Wed, May 15, 2019 at 11:54 AM Andres Freund wrote: > Hi, > > On 2019-04-25 15:43:15 -0700, Andres Freund wrote: > > > > 3) nodeTidscan, skipping over too large tids > > >I think this should just be moved into the AMs, there's no need to > > >have this in nodeTidscan.c > > > > I think h

Re: Pluggable Storage - Andres's take

Hi, On 2019-04-25 15:43:15 -0700, Andres Freund wrote: > Hm. I think some of those changes would be a bit bigger than I initially > though. Attached is a more minimal fix that'd route > RelationGetNumberOfBlocksForFork() through tableam if necessary. I > think it's definitely the right answer for

Re: Pluggable Storage - Andres's take

On Wed, May 8, 2019 at 2:46 PM Andres Freund wrote: > > Hi, > > On 2019-05-07 23:18:39 -0700, Ashwin Agrawal wrote: > > On Mon, May 6, 2019 at 1:39 PM Ashwin Agrawal wrote: > > > Also wish to point out, while working on Zedstore, we realized that > > > TupleDesc from Relation object can be truste

Re: Pluggable Storage - Andres's take

Hi, On 2019-05-07 23:18:39 -0700, Ashwin Agrawal wrote: > On Mon, May 6, 2019 at 1:39 PM Ashwin Agrawal wrote: > > Also wish to point out, while working on Zedstore, we realized that > > TupleDesc from Relation object can be trusted at AM layer for > > scan_begin() API. As for ALTER TABLE rewrite

Re: Pluggable Storage - Andres's take

Hi, On 2019-04-29 16:17:41 -0700, Ashwin Agrawal wrote: > On Thu, Apr 25, 2019 at 3:43 PM Andres Freund wrote: > > Hm. I think some of those changes would be a bit bigger than I initially > > though. Attached is a more minimal fix that'd route > > RelationGetNumberOfBlocksForFork() through tablea

Re: Pluggable Storage - Andres's take

On Mon, May 6, 2019 at 1:39 PM Ashwin Agrawal wrote: > > Also wish to point out, while working on Zedstore, we realized that > TupleDesc from Relation object can be trusted at AM layer for > scan_begin() API. As for ALTER TABLE rewrite case (ATRewriteTables()), > catalog is updated first and hence

Re: Pluggable Storage - Andres's take

On Mon, 6 May 2019 at 22:39, Ashwin Agrawal wrote: > > On Mon, May 6, 2019 at 7:14 AM Andres Freund wrote: > > > > Hi, > > > > On May 6, 2019 3:40:55 AM PDT, Rafia Sabih > > wrote: > > >I was trying the toyam patch and on make check it failed with > > >segmentation fault at > > > > > >static vo

Re: Pluggable Storage - Andres's take

On Mon, 6 May 2019 at 16:14, Andres Freund wrote: > > Hi, > > On May 6, 2019 3:40:55 AM PDT, Rafia Sabih wrote: > >On Tue, 9 Apr 2019 at 15:17, Heikki Linnakangas > >wrote: > >> > >> On 08/04/2019 20:37, Andres Freund wrote: > >> > On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: > >> >>

Re: Pluggable Storage - Andres's take

On Mon, May 6, 2019 at 7:14 AM Andres Freund wrote: > > Hi, > > On May 6, 2019 3:40:55 AM PDT, Rafia Sabih wrote: > >I was trying the toyam patch and on make check it failed with > >segmentation fault at > > > >static void > >toyam_relation_set_new_filenode(Relation rel, > > char persistence, > >

Re: Pluggable Storage - Andres's take

Hi, On May 6, 2019 3:40:55 AM PDT, Rafia Sabih wrote: >On Tue, 9 Apr 2019 at 15:17, Heikki Linnakangas >wrote: >> >> On 08/04/2019 20:37, Andres Freund wrote: >> > On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: >> >> There's a little bug in index-only scan executor node, where it >mixes

Re: Pluggable Storage - Andres's take

On Tue, 9 Apr 2019 at 15:17, Heikki Linnakangas wrote: > > On 08/04/2019 20:37, Andres Freund wrote: > > On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: > >> There's a little bug in index-only scan executor node, where it mixes up > >> the > >> slots to hold a tuple from the index, and fr

Re: Pluggable Storage - Andres's take

On Thu, Apr 25, 2019 at 3:43 PM Andres Freund wrote: > Hm. I think some of those changes would be a bit bigger than I initially > though. Attached is a more minimal fix that'd route > RelationGetNumberOfBlocksForFork() through tableam if necessary. I > think it's definitely the right answer for 1

Re: Pluggable Storage - Andres's take

Hi Heikki, Ashwin, Tom, On 2019-04-23 15:52:01 -0700, Andres Freund wrote: > On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: > > index_update_stats() calls RelationGetNumberOfBlocks(). If the AM > > doesn't use normal data files, that won't work. I bumped into that with my > > toy implemen

Re: Pluggable Storage - Andres's take

On Tue, Apr 23, 2019 at 6:55 PM Tom Lane wrote: > Andres Freund writes: > > ... I think none of these are critical issues for tableam, but we should fix > > them. > > > I'm not sure about doing so for v12 though. 1) and 3) are fairly > > trivial, but 2) would involve changing the FDW interface, b

Re: Pluggable Storage - Andres's take

Andres Freund writes: > ... I think none of these are critical issues for tableam, but we should fix > them. > I'm not sure about doing so for v12 though. 1) and 3) are fairly > trivial, but 2) would involve changing the FDW interface, by changing > the AnalyzeForeignTable, AcquireSampleRowsFunc

Re: Pluggable Storage - Andres's take

Hi, On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: > index_update_stats() calls RelationGetNumberOfBlocks(). If the AM > doesn't use normal data files, that won't work. I bumped into that with my > toy implementation, which wouldn't need to create any data files, if it > wasn't for this.

Re: Pluggable Storage - Andres's take

Hi, On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: > The comments for relation_set_new_relfilenode() callback say that the AM can > set *freezeXid and *minmulti to invalid. But when I did that, VACUUM hits > this assertion: > > TRAP: FailedAssertion("!(((classForm->relfrozenxid) >= ((Tra

Re: Pluggable Storage - Andres's take

Andres Freund writes: > On 2019-04-11 14:52:40 +0300, Heikki Linnakangas wrote: >> + * HEIKKI: A flags bitmask argument would be more readable than 6 >> booleans > I honestly don't have strong feelings about it. Not sure that I buy that > bitmasks would be much more readable Sure they would

Re: Pluggable Storage - Andres's take

On Thu, Apr 11, 2019 at 12:49 PM Andres Freund wrote: > > @@ -179,6 +184,12 @@ typedef struct TableAmRoutine > >* > >* if temp_snap is true, the snapshot will need to be deallocated at > >* scan_end. > > + * > > + * HEIKKI: table_scan_update_snapshot() changes the

Re: Pluggable Storage - Andres's take

Hi, On 2019-04-11 14:52:40 +0300, Heikki Linnakangas wrote: > Here is another iteration on the comments. The patch is a mix of > copy-editing and questions. The questions are marked with "HEIKKI:". I can > continue the copy-editing, if you can reply to the questions, clarifying the > intention on

Re: Pluggable Storage - Andres's take

On 08/04/2019 20:37, Andres Freund wrote: Hi, On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: There were a bunch of typos in the comments in tableam.h, see attached. Some of the comments could use more copy-editing and clarification, I think, but I stuck to fixing just typos and such fo

Re: Pluggable Storage - Andres's take

On 08/04/2019 20:37, Andres Freund wrote: On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: There's a little bug in index-only scan executor node, where it mixes up the slots to hold a tuple from the index, and from the table. That doesn't cause any ill effects if the AM uses TTSOpsHeapTup

Re: Pluggable Storage - Andres's take

Hi, On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: > There were a bunch of typos in the comments in tableam.h, see attached. Some > of the comments could use more copy-editing and clarification, I think, but > I stuck to fixing just typos and such for now. I pushed these after adding thr

Re: Pluggable Storage - Andres's take

On Mon, Apr 8, 2019 at 9:34 AM Heikki Linnakangas wrote: > > I wrote a little toy implementation that just returns constant data to > play with this a little. Looks good overall. > > There were a bunch of typos in the comments in tableam.h, see attached. > Some of the comments could use more copy-

Re: Pluggable Storage - Andres's take

I wrote a little toy implementation that just returns constant data to play with this a little. Looks good overall. There were a bunch of typos in the comments in tableam.h, see attached. Some of the comments could use more copy-editing and clarification, I think, but I stuck to fixing just ty

Re: Pluggable Storage - Andres's take

Hi, On 2019-04-04 00:51:38 -0500, Justin Pryzby wrote: > I reviewed new docs for $SUBJECT. > Find attached proposed changes. > There's one XXX item I'm unsure what it's intended to say. Thanks! I applied most of these, and filled in the XXX. I didn't like the s/allow to to specify properties/al

Re: Pluggable Storage - Andres's take

I reviewed new docs for $SUBJECT. Find attached proposed changes. There's one XXX item I'm unsure what it's intended to say. Justin >From a3d290bf67af2a34e44cd6c160daf552b56a13b5 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 4 Apr 2019 00:48:09 -0500 Subject: [PATCH v1] Fine tune doc

Re: Pluggable Storage - Andres's take

Hi, On 2019-04-02 17:11:07 +1100, Haribabu Kommi wrote: > From a72cfcd523887f1220473231d7982928acc23684 Mon Sep 17 00:00:00 2001 > From: Hari Babu > Date: Tue, 2 Apr 2019 15:41:17 +1100 > Subject: [PATCH 1/2] tableam : doc update of table access methods > > Providing basic explanation of table a

Re: Pluggable Storage - Andres's take

Hi, On 2019-04-02 17:11:07 +1100, Haribabu Kommi wrote: > + xreflabel="default_table_access_method"> > + default_table_access_method > (string) > + > + default_table_access_method configuration > parameter > + > + > + > + > +The value is ei

Re: Pluggable Storage - Andres's take

On Tue, Apr 2, 2019 at 11:53 AM Andres Freund wrote: > Hi, > > On 2019-04-02 11:39:57 +1100, Haribabu Kommi wrote: > > > What made you rename indexam.sgml to am.sgml, instead of creating a > > > separate tableam.sgml? Seems easier to just have a separate file? > > > > > > > No specific reason, I

Re: Pluggable Storage - Andres's take

Hi, On 2019-04-02 11:39:57 +1100, Haribabu Kommi wrote: > > What made you rename indexam.sgml to am.sgml, instead of creating a > > separate tableam.sgml? Seems easier to just have a separate file? > > > > No specific reason, I just thought of adding all the access methods under > one file. > I

Re: Pluggable Storage - Andres's take

On Tue, Apr 2, 2019 at 10:18 AM Andres Freund wrote: > Hi, > > On 2019-03-16 23:21:31 +1100, Haribabu Kommi wrote: > > updated patches are attached. > > Now that nearly all of the tableam patches are committed (with the > exception of the copy.c changes which are for bad reasons discussed at > [1

Re: Pluggable Storage - Andres's take

Hi, On 2019-03-16 23:21:31 +1100, Haribabu Kommi wrote: > updated patches are attached. Now that nearly all of the tableam patches are committed (with the exception of the copy.c changes which are for bad reasons discussed at [1]) I'm looking at the docs changes. What made you rename indexam.sgm

Re: Pluggable Storage - Andres's take

On 2019-03-29 18:38:46 +1100, Haribabu Kommi wrote: > As I see that your are fixing some typos of the code that is committed, > I just want to share some more corrections that I found in the patches > that are committed till now. Pushed both, thanks!

Re: Pluggable Storage - Andres's take

On Wed, Mar 27, 2019 at 11:17 AM Andres Freund wrote: > Hi, > > On 2019-02-22 14:52:08 -0500, Robert Haas wrote: > > On Fri, Feb 22, 2019 at 11:19 AM Amit Khandekar > wrote: > > > Thanks for the review. Attached v2. > > > > Thanks. I took this, combined it with Andres's > > v12-0040-WIP-Move-xi

Re: Pluggable Storage - Andres's take

Hi, On 2019-02-22 14:52:08 -0500, Robert Haas wrote: > On Fri, Feb 22, 2019 at 11:19 AM Amit Khandekar > wrote: > > Thanks for the review. Attached v2. > > Thanks. I took this, combined it with Andres's > v12-0040-WIP-Move-xid-horizon-computation-for-page-level-.patch, did > some polishing of

Re: Pluggable Storage - Andres's take

Hi, On 2019-03-23 20:16:30 -0700, Andres Freund wrote: > I'm pretty happy with that last version (of the first patch). I'm > planning to do one more pass, and then push. And done, after a bunch of mostly cosmetic changes (renaming ExecCheckHeapTupleVisible to ExecCheckTupleVisible, removing an un

Re: Pluggable Storage - Andres's take

Hi, On 2019-03-21 11:15:57 -0700, Andres Freund wrote: > Pending work: > - Wondering if table_insert/delete/update should rather be > table_tuple_insert etc. Would be a bit more consistent with the > callback names, but a bigger departure from existing code. I've left this as is. > - I'm no

Re: Pluggable Storage - Andres's take

On Fri, Mar 22, 2019 at 5:16 AM Andres Freund wrote: > Hi, > > Attached is a version of just the first patch. I'm still updating it, > but it's getting closer to commit: > > - There were no tests testing EPQ interactions with DELETE, and only an > accidental test for EPQ in UPDATE with a concur

Re: Pluggable Storage - Andres's take

Hi, The psql \dA commands currently doesn't show the type of the access methods of type 'Table'. postgres=# \dA heap List of access methods Name | Type --+--- heap | (1 row) Attached a simple patch that fixes the problem and outputs as follows. postgres=# \dA heap List of access metho

Re: Pluggable Storage - Andres's take

On Sat, Mar 16, 2019 at 5:43 PM Haribabu Kommi wrote: > > > On Sat, Mar 9, 2019 at 2:13 PM Andres Freund wrote: > >> Hi, >> >> While 0001 is pretty bulky, the interesting bits concentrate on a >> comparatively small area. I'd appreciate if somebody could give the >> comments added in tableam.h a

Re: What to name the current heap after pluggable storage / what to rename?

Hi, On 2019-03-19 15:25:44 +1100, Haribabu Kommi wrote: > On Tue, Mar 19, 2019 at 2:32 PM Andres Freund wrote: > > I'm kinda wondering about replacing the TableTuple prefix with TableMod, > > seems less confusing to me. > One more way, how about just TupleUpdated and etc. Removing of Table? > Th

Re: What to name the current heap after pluggable storage / what to rename?

On Tue, Mar 19, 2019 at 2:32 PM Andres Freund wrote: > On 2019-03-18 16:24:40 -0700, Andres Freund wrote: > > Hi, > > > > On 2019-03-13 08:29:47 -0400, Robert Haas wrote: > > > On Tue, Mar 12, 2019 at 8:39 PM Andres Freund > wrote: > > > > > I like that option. > > > > > > > > In that vein, does

Re: What to name the current heap after pluggable storage / what to rename?

On 2019-03-18 16:24:40 -0700, Andres Freund wrote: > Hi, > > On 2019-03-13 08:29:47 -0400, Robert Haas wrote: > > On Tue, Mar 12, 2019 at 8:39 PM Andres Freund wrote: > > > > I like that option. > > > > > > In that vein, does anybody have an opinion about the naming of > > > a) HeapUpdateFailureD

Re: What to name the current heap after pluggable storage / what to rename?

Hi, On 2019-03-13 08:29:47 -0400, Robert Haas wrote: > On Tue, Mar 12, 2019 at 8:39 PM Andres Freund wrote: > > > I like that option. > > > > In that vein, does anybody have an opinion about the naming of > > a) HeapUpdateFailureData, which will be used for different AMs > > b) HTSU_Result itself

Re: Pluggable Storage - Andres's take

On Sat, Mar 9, 2019 at 2:13 PM Andres Freund wrote: > Hi, > > While 0001 is pretty bulky, the interesting bits concentrate on a > comparatively small area. I'd appreciate if somebody could give the > comments added in tableam.h a read (both on callbacks, and their > wrappers, as they have differe

Re: What to name the current heap after pluggable storage / what to rename?

On Tue, Mar 12, 2019 at 8:39 PM Andres Freund wrote: > > I like that option. > > In that vein, does anybody have an opinion about the naming of > a) HeapUpdateFailureData, which will be used for different AMs > b) HTSU_Result itself, which'll be the return parameter for >update/delete via tabl

Re: What to name the current heap after pluggable storage / what to rename?

On 2018-12-19 14:21:29 -0500, Robert Haas wrote: > On Tue, Dec 18, 2018 at 11:17 PM Andres Freund wrote: > > The current pluggable table storage patchset [1] introduces the ability > > to specify the access method of a table (CREATE TABLE ... USING > > "ident"). The patchset currently names the cu

Re: Pluggable Storage - Andres's take

Hello. I had a look on the patch set. I cannot see the thread structure due to the depth and cannot get the picture on the all patches, but I have some comments. I apologize in advance for possible duplicate with upthread. 0001-Reduce-the... This doesn't apply master. > TupleTableSlot * > Ex

Re: Pluggable Storage - Andres's take

On 2019-03-11 13:31:26 -0700, Andres Freund wrote: > On 2019-03-11 12:37:46 -0700, Andres Freund wrote: > > Hi, > > > > On 2019-03-08 19:13:10 -0800, Andres Freund wrote: > > > Changes: > > > - I've added comments to all the callbacks in the first commit / the > > > scan commit > > > - I've rena

Re: Pluggable Storage - Andres's take

On 2019-03-11 12:37:46 -0700, Andres Freund wrote: > Hi, > > On 2019-03-08 19:13:10 -0800, Andres Freund wrote: > > Changes: > > - I've added comments to all the callbacks in the first commit / the > > scan commit > > - I've renamed table_gimmegimmeslot to table_slot_create > > - I've made the c

Re: Pluggable Storage - Andres's take

On Sat, Mar 9, 2019 at 2:18 PM Andres Freund wrote: > Hi, > > On 2019-03-09 11:03:21 +1100, Haribabu Kommi wrote: > > Here I attached the rebased patches that I shared earlier. I am adding > the > > comments to explain the API's in the code, will share those patches > later. > > I've started to a

Re: Pluggable Storage - Andres's take

Hi, On 2019-03-10 05:49:26 +0100, Dmitry Dolgov wrote: > > On Sat, Mar 9, 2019 at 4:13 AM Andres Freund wrote: > > > > While 0001 is pretty bulky, the interesting bits concentrate on a > > comparatively small area. I'd appreciate if somebody could give the > > comments added in tableam.h a read (

Re: Pluggable Storage - Andres's take

> On Sat, Mar 9, 2019 at 4:13 AM Andres Freund wrote: > > While 0001 is pretty bulky, the interesting bits concentrate on a > comparatively small area. I'd appreciate if somebody could give the > comments added in tableam.h a read (both on callbacks, and their > wrappers, as they have different au

Re: Pluggable Storage - Andres's take

Hi, On 2019-03-09 11:03:21 +1100, Haribabu Kommi wrote: > Here I attached the rebased patches that I shared earlier. I am adding the > comments to explain the API's in the code, will share those patches later. I've started to add those for the callbacks in the first commit. I'd appreciate a look!

Re: Pluggable Storage - Andres's take

On Thu, Mar 7, 2019 at 6:33 AM Andres Freund wrote: > Hi, > > On 2019-03-05 23:07:21 -0800, Andres Freund wrote: > > My next steps are: > > - final polish & push the basic DDL and pg_dump patches > > Done and pushed. Some collation dependent fallout, I'm hoping I've just > fixed that. > Thanks f

Re: Pluggable Storage - Andres's take

Andres Freund writes: > On 2019-03-07 08:52:21 -0500, Robert Haas wrote: >> On Wed, Mar 6, 2019 at 6:11 PM Andres Freund wrote: >> > slot that's compatible with the "target" table. You can get compatible >> > slot callbakcs by calling table_slot_callbacks(), or directly create one >> > by callin

Re: Pluggable Storage - Andres's take

On Thu, Mar 7, 2019 at 12:49 PM Andres Freund wrote: > On 2019-03-07 08:52:21 -0500, Robert Haas wrote: > > On Wed, Mar 6, 2019 at 6:11 PM Andres Freund wrote: > > > slot that's compatible with the "target" table. You can get compatible > > > slot callbakcs by calling table_slot_callbacks(), or d

Re: Pluggable Storage - Andres's take

On 2019-03-07 08:52:21 -0500, Robert Haas wrote: > On Wed, Mar 6, 2019 at 6:11 PM Andres Freund wrote: > > slot that's compatible with the "target" table. You can get compatible > > slot callbakcs by calling table_slot_callbacks(), or directly create one > > by calling table_gimmegimmeslot() (like

Re: Pluggable Storage - Andres's take

On Wed, Mar 6, 2019 at 6:11 PM Andres Freund wrote: > slot that's compatible with the "target" table. You can get compatible > slot callbakcs by calling table_slot_callbacks(), or directly create one > by calling table_gimmegimmeslot() (likely to be renamed :)). Hmm. I assume the issue is that t

Re: Pluggable Storage - Andres's take

Hi, On 2019-03-07 11:56:57 +1300, David Rowley wrote: > On Thu, 7 Mar 2019 at 08:33, Andres Freund wrote: > > Here's a cleaned up version of that patch. David, Alvaro, you also > > played in that area, any objections? I think this makes that part of the > > code easier to read actually. Robert,

Re: Pluggable Storage - Andres's take

On Thu, 7 Mar 2019 at 08:33, Andres Freund wrote: > Here's a cleaned up version of that patch. David, Alvaro, you also > played in that area, any objections? I think this makes that part of the > code easier to read actually. Robert, thanks for looking at that patch > already. I only had a quick

Re: Pluggable Storage - Andres's take

Hi, On 2019-03-05 23:07:21 -0800, Andres Freund wrote: > My next steps are: > - final polish & push the basic DDL and pg_dump patches Done and pushed. Some collation dependent fallout, I'm hoping I've just fixed that. > - cleanup & polish the ON CONFLICT refactoring Here's a cleaned up version

Re: Pluggable Storage - Andres's take

Hi, Thanks for looking! On 2019-03-05 18:27:45 -0800, Ashwin Agrawal wrote: > While playing with the tableam, usage of which starts with commit > v12-0023-tableam-Introduce-and-use-begin-endscan-and-do-i.patch, should we > check for NULL function pointer before actually calling the same and ERROR

Re: Pluggable Storage - Andres's take

Hi, Thanks for looking! On 2019-03-05 18:27:45 -0800, Ashwin Agrawal wrote: > While playing with the tableam, usage of which starts with commit > v12-0023-tableam-Introduce-and-use-begin-endscan-and-do-i.patch, should we > check for NULL function pointer before actually calling the same and ERROR

Re: Pluggable Storage - Andres's take

Hi, While playing with the tableam, usage of which starts with commit v12-0023-tableam-Introduce-and-use-begin-endscan-and-do-i.patch, should we check for NULL function pointer before actually calling the same and ERROR out instead as NOT_SUPPORTED or something on those lines. Understand its kind

Re: Pluggable Storage - Andres's take

hink we probably ought to revise that, at the very least because it'll otherwise be noticably less efficient to have triggers on !heap tables, but also because it's just cleaner. But I feel like I don't want more significantly sized patches on my plate right now, so my current goal is to just p

Re: Pluggable Storage - Andres's take

I haven't been following this thread closely, but I looked briefly at some of the patches posted here: On 21/01/2019 11:01, Andres Freund wrote: The patchset is now pretty granularly split into individual pieces. Wow, 42 patches, very granular indeed! That's nice for reviewing, but are you p

Re: Pluggable Storage - Andres's take

On Wed, Feb 27, 2019 at 11:10 AM Andres Freund wrote: > Hi, > > On 2019-01-21 10:32:37 +1100, Haribabu Kommi wrote: > > I am not able to remove the complete t_tableOid from HeapTuple, > > because of its use in triggers, as the slot is not available in triggers > > and I need to store the tableOid

Re: Pluggable Storage - Andres's take

Hi, On 2019-01-21 10:32:37 +1100, Haribabu Kommi wrote: > I am not able to remove the complete t_tableOid from HeapTuple, > because of its use in triggers, as the slot is not available in triggers > and I need to store the tableOid also as part of the tuple. What precisely do you man by "use in t

Re: Pluggable Storage - Andres's take

On Sat, 23 Feb 2019 at 01:22, Robert Haas wrote: > > On Fri, Feb 22, 2019 at 11:19 AM Amit Khandekar > wrote: > > Thanks for the review. Attached v2. > > Thanks. I took this, combined it with Andres's > v12-0040-WIP-Move-xid-horizon-computation-for-page-level-.patch, did > some polishing of the

Re: Pluggable Storage - Andres's take

On Fri, Feb 22, 2019 at 11:19 AM Amit Khandekar wrote: > Thanks for the review. Attached v2. Thanks. I took this, combined it with Andres's v12-0040-WIP-Move-xid-horizon-computation-for-page-level-.patch, did some polishing of the code and comments, and pgindented. Here's what I ended up with;

Re: Pluggable Storage - Andres's take

On Thu, 21 Feb 2019 at 18:06, Robert Haas wrote: > > On Thu, Feb 21, 2019 at 6:44 AM Amit Khandekar wrote: > > Ok, so something like XidHorizonPrefetchState ? On similar lines, does > > prefetch_buffer() function name sound too generic as well ? > > Yeah, that sounds good. > And, yeah, then mayb

Re: Pluggable Storage - Andres's take

On Thu, Feb 21, 2019 at 6:44 AM Amit Khandekar wrote: > Ok, so something like XidHorizonPrefetchState ? On similar lines, does > prefetch_buffer() function name sound too generic as well ? Yeah, that sounds good. And, yeah, then maybe rename the function too. > > +/* > > + * An arbitrary way to

Re: Pluggable Storage - Andres's take

On Thu, 21 Feb 2019 at 04:17, Robert Haas wrote: > > On Fri, Feb 8, 2019 at 5:18 AM Amit Khandekar wrote: > > In the attached v1 patch, the prefetch_distance is calculated as > > effective_io_concurrency + 10. Also it has some cosmetic changes. > > I did a little brief review of this patch and no

Re: Pluggable Storage - Andres's take

On Fri, Feb 8, 2019 at 5:18 AM Amit Khandekar wrote: > In the attached v1 patch, the prefetch_distance is calculated as > effective_io_concurrency + 10. Also it has some cosmetic changes. I did a little brief review of this patch and noticed the following things. +} PrefetchState; That name see

Re: Pluggable Storage - Andres's take

On Tue, Nov 27, 2018 at 4:59 PM Amit Langote wrote: > Hi, > > On 2018/11/02 9:17, Haribabu Kommi wrote: > > Here I attached the cumulative fixes of the patches, new API additions > for > > zheap and > > basic outline of the documentation. > > I've read the documentation patch while also looking a

Re: Pluggable Storage - Andres's take

n > 4. Manipulation of physical tuples > 5. Non-modifying operations on individual tuples > 6. DDL > 7. Planner > 8. Executor > > Suggestions? > Here I attached the doc patches for the pluggable storage, I divided the API's into the above specified groups and explai

Re: Pluggable Storage - Andres's take

ote: > > > On 2018-11-26 17:55:57 -0800, Andres Freund wrote: > > > > FWIW, now that oids are removed, and the tuple table slot abstraction > > > > got in, I'm working on rebasing the pluggable storage patchset ontop of > > > > that. >

Re: Pluggable Storage - Andres's take

w that oids are removed, and the tuple table slot abstraction > > > got in, I'm working on rebasing the pluggable storage patchset ontop of > > > that. > > > > I've pushed a version to that to the git tree, including a rebased > > version of zheap: > >

Re: Pluggable Storage - Andres's take

On Tue, Jan 22, 2019 at 1:43 PM Haribabu Kommi wrote: > > > OK. I will work on the doc changes. > Sorry for the delay. Attached a draft patch of doc and comments changes that I worked upon. Currently I added comments to the callbacks that are present in the TableAmRoutine structure and I copied

Re: Pluggable Storage - Andres's take

types that support table access method. This patch is based on commit 1bc7e6a4838 in https://github.com/anarazel/postgres-pluggable-storage Thanks -Amit Khandekar test_tableam_dependency.patch Description: Binary data

Re: Pluggable Storage - Andres's take

> On Sun, Jan 20, 2019 at 6:17 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Fri, Jan 18, 2019 at 11:22 AM Amit Khandekar > > wrote: > > > > I believe you are going to add a new regression testcase for the change ? > > Yep. So, here are these two patches for pg_dump/psql with a few re

Re: Pluggable Storage - Andres's take

On Tue, 22 Jan 2019 at 15:29, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Mon, Jan 21, 2019 at 9:33 AM Amit Khandekar > > wrote: > > > > Regression tests that use \d+ to show the table details might > > not be interested specifically in table access method. But these will > > fail if run

Re: Pluggable Storage - Andres's take

> On Mon, Jan 21, 2019 at 3:01 AM Andres Freund wrote: > > The patchset is now pretty granularly split into individual pieces. Wow, thanks! > On Mon, Jan 21, 2019 at 9:33 AM Amit Khandekar wrote: > > Regression tests that use \d+ to show the table details might > not be interested specifically

Re: Pluggable Storage - Andres's take

On Tue, Jan 22, 2019 at 12:15 PM Andres Freund wrote: > Hi, > > Thanks! > > On 2019-01-22 11:51:57 +1100, Haribabu Kommi wrote: > > Attached the patch for removal of scan_update_snapshot > > and also the rebased patch of reduction in use of t_tableOid. > > I'll soon look at the latter. > Thanks.

Re: Pluggable Storage - Andres's take

Hi, Thanks! On 2019-01-22 11:51:57 +1100, Haribabu Kommi wrote: > Attached the patch for removal of scan_update_snapshot > and also the rebased patch of reduction in use of t_tableOid. I'll soon look at the latter. > > - consider removing table_gimmegimmeslot() > > - add substantial docs for e

  1   2   3   >