Re: Bug in either collation docs or code

2018-06-08 Thread Melanie Plageman
> > I tried inserting data and did not get an error: CREATE TABLE test1 ( a text COLLATE "de_DE", b text COLLATE "es_ES" ); INSERT INTO test1 VALUES('b','b'), ('c','c'), ('g','g'), ('h','h'); SELECT a < (select 'foo' COLLATE "fr_FR") FROM test1; -- Melanie Plageman

Re: Bug in either collation docs or code

2018-06-08 Thread Melanie Plageman
g planning EXPLAIN SELECT 'c' COLLATE "de_DE" > 'ç' COLLATE "es_ES"; -- error during planning It seems like this would not allow the function/operator to decide if it cares about a determinate collation during execution, since it would already have errored out during planning. -- Melanie Plageman

Re: Bug in either collation docs or code

2018-06-08 Thread Melanie Plageman
A'); INSERT 0 1 SELECT a < (SELECT 'foo'::TEXT COLLATE "fr_FR") FROM test_col_mac; ?column? -- t (1 row) -- Melanie Plageman

Re: Bug in either collation docs or code

2018-06-07 Thread Melanie Plageman
You could mark the subquery's result with a collation like this: > > postgres=# SELECT 'c' COLLATE "de_DE" > (SELECT 'ç') COLLATE "es_ES"; > ERROR: collation mismatch between explicit collations "de_DE" and "es_ES" > > I'm not sure if this behavior is considered a bug, but I also can't imagine >>

Bug in either collation docs or code

2018-06-04 Thread Melanie Plageman
" > ANY(select 'ç' COLLATE "es_ES"); I'm not sure if this behavior is considered a bug, but I also can't imagine how it would be expected given the current documentation. It seems to me one or the other should be updated. -- Melanie Plageman subquery_opexpr_explicit_collation_hack.patch Description: Binary data

Re: Making "COPY partitioned_table FROM" faster

2018-07-28 Thread Melanie Plageman
On Thu, Jul 26, 2018 at 7:26 PM, David Rowley wrote: > Fixed in the attached v4. That's the only change. > I don't see an attachment. > So, I thinking I'm missing something. Which of the changes would cause the > > performance improvement from patch v2 to v3? My understanding is: > > You could

Re: Making "COPY partitioned_table FROM" faster

2018-07-30 Thread Melanie Plageman
On Mon, Jul 30, 2018 at 11:21 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 30/07/2018 15:26, David Rowley wrote: > >> - Add some tests. The if (nBufferedTuples > 0) that flushes the tuples > >> when the partition changes is not currently exercised. > > > > That seems like

Re: Making "COPY partitioned_table FROM" faster

2018-07-26 Thread Melanie Plageman
On Wed, Jul 25, 2018 at 7:24 PM, David Rowley wrote: On patched code line 2564, there is a missing parenthesis. It might be better to remove the parentheses entirely and, while you're at it, there is a missing comma. /* * For partitioned tables we can't support multi-inserts when there * are

Re: Making "COPY partitioned_table FROM" faster

2018-07-29 Thread Melanie Plageman
On Sat, Jul 28, 2018 at 6:00 PM, David Rowley wrote: > Oops. Must've fallen off in transit :) Hopefully, it's more firmly > attached this time. > LGTM. Status changed to "ready for committer"

Re: Making "COPY partitioned_table FROM" faster

2018-07-24 Thread Melanie Plageman
On Fri, Jul 20, 2018 at 7:57 AM, David Rowley wrote: > > So, overall, we feel that the code from lines 2689 until 2691 and from > 2740 to 2766 could use further clarification with regard to switching from > parent to child partition and sibling to sibling partition as well as > regarding saving

Re: Removing useless DISTINCT clauses

2018-04-05 Thread Melanie Plageman
Hi David, The updated patch looks good to me. I've changed the status to "ready for committer" Thanks

Re: Removing useless DISTINCT clauses

2018-03-20 Thread Melanie Plageman
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed This is a review of the patch to remove useless DISTINCT

Re: Removing useless DISTINCT clauses

2018-03-23 Thread Melanie Plageman
* to *constraintDeps. FIXME whenever not-null constraints get represented * in pg_constraint. */ -- Melanie Plageman

Re: [HACKERS] [PATCH v2] Add and report the new "session_read_only" GUC pseudo-variable.

2018-11-11 Thread Melanie Plageman
On Mon, Sep 24, 2018 at 9:28 AM Elvis Pranskevichus wrote: > On Thursday, March 1, 2018 4:25:16 AM EDT Andres Freund wrote: > > A CF entry for this patch has been created yesterday, without any > > changes having happend since the above status update. Given that > > there's been no progress for

Re: BUG #15160: planner overestimates number of rows in join when there are more than 200 rows coming from CTE

2018-11-15 Thread Melanie Plageman
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested This patch applies cleanly and works for the case described in the

Re: BUG #15160: planner overestimates number of rows in join when there are more than 200 rows coming from CTE

2018-11-16 Thread Melanie Plageman
Thanks for the quick responses. I've put some inline follow-up questions. On a separate note, I had one additional code clarity feedback. I felt that eqjoinsel could be reorganized a bit for readability/clarity for the reader. For example, eqjoinsel_inner uses only the AttStatsSlots up until here

Re: BUG #15160: planner overestimates number of rows in join when there are more than 200 rows coming from CTE

2018-11-20 Thread Melanie Plageman
Given that you have addressed all of my feedback and that it's a pretty low-risk change, I will change the status to "ready for committer". There are a couple of minor follow-up clarifications inline that relate mostly to the questions that I asked in previous emails. I did have one other

Re: Adding a test for speculative insert abort case

2019-05-16 Thread Melanie Plageman
the wording of the comments overall in the above email I sent). -- Melanie Plageman 0002-Add-test-to-validate-speculative-wait-is-performed.patch Description: Binary data

Re: Adding a test for speculative insert abort case

2019-05-16 Thread Melanie Plageman
On Thu, May 16, 2019 at 2:03 PM Andres Freund wrote: > Hi, > > On 2019-05-16 13:59:47 -0700, Melanie Plageman wrote: > > On Wed, May 15, 2019 at 10:32 PM Ashwin Agrawal > wrote: > > > > > > > > The second index would help to hold the session after i

Re: Adding a test for speculative insert abort case

2019-05-15 Thread Melanie Plageman
ot;controller_show" I was also wondering: Is it possible that one of the "controller_unlock_*" functions will get called before the session with the upsert has had a chance to move forward in its progress and be waiting on that lock? That is, given that we don't check that the sessions are waiting on the locks before unlocking them, is there a race condition? I noticed that there is not a test case which would cover the speculative wait codepath. This seems much more challenging, however, it does seem like a worthwhile test to have. -- Melanie Plageman

Re: Adding a test for speculative insert abort case

2019-05-15 Thread Melanie Plageman
t;s2_upsert" "controller_show" "controller_unlock_1_1" "controller_unlock_2_1" "controller_unlock_1_3" "controller_unlock_2_3" "controller_unlock_1_2" "s1_magically_pause_before_complete_speculative" # put s2 in speculative wait "controller_unlock_2_2" "s1_magically_unpause_before_complete_speculative" So, how would another lock on another index keep s1 from clearing the speculative token after it has updated the index? -- Melanie Plageman specconflict_permutation_comment_update.patch Description: Binary data

Re: Adding a test for speculative insert abort case

2019-05-17 Thread Melanie Plageman
On Thu, May 16, 2019 at 8:46 PM Melanie Plageman wrote: > > I squashed the changes I suggested in previous emails, Ashwin's patch, my > suggested updates to that patch, and the index order check all into one > updated > patch attached. > > I realized that the numbers

Re: Sort support for macaddr8

2019-06-04 Thread Melanie Plageman
p_internal() is just specified as an int. I was wondering why. I also noticed that the prototype for macaddr8_cmp_internal() was not at the top of the file with the other static function prototypes. I added it there, but I wasn't sure if there was some reason that it was like tha

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-06-04 Thread Melanie Plageman
given inner chunk, would you load the bitmaps from each worker into memory, OR them together, and then write the updated bitmap back out so that each worker starts with the updated bitmap? -- Melanie Plageman

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-06-04 Thread Melanie Plageman
On Tue, Jun 4, 2019 at 5:43 AM Robert Haas wrote: > On Mon, Jun 3, 2019 at 5:10 PM Melanie Plageman > wrote: > > I was talking to Jeff Davis about this on Saturday, and, he felt that > > there might be a way to solve the problem differently if we thought of > > the left

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-06-06 Thread Melanie Plageman
do that was kind of like mmap'ing the outer side file to give the workers in parallel hashjoin the ability to update a match bit in the tuple in place and avoid writing the whole outer side out each time. -- Melanie Plageman

Re: Sort support for macaddr8

2019-06-05 Thread Melanie Plageman
ope, Melanie? > > I can take on making macaddr8 pass-by-value I tinkered a bit last night and got in/out mostly working (I think). I'm not sure about macaddr, TID, and user-defined types. -- Melanie Plageman

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-06-06 Thread Melanie Plageman
On Tue, Jun 4, 2019 at 12:08 PM Robert Haas wrote: > On Tue, Jun 4, 2019 at 2:47 PM Melanie Plageman > wrote: > > Oops! You are totally right. > > I will amend the idea: > > For each chunk on the inner side, loop through both the original batch > > file and t

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-06-06 Thread Melanie Plageman
On Tue, Jun 4, 2019 at 12:15 PM Robert Haas wrote: > On Tue, Jun 4, 2019 at 3:09 PM Melanie Plageman > wrote: > > One question I have is, how would the OR'd together bitmap be > > propagated to workers after the first chunk? That is, when there are > > no tuples

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-06-03 Thread Melanie Plageman
On Sun, May 19, 2019 at 4:07 PM Thomas Munro wrote: > On Sat, May 18, 2019 at 12:15 PM Melanie Plageman > wrote: > > On Thu, May 16, 2019 at 3:22 PM Thomas Munro > wrote: > >> Admittedly I don't have a patch, just a bunch of handwaving. One > >> re

Re: Adding a test for speculative insert abort case

2019-06-05 Thread Melanie Plageman
On Thu, May 16, 2019 at 8:46 PM Melanie Plageman wrote: > > Good idea. > I squashed the changes I suggested in previous emails, Ashwin's patch, my > suggested updates to that patch, and the index order check all into one > updated > patch attached. > > I've updated thi

Extracting only the columns needed for a query

2019-06-14 Thread Melanie Plageman
While hacking on zedstore, we needed to get a list of the columns to be projected--basically all of the columns needed to satisfy the query. The two use cases we have for this is 1) to pass this column list down to the AM layer for the AM to leverage it 2) for use during planning to improving

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-06-11 Thread Melanie Plageman
On Fri, Jun 7, 2019 at 7:30 AM Robert Haas wrote: > On Thu, Jun 6, 2019 at 7:31 PM Melanie Plageman > wrote: > > I'm not sure I understand why you would need to compare the original > > tuples to the unmatched tuples file. > > I think I was confused. Actually, I'm sti

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-06-18 Thread Melanie Plageman
On Thu, Jun 13, 2019 at 7:10 AM Robert Haas wrote: > On Tue, Jun 11, 2019 at 2:35 PM Melanie Plageman > wrote: > > Let me try to articulate what I think the bitmap implementation would > look > > like: > > > > Before doing chunked hashloop join for any batch, w

Re: accounting for memory used for BufFile during hash joins

2019-05-21 Thread Melanie Plageman
On Wed, May 8, 2019 at 8:08 AM Tomas Vondra wrote: > On Tue, May 07, 2019 at 05:30:27PM -0700, Melanie Plageman wrote: > > One thing I was a little confused by was the nbatch_inmemory member > > of the hashtable. The comment in ExecChooseHashTableSize says that > >

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-05-17 Thread Melanie Plageman
hes for the outer side, make one "batch" that has all the tuples from the outer side which the inner side batch which was flagged will do NLJ with. -- Melanie Plageman

describe working as intended?

2019-05-17 Thread Melanie Plageman
---+---+- 23609 | public| t1 23612 | pg_temp_4 | t1 (2 rows) So, without much more digging, is the current behavior of describe intended? I couldn't find an email thread discussing this with the search terms I tried. (I noticed it on master and checked 11 as well and got the same behavior.) -- Melanie

Re: describe working as intended?

2019-05-21 Thread Melanie Plageman
to display tables with such name in all schemas. > > regards, Sergei > Thanks! I suppose it would behoove me to check the documentation before resorting to looking at the source code :) -- Melanie Plageman

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-05-20 Thread Melanie Plageman
On Sun, May 19, 2019 at 4:07 PM Thomas Munro wrote: > On Sat, May 18, 2019 at 12:15 PM Melanie Plageman > wrote: > > On Thu, May 16, 2019 at 3:22 PM Thomas Munro > wrote: > >> Admittedly I don't have a patch, just a bunch of handwaving. One > >> re

Adding a test for speculative insert abort case

2019-04-30 Thread Melanie Plageman
Today, while poking around the table_complete_speculative code which Ashwin mentioned in [1], we were trying to understand when exactly we would complete a speculative insert by aborting. We added a logging message to heapam_tuple_complete_speculative before it calls heap_abort_speculative and

Re: Adding a test for speculative insert abort case

2019-04-30 Thread Melanie Plageman
tLockTableWait is called with reason_wait as XLTW_InsertIndex (even though we are just trying to check it, so maybe it knows our intentions:)) Is there something I can do in the test to allow my check to go through but the insert to have to wait? -- Melanie Plageman

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-30 Thread Melanie Plageman
r API should do catalog lookups. Is this correct? [1] https://www.postgresql.org/message-id/11777.1556133426%40sss.pgh.pa.us -- Melanie Plageman

Re: Adding a test for speculative insert abort case

2019-05-01 Thread Melanie Plageman
t how to use it [1]. The Greenplum implementation is not documented particularly well in the code, but, it is something that folks working on Greenplum have talked about modifying and proposing to Postgres. [1] http://engineering.pivotal.io/post/testing_greenplum_database_using_fault_injection/ -- Melanie Plageman

Re: accounting for memory used for BufFile during hash joins

2019-05-06 Thread Melanie Plageman
k at the patches is that I prefer the first approach--increasing the resize threshhold. The second patch, the per-slice-overflow patch, adds a major new mechanic to hashjoin in order to address what is, based on my understanding, an edge case. -- Melanie Plageman

Re: accounting for memory used for BufFile during hash joins

2019-05-07 Thread Melanie Plageman
ry was 2 for me, thus, nbatch_tmp was 2 so, I didn't meet this condition if (nbatch_tmp > hashtable->nbatch_inmemory) since I just set nbatch_tmp using hashtable->nbatch_inmemory So, I didn't increase the number of slices, which is what I was expecting. What happens when hashtable->nbatch_inmemory is equal to nbatch_tmp? -- Melanie Plageman

Re: accounting for memory used for BufFile during hash joins

2019-05-07 Thread Melanie Plageman
by-batch basis and did it before starting execution of the join but after building the inner side of the hash table. That way, no tuples will have been sent to other nodes yet. -- Melanie Plageman

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-10 Thread Melanie Plageman
ule called atmsort [2] to deal with the specific ORDER BY case discussed here. [1] https://github.com/greenplum-db/gpdb/blob/master/src/test/regress/gpdiff.pl [2] https://github.com/greenplum-db/gpdb/blob/master/src/test/regress/atmsort.pl -- Melanie Plageman

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-07-03 Thread Melanie Plageman
On Tue, Jun 18, 2019 at 3:24 PM Melanie Plageman wrote: > > These questions will probably make a lot more sense with corresponding > code, so I will follow up with the second version of the state machine > patch once I finish it. > > I have changed the state machine and resolv

Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-22 Thread Melanie Plageman
the temp_tablespaces GUC to ensure I create the spill files there if it is set. Thanks, Melanie Plageman

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-24 Thread Melanie Plageman
lers of the > second and third that I'm not sure that'd be better. Thoughts? > > I think an assertion is sufficiently clear for GetNextTempTableSpace based on what it does and its current callers. The same is probably true for GetTempTableSpaces. -- Melanie Plageman

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-25 Thread Melanie Plageman
the two. > > Would an existing test cover the code after moving PrepareTempTablespaces into OpenTemporaryFile? -- Melanie Plageman

Re: Extracting only the columns needed for a query

2019-07-16 Thread Melanie Plageman
We implemented Approach B in the attached patch set (patch 0001) and then implemented Approach A (patch 0002) to sanity check the pruned list of columns to scan we were getting at plan-time. We emit a notice in SeqNext() if the two column sets differ. Currently, for all of the queries in the

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Melanie Plageman
o the check that all steps have been used in dry_run mode instead of when running the tests for real? -- Melanie Plageman

Re: Adding a test for speculative insert abort case

2019-08-21 Thread Melanie Plageman
On Wed, Aug 7, 2019 at 1:47 PM Melanie Plageman wrote: > > > On Wed, Jul 24, 2019 at 11:48 AM Andres Freund wrote: > >> > diff --git a/src/test/isolation/specs/insert-conflict-specconflict.spec >> b/src/test/isolation/specs/insert-conflict-specconflict.spec >&g

Re: Cleanup isolation specs from unused steps

2019-08-21 Thread Melanie Plageman
ce really much extra maintenance cost. > So, I think I completely misunderstood the purpose of 'dry-run'. If no one is using it, having a check for unused steps in dry-run may not be useful. +1 to renaming it to --print-permutations and, potentially, adding --print-all-permutations -- Melanie Plageman

Re: Cleanup isolation specs from unused steps

2019-08-21 Thread Melanie Plageman
fferent syntax. It doesn't use isolationtester at all. So, I haven't had a use case to add long options to isolationtester yet :) -- Melanie Plageman

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Melanie Plageman
On Wed, Aug 21, 2019 at 12:16 PM Alvaro Herrera wrote: > On 2019-Aug-21, Melanie Plageman wrote: > > > In Greenplum, we mainly add new tests to a separate isolation > > framework (called isolation2) which uses a completely different > > syntax. It doesn't use isola

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Melanie Plageman
On Thu, Aug 22, 2019 at 6:53 PM Michael Paquier wrote: > On Thu, Aug 22, 2019 at 10:20:48AM -0700, Melanie Plageman wrote: > > So, there is some historical context as to why it is a separate test > suite. > > And some of the differences are specific to Greenplum -- e.g. need

Re: accounting for memory used for BufFile during hash joins

2019-09-05 Thread Melanie Plageman
On Tue, Sep 3, 2019 at 9:36 AM Alvaro Herrera wrote: > On 2019-Jul-11, Tomas Vondra wrote: > > > On Wed, Jul 10, 2019 at 04:51:02PM -0700, Melanie Plageman wrote: > > > > I think implementing support for parallel hashjoin or explicitly > > > disabling it would b

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-09-06 Thread Melanie Plageman
ker has scanned their outer match status files. All the probe loops would be done, and the worker that is emitting tuples is not referencing the inner side hashtable at all and only the outer batch file and the combined bitmap. -- Melanie Plageman

Re: Memory Accounting

2019-09-18 Thread Melanie Plageman
I wanted to address a couple of questions Jeff asked me off-list about Greenplum's implementations of Memory Accounting. Greenplum has two memory accounting sub-systems -- one is the MemoryContext-based system proposed here. The other memory accounting system tracks "logical" memory owners in

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-07-30 Thread Melanie Plageman
22:09PM -0700, Melanie Plageman wrote: > >On Tue, Jun 18, 2019 at 3:24 PM Melanie Plageman < > melanieplage...@gmail.com> > > > >Before doing that, I wanted to ask what a desirable fallback condition > >would be. In this patch, fallback to hashloop join happens only whe

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-07-30 Thread Melanie Plageman
On Tue, Jul 30, 2019 at 4:36 PM Robert Haas wrote: > On Tue, Jul 30, 2019 at 2:47 PM Melanie Plageman > wrote: > > I did the "needlessly dumb implementation" Robert mentioned, though, > > I thought about it and couldn't come up with a much smarter way to > > w

Re: Adding a test for speculative insert abort case

2019-08-07 Thread Melanie Plageman
On Wed, Jul 24, 2019 at 11:48 AM Andres Freund wrote: > > diff --git a/src/test/isolation/specs/insert-conflict-specconflict.spec > b/src/test/isolation/specs/insert-conflict-specconflict.spec > > index 3a70484fc2..7f29fb9d02 100644 > > ---

Re: Extracting only the columns needed for a query

2019-07-17 Thread Melanie Plageman
On Sat, Jun 15, 2019 at 10:01 AM Tom Lane wrote: > Melanie Plageman writes: > > While hacking on zedstore, we needed to get a list of the columns to > > be projected--basically all of the columns needed to satisfy the > > query. The two use cases we have for this is >

Re: Memory Accounting

2019-07-23 Thread Melanie Plageman
uld be good for memory intensive operators which use a large, representative context. I think the HashTableContext for HashJoin might be one? -- Melanie Plageman

Re: Memory Accounting

2019-09-24 Thread Melanie Plageman
On Thu, Sep 19, 2019 at 11:00 AM Jeff Davis wrote: > On Wed, 2019-09-18 at 13:50 -0700, Soumyadeep Chakraborty wrote: > > Hi Jeff, > > Hi Soumyadeep and Melanie, > > Thank you for the review! > > > max_stack_depth max level lazy (ms) eager (ms) > (eage > > r/lazy) > > 2MB 82

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-09 Thread Melanie Plageman
results files to ignore row ordering in some cases to allow for easier comparison. Has this been proposed in the past? -- Melanie Plageman

Re: Parallel leader process info in EXPLAIN

2019-10-30 Thread Melanie Plageman
me think about other explain wishlist items. For parallel hashjoin, I would find it useful to know which batches each worker participated in (maybe just probing to start with, but loading would be great too). I'm not sure anyone else (especially users) would care about this, though. -- Melanie Plageman

Re: Memory-Bounded Hash Aggregation

2019-12-04 Thread Melanie Plageman
On Thu, Nov 28, 2019 at 9:47 AM Tomas Vondra wrote: > On Wed, Nov 27, 2019 at 02:58:04PM -0800, Jeff Davis wrote: > >On Wed, 2019-08-28 at 12:52 -0700, Taylor Vesely wrote: > >> Right now the patch always initializes 32 spill partitions. Have you > >> given > >> any thought into how to

Re: Adding a test for speculative insert abort case

2020-02-11 Thread Melanie Plageman
> > Thanks so much for finishing the patch and checking for race conditions! -- Melanie Plageman

Re: Extracting only the columns needed for a query

2020-01-31 Thread Melanie Plageman
le of how a table AM API user like Zedstore uses the columns during planning. [1] https://www.postgresql.org/message-id/e5566f7def33a9e9fdff337cca32d07155d7b635.camel%40j-davis.com -- Melanie Plageman

Re: BufFileRead() error signalling

2020-01-30 Thread Melanie Plageman
ker reading one SharedTuplestoreChunk encounters an error and another worker on a different SharedTuplstoreChunk of the same file does not, I would find it useful to know more about which block it was on when the error was encountered. -- Melanie Plageman

Re: Parallel leader process info in EXPLAIN

2020-01-24 Thread Melanie Plageman
So, I think from a code review perspective the code in the patches LGTM. As for the EXPLAIN ANALYZE tests--I don't see that many of them in regress, so maybe that's because they aren't normally very useful. In this case, it would only be to protect against regressions in printing the leader

Re: Memory-Bounded Hash Aggregation

2020-02-18 Thread Melanie Plageman
I'll let Jeff decide what he wants to do about the patch at all (e.g. include it in his overall patch or exclude it for now). Anyway it is trivial to move those declarations up, were he to decide to include it. -- Melanie Plageman

Re: Extracting only the columns needed for a query

2020-02-18 Thread Melanie Plageman
On Fri, Jan 31, 2020 at 9:52 AM Melanie Plageman wrote: > I'm bumping this to the next commitfest because I haven't had a chance > to address the questions posed by Dmitry. I'm sure I'll get to it in > the next few weeks. > > > I believe it would be beneficial to add this poten

Re: Memory-Bounded Hash Aggregation

2020-02-13 Thread Melanie Plageman
s.com [2] https://www.postgresql.org/message-id/flat/CAAKRu_Yj%3DQ_ZxiGX%2BpgstNWMbUJApEJX-imvAEwryCk5SLUebg%40mail.gmail.com -- Melanie Plageman v1-0001-aggregated-unaggregated-cols-together.patch Description: Binary data

Re: Parallel leader process info in EXPLAIN

2020-01-17 Thread Melanie Plageman
with xxx the actual numbers, I would have thought that there would be an EXPLAIN VERBOSE with leader participation somewhere in regress). -- Melanie Plageman 0001-Show-parallel-leader-stats-in-EXPLAIN-output-v4.patch Description: Binary data

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2020-01-09 Thread Melanie Plageman
t; that, blocks are also super overloaded). > Hmmm. I think loop is kinda confusing. "fragment" has potential. I also thought of "piece". That is actually where I am leaning now. What do you think? [1] https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BA6ftXPz4oe92%2Bx8Er%2BxpGZqto70-Q_ERwRaSyA%3DafNg%40mail.gmail.com -- Melanie Plageman

Re: Extracting only the columns needed for a query

2020-01-02 Thread Melanie Plageman
when IS_DUMMY_REL is true for a relation, do we reference the associated RTE later? It seems like if it is a dummy rel, we wouldn't scan it. It still makes sense to add it to extract_used_columns(), I think, to avoid any wasted loops through the rel's expressions. Thanks for the idea! -- Melanie Plageman v2-0001-Plan-time-extraction-of-scan-cols.patch Description: Binary data

Re: accounting for memory used for BufFile during hash joins

2020-01-03 Thread Melanie Plageman
tps://www.postgresql.org/message-id/CAAKRu_YsWm7gc_b2nBGWFPE6wuhdOLfc1LBZ786DUzaCPUDXCA%40mail.gmail.com -- Melanie Plageman

Re: Extracting only the columns needed for a query

2020-01-07 Thread Melanie Plageman
I just wanted to address a question we got about making scanCols instead of using RelOptInfo->attr_needed. David Rowley had asked: > For planning, isn't this information already available via either > targetlists or from the RelOptInfo->attr_needed array combined with > min_attr and max_attr?

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2020-04-30 Thread Melanie Plageman
On Tue, Apr 28, 2020 at 11:50 PM Heikki Linnakangas wrote: > On 29/04/2020 05:03, Melanie Plageman wrote: > > I've attached a patch which should address some of the previous feedback > > about code complexity. Two of my co-workers and I wrote what is > > essentially a new

Re: Assertion failure with barriers in parallel hash join

2020-10-12 Thread Melanie Plageman
On Thu, Oct 1, 2020 at 8:08 PM Thomas Munro wrote: > On Tue, Sep 29, 2020 at 9:12 PM Thomas Munro > wrote: > > On Tue, Sep 29, 2020 at 7:11 PM Michael Paquier > wrote: > > > #2 0x009027d2 in ExceptionalCondition > > > (conditionName=conditionName@entry=0xa80846

Re: Parallel Full Hash Join

2020-09-29 Thread Melanie Plageman
On Mon, Sep 21, 2020 at 8:34 PM Thomas Munro wrote: > On Tue, Sep 22, 2020 at 8:49 AM Melanie Plageman > wrote: > > On Wed, Sep 11, 2019 at 11:23 PM Thomas Munro > wrote: > > I took it for a very quick spin and saw simple cases working nicely, > but TPC-DS queries 51 an

Re: Parallel Full Hash Join

2020-09-21 Thread Melanie Plageman
On Wed, Sep 11, 2019 at 11:23 PM Thomas Munro wrote: > > While thinking about looping hash joins (an alternative strategy for > limiting hash join memory usage currently being investigated by > Melanie Plageman in a nearby thread[1]), the topic of parallel query > deadlock ha

Re: Trouble with hashagg spill I/O pattern and costing

2020-05-26 Thread Melanie Plageman
for all the cases I tried. Is that where you made the change? And then are you proposing to set it based on the aggstrategy to either CP_LABEL_TLIST or CP_SMALL_TLIST here? -- Melanie Plageman

Reigning in ExecParallelHashRepartitionFirst

2020-07-08 Thread Melanie Plageman
tgresql.org/message-id/flat/CA%2BhUKGJvYFCcF8vTHFSQQB_F8oGRsBp3JdZAPWbORZgfAPk5Sw%40mail.gmail.com#1156516651bb2587da3909cf1db29952 -- Melanie Plageman From 22b01b7e514cf975bb70d14918dcb6611a09bbd4 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Thu, 2 Jul 2020 17:02:48 -0700 Subject: [PATCH v1] Bail out of rep

Re: Reigning in ExecParallelHashRepartitionFirst

2020-07-08 Thread Melanie Plageman
s/reign/rein/ in $subject https://www.merriam-webster.com/words-at-play/do-you-rein-in-or-reign-in-something

Extra target list entries for child partitions in DELETE...USING...RETURNING

2020-06-22 Thread Melanie Plageman
ll correctly update * subroot->partColsUpdated.) */ [1] https://www.postgresql.org/message-id/flat/CAAKRu_ZQ0Jy7LfZDCY0JdxChdpja9rf-S8Y5%2BU4vX7cYJd62dA%40mail.gmail.com#f16fb3bdf33519c0d547a4b7ae2fc3c3 [2] https://www.postgresql.org/message-id/CAAKRu_ZQ0Jy7LfZDCY0JdxChdpja9rf-S8Y5%2BU4vX7cYJd62dA%40mail.gmail.com -- Melanie Plageman

Re: hashagg slowdown due to spill changes

2020-06-23 Thread Melanie Plageman
**unaggregated) > > { > > It's not this patch's fault, but none, really none, of this stuff should > be in the executor. > > Were you thinking it could be done in grouping_planner() and then the bitmaps could be saved in the PlannedStmt? Or would you have to wait until query_planner()? Or are you imagining somewhere else entirely? -- Melanie Plageman

Re: hashagg slowdown due to spill changes

2020-06-24 Thread Melanie Plageman
On Tue, Jun 23, 2020 at 10:06 AM Andres Freund wrote: > Hi, > > On 2020-06-23 09:23:57 -0700, Melanie Plageman wrote: > > On Mon, Jun 22, 2020 at 9:02 PM Andres Freund > wrote: > > > It's not this patch's fault, but none, really none, of this stuff > s

Re: Extracting only the columns needed for a query

2020-06-18 Thread Melanie Plageman
On Fri, Mar 13, 2020 at 12:09 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > In general implemented functionality looks good. I've checked how it > works on the existing tests, almost everywhere required columns were not > missing in scanCols (which is probably the most important part). >

Re: Improve planner cost estimations for alternative subplans

2020-06-17 Thread Melanie Plageman
nt because instead of comparing alternatives you are blending them. I don't have any academic basis for saying that the alternatives costs shouldn't be averaged together for use in the rest of the plan, so I could definitely be wrong. -- Melanie Plageman

Re: Extracting only the columns needed for a query

2020-06-23 Thread Melanie Plageman
On Fri, Mar 13, 2020 at 12:09 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > I've also noticed that for partitioned tables every partition is > evaluated separately. IIRC they structure cannot differ, does it > makes sense then? Good point. At some point, we had discussed only collecting

Re: Improve planner cost estimations for alternative subplans

2020-06-16 Thread Melanie Plageman
onek a group by grouping sets((ten,four),(ten)) having exists (select 1 from onek b where sum(distinct a.four) = b.four); But, the chosen plan for this query stays the same. It would be helpful to see a query where a different plan is chosen because of this change that is not from updatable

Re: Default setting for enable_hashagg_disk

2020-06-09 Thread Melanie Plageman
sh Key: g10, g1000 Hash Key: g10 Group Key: () Peak Memory Usage: 405 kB Disk Usage: 59712 kB HashAgg Batches: 4209 -> Seq Scan on gs_hash_1 (actual rows=20 loops=1) I'm not sure if this is more what you were looking for--or maybe I am misunderstanding the guc. -- Melanie Plageman

Re: Default setting for enable_hashagg_disk

2020-06-10 Thread Melanie Plageman
On Tue, Jun 9, 2020 at 7:15 PM Justin Pryzby wrote: > On Tue, Jun 09, 2020 at 06:20:13PM -0700, Melanie Plageman wrote: > > On Thu, Apr 9, 2020 at 1:02 PM Jeff Davis wrote: > > > > > 2. enable_groupingsets_hash_disk (default false): > > > > > > This

Re: Default setting for enable_hashagg_disk

2020-06-10 Thread Melanie Plageman
On Wed, Jun 10, 2020 at 10:39 AM Jeff Davis wrote: > On Tue, 2020-06-09 at 18:20 -0700, Melanie Plageman wrote: > > So, I was catching up on email and noticed the last email in this > > thread. > > > > I think I am not fully understanding what > > enabl

Re: Find query characters in respect of optimizer for develop purpose

2020-06-11 Thread Melanie Plageman
hat it seems like this struct would have to be updated throughout planning and that it would be easy to break it with the addition of new code. Couldn't every new optimization added to planner potentially affect the accuracy of the information in the struct? -- Melanie Plageman

  1   2   3   4   5   6   7   >