h to have to give out
>> privileges to both at once.
>
> Good idea!
I hope that you follow up on this soon.
--
Peter Geoghegan
code. I had to push to get
us to give external sorts test coverage at one point about 18 months
ago, because of concerns about the overhead/duration of external
sorts. Not that I feel strongly about this myself.
--
Peter Geoghegan
On Wed, Feb 21, 2018 at 10:55 AM, Peter Geoghegan <p...@bowt.ie> wrote:
> Sure, but it looks like it has the exact same underlying cause to the
> LogicalTapeFreeze() issue. It shouldn't be very hard to write an
> equivalent patch for LogicalTapeRewindForRead() -- I pointed out th
osting a patch that I'm going to mark "ready for
committer". I've already made the change above, and once I spend time
on trying to break the few small changes needed within buffile.c I'll
have taken it as far as I can, most likely.
--
Peter Geoghegan
s not a priority).
What we really cared about was the fact that it was possible to make a
backend's relcache irrecoverably corrupt. That should never be allowed
to happen, even when the user is determined to do something
unreasonable.
--
Peter Geoghegan
Can I really fix space utilization in a piecemeal fashion? I strongly doubt it.
--
Peter Geoghegan
On Thu, Jul 26, 2018 at 7:33 PM, Michael Paquier wrote:
> The maintenance load is not high as well, so I see no real point in
> removing it, and that it would likely make people using it unhappy.
Why, specifically, would it make them unhappy?
--
Peter Geoghegan
nth. Let us know what patches you think are scariest:
Is there going to be an announcement of the results?
--
Peter Geoghegan
ticularly ok with just ignoring that?
+1
--
Peter Geoghegan
original -bug thread.
I certainly would not say that you were in any way inept. Perhaps
there is an argument for *also* doing what you propose. I am not
dismissive of your idea.
It seems like a question that should be considered separately, on
another thread. If you still want to pursue it.
--
Peter Geoghegan
ug that I just
pushed a fix for. It isn't known to cause an issue on releases prior
to v11, so I'm not sure on the best way of phrasing it for users.
Thanks
--
Peter Geoghegan
t have a "c" column.
Do you have some particular practical problem in mind here, or are you
just concerned about the semantics being exactly consistent?
--
Peter Geoghegan
ere we paper over the problem in a similar way. I
notice that it tends to happen when the machine running the regression
tests is heavily loaded.
--
Peter Geoghegan
heck-world passed
How does this affect ordinary opportunistic pruning?
--
Peter Geoghegan
g to their notes. That actually
isn't enough to force parallel CREATE INDEX, though I tend to think it
should be. I don't see anything interesting in their "extra_config",
but perhaps "min_parallel_table_scan_size = 0" got in some other way.
I don't know all that much about the buildfarm client code, and it's
late.
--
Peter Geoghegan
/CAH2-Wzn=j0i8rxCAo6E=tbo9xuyxb8hbusnw7j_stkon8dd...@mail.gmail.com
--
Peter Geoghegan
d with it yourself, I suggest modifying the
DEBUG1 elog()s within index_build() to be LOG elevel.
--
Peter Geoghegan
tions, given the error occurs before any subscriptions are
> created in the schedule.
+1. I like that idea.
--
Peter Geoghegan
e us something new to chase.
Thanks for pointing out that Tomas had a lead - I missed that.
I'm concerned that this item has the potential to delay the release,
since, as you said, we're back to the drawing board.
--
Peter Geoghegan
at all advisable to make an executive decision to
release, while still not having the foggiest idea what's really going
on.
--
Peter Geoghegan
e entire buffer noaccess when there is no pin held.
--
Peter Geoghegan
that were just stamped + tagged.
They should be released shortly.
--
Peter Geoghegan
all functions
in practice.
I think that this has the potential to confuse users, since, of
course, a function is a distinct variety of object to a procedure in
v11. Tightening up the wording seems like a good idea. I'm not sure if
this was discussed already, but didn't find anything during a quick
search.
--
Peter Geoghegan
On Tue, Aug 14, 2018 at 10:18 AM, Tom Lane wrote:
> Agreed, we'd better stop being cavalier about whether that's an
> exact synonym or not.
I made an open item for this.
--
Peter Geoghegan
On Tue, Aug 14, 2018 at 1:39 PM, Tom Lane wrote:
> We're kinda stuck with that. We could add FUNCTION as a preferred
> synonym, perhaps, but I doubt that'd really be worth the trouble.
Seems sufficient to note in the relevant docs that it's a legacy thing.
--
Peter Geoghegan
ight now.)
We've already been able to rule that out as a factor here. Thanks all
the same, though.
--
Peter Geoghegan
maybe out of range)
FWIW, I suspect that we're going to have the biggest problems in the
optimizer. It's not as if ndistinct is in any way reliable. That may
matter more on average than it has with other path types.
--
Peter Geoghegan
On Wed, Aug 8, 2018 at 10:08 PM, Peter Geoghegan wrote:
>> Hmmm ... maybe we should temporarily stick in an elog(LOG) showing whether
>> a parallel build happened or not, so that we can check the buildfarm logs
>> next time we see that failure?
>
> I can do that tomor
gh number of skips, you
> could fall back to regular next-tuple mode. Unfortunately that's
> require the parent plan node to tolerate non-unique results.
I like the idea of dynamic fallback in certain situations, but the
details always seem complicated.
--
Peter Geoghegan
On Thu, Aug 23, 2018 at 5:20 PM, Peter Geoghegan wrote:
> This patch adds an enhancement that is an example of a broader class
> of optimizer enhancement primarily aimed at making star-schema queries
> have more efficient plans, by arranging to use several independent
> nested loop
cerned that the optimization added by this patch may
be too much of a special case, which is understandable. It may be that
we're failing to identify some greater opportunity to add DAG-like
plans for star schema queries.
--
Peter Geoghegan
On Fri, Aug 24, 2018 at 2:15 PM, Andres Freund wrote:
> Do they persist after you do re-configure? If so, could you send your
> config.log?
I should point out to Bruce that this is clearly related to commit
143290efd0795b61ed2c8358fc1767e799140047.
--
Peter Geoghegan
just
>
> I don't do my work in a vacuum. My behavior is based on what feedback I
> have gotten from the community on what to include/exclude.
FWIW, I agree with what Andres said about planner changes and the release notes.
--
Peter Geoghegan
On Fri, Aug 17, 2018 at 7:15 AM, Peter Eisentraut
wrote:
> Attached are my proposed patches.
I take it that you propose all 3 for backpatch to v11?
--
Peter Geoghegan
t idea to what I talked about, since it could perhaps
work in a way that's much closer to LP_DEAD/kill_prior_tuple (no extra
executor stuff is required). I'm not sure if your idea is better or
worse than what I suggested, though. It would certainly be easier to
implement.
--
Peter Geoghegan
uce any two-decade-old problems.
A mountain of work remains to validate the performance of the patch.
--
Peter Geoghegan
by most
or all real-world B-Tree implementations.
[1] https://www.postgresql.org/message-id/18788.963953...@sss.pgh.pa.us
--
Peter Geoghegan
thing.
I made one small mistake with my test case: It actually *is* perfectly
efficient at recycling space even at the end, since I don't delete all
the duplicates (just 90% of them). Getting tired might have been a
contributing factor there, too.
--
Peter Geoghegan
@telsasoft.com
[3]
https://postgr.es/m/520D6610.8040907%40emulex.com#520d6610.8040...@emulex.com
[4] https://postgr.es/m/22002.1487099934%40sss.pgh.pa.us
--
Peter Geoghegan
-gnu/libc.so.6 (linked with --as-needed)
I didn't have this problem.
--
Peter Geoghegan
s.
[1]
https://wiki.postgresql.org/wiki/Key_normalization#Using_existing_.22minus_infinity.22_index_tuple_as_a_low_key
[2]
https://www.postgresql.org/message-id/CAH2-Wz=mv4dmoapficrsyntv2kinxeotbwuy5r7fxxoc-oe...@mail.gmail.com
[3] https://openproceedings.org/2013/conf/edbt/TozunPKJA13.pdf --
"6.1.2 Core stalls"
--
Peter Geoghegan
rsuing.
I'm not dismissing your idea. I'm just pointing out that the burden of
proving that explicit prefetching is a good idea is rather
significant. We especially want to avoid something that needs to be
reassessed every couple of years.
--
Peter Geoghegan
or two other small things along those
lines.
[1] https://wiki.postgresql.org/wiki/Key_normalization
--
Peter Geoghegan
tial.
I also think it's important to know *why* you're doing something.
Ideally, you'll have multiple reasons for working on something, and
not just one. Ideally, you'll be able to pick a project that could
easily lead to another project, and then another. A "virtuous circle"
can be created.
--
Peter Geoghegan
ich extracts the WAL information and updates the page.
I wonder how you managed to detect it in the first place. Were you
using something like wal_consistency_checking=all, with a custom
stress-test?
--
Peter Geoghegan
pe that this
problem can be fixed.
--
Peter Geoghegan
look at
the code that picks a split point, though. Let me know if you think
that that makes sense. I wouldn't want you to spend too much time on
old-ish code.
--
Peter Geoghegan
On Wed, Jul 4, 2018 at 7:53 PM, Michael Paquier wrote:
> On Wed, Jul 04, 2018 at 06:54:05PM -0700, Peter Geoghegan wrote:
>> I don't know about any of that, but something has to give. How much
>> more time has to pass before we admit defeat? At a certain point, that
>> is
. Often,
being sure that dropping an index won't have any ramifications is an
unobtainable luxury, because knowledge about how the app works isn't
centralized in one place. If it's a very large index, why even take a
very small chance?
--
Peter Geoghegan
's what Andrew had in mind
when reading his original message. I only realized later that it
wasn't.
--
Peter Geoghegan
if possible instead of
> throwing an error. Perhaps logging a WARNING could make sense.
I don't know about any of that, but something has to give. How much
more time has to pass before we admit defeat? At a certain point, that
is the responsible thing to do.
--
Peter Geoghegan
e be?
>
> It seems like a good idea to me.
+1
--
Peter Geoghegan
vely
addressed by suffix truncation -- why not just make it so that those
pages don't become half empty to begin with?
I could perhaps work on prefix compression. But probably not in the
next 3 years. It's way down the list for me.
--
Peter Geoghegan
On Wed, Jul 4, 2018 at 9:43 AM, Peter Geoghegan wrote:
> I'm starting this new thread to discuss the benefits of B-Tree suffix
> truncation, and to demonstrate how effective it can be at increasing
> fan-out and space utilization with a real-world example. I haven't
> explaine
a lock, since in practice
nobody gets starved out. I'm guessing (and it is very much a guess)
that it could be something like that, since the behavior of
_bt_findinsertloc() can be FIFO-like, whereas the behavior of
_bt_moveright() may not be.
Again, the actual queries would help a lot. It's just a guess.
--
Peter Geoghegan
er. ReleaseAndReadBuffer()
could end up repeatedly pinning and unpinning the same heap pages when
they appear again and again out of order on an nbtree leaf page. If
that's true, then it might not matter how smart our algorithm is.
--
Peter Geoghegan
tive type, even the same type as
> the original table.
tuple_data_split() can do this (split the data into individual
attributes). If you want a friendlier, more visual way of getting this
information, then you may find pg_hexedit helpful:
https://github.com/petergeoghegan/pg_hexedit
--
Peter Geoghegan
On Tue, Jul 10, 2018 at 6:44 PM, Yang Jie wrote:
> my question is not split the data into individual attributes.
> I want to see the data in the table, but I don't want to be a bytea type.
What's wrong with simply using an SQL query?
--
Peter Geoghegan
ted sense, but querying old data like
that seems totally impractical. You might do something like that for
forensic purposes, as a once off thing, but you didn't say anything
about your high level goals here.
--
Peter Geoghegan
Though, of course, it's
still very complicated.
--
Peter Geoghegan
On Mon, Jul 24, 2017 at 11:12 AM, Peter Geoghegan wrote:
> You can get about a 1/3 loss of space by inserting randomly, rather
> than inserting in sorted order, which is what REINDEX will more or
> less do for you. That's because random workloads almost entirely get
> 50:50 page spl
checklist. Still, it wouldn't hurt
> to have a wiki page of checklist entry ideas from which folks cherry-pick the
> entries they like.
You've convinced me that we should definitely have such a list. I've
put it on my TODO list.
--
Peter Geoghegan
clusive mode.
_bt_findinsertloc() couples/crabs exclusive buffer locks because the
unique case requires it, even when we're not inserting into a unique
index. Whereas _bt_moveright() holds at most one buffer lock at a
time.
--
Peter Geoghegan
indistinguishable from zero.
--
Peter Geoghegan
need buy-in from
the user. No new GUCs required.
[1] https://wiki.postgresql.org/wiki/Sample_Databases#Other_Samples
--
Peter Geoghegan
runcation:
https://postgr.es/m/CAH2-Wzn5XbCzk6u0GL+uPnCp1tbrp2pJHJ=3byt4yq0_zzh...@mail.gmail.com
Perhaps this is related in some way, since in both cases we're talking
about a composite index on varlena-type columns, where the types have
expensive comparisons.
--
Peter Geoghegan
lesort.c, even
though they may be equal in other contexts. This is likely to defeat
things like the Bentley-McIlroy optimization where equal keys are
swapped, which is very effective in the event of many equal keys.
(Could also be parallelism, though I suppose you probably accounted for that.)
--
Peter Geoghegan
(or even
WaitForParallelWorkersToFinish()). If we did add a
WaitForParallelWorkersToAttach() function, then the performance hit
would probably not be noticeable in practice. The
parallel_leader_participates case would still work without special
care (that's the main hazard that makes using a barrier seem
unappealing).
--
Peter Geoghegan
On Tue, Jan 23, 2018 at 9:38 PM, Amit Kapila <amit.kapil...@gmail.com> wrote:
> On Wed, Jan 24, 2018 at 10:38 AM, Peter Geoghegan <p...@bowt.ie> wrote:
>> The leader can go ahead and sort before calling something like a new
>> WaitForParallelWorkersT
fork
> failure (ie kill SIGUSR1) and after GetBackgroundWorkerPid() is
> guaranteed to return BGWH_STOPPED after that, and as long as we only
> ever use latch/CFI loops to wait, and as long as we try to read from a
> shm_mq, then I don't see a failure mode that hangs.
What about the parallel_leader_participation=off case?
--
Peter Geoghegan
On Tue, Jan 23, 2018 at 10:50 AM, Peter Geoghegan <p...@bowt.ie> wrote:
> On Tue, Jan 23, 2018 at 10:36 AM, Robert Haas <robertmh...@gmail.com> wrote:
>> I am going to repeat my previous suggest that we use a Barrier here.
>> Given the discussion subsequent to my
And, more importantly, it would be tricky
to use a barrier even for this, because we still have that baked-in
assumption that nworkers_launched is the single source of truth about
the number of participants.
--
Peter Geoghegan
we can do about
that, though.
--
Peter Geoghegan
nd. I do not know did the patch had all necessary
> reviewers attention. Please, feel free to change status if you think that
> patch is ready. From my point of view, the patch is in a good shape.
Michael said he'd do more review. I generally feel this is close, though.
Thanks for the review
--
Peter Geoghegan
lization, and therefore can be
expected to show up.
Is there actually a meaningful difference between the way
nworkers_launched is depended upon in each case, though?
--
Peter Geoghegan
der_participation=off (or some way of getting the
same behavior through a #define) is to be retained, then an artificial
wait is required as a substitute for the leader's participation as a
worker.
--
Peter Geoghegan
us something.
Moreover, parallel_leader_participation=off is always going to be
wonky under a regime based on "see who shows up", because failure
becomes indistinguishable from high latency -- I'd rather avoid having
to think about nbtsort.c in distributed systems terms.
--
Peter Geoghegan
On Thu, Mar 1, 2018 at 2:47 PM, Peter Geoghegan <p...@bowt.ie> wrote:
> No. Just an oversight. Looks like _bt_parallel_build_main() should
> call pgstat_report_activity(), just like ParallelQueryMain().
>
> I'll come up with a patch soon.
Attached patch has parallel CREA
st enforcing !IsInParallelMode() in the set functions is
sufficient. Attached patch does this.
--
Peter Geoghegan
From 79f6708165c83c39b3e1bf785539bee84a28f650 Mon Sep 17 00:00:00 2001
From: Peter Geoghegan <p...@bowt.ie>
Date: Sun, 11 Mar 2018 12:18:25 -0700
Subject: [PATCH] Fix corr
On Wed, Feb 28, 2018 at 8:53 AM, Robert Haas <robertmh...@gmail.com> wrote:
> On Tue, Feb 27, 2018 at 5:07 PM, Peter Geoghegan <p...@bowt.ie> wrote:
>> I now feel like Simon's suggestion of throwing an error in corner
>> cases isn't so bad. It still seems like we could d
e in the update chain.
Maybe we should actually formalize that you're only supposed to do a
PK or unique index equi-join within the main ON join, though -- you
can do something fancy with the source table, but not the target
table.
--
Peter Geoghegan
I wonder why DefineCustomStringVariable() does not set var->reset_val.
We see that within DefineCustomEnumVariable(),
DefineCustomRealVariable(), DefineCustomIntVariable(), etc.
--
Peter Geoghegan
t someone would make that mistake, just
about, but they'd have to be writing their DML statement on
auto-pilot.
--
Peter Geoghegan
ng the same tuples N times.
> +*/
> + if (node->mt_whichplan < node->mt_nplans && (operation !=
> CMD_MERGE))
> {
> resultRelInfo++;
> subplanstate = node->mt_plans[node->mt_whichplan];
* Is there a way to make what I describe (having a single target RTE)
work with partitioning, without any big new special cases, especially
in the executor?
* Any thoughts on this multiple-RTEs-for-target-rel business more generally?
[1]
https://postgr.es/m/CABOikdPjjG+JcdNeegrL7=btpdj6yev--v4hu8kzjttwx1s...@mail.gmail.com
--
Peter Geoghegan
O UPDATE + partitioning patch in
the works from Alvaro. In your explanation about that approach that
you cited, you wondered what the trouble might have been with ON
CONFLICT + partitioning, and supposed that the issues were similar
there. Are they? Has that turned up much?
--
Peter Geoghegan
code).
If we're going to do this, we'd have to do the same with UPDATE, IMV.
And, well, we're not gonna do that.
--
Peter Geoghegan
, which happens
> to be exactly the fix you've used in the patch.
I didn't mean to detract from that, but I can see how my wording had
that effect. I apologize.
--
Peter Geoghegan
fact
> that tuples now have different size shouldn't affect SSI. Right now,
> I'm not sure if CheckForSerializableConflictIn() just above the
> _bt_doinsert() is good idea. But even if so, I think that should be
> a subject of separate patch.
My point was that that nothing changes, because we already use what
_bt_doinsert() calls the "first valid" page. Maybe just add: "(This
reasoning also applies to INCLUDE indexes, whose extra attributes are
not considered part of the key space.)".
That's it for today.
--
Peter Geoghegan
t.
+1
Andres' remarks need to be seen in the context of the past couple of
weeks, and in the context of this being a relatively high risk patch
that was submitted quite late in the cycle.
--
Peter Geoghegan
re that amcheck gains acceptance as a way of validating a
complicated patch like this one after commit.
[1] https://www.postgresql.org/message-id/15195.1490988897%40sss.pgh.pa.us
--
Peter Geoghegan
On Fri, Apr 6, 2018 at 4:39 PM, Robert Haas <robertmh...@gmail.com> wrote:
> Committed. Thanks to David for the report and analysis and to Peter
> for the patch and study.
Thanks!
--
Peter Geoghegan
o take a step back?
I wish it was just one patch. I can think of several.
--
Peter Geoghegan
g separate 0003 +
0004 patches. For the next revision, please squash those down into
0002. Actually, maybe there should be only one patch for the next
revision. Up to you.
* Please write commit messages for your patches. I like to make these
part of the review process.
That's all for now.
--
Peter Geoghegan
h massive usage of
> ItemPointerGetBlockNumberNoCheck(&(itup->t_tid)), suggest to wrap it to
> macro something like this:
> #define BTreeInnerTupleGetDownLink(itup) \
> ItemPointerGetBlockNumberNoCheck(&(itup->t_tid))
Agreed. We do that with GIN.
--
Peter Geoghegan
On Sat, Apr 7, 2018 at 1:02 PM, Teodor Sigaev <teo...@sigaev.ru> wrote:
> Thanks to everyone, pushed.
I'll keep an eye on the buildfarm, since it's late in Russia.
--
Peter Geoghegan
On Sun, Apr 8, 2018 at 11:18 AM, Teodor Sigaev <teo...@sigaev.ru> wrote:
> Thank you, fixed
I suggest that we remove some unneeded amcheck tests, as in the
attached patch. They don't seem to add anything.
--
Peter Geoghegan
From 0dbbee5bfff8816cddf86961bf4959192f62f1ff Mon Sep 17 00:0
spicious place for you opinion?
_bt_mark_page_halfdead() looked like it had a problem, but it now
looks like I was wrong. I also verified every other
BTreeInnerTupleGetDownLink() caller. It now looks like everything is
good here.
--
Peter Geoghegan
would be better.
I agree with Michael -- shutting down the server using immediate mode
could lead to torn pages, that crash recovery will need to repair at a
later stage. I think that some strong caveats around this are required
in the pg_verify_checksums docs, at a minimum.
--
Peter Geoghegan
credibly rushed
> feel to it.
I agree that this was handled in a way that was just unsatisfactory.
It was also unnecessary, since we still have a few days left until
feature freeze.
--
Peter Geoghegan
e to being totally untested. That was a consequence
of the representation in the executor.
--
Peter Geoghegan
201 - 300 of 3113 matches
Mail list logo
