Hi,
> Aleksander, thank you for reminding me of this patch, try to do it in a few
> days.
A consensus was reached [1] to mark this patch as RwF for now. There
are many patches to be reviewed and this one doesn't seem to be in the
best shape, so we have to prioritise. Please feel free
Hi!
Aleksander, thank you for reminding me of this patch, try to do it in a few
days.
--
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company
https://postgrespro.ru/
Hi Nikita,
> I've prepared a patch with a 64-bit TOAST Value ID. It is a kind of prototype
> and needs some further work, but it is already working and ready to play with.
Unfortunately the patch rotted a bit. Could you please submit an
updated/rebased patch so that it could be reviewed in the
Hi!
Widening of a TOAST pointer is possible if we keep backward compatibility
with
old-fashioned TOAST tables - I mean differ 'long' and 'short' TOAST
pointers and
process them accordingly on insert and delete cases, and vacuum with logical
replication. It is not very difficult, however it takes
Hi,
> I agree that we can't simply widen varatt_external to use 8 bytes for
> the toast ID in all cases.
+1
Note that the user may have a table with multiple TOASTable
attributes. If we simply widen the TOAST pointer it may break the
existing tables in the edge case. Also this may be a reason
Hi!
No, it wasn't. It was a proposal, I thought I'd get some feedback on it
before sending it to commitfest.
On Sat, Apr 22, 2023 at 6:17 PM Gurjeet Singh wrote:
> On Thu, Dec 22, 2022 at 10:07 AM Nikita Malakhov
> wrote:
> > Any suggestions on the previous message (64-bit toast value ID with
On Thu, Dec 22, 2022 at 10:07 AM Nikita Malakhov wrote:
> Any suggestions on the previous message (64-bit toast value ID with
> individual counters)?
Was this patch ever added to CommitFest? I don't see it in the current
Open Commitfest.
https://commitfest.postgresql.org/43/
Best regards,
Hi hackers!
Any suggestions on the previous message (64-bit toast value ID with
individual counters)?
On Tue, Dec 13, 2022 at 1:41 PM Nikita Malakhov wrote:
> Hi hackers!
>
> I've prepared a patch with a 64-bit TOAST Value ID. It is a kind of
> prototype
> and needs some further work, but it
Hi hackers!
I've prepared a patch with a 64-bit TOAST Value ID. It is a kind of
prototype
and needs some further work, but it is already working and ready to play
with.
I've introduced 64-bit value ID field in varatt_external, but to keep it
compatible
with older bases 64-bit value is stored as
Hi hackers!
Here's some update on the subject - I've made a branch based on master from
28/11 and introduced 64-bit TOAST value ID there. It is not complete now but
is already working and has some features:
- extended structure for TOAST pointer (varatt_long_external) with 64-bit
TOAST value ID;
Hi!
I'm working on this issue according to the plan Tom proposed above -
>I agree that we can't simply widen varatt_external to use 8 bytes for
>the toast ID in all cases. Also, I now get the point about avoiding
>use of globally assigned OIDs here: if the counter starts from zero
>for each
I've missed that -
>4 billion external datums
>typically use a lot of space.
Not quite so. It's 8 Tb for the minimal size of toasted data (about 2 Kb).
In my practice tables with more than 5 billions of rows are not of
something out
of the ordinary (highly loaded databases with large amounts of
Andres Freund writes:
> On 2022-11-28 16:57:53 -0500, Tom Lane wrote:
>> As I said before, I think there's a decent argument that some people
>> will want the option to stay with 4-byte TOAST OIDs indefinitely,
>> at least for smaller tables.
> And as you said earlier, the increased overhead
Hi,
I'll check that tomorrow. If it is so then there won't be a problem keeping
old tables without re-toasting.
On Tue, Nov 29, 2022 at 1:10 AM Andres Freund wrote:
> Hi,
>
> On 2022-11-28 16:57:53 -0500, Tom Lane wrote:
> > As I said before, I think there's a decent argument that some people
Hi,
On 2022-11-28 16:57:53 -0500, Tom Lane wrote:
> As I said before, I think there's a decent argument that some people
> will want the option to stay with 4-byte TOAST OIDs indefinitely,
> at least for smaller tables.
I think we'll need to do something about the width of varatt_external to
Andres Freund writes:
> On 2022-11-29 00:24:49 +0300, Nikita Malakhov wrote:
>> 2) Upgrading to 64-bit id would require re-toasting old TOAST tables. Or
>> there is some way to distinguish old tables from new ones?
> The catalog / relcache entry should suffice to differentiate between the two.
Hi,
On 2022-11-29 00:24:49 +0300, Nikita Malakhov wrote:
> 2) Upgrading to 64-bit id would require re-toasting old TOAST tables. Or
> there is some way to distinguish old tables from new ones?
The catalog / relcache entry should suffice to differentiate between the two.
Greetings,
Andres
Andres Freund writes:
> On 2022-11-28 16:04:12 -0500, Tom Lane wrote:
>> And I don't buy that either. An extra 4 bytes with a 2K payload is not
>> "prohibitive", it's more like "down in the noise".
> The space usage for the the the toast relation + index itself is indeed
> irrelevant. Where
Hi,
Andres Freund writes:
>Was the issue that you exceeded 4 billion toasted datums, or that
assignment
>took a long time? How many toast datums did you actually have? Was this
due to
>very wide rows leading to even small datums getting toasted?
Yep, we had 4 billion toasted datums. I remind
Hi,
On 2022-11-28 16:04:12 -0500, Tom Lane wrote:
> Andres Freund writes:
> > - to
> > combat the space usage we likely should switch to representing the ids as
> > variable width integers or such, otherwise the space increase would likely
> > be
> > prohibitive.
>
> And I don't buy that
Andres Freund writes:
> I think the first step to improve the situation is to not use a global oid
> counter for toasted values. One way to do that would be to use the sequence
> code to do oid assignment, but we likely can find a more efficient
> representation.
I don't particularly buy that,
Hi,
On 2022-11-28 23:54:53 +0300, Nikita Malakhov wrote:
> We've already encountered this issue on large production databases, and
> 4 billion rows is not so much for modern bases, so this issue already arises
> from time to time and would arise more and more often.
Was the issue that you
Hi!
We've already encountered this issue on large production databases, and
4 billion rows is not so much for modern bases, so this issue already arises
from time to time and would arise more and more often. I agree that global
oid counter is the main issue, and better solution would be local
Hi,
On 2022-11-28 18:34:20 +0300, Nikita Malakhov wrote:
> While working on Pluggable TOAST we've detected a defective behavior
> on tables with large amounts of TOASTed data - queries freeze and DB stalls.
> Further investigation led us to the loop with GetNewOidWithIndex function
> call - when
Hi hackers!
While working on Pluggable TOAST we've detected a defective behavior
on tables with large amounts of TOASTed data - queries freeze and DB stalls.
Further investigation led us to the loop with GetNewOidWithIndex function
call - when all available Oid already exist in the related TOAST
25 matches
Mail list logo
