Hello -hackers!
Please have a look at the attached patch, which implements some
statistics for TOAST.
The idea (and patch) have been lurking here for quite a while now, so I
decided to dust it off, rebase it to HEAD and send it out for review today.
A big shoutout to Georgios Kokolatos, who
Hi,
On 2021-12-12 17:20:58 +0100, Gunnar "Nick" Bluth wrote:
> Please have a look at the attached patch, which implements some statistics
> for TOAST.
>
> The idea (and patch) have been lurking here for quite a while now, so I
> decided to dust it off, rebase it to HEAD and send it out for review
Am 12.12.21 um 22:52 schrieb Andres Freund:
Hi,
On 2021-12-12 17:20:58 +0100, Gunnar "Nick" Bluth wrote:
Please have a look at the attached patch, which implements some statistics
for TOAST.
The idea (and patch) have been lurking here for quite a while now, so I
decided to dust it off, rebase
Hi,
On 2021-12-13 00:00:23 +0100, Gunnar "Nick" Bluth wrote:
> Regarding stats size; it adds one PgStat_BackendToastEntry
> (PgStat_BackendAttrIdentifier + PgStat_ToastCounts, should be 56-64 bytes or
> something in that ballpark) per TOASTable attribute, I can't see that make
> any system break s
Am 13.12.21 um 00:41 schrieb Andres Freund:
Hi,
On 2021-12-13 00:00:23 +0100, Gunnar "Nick" Bluth wrote:
Regarding stats size; it adds one PgStat_BackendToastEntry
(PgStat_BackendAttrIdentifier + PgStat_ToastCounts, should be 56-64 bytes or
something in that ballpark) per TOASTable attribute, I
Dear Gunnar,
> postgres=# CREATE TABLE test (i int, lz4 text COMPRESSION lz4, std text);
> postgres=# INSERT INTO test SELECT
> i,repeat(md5(i::text),100),repeat(md5(i::text),100) FROM
> generate_series(0,10) x(i);
> postgres=# SELECT * FROM pg_stat_toast WHERE schemaname = 'public';
> -[ REC
Am 20.12.2021 um 04:20 schrieb kuroda.hay...@fujitsu.com:
Dear Gunnar,
Hi Kuroda-San!
postgres=# CREATE TABLE test (i int, lz4 text COMPRESSION lz4, std text);
postgres=# INSERT INTO test SELECT
i,repeat(md5(i::text),100),repeat(md5(i::text),100) FROM
generate_series(0,10) x(i);
postgres
uth
Eimermacherweg 106
D-48159 Münster
Mobil +49 172 8853339
Email: gunnar.bl...@pro-open.de
__
"Ceterum censeo SystemD esse delendam" - CatoFrom e4f6b8a9b398ff08f76e0c02ca7a1201062beb78 Mon Sep 17 00:00:00 2001
From: "G
Am 05.04.22 um 18:17 schrieb Robert Haas:
> On Thu, Mar 31, 2022 at 9:16 AM Gunnar "Nick" Bluth
> wrote:
>> That was meant to say "v10", sorry!
>
> Hi,
Hi Robert,
and thx for looking at this.
> From my point of view, at least, it would be preferable if you'd stop
> changing the subject line ev
Hi,
On 2022-04-06 00:08:13 +0200, Gunnar "Nick" Bluth wrote:
> AFAICT, Andres' work is more about the structure (e.g.
> 13619598f1080d7923454634a2570ca1bc0f2fec). Or I've missed something...?
That was just the prep work... I'm about to send slightly further polished
version, but here's the patchs
On Tue, Apr 5, 2022 at 10:34 PM Andres Freund wrote:
> > Anyway, my (undisputed up to now!) understanding still is that only
> > backends _looking_ at these stats (so, e.g., accessing the pg_stat_toast
> > view) actually read the data. So, the 10-15% more space used for pg_stat
> > only affect the
On 2022-Apr-06, Robert Haas wrote:
> Now if we're only incurring that overhead when this feature is
> enabled, then in fairness that problem is a lot less of an issue,
> especially if this is also disabled by default. People who want the
> data can get it and pay the cost, and others aren't much i
On Tue, Apr 5, 2022 at 6:08 PM Gunnar "Nick" Bluth
wrote:
> At the end of the day, one could argue like you did there for almost all
> (non-attribute) stats. "Why track function execution times? Just set up
> a benchmark and call the function 1 mio times and you'll know how long
> it takes on aver
On Wed, Apr 6, 2022 at 11:49 AM Alvaro Herrera wrote:
> On 2022-Apr-06, Robert Haas wrote:
> > Now if we're only incurring that overhead when this feature is
> > enabled, then in fairness that problem is a lot less of an issue,
> > especially if this is also disabled by default. People who want th
Am 06.04.22 um 17:22 schrieb Robert Haas:
> On Tue, Apr 5, 2022 at 10:34 PM Andres Freund wrote:
>>> Anyway, my (undisputed up to now!) understanding still is that only
>>> backends _looking_ at these stats (so, e.g., accessing the pg_stat_toast
>>> view) actually read the data. So, the 10-15% mor
Am 06.04.22 um 17:49 schrieb Robert Haas:
> I feel like if you want to know whether externalization made a
> difference, you can look at the size of the TOAST table. And by
> selecting directly from that table, you can even see how many chunks
> it contains, and how many are full-sized chunks vs.
On Wed, Apr 6, 2022 at 12:01 PM Gunnar "Nick" Bluth
wrote:
> Fair enough. At that point, a lot of things become unexpectedly painful.
> How many % of the installed base may that be though?
I don't have statistics on that, but it's large enough that the
expense associated with the statistics colle
Hi,
On 2022-04-06 12:24:20 -0400, Robert Haas wrote:
> On Wed, Apr 6, 2022 at 12:01 PM Gunnar "Nick" Bluth
> wrote:
> > Fair enough. At that point, a lot of things become unexpectedly painful.
> > How many % of the installed base may that be though?
>
> I don't have statistics on that, but it's
Am 06.04.22 um 18:55 schrieb Andres Freund:
> Hi,
>
> On 2022-04-06 12:24:20 -0400, Robert Haas wrote:
>> On Wed, Apr 6, 2022 at 12:01 PM Gunnar "Nick" Bluth
>> wrote:
>>> Fair enough. At that point, a lot of things become unexpectedly painful.
>>> How many % of the installed base may that be tho
Am 06.04.22 um 17:49 schrieb Alvaro Herrera:
> On 2022-Apr-06, Robert Haas wrote:
>
>> Now if we're only incurring that overhead when this feature is
>> enabled, then in fairness that problem is a lot less of an issue,
>> especially if this is also disabled by default. People who want the
>> data
Dear Gunnar,
> The attached v0.3 includes
> * columns "storagemethod" and "compressmethod" added as per Hayato
> Kuroda's suggestion
I prefer your implementation that referring another system view.
> * gathering timing information
Here is a minor comment:
I'm not sure when we should start to
Hi,
On 2022-03-08 19:32:03 +0100, Gunnar "Nick" Bluth wrote:
> v8 (applies cleanly to today's HEAD/master) attached.
This doesn't apply anymore, likely due to my recent pgstat changes - which
you'd need to adapt to...
http://cfbot.cputube.org/patch_37_3457.log
Marked as waiting on author.
Gree
Am 22.03.22 um 02:17 schrieb Andres Freund:
> Hi,
>
> On 2022-03-08 19:32:03 +0100, Gunnar "Nick" Bluth wrote:
>> v8 (applies cleanly to today's HEAD/master) attached.
>
> This doesn't apply anymore, likely due to my recent pgstat changes - which
> you'd need to adapt to...
Now, that's been quit
Am 22.03.22 um 12:23 schrieb Gunnar "Nick" Bluth:
> Am 22.03.22 um 02:17 schrieb Andres Freund:
>> Hi,
>>
>> On 2022-03-08 19:32:03 +0100, Gunnar "Nick" Bluth wrote:
>>> v8 (applies cleanly to today's HEAD/master) attached.
>>
>> This doesn't apply anymore, likely due to my recent pgstat changes -
Am 31.03.22 um 15:14 schrieb Gunnar "Nick" Bluth:
> Am 22.03.22 um 12:23 schrieb Gunnar "Nick" Bluth:
>> Am 22.03.22 um 02:17 schrieb Andres Freund:
>>> Hi,
>>>
>>> On 2022-03-08 19:32:03 +0100, Gunnar "Nick" Bluth wrote:
v8 (applies cleanly to today's HEAD/master) attached.
>>>
>>> This doesn
On Thu, Mar 31, 2022 at 9:16 AM Gunnar "Nick" Bluth
wrote:
> That was meant to say "v10", sorry!
Hi,
>From my point of view, at least, it would be preferable if you'd stop
changing the subject line every time you post a new version.
Based on the test results in
http://postgr.es/m/42bfa680-7998-
Am 21.12.21 um 13:51 schrieb kuroda.hay...@fujitsu.com:
Dear Gunnar,
Hayato-san, all,
thanks for the feedback!
* gathering timing information
Here is a minor comment:
I'm not sure when we should start to measure time.
If you want to record time spent for compressing, toast_compress_datum()
Am 03.01.22 um 16:52 schrieb Gunnar "Nick" Bluth:
pg_stat_toast_v0.4.patch attached.
Aaaand I attached a former version of the patch file... sorry, I'm kind
of struggling with all the squashing/rebasing...
--
Gunnar "Nick" Bluth
Eimermacherweg 106
D-48159 Münster
Mobil +49 172 8853339
Emai
On Mon, Jan 03, 2022 at 05:00:45PM +0100, Gunnar "Nick" Bluth wrote:
> Am 03.01.22 um 16:52 schrieb Gunnar "Nick" Bluth:
>
> > pg_stat_toast_v0.4.patch attached.
Note that the cfbot says this fails under windows
http://cfbot.cputube.org/gunnar-quotnickquot-bluth.html
...
[16:47:05.347] Could not
Am 03.01.22 um 17:50 schrieb Justin Pryzby:
On Mon, Jan 03, 2022 at 05:00:45PM +0100, Gunnar "Nick" Bluth wrote:
Am 03.01.22 um 16:52 schrieb Gunnar "Nick" Bluth:
pg_stat_toast_v0.4.patch attached.
Note that the cfbot says this fails under windows
Thanks for the heads up!
http://cfbot.
On 2022-Jan-03, Gunnar "Nick" Bluth wrote:
> Am 03.01.22 um 17:50 schrieb Justin Pryzby:
> > Soon you will think this is fun :)
>
> As long as you're happy with plain patches like the attached one, I may ;-)
Well, with a zero-byte patch, not very much ...
BTW why do you number with a "0." pref
Am 03.01.22 um 19:30 schrieb Alvaro Herrera:
On 2022-Jan-03, Gunnar "Nick" Bluth wrote:
Am 03.01.22 um 17:50 schrieb Justin Pryzby:
Soon you will think this is fun :)
As long as you're happy with plain patches like the attached one, I may ;-)
Well, with a zero-byte patch, not very much .
On 2022-Jan-03, Gunnar "Nick" Bluth wrote:
> 9:38 $ git format-patch PGDG/master -v5 -o ..
> ../v5-0001-ping-pong-of-thougths.patch
> ../v5-0002-ping-pong-of-thougths.patch
> ../v5-0003-adds-some-debugging-messages-in-toast_helper.c.patch
> ...
Hmm, in such cases I would suggest to create a sepa
Am 03.01.22 um 20:11 schrieb Alvaro Herrera:
On 2022-Jan-03, Gunnar "Nick" Bluth wrote:
9:38 $ git format-patch PGDG/master -v5 -o ..
../v5-0001-ping-pong-of-thougths.patch
../v5-0002-ping-pong-of-thougths.patch
../v5-0003-adds-some-debugging-messages-in-toast_helper.c.patch
...
Hmm, in suc
On Mon, Jan 03, 2022 at 08:40:50PM +0100, Gunnar "Nick" Bluth wrote:
> I wonder why "track_toast.sql" test fails on Windows (with "ERROR:
> compression method lz4 not supported"), but "compression.sql" doesn't.
> Any hints?
The windows CI doesn't have LZ4, so the SQL command fails, but there's an
> +pgstat_report_toast_activity(Oid relid, int attr,
> + bool externalized,
> + bool compressed,
> + int32 old_size,
> +
Overall I think this is a good feature to have; assessing the need for
compression is important for tuning, so +1 for the goal of the patch.
I didn't look into the patch carefully, but here are some minor
comments:
On 2022-Jan-03, Gunnar "Nick" Bluth wrote:
> @@ -229,7 +230,9 @@ toast_tuple_try_
Am 03.01.22 um 22:03 schrieb Justin Pryzby:
+pgstat_report_toast_activity(Oid relid, int attr,
+ bool externalized,
+ bool compressed,
+
Am 03.01.22 um 22:23 schrieb Alvaro Herrera:
Overall I think this is a good feature to have; assessing the need for
compression is important for tuning, so +1 for the goal of the patch.
Much appreciated!
I didn't look into the patch carefully, but here are some minor
comments:
On 2022-Jan-0
Am 04.01.22 um 12:29 schrieb Gunnar "Nick" Bluth:
> Am 03.01.22 um 22:23 schrieb Alvaro Herrera:
>> Overall I think this is a good feature to have; assessing the need for
>> compression is important for tuning, so +1 for the goal of the patch.
>
> Much appreciated!
>
>
>> I didn't look into the
40 matches
Mail list logo
