Re: meson and check-tests

2024-09-26 Thread Nazir Bilal Yavuz
Hi, On Thu, 26 Sept 2024 at 08:45, Ashutosh Bapat wrote: > > On Wed, Sep 25, 2024 at 8:24 PM Nazir Bilal Yavuz wrote: > > > > Additionally, the patch I shared earlier was only for regress/regress > > tests. From what I understand from here [1], only regress/regress >

Re: meson and check-tests

2024-09-25 Thread Nazir Bilal Yavuz
Hi, Thanks for looking into this! On Wed, 25 Sept 2024 at 13:27, Ashutosh Bapat wrote: > > On Mon, Sep 23, 2024 at 2:16 PM Nazir Bilal Yavuz wrote: > > > > On Sat, 21 Sept 2024 at 09:01, jian he wrote: > > > > > > so > > > TESTS="copy json

Re: meson and check-tests

2024-09-23 Thread Nazir Bilal Yavuz
Hi, On Mon, 23 Sept 2024 at 11:46, Nazir Bilal Yavuz wrote: > On Sat, 21 Sept 2024 at 09:01, jian he wrote: > > in [1] you mentioned "setup", but that "setup" is more or less like > > "meson test --suite setup --suite regress" > > but or

Re: meson and check-tests

2024-09-23 Thread Nazir Bilal Yavuz
issue. alter_table depending on create_index. > > TESTS="test_setup alter_table" meson test --suite regress > will fail. > TESTS="test_setup create_index alter_table" meson test --suite regress > will work. Yes, I realized that but since that is how it is done in the make builds, I didn't want to change the behaviour. Also, I think it makes sense to leave it to the tester. It is more flexible in that way. -- Regards, Nazir Bilal Yavuz Microsoft

Re: meson and check-tests

2024-09-20 Thread Nazir Bilal Yavuz
up select_into' meson test --suite regress -> passes (0.52s) $ TESTS='rangetypes multirangetypes' meson test --suite regress -> fails $ TESTS='test_setup multirangetypes rangetypes' meson test --suite regres -> fails $ TESTS='test_setup rangetypes mu

Re: per backend I/O statistics

2024-09-17 Thread Nazir Bilal Yavuz
Hi, On Tue, 17 Sept 2024 at 16:07, Bertrand Drouvot wrote: > On Tue, Sep 17, 2024 at 02:52:01PM +0300, Nazir Bilal Yavuz wrote: > > Could we remove pg_stat_get_my_io() completely and use > > pg_stat_get_backend_io() with the current backend's pid to get the > > curren

Re: per backend I/O statistics

2024-09-17 Thread Nazir Bilal Yavuz
Hi, On Fri, 13 Sept 2024 at 19:09, Bertrand Drouvot wrote: > On Fri, Sep 13, 2024 at 04:45:08PM +0300, Nazir Bilal Yavuz wrote: > > - The pgstat_reset_io_counter_internal() is called in the > > pgstat_shutdown_hook(). This causes the stats_reset column showing the > > termin

Re: per backend I/O statistics

2024-09-13 Thread Nazir Bilal Yavuz
_reset column showing the termination time of the old backend when its proc num is reassigned to a new backend. -- Regards, Nazir Bilal Yavuz Microsoft

Re: PG_TEST_EXTRA and meson

2024-09-12 Thread Nazir Bilal Yavuz
a to --pg-test-extra using > convention similar to other arguments. > 3. few other cosmetic changes. > > Please review and incorporate those in the respective patches and > tests. Sorry for a single diff. > > Once this is done, I think we can mark this CF entry as RFC. Thanks

Re: Avoid dead code (contrib/pg_visibility/pg_visibility.c)

2024-09-11 Thread Nazir Bilal Yavuz
ersight. > > > > The report is: > > CID 1559993: (#1 of 1): Logically dead code (DEADCODE) Thanks for the report! > I am not sure to understand what you mean here and if this is still > relevant as of Noah's latest commit in 65c310b310a6. This should be fixed in 65c310b310a6. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Allow CI to only run the compiler warnings task

2024-09-11 Thread Nazir Bilal Yavuz
it > it renames ci-os-only to ci-task-only. I think this change makes sense. I gave a quick try to your patch with ci-task-only: ["", "linux", "compilerwarnings"] and it worked as expected. -- Regards, Nazir Bilal Yavuz Microsoft

Re: PG_TEST_EXTRA and meson

2024-09-11 Thread Nazir Bilal Yavuz
"export PG_TEST_EXTRA" in /src/test/Makefile. > Even if I remove it, it doesn't affect anything. Commands a. > PG_TEST_EXTRA=xid_wraparound make check, b. > PG_TEST_EXTRA=xid_wraparound make -C $XID_MODULE_DIR check run the > tests (and don't skip them). Yes, it

Make pg_stat_io view count IOs as bytes instead of blocks

2024-09-11 Thread Nazir Bilal Yavuz
ject, context, TRUNC((read_bytes / reads / (SELECT current_setting('block_size')::numeric)), 2) as avg_io_blocks FROM pg_stat_io WHERE reads > 0; [3] SET io_combine_limit TO 32; -- Regards, Nazir Bilal Yavuz Microsoft From f02b0d261880aa3f933a9350b6b1557f6b14f292 Mon Sep 17 00:00:00 2

Re: Use read streams in pg_visibility

2024-09-10 Thread Nazir Bilal Yavuz
Hi, On Wed, 11 Sept 2024 at 01:38, Noah Misch wrote: > > On Tue, Sep 10, 2024 at 02:35:46PM +0300, Nazir Bilal Yavuz wrote: > > Your patch is correct. I wrongly assumed it would catch blockno bug, > > the attached version catches it. I made blockno = 0 invisible and not > &g

Re: Use read streams in pg_visibility

2024-09-10 Thread Nazir Bilal Yavuz
1 blockno difference between expected and result output. -- Regards, Nazir Bilal Yavuz Microsoft From 342bd8ede69942424b6d0568574db52cb574c479 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Tue, 10 Sep 2024 13:15:11 +0300 Subject: [PATCH v7 1/2] Add tests that pg_check_[visible|frozen] repo

Re: Use read streams in pg_visibility

2024-09-09 Thread Nazir Bilal Yavuz
Hi, On Thu, 5 Sept 2024 at 18:54, Noah Misch wrote: > > On Thu, Sep 05, 2024 at 03:59:53PM +0300, Nazir Bilal Yavuz wrote: > > On Wed, 4 Sept 2024 at 21:43, Noah Misch wrote: > > > https://postgr.es/m/caeudqaozv3wty5tv2t29jcwpydbmkbiwqkzd42s2ogzdixp...@mail.gmail.com >

Re: Use read streams in pg_visibility

2024-09-05 Thread Nazir Bilal Yavuz
visibility.c changes. For > the next try, could you add that testing we discussed? Do you think that corrupting the visibility map and then seeing if pg_check_visible() and pg_check_frozen() report something is enough? -- Regards, Nazir Bilal Yavuz Microsoft

Re: meson vs. llvm bitcode files

2024-09-05 Thread Nazir Bilal Yavuz
ch) a while ago [1]. [1] https://www.postgresql.org/message-id/20220927011951.j3h4o7n6bhf7dwau%40awork3.anarazel.de -- Regards, Nazir Bilal Yavuz Microsoft

Re: Use read streams in pg_visibility

2024-09-03 Thread Nazir Bilal Yavuz
Hi, On Tue, 3 Sept 2024 at 22:20, Noah Misch wrote: > > On Tue, Sep 03, 2024 at 10:50:11AM -0700, Noah Misch wrote: > > On Mon, Sep 02, 2024 at 03:20:12PM +0300, Nazir Bilal Yavuz wrote: > > > Thanks, updated patches are attached. > > > > > +/* > > &

Re: PG_TEST_EXTRA and meson

2024-09-02 Thread Nazir Bilal Yavuz
Hi, On Fri, 30 Aug 2024 at 21:36, Jacob Champion wrote: > > On Wed, Aug 28, 2024 at 8:21 AM Nazir Bilal Yavuz wrote: > > I do not exactly remember the reason but I think I copied the same > > behavior as before, PG_TEST_EXTRA variable was checked in the > > src/test/M

Re: Virtual generated columns

2024-09-02 Thread Nazir Bilal Yavuz
ntioned, I'll look into improving the > rewriting. xid_wraparound test started to fail after edee0c621d. It seems the error message used in xid_wraparound/002_limits is updated. The patch that applies the same update to the test file is attached. -- Regards, Nazir

Re: Use read streams in pg_visibility

2024-09-02 Thread Nazir Bilal Yavuz
techniques like corrupt_page_checksum() to create > the corrupt state. That wouldn't be a bad thing to have, but it's big enough > that I'll consider it out of scope for $SUBJECT. With the callback change > above, I'll be ready to push all this. Thanks, updated patches

Re: PG_TEST_EXTRA and meson

2024-08-28 Thread Nazir Bilal Yavuz
ied the same behavior as before, PG_TEST_EXTRA variable was checked in the src/test/Makefile so I exported it there. -- Regards, Nazir Bilal Yavuz Microsoft

Re: PG_TEST_EXTRA and meson

2024-08-28 Thread Nazir Bilal Yavuz
ST_EXTRA to the make command. 110 should be: 'cd $XID_MODULE_DIR && PG_TEST_EXTRA=xid_wraparound make check' instead of 'PG_TEST_EXTRA=xid_wraparound cd $XID_MODULE_DIR && make check' 170 should be: 'cd $XID_MODULE_DIR && PG_TEST_EXTRA="" make check && cd $PGDir' instead of 'PG_TEST_EXTRA="" cd $XID_MODULE_DIR && make check && cd $PGDir' -- Regards, Nazir Bilal Yavuz Microsoft

Re: Use read streams in pg_visibility

2024-08-27 Thread Nazir Bilal Yavuz
Hi, On Fri, 23 Aug 2024 at 22:01, Noah Misch wrote: > > On Fri, Aug 23, 2024 at 02:20:06PM +0300, Nazir Bilal Yavuz wrote: > > On Tue, 20 Aug 2024 at 21:47, Noah Misch wrote: > > > On Tue, Aug 13, 2024 at 03:22:27PM +0300, Nazir Bilal Yavuz wrote: > > > > The d

Re: Use read streams in pg_visibility

2024-08-23 Thread Nazir Bilal Yavuz
Hi, Thanks for the review and feedback! On Tue, 20 Aug 2024 at 21:47, Noah Misch wrote: > > On Tue, Aug 13, 2024 at 03:22:27PM +0300, Nazir Bilal Yavuz wrote: > > 2- collect_corrupt_items() > > > > This one is more complicated. The read stream callback function lo

Re: Use read streams in pg_visibility

2024-08-19 Thread Nazir Bilal Yavuz
Hi, On Mon, 19 Aug 2024 at 09:30, Michael Paquier wrote: > > On Tue, Aug 13, 2024 at 03:22:27PM +0300, Nazir Bilal Yavuz wrote: > > Hi, > > > > I am working on using the read stream in pg_visibility. There are two > > places to use it: > > > > 1- col

Use read streams in pg_visibility

2024-08-13 Thread Nazir Bilal Yavuz
tached. Any feedback would be appreciated. -- Regards, Nazir Bilal Yavuz Microsoft From c178e010759f042dc089fd4c5b2283a04b95f017 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Fri, 9 Aug 2024 11:41:52 +0300 Subject: [PATCH v1 1/2] Use read stream in pg_visibility in collect_visibility_data()

Re: CREATE DATABASE with filesystem cloning

2024-08-08 Thread Nazir Bilal Yavuz
Hi, Rebased version of the patch is attached. -- Regards, Nazir Bilal Yavuz Microsoft From a419004a26410c9ad9348e2f1420695db8ca35b6 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Thu, 8 Aug 2024 15:01:48 +0300 Subject: [PATCH v9] Introduce file_copy_method GUC This GUC can be set to

Using read stream in autoprewarm

2024-08-08 Thread Nazir Bilal Yavuz
the OS cache before each restart. Also, I ensured that the block number of the buffer returning from the read stream API is correct. I am not sure if that much testing is enough for this kind of change. Any feedback would be appreciated. -- Regards, Nazir Bilal Yavuz Microsoft From

Re: PG_TEST_EXTRA and meson

2024-07-23 Thread Nazir Bilal Yavuz
Hi, On Tue, 23 Jul 2024 at 13:40, Ashutosh Bapat wrote: > > On Tue, Jul 23, 2024 at 4:02 PM Nazir Bilal Yavuz wrote: > > > > > I wonder whether we really require pg_test_extra argument to testwrap. > > > Why can't we use the logic in testwrap, to set run time

Re: PG_TEST_EXTRA and meson

2024-07-23 Thread Nazir Bilal Yavuz
iron[;PG_TEST_EXTRA'] if the latter is set, otherwise set the > first to get_option('PG_TEST_EXTRA'). When test_env('PG_TEST_EXTRA') is set, it could not be overridden afterwards. Perhaps there is a way to override test_env() but I do not know how. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Use read streams in CREATE DATABASE command when the strategy is wal_log

2024-07-22 Thread Nazir Bilal Yavuz
Hi, On Sat, 20 Jul 2024 at 21:14, Noah Misch wrote: > > On Sat, Jul 20, 2024 at 03:01:31PM +0300, Nazir Bilal Yavuz wrote: > > > > With the separate commit (e00c45f685), does it make sense to rename > > the smgr_persistence parameter of the ReadBuffer_common() to

Re: Use read streams in CREATE DATABASE command when the strategy is wal_log

2024-07-20 Thread Nazir Bilal Yavuz
Hi, On Sat, 20 Jul 2024 at 14:27, Noah Misch wrote: > > On Thu, Jul 18, 2024 at 02:11:13PM +0300, Nazir Bilal Yavuz wrote: > > v4 is attached. > > Removal of the PinBufferForBlock() comment about the "persistence = > RELPERSISTENCE_PERMANENT" fallback started to

Re: PG_TEST_EXTRA and meson

2024-07-19 Thread Nazir Bilal Yavuz
ks.yml file should be removed as they are useless now. I added that as a second patch. [1] https://commitfest.postgresql.org/49/5134/ -- Regards, Nazir Bilal Yavuz Microsoft From 7e6c31c73d14f8e50d13ad7ce4aa1aa167193afd Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Fri, 19 Jul 2024 09:48:47

Re: CI, macports, darwin version problems

2024-07-18 Thread Nazir Bilal Yavuz
Hi, On Thu, 18 Jul 2024 at 17:01, Joe Conway wrote: > > On 7/18/24 08:55, Nazir Bilal Yavuz wrote: > > Hi, > > > > On Thu, 18 Jul 2024 at 15:00, Joe Conway wrote: > >> > >> On 7/18/24 07:55, Joe Conway wrote: > >> > On 7/18/24 04:12, Na

Re: CI, macports, darwin version problems

2024-07-18 Thread Nazir Bilal Yavuz
Hi, On Thu, 18 Jul 2024 at 15:00, Joe Conway wrote: > > On 7/18/24 07:55, Joe Conway wrote: > > On 7/18/24 04:12, Nazir Bilal Yavuz wrote: > >> Could it be pulling the ''macos-runner:sonoma' image on every run? > > > > Or perhaps since thi

Re: Use read streams in CREATE DATABASE command when the strategy is wal_log

2024-07-18 Thread Nazir Bilal Yavuz
Hi, On Wed, 17 Jul 2024 at 23:41, Noah Misch wrote: > > On Wed, Jul 17, 2024 at 12:22:49PM +0300, Nazir Bilal Yavuz wrote: > > On Tue, 16 Jul 2024 at 15:19, Noah Misch wrote: > > > On Tue, Jul 16, 2024 at 02:11:20PM +0300, Nazir Bilal Yavuz wrote: > > > > O

Re: CI, macports, darwin version problems

2024-07-18 Thread Nazir Bilal Yavuz
age was released a day ago [3] https://github.com/cirruslabs/macos-image-templates/pkgs/container/macos-sonoma-base/245087497?tag=latest https://github.com/orgs/cirruslabs/packages/container/macos-runner/242649219?tag=sonoma -- Regards, Nazir Bilal Yavuz Microsoft

Re: PG_TEST_EXTRA and meson

2024-07-17 Thread Nazir Bilal Yavuz
Hi, On Wed, 17 Jul 2024 at 13:23, Nazir Bilal Yavuz wrote: > > Hi, > > On Wed, 17 Jul 2024 at 13:13, Ashutosh Bapat > wrote: > > xid_wraparound tests are run if PG_TEST_EXTRA contains xid_wraparound > > or it is not set. Any other setting will not run xid_wraparou

Re: PG_TEST_EXTRA and meson

2024-07-17 Thread Nazir Bilal Yavuz
commitfest. I think I know why there is confusion. Could you try to set PG_TEST_EXTRA with quotes? Like PG_TEST_EXTRA="ldap mts xid_wraparound". -- Regards, Nazir Bilal Yavuz Microsoft

Re: Use read streams in CREATE DATABASE command when the strategy is wal_log

2024-07-17 Thread Nazir Bilal Yavuz
Hi, On Tue, 16 Jul 2024 at 15:19, Noah Misch wrote: > > On Tue, Jul 16, 2024 at 02:11:20PM +0300, Nazir Bilal Yavuz wrote: > > On Fri, 12 Jul 2024 at 02:52, Noah Misch wrote: > > > On Tue, Apr 16, 2024 at 02:12:19PM +0300, Nazir Bilal Yavuz wrote: > > > &

Re: PG_TEST_EXTRA and meson

2024-07-16 Thread Nazir Bilal Yavuz
Hi, On Wed, 17 Jul 2024 at 00:27, Jacob Champion wrote: > > On Tue, Jul 16, 2024 at 2:12 PM Nazir Bilal Yavuz wrote: > > > > 2- If PG_TEST_EXTRA is set from the setup command, use it from the > > setup command and discard the environment variable. If PG_TEST_EXTRA >

Re: PG_TEST_EXTRA and meson

2024-07-16 Thread Nazir Bilal Yavuz
. I hope these patches help. -- Regards, Nazir Bilal Yavuz Microsoft From 5eabb10189b635ddf1969a6f85ba80d70fec3a83 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Wed, 17 Jul 2024 00:01:48 +0300 Subject: [PATCH] Solution 1 --- meson.build | 5 - meson_options.txt | 3 --- 2 files

Re: Use read streams in CREATE DATABASE command when the strategy is wal_log

2024-07-16 Thread Nazir Bilal Yavuz
Hi, Thank you for the review! On Fri, 12 Jul 2024 at 02:52, Noah Misch wrote: > > On Tue, Apr 16, 2024 at 02:12:19PM +0300, Nazir Bilal Yavuz wrote: > > I am working on using read streams in the CREATE DATABASE command when the > > strategy is wal_log. RelationCopyStorageUsin

Re: CFbot failed on Windows platform

2024-07-11 Thread Nazir Bilal Yavuz
ights/all.html#4337) > > Any idea? I think It is related to the '628c1d1f2c' commit. This commit changed the output of the regress test in Windows. -- Regards, Nazir Bilal Yavuz Microsoft

Re: tests fail on windows with default git settings

2024-07-11 Thread Nazir Bilal Yavuz
r 2019, VS 2019 - Meson & ninja' [1] task started to fail after this commit, there is one extra space at the end of line in regress test's output. [1] https://cirrus-ci.com/task/6753781205958656 -- Regards, Nazir Bilal Yavuz Microsoft

Re: Streaming I/O, vectored I/O (WIP)

2024-07-10 Thread Nazir Bilal Yavuz
Hi, It seems that Heikki's 'v9.heikki-0007-Trivial-comment-fixes.patch' [1] is partially applied, the top comment is not updated. The attached patch just updates it. [1] https://www.postgresql.org/message-id/289a1c0e-8444-4009-a8c2-c2d77ced6f07%40iki.fi -- Regards, Nazir Bilal

Re: Doc: fix track_io_timing description to mention pg_stat_io

2024-06-27 Thread Nazir Bilal Yavuz
cs collection configuration section [1] only mentions reads > and writes -- perhaps it should also mention extends and fsyncs? Both suggestions look good to me. If what you said will be implemented, maybe track_wal_io_timing too should mention fsyncs? -- Regards, Nazir Bilal Yavuz Microsoft

Re: CI and test improvements

2024-06-14 Thread Nazir Bilal Yavuz
https://github.com/anarazel/pg-vm-images/commit/03a9225ac962fb30b5c0722c702941e2d7c1e81e -- Regards, Nazir Bilal Yavuz Microsoft

Re: CI and test improvements

2024-06-13 Thread Nazir Bilal Yavuz
ats log has been configured'. Perhaps you forgot to add 'CCACHE_STATSLOG: $CCACHE_DIR.stats.log' to 0002? After adding that line, CI finished successfully. And, I confirm that the build step takes ~30 seconds now; it was ~90 seconds before that. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2024-06-13 Thread Nazir Bilal Yavuz
memory so I believe they can be counted as hits in the shared buffers. Could you please explain how this change can 'lead to misleading interpretations for database tuning' a bit more? -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2024-06-13 Thread Nazir Bilal Yavuz
Hi, Thank you for looking into this! And, sorry for the late answer. On Mon, 13 May 2024 at 17:12, Bharath Rupireddy wrote: > > On Fri, Apr 19, 2024 at 1:32 PM Nazir Bilal Yavuz wrote: > > > > > I wanted to inform you that the 73f0a13266 commit changed all WALRead > &g

Re: CREATE DATABASE with filesystem cloning

2024-06-03 Thread Nazir Bilal Yavuz
Hi, On Tue, 21 May 2024 at 15:08, Ranier Vilela wrote: > > Em ter., 21 de mai. de 2024 às 05:18, Nazir Bilal Yavuz > escreveu: >> >> Hi, >> >> On Thu, 16 May 2024 at 17:47, Robert Haas wrote: >> > >> > On Thu, May 16, 2024 at 9:43 AM N

Re: meson and check-tests

2024-06-02 Thread Nazir Bilal Yavuz
for the tests. 2- Although the initdb executable exists, it is not enough by itself. Regress tests copies its database contents from the template directory, instead of running initdb for each test [1] (This is the default behaviour in the meson builds' regress tests). This template directory is created by the 'initdb_cache' test. [1] 252dcb3239 -- Regards, Nazir Bilal Yavuz Microsoft

Re: zlib detection in Meson on Windows broken?

2024-05-22 Thread Nazir Bilal Yavuz
Hi, On Wed, 22 May 2024 at 17:21, Dave Page wrote: > > Hi > > On Wed, 22 May 2024 at 14:11, Nazir Bilal Yavuz wrote: >> >> >> I tried to install your latest zlib artifact (nmake one) to the >> Windows CI images (not the official ones) [1]. Then, I used the

Re: zlib detection in Meson on Windows broken?

2024-05-22 Thread Nazir Bilal Yavuz
ary' method with your old artifact (cmake one). It was able to find zlib but all tests failed [4]. Experimental zlib meson.build diff is attached. [1] https://cirrus-ci.com/task/6736867247259648 [2] https://cirrus-ci.com/build/5286228755480576 [3] Run-time dependency zlib found: NO (tried pkgcon

Re: zlib detection in Meson on Windows broken?

2024-05-21 Thread Nazir Bilal Yavuz
eson setup ... --pkg-config-path $ZLIB_PC_PATH' might help. -- Regards, Nazir Bilal Yavuz Microsoft

Re: CREATE DATABASE with filesystem cloning

2024-05-21 Thread Nazir Bilal Yavuz
Hi, On Thu, 16 May 2024 at 17:47, Robert Haas wrote: > > On Thu, May 16, 2024 at 9:43 AM Nazir Bilal Yavuz wrote: > > Actually, the documentation for the file_copy_method was mentioning > > the things it controls; I converted it to an itemized list now. Also, > >

Re: Requiring LLVM 14+ in PostgreSQL 18

2024-05-16 Thread Nazir Bilal Yavuz
.es/m/CAN55FZ0fY5EFHXLKCO_=p4pwfmhrovom_qse_7b48gpchfa...@mail.gmail.com [2] https://cirrus-ci.com/task/4969910856581120 -- Regards, Nazir Bilal Yavuz Microsoft From 976d2c7ad0e470b24875ee27171359f54078a761 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Mon, 13 May 2024 10:56:28 +0300 Subject: [PATCH v5

Re: CREATE DATABASE with filesystem cloning

2024-05-16 Thread Nazir Bilal Yavuz
Hi, On Thu, 16 May 2024 at 15:40, Robert Haas wrote: > > On Thu, May 16, 2024 at 8:35 AM Nazir Bilal Yavuz wrote: > > I updated the documentation and put a comment on top of the copydir() > > function to inform that further changes and uses of this function may > > requi

Re: CREATE DATABASE with filesystem cloning

2024-05-16 Thread Nazir Bilal Yavuz
Hi, On Thu, 9 May 2024 at 19:29, Robert Haas wrote: > > On Wed, May 8, 2024 at 10:34 AM Nazir Bilal Yavuz wrote: > > > I'm not so sure about the GUC name. On the one hand, it feels like > > > createdb should be spelled out as create_database, but on the other >

Re: Use streaming read API in ANALYZE

2024-05-15 Thread Nazir Bilal Yavuz
Hi, On Mon, 29 Apr 2024 at 18:41, Nazir Bilal Yavuz wrote: > > Hi, > > On Mon, 8 Apr 2024 at 04:21, Thomas Munro wrote: > > > > Pushed. Thanks Bilal and reviewers! > > I wanted to discuss what will happen to this patch now that > 27bc1772fc8 is reverted. I am

Re: Build the docs if there are changes in docs and don't run other tasks if the changes are only in docs

2024-05-15 Thread Nazir Bilal Yavuz
Hi, On Sun, 12 May 2024 at 14:53, Peter Eisentraut wrote: > > On 14.12.23 14:40, Nazir Bilal Yavuz wrote: > > On Fri, 6 Oct 2023 at 17:07, Tom Lane wrote: > >> > >> As a quick cross-check, I searched our commit log to see how many > >> README-only commit

Re: Fix parallel vacuum buffer usage reporting

2024-05-13 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 May 2024 at 19:09, Masahiko Sawada wrote: > > On Fri, May 10, 2024 at 7:26 PM Nazir Bilal Yavuz wrote: > > > > Hi, > > > > Thank you for working on this! > > > > On Wed, 1 May 2024 at 06:37, Masahiko Sawada wrote: > > > > &g

Upgrade Debian CI images to Bookworm

2024-05-13 Thread Nazir Bilal Yavuz
narazel/pg-vm-images/pull/91 [1] postgr.es/m/CAN55FZ0o9wqVoMTh_gJCmj_%2B4XbX9VXzQF8OySPZ0R1saxV3bA%40mail.gmail.com -- Regards, Nazir Bilal Yavuz Microsoft From 26f4b2425cb04dc7218142f24f151d3698f33191 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Mon, 13 May 2024 10:56:28 +0300 Subject

Re: CFbot does not recognize patch contents

2024-05-12 Thread Nazir Bilal Yavuz
UKGLiY1e%2B1%3DpB7hXJOyGj1dJOfgde%2BHmiSnv3gDKayUFJMA%40mail.gmail.com -- Regards, Nazir Bilal Yavuz Microsoft

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 May 2024 at 16:55, Nazir Bilal Yavuz wrote: > > Hi, > > On Fri, 10 May 2024 at 16:21, Nazir Bilal Yavuz wrote: > > > > Hi, > > > > On Fri, 10 May 2024 at 14:49, Alena Rybakina > > wrote: > > > > > > Hi! I coul

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 May 2024 at 16:21, Nazir Bilal Yavuz wrote: > > Hi, > > On Fri, 10 May 2024 at 14:49, Alena Rybakina wrote: > > > > Hi! I could try to check it with the test, but I want to ask you about > > details, because I'm not sure that I completely un

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
cans: 1 ... ... buffer usage: 29343 hits, 9580 misses in the previous version, 14165 misses in the patched version, 14262 dirtied Patched version counts 14165 misses but the previous version counts 9580 misses in this specific example. -- Regards, Nazir Bilal Yavuz Microsoft diff --git a/src/

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
/* * Skip this block if someone else has already completed it. If an * I/O is already in progress in another backend, this will wait for * the outcome: either done, or something went wrong and we will * retry. */ -- Regards, Nazir Bilal Yavuz Microsoft

Re: gcc 12.1.0 warning

2024-05-10 Thread Nazir Bilal Yavuz
appen in newer versions of postgres is that we aren't > using guc_var_compare() in the relevant places anymore... The fix is attached. It cleanly applies from REL_15_STABLE to REL_12_STABLE, fixes the warnings and the tests pass. -- Regards, Nazir Bilal Yavuz Microsoft From 588c99f5c402fc414147

Re: CREATE DATABASE with filesystem cloning

2024-05-08 Thread Nazir Bilal Yavuz
Hi, On Wed, 8 May 2024 at 16:58, Robert Haas wrote: > > On Tue, May 7, 2024 at 8:00 AM Nazir Bilal Yavuz wrote: > > We had an off-list talk with Thomas and we thought making this option > > GUC instead of SQL command level could solve this problem. > > > > I am po

Re: CREATE DATABASE with filesystem cloning

2024-05-08 Thread Nazir Bilal Yavuz
Hi, On Wed, 8 May 2024 at 15:23, Ranier Vilela wrote: > > Em qua., 8 de mai. de 2024 às 08:42, Nazir Bilal Yavuz > escreveu: >> >> Hi, >> >> On Wed, 8 May 2024 at 14:16, Ranier Vilela wrote: >> > >> > >> > Em qua., 8 de mai. de 2024

Re: CREATE DATABASE with filesystem cloning

2024-05-08 Thread Nazir Bilal Yavuz
Hi, On Wed, 8 May 2024 at 14:16, Ranier Vilela wrote: > > > Em qua., 8 de mai. de 2024 às 04:37, Nazir Bilal Yavuz > escreveu: >> >> Hi Ranier, >> >> Thanks for looking into this! >> >> I am not sure why but your reply does not show up in the t

Re: CREATE DATABASE with filesystem cloning

2024-05-08 Thread Nazir Bilal Yavuz
OID %u already exists", dboid)); } else { /* Select an OID for the new database if is not explicitly configured. */ do { dboid = GetNewOidWithIndex(pg_database_rel, DatabaseOidIndexId, Anum_pg_database_oid);

Re: CREATE DATABASE with filesystem cloning

2024-05-07 Thread Nazir Bilal Yavuz
ing that since this is a GUC now, the user will have the full responsibility. Any kind of feedback would be appreciated. -- Regards, Nazir Bilal Yavuz Microsoft From 404e301dbdb252c23ea9d451b817cf6e372d0d9a Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Tue, 7 May 2024 14:16:09 +030

Re: Use streaming read API in ANALYZE

2024-04-29 Thread Nazir Bilal Yavuz
lved or can we resume working on this patch? Any kind of feedback would be appreciated. [1] https://www.postgresql.org/message-id/CAAKRu_ZxU6hucckrT1SOJxKfyN7q-K4KU1y62GhDwLBZWG%2BROg%40mail.gmail.com [2] https://www.postgresql.org/message-id/CAAKRu_YkphAPNbBR2jcLqnxGhDEWTKhYfLFY%3D0R_oG5LHBH7Gw%40mail.gmail.com [3] https://www.postgresql.org/message-id/flat/CAPpHfdurb9ycV8udYqM%3Do0sPS66PJ4RCBM1g-bBpvzUfogY0EA%40mail.gmail.com -- Regards, Nazir Bilal Yavuz Microsoft

Re: New committers: Melanie Plageman, Richard Guo

2024-04-26 Thread Nazir Bilal Yavuz
ss and few reverts! Congratulations to both of you! -- Regards, Nazir Bilal Yavuz Microsoft

Re: ci: Allow running mingw tests by default via environment variable

2024-04-25 Thread Nazir Bilal Yavuz
On the code level, I thought if it'd be good to have a common prefix for all > the automatically set variables. Right now that's CI_, but I'm not at all > wedded to that. I agree with your thoughts and CI_ prefix. I tested both patches and they work as expected. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2024-04-24 Thread Nazir Bilal Yavuz
Hi, On Fri, 19 Apr 2024 at 11:01, Nazir Bilal Yavuz wrote: > > On Thu, 18 Jan 2024 at 04:22, Michael Paquier wrote: > > > > > > On Wed, Jan 17, 2024 at 03:20:39PM +0300, Nazir Bilal Yavuz wrote: > > > > I agree with your points. While the other I/O rela

Re: slightly misleading Error message in guc.c

2024-04-22 Thread Nazir Bilal Yavuz
e valid range for parameter "work_mem" (64 > kB .. 2147483647 kB) > ? > since the units for work_mem are { "B", "kB", "MB", "GB", and "TB"} > > search `outside the valid range for parameter`, > there are two occurrences in guc.c. Nice find. I agree it could cause confusion. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Show WAL write and fsync stats in pg_stat_io

2024-04-19 Thread Nazir Bilal Yavuz
Hi, On Mon, 19 Feb 2024 at 10:28, Nazir Bilal Yavuz wrote: > > Hi, > > On Thu, 18 Jan 2024 at 04:22, Michael Paquier wrote: > > > > On Wed, Jan 17, 2024 at 03:20:39PM +0300, Nazir Bilal Yavuz wrote: > > > I agree with your points. While the other I/O relat

Re: Use streaming read API in ANALYZE

2024-04-16 Thread Nazir Bilal Yavuz
Hi, On Wed, 3 Apr 2024 at 22:25, Nazir Bilal Yavuz wrote: > > Hi, > > Thank you for looking into this! > > On Wed, 3 Apr 2024 at 20:17, Heikki Linnakangas wrote: > > > > On 03/04/2024 13:31, Nazir Bilal Yavuz wrote: > > > Streaming API has been committe

Re: Typos in the code and README

2024-04-16 Thread Nazir Bilal Yavuz
and duplicate words fixups, 0002 fixes a parameter > with > the wrong name in the prototype and 0003 removes a leftover prototype which > was > accidentally left in a refactoring. I realized two small typos: 'sgmr' -> 'smgr'. You may want to include them in 0001. -- Regards, Nazir Bilal Yavuz Microsoft

Use read streams in CREATE DATABASE command when the strategy is wal_log

2024-04-16 Thread Nazir Bilal Yavuz
rom generate_series(1, 900) as i; [2] CREATE DATABASE test_1 STRATEGY 'wal_log' TEMPLATE test; [3] https://www.postgresql.org/message-id/CAN55FZ1yGvCzCW_aufu83VimdEYHbG_zuOY3J9JL-nBptyJyKA%40mail.gmail.com -- Regards, Nazir Bilal Yavuz Microsoft From edcfacec40a70a8747c5f18777b6c2

Re: Table AM Interface Enhancements

2024-04-15 Thread Nazir Bilal Yavuz
Sampler_Init(), BlockSampler_Next() etc.) need to be edited as well since these functions assume block numbers are actual physical on-disk location, right? I mean if the block number is something different than the actual physical on-disk location, the acquire_sample_rows() function looks wrong to me before c6fc50cb4028 as well. -- Regards, Nazir Bilal Yavuz Microsoft

Re: gcc 12.1.0 warning

2024-04-15 Thread Nazir Bilal Yavuz
Hi, On Sat, 13 Apr 2024 at 05:25, Andres Freund wrote: > > Hi, > > On 2023-10-27 13:09:01 +0300, Nazir Bilal Yavuz wrote: > > I was testing 'upgrading CI Debian images to bookworm'. I tested the > > bookworm on REL_15, REL_16 and upstream. REL_16 and upstream

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Nazir Bilal Yavuz
st way forward. > > > > As I wrote in [1], I didn't observe the issue with clang-18, so maybe it > is fixed already. > Perhaps it's worth rechecking... > > [1] > https://www.postgresql.org/message-id/d2bf3727-bae4-3aee-65f6-caec2c4ebaa8%40gmail.com I had this probl

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Nazir Bilal Yavuz
Hi, On Mon, 8 Apr 2024 at 00:01, Nazir Bilal Yavuz wrote: > > Hi, > > On Sun, 7 Apr 2024 at 20:33, Nazir Bilal Yavuz wrote: > > > > Hi, > > > > On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > > > > > I had been planning to commit v14 t

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Nazir Bilal Yavuz
Hi, On Sun, 7 Apr 2024 at 20:33, Nazir Bilal Yavuz wrote: > > Hi, > > On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > > > I had been planning to commit v14 this morning but got cold feet with > > the BMR-based interface. Heikki didn't like it much, an

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Nazir Bilal Yavuz
Read stream objects can be created only using Relations now. There could be read stream users which do not have a Relation but SMgrRelations. So, I created another constructor for the read streams which use SMgrRelations instead of Relations. Related patch is attached. -- R

Re: Change prefetch and read strategies to use range in pg_prewarm ... and raise a question about posix_fadvise WILLNEED

2024-04-07 Thread Nazir Bilal Yavuz
Hi Andrey, On Sun, 7 Apr 2024 at 08:29, Andrey M. Borodin wrote: > > > > > On 15 Mar 2024, at 17:12, Nazir Bilal Yavuz wrote: > > > > I did not have the time to check other things you mentioned but I > > tested the read performance. The table size is 5.5GB, I

Re: Use streaming read API in ANALYZE

2024-04-04 Thread Nazir Bilal Yavuz
Hi, On Wed, 3 Apr 2024 at 23:44, Melanie Plageman wrote: > > > I've reviewed the patches inline below and attached a patch that has > some of my ideas on top of your patch. Thank you! > > > From 8d396a42186325f920d5a05e7092d8e1b66f3cdf Mon Sep 17 00:00:00 2001 &

Re: Use streaming read API in ANALYZE

2024-04-03 Thread Nazir Bilal Yavuz
Hi, Thank you for looking into this! On Wed, 3 Apr 2024 at 20:17, Heikki Linnakangas wrote: > > On 03/04/2024 13:31, Nazir Bilal Yavuz wrote: > > Streaming API has been committed but the committed version has a minor > > change, the read_stream_begin_relation function take

Re: Use streaming read API in ANALYZE

2024-04-03 Thread Nazir Bilal Yavuz
Hi Jakub, Thank you for looking into this and doing a performance analysis. On Wed, 3 Apr 2024 at 11:42, Jakub Wartak wrote: > > On Tue, Apr 2, 2024 at 9:24 AM Nazir Bilal Yavuz wrote: > [..] > > v4 is rebased on top of v14 streaming read API changes. > > Hi Nazir,

Re: Add missing error codes to PANIC/FATAL error reports in xlog.c and relcache.c

2024-04-03 Thread Nazir Bilal Yavuz
ommitted it. Thanks! Thanks for the commit! -- Regards, Nazir Bilal Yavuz Microsoft

Re: Use streaming read API in ANALYZE

2024-04-03 Thread Nazir Bilal Yavuz
Hi, On Tue, 2 Apr 2024 at 10:23, Nazir Bilal Yavuz wrote: > > v4 is rebased on top of v14 streaming read API changes. Streaming API has been committed but the committed version has a minor change, the read_stream_begin_relation function takes Relation instead of BufferManagerRelation n

Re: Use streaming read API in ANALYZE

2024-04-02 Thread Nazir Bilal Yavuz
Hi, Thanks for the review! On Wed, 27 Mar 2024 at 23:15, Melanie Plageman wrote: > > On Tue, Mar 26, 2024 at 02:51:27PM +0300, Nazir Bilal Yavuz wrote: > > Hi, > > > > On Wed, 28 Feb 2024 at 14:42, Nazir Bilal Yavuz wrote: > > > > > > > > &g

Re: Building with meson on NixOS/nixpkgs

2024-04-01 Thread Nazir Bilal Yavuz
Hi, >From your prior reply: On Thu, 21 Mar 2024 at 23:44, Wolfgang Walther wrote: > > Nazir Bilal Yavuz: > > 0001 & 0002: Adding code comments to explain why they have fallback > > could be nice. > > 0003: Looks good to me. > > Added some comments in the

  1   2   3   >