Re: C_C_A animal on HEAD gets stuck in initdb

[Christian Ullrich]

* Christian Ullrich wrote:

> * Thomas Munro wrote:
> 
>> On Mon, Apr 1, 2019 at 9:39 PM Christian Ullrich 
>>  wrote:

>>> my CLOBBER_CACHE_ALWAYS animal, jaguarundi, has gotten stuck in "make
>>> check"'s initdb three times in a row now.
>>
>> Could it be the same as this?
>>
>> https://www.postgresql.org/message-id/CA%2BhUKGLCwPF0S4Mk7S8qw%2BDK0Bq65LueN9rofAA3HHSYikW-Zw%40mail.gmail.com
>>  
>>
>>
>> I see that its first failure was after commit 558a9165e0 (along with 
>> others).
> 
> It does look very similar. I don't have a working gdb on the box, hence 
> this is from lldb.

I think the patch in the linked message works; it doesn't get stuck 
anymore. It's still slow as molasses with C_C_A; this animal can take 
12+ hours to complete.

-- 
Christian

Re: C_C_A animal on HEAD gets stuck in initdb

[Christian Ullrich]

* Thomas Munro wrote:

> On Mon, Apr 1, 2019 at 9:39 PM Christian Ullrich  wrote:
>> my CLOBBER_CACHE_ALWAYS animal, jaguarundi, has gotten stuck in "make
>> check"'s initdb three times in a row now.
> 
> Could it be the same as this?
> 
> https://www.postgresql.org/message-id/CA%2BhUKGLCwPF0S4Mk7S8qw%2BDK0Bq65LueN9rofAA3HHSYikW-Zw%40mail.gmail.com
> 
> I see that its first failure was after commit 558a9165e0 (along with others).

It does look very similar. I don't have a working gdb on the box, hence 
this is from lldb.

(lldb) bt
* thread #1, name = 'postgres'
   * frame #0: 0x0008020e4ce8 libc.so.7`_umtx_op + 8
 frame #1: 0x0008020d0e5e libc.so.7`_sem_clockwait_np [inlined] 
usem_wait(sem=, clock_id=0, rmtp=) at 
cancelpoints_sem_new.c:365
 frame #2: 0x0008020d0e4f 
libc.so.7`_sem_clockwait_np(sem=, clock_id=0, 
flags=, rqtp=0x, rmtp=) at 
cancelpoints_sem_new.c:424
 frame #3: 0x007104e8 
postgres`PGSemaphoreLock(sema=0x0008032031b0) at pg_sema.c:316
 frame #4: 0x00796889 
postgres`LWLockAcquire(lock=0x000803725924, mode=LW_SHARED) at 
lwlock.c:1244
 frame #5: 0x0077157a 
postgres`LockBuffer(buffer=, mode=1) at bufmgr.c:0
 frame #6: 0x004f54f1 postgres`_bt_getroot [inlined] 
_bt_getbuf(rel=, blkno=, access=1) at 
nbtpage.c:806
 frame #7: 0x004f54cd 
postgres`_bt_getroot(rel=0x00080314d080, access=1) at nbtpage.c:323
 frame #8: 0x004fa7fa 
postgres`_bt_search(rel=0x00080314d080, key=0x7fffb508, 
bufP=0x7fffbf28, access=1, snapshot=0x00dccc48) at 
nbtsearch.c:99
 frame #9: 0x004fbe5a 
postgres`_bt_first(scan=0x0008031bb4f0, dir=) at 
nbtsearch.c:1247
 frame #10: 0x004f9736 
postgres`btgettuple(scan=0x0008031bb4f0, dir=ForwardScanDirection) 
at nbtree.c:245
 frame #11: 0x004efec1 
postgres`index_getnext_tid(scan=0x0008031bb4f0, 
direction=) at indexam.c:550
 frame #12: 0x004f0052 
postgres`index_getnext_slot(scan=0x0008031bb4f0, 
direction=ForwardScanDirection, slot=0x0008031bcf80) at indexam.c:642
 frame #13: 0x004eefc2 
postgres`systable_getnext(sysscan=0x0008031bdcd0) at genam.c:450
 frame #14: 0x008ccaf0 
postgres`ScanPgRelation(targetRelId=, 
indexOK=, force_non_historic=false) at relcache.c:365
 frame #15: 0x008c5ea6 postgres`RelationClearRelation at 
relcache.c:2288
 frame #16: 0x008c5e51 
postgres`RelationClearRelation(relation=0x0008031460b0, 
rebuild=true) at relcache.c:2421
 frame #17: 0x008c788d postgres`RelationCacheInvalidate at 
relcache.c:2854
 frame #18: 0x008bd0aa postgres`AcceptInvalidationMessages 
[inlined] InvalidateSystemCaches at inval.c:649
 frame #19: 0x008bd09b postgres`AcceptInvalidationMessages 
at inval.c:708
 frame #20: 0x0078a929 postgres`LockRelationOid(relid=1213, 
lockmode=) at lmgr.c:133
 frame #21: 0x0049baa2 
postgres`relation_open(relationId=1213, lockmode=1) at relation.c:56
 frame #22: 0x0051624c 
postgres`table_open(relationId=, lockmode=) at 
table.c:43
 frame #23: 0x008bc707 
postgres`SearchCatCacheMiss(cache=0x00080313e900, nkeys=1, 
hashValue=1761185739, hashIndex=3, v1=1663, v2=0, v3=0, v4=0) at 
catcache.c:1357
 frame #24: 0x008bae3b 
postgres`SearchCatCacheInternal(cache=0x00080313e900, 
nkeys=, v1=, v2=, 
v3=, v4=0) at catcache.c:1299
 frame #25: 0x008ce406 
postgres`get_tablespace(spcid=) at spccache.c:136
 frame #26: 0x008ce4a9 
postgres`get_tablespace_io_concurrency(spcid=) at 
spccache.c:217
 frame #27: 0x004db9c6 
postgres`heap_compute_xid_horizon_for_tuples(rel=0x0008031460b0, 
tids=0x0008031ba7f8, nitems=) at heapam.c:6980
 frame #28: 0x004eed06 
postgres`index_compute_xid_horizon_for_tuples [inlined] 
table_compute_xid_horizon_for_tuples(rel=, 
items=, nitems=) at tableam.h:973
 frame #29: 0x004eecf1 
postgres`index_compute_xid_horizon_for_tuples(irel=, 
hrel=0x0008031460b0, ibuf=, itemnos=0x7fffc7a0, 
nitems=3) at genam.c:306
 frame #30: 0x004f6b14 
postgres`_bt_delitems_delete(rel=0x00080314d080, buf=49, 
itemnos=, nitems=3, heapRel=) at nbtpage.c:
 frame #31: 0x004f4c2c 
postgres`_bt_vacuum_one_page(rel=, buffer=, 
heapRel=) at nbtinsert.c:2270
 frame #32: 0x004f13a2 postgres`_bt_doinsert [inlined] 
_bt_findinsertloc(rel=, heapRel=0x0008031460b0) at 
nbtinsert.c:736
 frame #33: 0x004f136d 
postgres`_bt_doinsert(rel=, itup=0x00080306b678, 
checkUnique=UNIQUE_CHECK_YES, heapRel=0x0008031460b0) at nbtinsert.c:281
 frame #34: 0x004f9017 
postgres`btinsert(rel=0x00080314d080, values=, 
isnull=, ht_ctid=0x0008031b85c8, heapRel=, 
checkUnique

C_C_A animal on HEAD gets stuck in initdb

[Christian Ullrich]

Hello,

my CLOBBER_CACHE_ALWAYS animal, jaguarundi, has gotten stuck in "make 
check"'s initdb three times in a row now.

I have trace output covering about the final minute of initdb. It mainly 
consists of ~9 iterations of

- Open base/1/1259 [pg_class]
- Seek to end [twice]
- Open global/pg_filenode.map
- Read 512 bytes
- Close global/filenode.map
- Open base/1/pg_filenode.map
- Read 512 bytes
- Close base/1/pg_filenode.map
- Close base/1/1259

with some operations on other heap files in between. At the very end, it 
writes 8K of zeros to 1259_fsm at offset 0x1, then it starts waiting 
on a semaphore and never finishes.

If someone would like the 0.5 GiB of trace output (FreeBSD ktrace), it 
compresses to 1.75 MiB.


All the best,

-- 
Christian

Re: Problem with EDB 11.0 Windows x64 distributions

[Christian Ullrich]

* Sandeep Thakkar wrote:

> On Thu, Oct 25, 2018 at 4:58 PM Christian Ullrich  <mailto:ch...@chrullrich.net>> wrote:
> The file download on your web site is still 11.0-2, and the files are
> still missing from it. Should it have updated (to -3?) by now?
> 
> You talking about Zip archive? I can see the missing DLLs in it:

Hm. There must be some strange caching effect at Cloudflare or somewhere 
(definitely not here). Two different browsers give me the old version, 
but on the command line (FreeBSD fetch) I get the new one from two 
different locations.

It's all HTTPS, so the only place that could have a caching issue is the 
origin server.

Anyway, I can see that an updated file exists. Thank you.

-- 
Christian

Re: Problem with EDB 11.0 Windows x64 distributions

[Christian Ullrich]

* Sandeep Thakkar wrote:

> There shouldn't be any problem with the installer (in non extract-only 
> mode) and all the dependencies should be in place. Let us know if any 
> binary is not working. For extract-only mode, let me confirm and fix the 
> issues if any.

Sorry, I do not use the installer in install mode; it's too black a box 
for me.

> Also, bin/libwinpthread-1.dll and bin/zilb1.dll were missing in zip 
> archive which I just added.

The file download on your web site is still 11.0-2, and the files are 
still missing from it. Should it have updated (to -3?) by now?

-- 
Christian

Problem with EDB 11.0 Windows x64 distributions

[Christian Ullrich]

Hello,

there are some problems with the 11.0 EDB installers and binary archives 
for Windows x64:

- bin/libwinpthread-1.dll is only in the installer

   This is a dependency of libintl-9.dll, so without it, nearly
   everything in the Zip archive does not work.

- zlib1.dll is only in the installer

   No backups, no replication, no logical replication.

- bin/libcurl.dll is only in the Zip archive

   Needed by bin/stackbuilder.exe , but ...

- StackBuilder (all of it) is only in the Zip archive

   ... and so are all the wxWidgets DLLs. At least I cannot get
   the installer to --extract-only it.

- bin/libcurl.lib is present in both

   It is not needed, is it?


Files involved:

- postgresql-11.0-2-windows-x64-binaries.zip
   SHA1: 167c37a61a60737d9e3461b845fc2baa8db34bd5

- postgresql-11.0-2-windows-x64.exe (--extract-only 1)
   SHA1: 67a0e27e69375404df8967ab81383d26f22c94b5

-- 
Christian

Re: late binding of shared libs for C functions

[Christian Ullrich]

* On 2018-06-12 16:35, Geoff Winkless wrote:


On Tue, 12 Jun 2018 at 13:41, Andrew Dunstan wrote:



UNBOUNDED would be terrible. It does not mean the same thing as UNBOUND.


Indeed. I agree.


Perhaps something like NO CHECK would meet the case, i.e. we're not
checking the link at function creation time.


I did wonder about "NO CHECK" but wasn't sure if having two words
would make the parser change more complex.


DEFERRED?

--
Christian

Re: pg_config.h.win32 missing a set of flags from pg_config.h.in added in v11 development

[Christian Ullrich]

*On 2018-06-12 16:21, Jonathan S. Katz wrote:


On Jun 3, 2018, at 4:29 AM, Michael Paquier  wrote:



A script which reports the version of OpenSSL should be simple enough
for MSVC.  And I am ready to bet that at least hamerkop is not using
OpenSSL 1.0.2, so a simple switch would most likely cause the buildfarm
to go red.  I am attaching in CC the maintainers of the Windows animals
so as they can comment.


Wanted to check in and see if any of the Windows maintainers had thoughts
on this issue so we can continue to move it forward.


I can't contribute much; mine all build without OpenSSL anyway.

If I enable it in the future, it will be with libraries from vcpkg [1], 
and they are at 1.0.2o now and unlikely to go backwards.


[1] https://github.com/Microsoft/vcpkg/

--
Christian

Re: A few warnings on Windows

[Christian Ullrich]

* Tom Lane wrote:


Thomas Munro  writes:

One more problem.  whelk builds against Python 3.6 and says:



c:\users\pgbf\appdata\local\programs\python\python36-32\include\pyconfig.h(174):
warning C4142: benign redefinition of type
(src/pl/plpython/plpy_elog.c)
[C:\buildfarm\buildenv\HEAD\pgsql.build\plpython3.vcxproj]



Does anyone know what line 174 of pyconfig.h happens to say?


typedef _W64 int ssize_t;

, in a "not for 64-bit" block.

, 3.6.3 is 
the installed version on whelk.


--
Christian

Re: pl/perl extension fails on Windows

[Christian Ullrich]

* Christian Ullrich wrote:


* Noah Misch wrote:


On Wed, Nov 29, 2017 at 11:45:35PM -0500, Tom Lane wrote:

Oh, OK.  In that case, we need to get some representatives of these
more modern builds into the buildfarm while we're at it.


Yep.  Among machines already in the buildfarm, the one running member
woodlouse is the best candidate for this.  Its owner could install
http://strawberryperl.com/download/5.26.1.1/strawberry-perl-5.26.1.1-32bit.msi
and setup another animal on the same machine that builds 32-bit and enables
Perl.  Christian, are you interested in doing this?


Ready to go, waiting for animal assignment. For now, I can confirm that it 
works, that is, the buildfarm --test run is successful.


Up and running now, name is whelk, first report on REL9_6_STABLE.

Sorry it took me another ten days to complete the configuration.

--
Christian

RE: pl/perl extension fails on Windows

[Christian Ullrich]

* From: Noah Misch [mailto:n...@leadboat.com]

> > > Ready to go, waiting for animal assignment. For now, I can
> confirm that it works, that is, the buildfarm --test run is
> successful.

> Did the animal assignment come through?  I don't see such an animal
> reporting.

No, not yet. Sorry, I lost track of it, or I would have mentioned it again 
earlier.

-- 
Christian

RE: pl/perl extension fails on Windows

[Christian Ullrich]

* Noah Misch wrote:

> On Wed, Nov 29, 2017 at 11:45:35PM -0500, Tom Lane wrote:
>> Oh, OK.  In that case, we need to get some representatives of these
>> more modern builds into the buildfarm while we're at it.
> 
> Yep.  Among machines already in the buildfarm, the one running member
> woodlouse is the best candidate for this.  Its owner could install
> http://strawberryperl.com/download/5.26.1.1/strawberry-perl-5.26.1.1-32bit.msi
> and setup another animal on the same machine that builds 32-bit and enables
> Perl.  Christian, are you interested in doing this?

Ready to go, waiting for animal assignment. For now, I can confirm that it 
works, that is, the buildfarm --test run is successful.

Although I have to admit, I fail to see the need for Windows x86 builds, too. 
Who in their right mind would want them today?

--
Christian