Re: [PATCHES] Synchronized scans

[Tom Lane]

Jeff Davis [EMAIL PROTECTED] writes:
  * For a large table, do lazy_scan_heap, scan_heap, and a sequential
 scan usually progress at approximately the same rate?

scan_heap would probably be faster than a regular seqscan, since it
isn't doing any where-clause-checking or data output.  Except if you've
got vacuum-cost-limit enabled, which I think is likely to be true by
default in future.  Another problem is that lazy_scan_heap stops every
so often to make a pass over the table's indexes, which'd certainly
cause it to fall out of sync with more typical seqscans.

The vacuum-cost-limit issue may be sufficient reason to kill this idea;
not sure.

  * Just adding in the syncscan to scan_heap and lazy_scan_heap seems
 very easy at first thought. Are there any complications that I'm
 missing?

I believe there are assumptions buried in both full and lazy vacuum that
blocks are scanned in increasing order.  Not sure how hard that would be
to fix or work around.  The only one I can specifically recall in lazy
vacuum is that we assume the list of deletable TIDs is sorted a priori.
Possibly you could deal with that by forcing an index-vacuum pass at the
instant where the scan would wrap around, so that the list could be
cleared before putting any lower-numbered blocks into it.

regards, tom lane

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster

Re: [PATCHES] Synchronized scans

[Gregory Stark]

Tom Lane [EMAIL PROTECTED] writes:

 The vacuum-cost-limit issue may be sufficient reason to kill this idea;
 not sure.

We already have a much higher cost for blocks that cause i/o than blocks which
don't. I think if we had zero cost for blocks which don't cause i/o it would
basically work unless the sleep time was so large that the other scans managed
to cycle through the entire ring in that time.

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com


---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly

Re: [PATCHES] Synchronized scans

[Tom Lane]

Gregory Stark [EMAIL PROTECTED] writes:
 Tom Lane [EMAIL PROTECTED] writes:
 The vacuum-cost-limit issue may be sufficient reason to kill this idea;
 not sure.

 We already have a much higher cost for blocks that cause i/o than
 blocks which don't. I think if we had zero cost for blocks which don't
 cause i/o it would basically work unless the sleep time was so large
 that the other scans managed to cycle through the entire ring in that
 time.

I'm unconvinced.  That could perhaps work as long as the vacuum process
never did any I/O, ie was always a follower and never the leader of the
syncscan pack.  But if lazy_heap_scan is faster than a regular seqscan,
as I suspect, then it'd frequently become the leader and have to do the
I/O.  A few iterations of that will cause it to hit the vacuum cost
sleep, and that will make it fall behind the pack (and probably out of
sync entirely, unless the sleep is really short).

regards, tom lane

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster

Re: [PATCHES] Regression tests

[Magnus Hagander]

Magnus Hagander wrote:
 Joachim Wieland attempted to post this patch, but it appears to be gone.
 I tried a repost, and notivced it got rejected because it was 100kb.
 Let me repeat previous objections that it really should be possible to
 post a patch 100kb.
 That said, here's a gzipped version.
 
 Joachim, once it comes through, feel free to post whatever comments you
 had in your original mail.

Here's an updated version of this patch. I reversed some parts of where
files went, which made the patch a whole lot smaller and easier to read,
and changed a couple of smaller things.

I have not yet looked at the actual ecpg parts (except noticed that they
are working on msvc at least), just the changes to main regression
tests. But I wanted to send off my new version for others to look at as
well before I do that.

This is still WIP, but comments appreciated.

//Magnus

Index: src/interfaces/ecpg/test/Makefile
===
RCS file: /projects/cvsroot/pgsql/src/interfaces/ecpg/test/Makefile,v
retrieving revision 1.67
diff -c -r1.67 Makefile
*** src/interfaces/ecpg/test/Makefile   29 Mar 2007 12:02:24 -  1.67
--- src/interfaces/ecpg/test/Makefile   9 Jun 2007 13:12:35 -
***
*** 8,13 
--- 8,16 
  # this is also defined in test/connect/Makefile
  TEMP_PORT = 5$(DEF_PGPORT)
  
+ # where to find psql for testing an existing installation
+ PSQLDIR = $(bindir)
+ 
  # default encoding
  MULTIBYTE = SQL_ASCII
  
***
*** 26,31 
--- 29,43 
  abs_builddir := $(shell pwd -W)
  endif
  
+ # stuff to pass into build of pg_regress
+ EXTRADEFS = '-DHOST_TUPLE=$(host_tuple)' \
+   '-DMAKEPROG=$(MAKE)' \
+   '-DSHELLPROG=$(SHELL)' \
+   '-DDLSUFFIX=$(DLSUFFIX)'
+ 
+ REGRESSINCLUDES = -I$(top_builddir)/src/test/regress
+ REGRESSDRIVER = $(top_builddir)/src/test/regress/pg_regress.o
+ 
  all install installdirs uninstall distprep:
$(MAKE) -C connect $@
$(MAKE) -C expected $@
***
*** 45,64 
$(MAKE) -C compat_informix $@
$(MAKE) -C thread $@
rm -rf tmp_check results log
!   rm -f pg_regress regression.diffs
  
! all: pg_regress
  
! pg_regress: pg_regress.sh $(top_builddir)/src/Makefile.global
!   sed -e 's,@bindir@,$(bindir),g' \
!   -e 's,@libdir@,$(libdir),g' \
!   -e 's,@pkglibdir@,$(pkglibdir),g' \
!   -e 's,@datadir@,$(datadir),g' \
!   -e 's/@VERSION@/$(VERSION)/g' \
!   -e 's/@host_tuple@/$(host_tuple)/g' \
!   -e 's,@GMAKE@,$(MAKE),g' \
!   -e 's/@enable_shared@/$(enable_shared)/g' \
! $ $@
  
  # When doing a VPATH build, copy over the .pgc, .stdout and .stderr
  # files so that the driver script can find them.  We have to use an
--- 57,77 
$(MAKE) -C compat_informix $@
$(MAKE) -C thread $@
rm -rf tmp_check results log
!   rm -f pg_regress regression.diffs pg_regress_ecpg.o
! 
! # Build regression test driver
! 
! all: pg_regress$(X)
! 
! pg_regress$(X): pg_regress_ecpg.o
!   $(CC) $(CFLAGS) $^ $(REGRESSDRIVER) $(LDFLAGS) $(LIBS) -o $@
  
! # dependencies ensure that path changes propagate
! pg_regress_ecpg.o: pg_regress_ecpg.c 
$(top_builddir)/src/port/pg_config_paths.h
!   $(CC) $(CFLAGS) $(CPPFLAGS) -I$(top_builddir)/src/port 
$(REGRESSINCLUDES) $(EXTRADEFS) -c -o $@ $
  
! $(top_builddir)/src/port/pg_config_paths.h: 
$(top_builddir)/src/Makefile.global
!   $(MAKE) -C $(top_builddir)/src/port pg_config_paths.h
  
  # When doing a VPATH build, copy over the .pgc, .stdout and .stderr
  # files so that the driver script can find them.  We have to use an
***
*** 78,88 
  
  
  check: all
!   sh ./pg_regress  --dbname=regress1 --temp-install 
--top-builddir=$(top_builddir) --temp-port=$(TEMP_PORT) 
--multibyte=$(MULTIBYTE) --load-language=plpgsql $(NOLOCALE) $(THREAD)
  
  # the same options, but with --listen-on-tcp
  checktcp: all
!   sh ./pg_regress  --dbname=regress1 --temp-install 
--top-builddir=$(top_builddir) --temp-port=$(TEMP_PORT) 
--multibyte=$(MULTIBYTE) --load-language=plpgsql $(NOLOCALE) --listen-on-tcp 
$(THREAD)
  
  installcheck: all
!   sh ./pg_regress  --dbname=regress1 --top-builddir=$(top_builddir) 
--load-language=plpgsql $(NOLOCALE)
--- 91,101 
  
  
  check: all
!   ./pg_regress  --dbname=regress1,connectdb 
--top-builddir=$(top_builddir) --temp-install=./tmp_check 
--temp-port=$(TEMP_PORT) --multibyte=$(MULTIBYTE) --load-language=plpgsql 
$(NOLOCALE) $(THREAD) --schedule=$(srcdir)/ecpg_schedule 
--create-role=connectuser,connectdb
  
  # the same options, but with --listen-on-tcp
  checktcp: all
!   ./pg_regress  --dbname=regress1,connectdb 
--top-builddir=$(top_builddir) --temp-install=./tmp_check 
--temp-port=$(TEMP_PORT) --multibyte=$(MULTIBYTE) --load-language=plpgsql 
$(NOLOCALE) $(THREAD) --schedule=$(srcdir)/ecpg_schedule_tcp 
--create-role=connectuser,connectdb