Re: [HACKERS] 8.3.0 Core with concurrent vacuum fulls
2008-03-04 05:45:47 EST [6698]: [1-1] LOG: process 6698 still waiting for AccessShareLock on relation 1247 of database 16385 after 1001.519 ms 2008-03-04 05:45:47 EST [6698]: [2-1] STATEMENT: VACUUM FULL autograph.autograph_creators 2008-03-04 05:46:28 EST [6730]: [1-1] LOG: process 6730 still waiting for AccessShareLock on relation 1247 of database 16385 after 1000.887 ms 2008-03-04 05:46:28 EST [6730]: [2-1] STATEMENT: VACUUM FULL lunchmoney.totals 2008-03-04 05:47:55 EST [3809]: [18-1] LOG: server process (PID 6742) was terminated by signal 6: Aborted 2008-03-04 05:47:55 EST [3809]: [19-1] LOG: terminating any other active server processes 2008-03-04 05:47:55 EST [6741]: [12-1] WARNING: terminating connection because of crash of another server process On Tue, Mar 4, 2008 at 9:56 PM, Tom Lane [EMAIL PROTECTED] wrote: Gavin M. Roy [EMAIL PROTECTED] writes: (gdb) where #0 0x003fe362e21d in raise () from /lib64/tls/libc.so.6 #1 0x003fe362fa1e in abort () from /lib64/tls/libc.so.6 #2 0x0063a2e3 in errfinish () #3 0x005974c4 in DeadLockReport () #4 0x0059381f in LockAcquire () #5 0x00592357 in LockRelationOid () #6 0x00457255 in relation_open () #7 0x004574c3 in heap_open () #8 0x0062cf41 in CatalogCacheInitializeCache () #9 0x0062dfad in PrepareToInvalidateCacheTuple () #10 0x0062e8c5 in CacheInvalidateHeapTuple () #11 0x0045c803 in heap_page_prune () #12 0x005086cd in vacuum_rel () #13 0x005096bb in vacuum () #14 0x005a163b in PortalRunUtility () #15 0x005a1714 in PortalRunMulti () #16 0x005a1d30 in PortalRun () #17 0x0059f4b6 in PostgresMain () #18 0x005760c0 in ServerLoop () #19 0x0050 in PostmasterMain () #20 0x0052fd3e in main () So what did DeadLockReport put in the server log before panic'ing? I'm wondering exactly why CatalogCacheInitializeCache is being called here --- seems like that should have been done long before we got to VACUUM. But it would be useful to know just what deadlock it saw. regards, tom lane
Re: [HACKERS] 8.3.0 Core with concurrent vacuum fulls
On Wed, Mar 5, 2008 at 10:13 AM, Tom Lane [EMAIL PROTECTED] wrote: I wrote: In particular, if that's the problem, why has this not been seen before? The fact that it's going through heap_page_prune doesn't seem very relevant --- VACUUM FULL has certainly always had to invoke CacheInvalidateHeapTuple someplace or other. So I still want to see the deadlock report ... we at least need to know which tables are involved in the deadlock. Actually, maybe it *has* been seen before. Gavin, are you in the habit of running concurrent VACUUM FULLs on system catalogs, and if so have you noted that they occasionally get deadlock failures? Generally no, I've never noticed deadlocks before, but I'll go back and look at some of the other the machines. A separate line of thought is whether it's a good idea that heap_page_prune calls the inval code inside a critical section. That's what's turning an ordinary deadlock failure into a PANIC. Even without the possibility of having to do cache initialization, that's a lot of code to be invoking, and it has obvious failure modes (eg, out of memory for the new inval list item). The more I think about this the more I don't like it. The critical section in heap_page_prune is *way* too big. Aside from the inval call, there are HeapTupleSatisfiesVacuum() calls, which could have failures during attempted clog references. The reason the critical section is so large is that we're manipulating the contents of a shared buffer, and we don't want a failure to leave a partially-modified page in the buffer. We could fix that if we were to memcpy the page into local storage and do all the pruning work there. Then the critical section would only surround copying the page back to the buffer and writing the WAL record. Copying the page is a tad annoying but heap_page_prune is an expensive operation anyway, and I think we really are at too much risk of PANIC the way it's being done now. Has anyone got a better idea? regards, tom lane
Re: [HACKERS] 8.3.0 Core with concurrent vacuum fulls
On Wed, Mar 5, 2008 at 10:31 AM, Tom Lane [EMAIL PROTECTED] wrote: Gavin M. Roy [EMAIL PROTECTED] writes: 2008-03-04 05:45:47 EST [6698]: [1-1] LOG: process 6698 still waiting for AccessShareLock on relation 1247 of database 16385 after 1001.519 ms 2008-03-04 05:45:47 EST [6698]: [2-1] STATEMENT: VACUUM FULL autograph.autograph_creators 2008-03-04 05:46:28 EST [6730]: [1-1] LOG: process 6730 still waiting for AccessShareLock on relation 1247 of database 16385 after 1000.887 ms 2008-03-04 05:46:28 EST [6730]: [2-1] STATEMENT: VACUUM FULL lunchmoney.totals 2008-03-04 05:47:55 EST [3809]: [18-1] LOG: server process (PID 6742) was terminated by signal 6: Aborted 2008-03-04 05:47:55 EST [3809]: [19-1] LOG: terminating any other active server processes 2008-03-04 05:47:55 EST [6741]: [12-1] WARNING: terminating connection because of crash of another server process How annoying ... the PANIC message doesn't seem to have reached the log. elog.c is careful to fflush(stderr) before abort(), so that isn't supposed to happen. But it looks like you are using syslog for logging (correct?) so maybe this is a problem with the syslog implementation you're using. What's the platform exactly? I wonder if it'd be reasonable to put a closelog() call just before the abort() ... regards, tom lane I'm using stderr, emulated to look like syslog for pgfouine. log_destination = 'stderr' logging_collector = on # Enable capturing of stderr and csvlog log_directory = '/var/log/postgres/' # Directory where log files are written log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' # Log file name pattern.
Re: [HACKERS] 8.3.0 Core with concurrent vacuum fulls
On Wed, Mar 5, 2008 at 10:31 AM, Tom Lane [EMAIL PROTECTED] wrote: Gavin M. Roy [EMAIL PROTECTED] writes: 2008-03-04 05:45:47 EST [6698]: [1-1] LOG: process 6698 still waiting for AccessShareLock on relation 1247 of database 16385 after 1001.519 ms 2008-03-04 05:45:47 EST [6698]: [2-1] STATEMENT: VACUUM FULL autograph.autograph_creators 2008-03-04 05:46:28 EST [6730]: [1-1] LOG: process 6730 still waiting for AccessShareLock on relation 1247 of database 16385 after 1000.887 ms 2008-03-04 05:46:28 EST [6730]: [2-1] STATEMENT: VACUUM FULL lunchmoney.totals 2008-03-04 05:47:55 EST [3809]: [18-1] LOG: server process (PID 6742) was terminated by signal 6: Aborted 2008-03-04 05:47:55 EST [3809]: [19-1] LOG: terminating any other active server processes 2008-03-04 05:47:55 EST [6741]: [12-1] WARNING: terminating connection because of crash of another server process How annoying ... the PANIC message doesn't seem to have reached the log. elog.c is careful to fflush(stderr) before abort(), so that isn't supposed to happen. But it looks like you are using syslog for logging (correct?) so maybe this is a problem with the syslog implementation you're using. What's the platform exactly? I wonder if it'd be reasonable to put a closelog() call just before the abort() ... regards, tom lane The panic may have made it if this is what you were looking for: 2008-03-04 05:45:20 EST [6742]: [7-1] PANIC: deadlock detected 2008-03-04 05:58:33 EST [8751]: [3-1] PANIC: deadlock detected (it cored twice before I lowered the concurrency of vacuums)
Re: [HACKERS] 8.3.0 Core with concurrent vacuum fulls
2008-03-04 05:45:20 EST [6742]: [7-1] PANIC: deadlock detected 2008-03-04 05:45:20 EST [6742]: [8-1] DETAIL: Process 6742 waits for AccessShareLock on relation 2619 of database 16385; blocked by process 6740. Process 6740 waits for AccessShareLock on relation 1247 of database 16385; blocked by process 6742. 2008-03-04 05:45:20 EST [6742]: [9-1] STATEMENT: VACUUM FULL pg_catalog.pg_type Sorry, been juggling too many things this morning! On Wed, Mar 5, 2008 at 10:45 AM, Tom Lane [EMAIL PROTECTED] wrote: Gavin M. Roy [EMAIL PROTECTED] writes: The panic may have made it if this is what you were looking for: 2008-03-04 05:45:20 EST [6742]: [7-1] PANIC: deadlock detected 2008-03-04 05:58:33 EST [8751]: [3-1] PANIC: deadlock detected That's what I expected to find, but where's the DETAIL for these? regards, tom lane
[HACKERS] 8.3.0 Core with concurrent vacuum fulls
This morning I had a postgres 8.3 install core this morning while multiple vacuum fulls were taking place. I saved the core file, would anyone be interested in dissecting it? I've otherwise had no issues with this machine or pgsql install. Gavin
Re: [HACKERS] 8.3.0 Core with concurrent vacuum fulls
[EMAIL PROTECTED] backup]$ cat /etc/redhat-release CentOS release 4.4 (Final) BINDIR = /usr/local/pgsql/bin DOCDIR = /usr/local/pgsql/doc INCLUDEDIR = /usr/local/pgsql/include PKGINCLUDEDIR = /usr/local/pgsql/include INCLUDEDIR-SERVER = /usr/local/pgsql/include/server LIBDIR = /usr/local/pgsql/lib PKGLIBDIR = /usr/local/pgsql/lib LOCALEDIR = MANDIR = /usr/local/pgsql/man SHAREDIR = /usr/local/pgsql/share SYSCONFDIR = /usr/local/pgsql/etc PGXS = /usr/local/pgsql/lib/pgxs/src/makefiles/pgxs.mk CONFIGURE = '--with-ldap' '--with-perl' '--enable-integer-datetimes' CC = gcc CPPFLAGS = -D_GNU_SOURCE CFLAGS = -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing CFLAGS_SL = -fpic LDFLAGS = -Wl,-rpath,'/usr/local/pgsql/lib' LDFLAGS_SL = LIBS = -lpgport -lz -lreadline -ltermcap -lcrypt -ldl -lm VERSION = PostgreSQL 8.3.0 (gdb) where #0 0x003fe362e21d in raise () from /lib64/tls/libc.so.6 #1 0x003fe362fa1e in abort () from /lib64/tls/libc.so.6 #2 0x0063a2e3 in errfinish () #3 0x005974c4 in DeadLockReport () #4 0x0059381f in LockAcquire () #5 0x00592357 in LockRelationOid () #6 0x00457255 in relation_open () #7 0x004574c3 in heap_open () #8 0x0062cf41 in CatalogCacheInitializeCache () #9 0x0062dfad in PrepareToInvalidateCacheTuple () #10 0x0062e8c5 in CacheInvalidateHeapTuple () #11 0x0045c803 in heap_page_prune () #12 0x005086cd in vacuum_rel () #13 0x005096bb in vacuum () #14 0x005a163b in PortalRunUtility () #15 0x005a1714 in PortalRunMulti () #16 0x005a1d30 in PortalRun () #17 0x0059f4b6 in PostgresMain () #18 0x005760c0 in ServerLoop () #19 0x0050 in PostmasterMain () #20 0x0052fd3e in main () On Tue, Mar 4, 2008 at 11:35 AM, Alvaro Herrera [EMAIL PROTECTED] wrote: Gavin M. Roy wrote: This morning I had a postgres 8.3 install core this morning while multiple vacuum fulls were taking place. I saved the core file, would anyone be interested in dissecting it? I've otherwise had no issues with this machine or pgsql install. Of course. Please post the backtrace. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Re: [HACKERS] 8.3devel slower than 8.2 under read-only load
On 11/24/07, Gregory Stark [EMAIL PROTECTED] wrote: This is a conflict which will affect Postgres in the future as well. Generally I/O costs win over cpu costs in databases since only relatively small systems are cpu-bound. Large systems are typically I/O-bound. That really depends on hardware doesn't it? I'd say that I'm more concerned with CPU than IO on high-end hardware as a generalization, especially with the scaling issues beyond 32 CPU's. Regards, Gavin
Re: [HACKERS] Machine available for community use
Just a follow-up to note that Red Hat has graciously donated a 1 year RHEL subscription and myYearbook is paying Command Prompt to setup the RHEL box for community use. We've not worked out a scheduling methodology, or how to best organize the use of said hardware, but I know that Tom and others are interested. Does anyone have a scheduling solution for things like this to make sure people aren't stepping on each others toes processor/ram/disk wise? Also, what should the policies be for making sure that people can use the box for what they need to use the box for? Should people clean up after themselves data usage wise after their scheduled time? Should people only be able to run PostgreSQL in the context of their own user? Do we have experience with such setups in the past? What has worked well and what hasn't? Gavin On 7/25/07, Gavin M. Roy [EMAIL PROTECTED] wrote: Recently I've been involved in or overheard discussions about SMP scalability at both the PA PgSQL get together and in some list traffic. myYearbook.com would ike to make one of our previous production machines available to established PgSQL Hackers who don't have access to this level of hardware for testing, benchmarking and development to work at improving SMP scalability and related projects. The machine is a HP 585 G1, 8 Core AMD, 32GB RAM with one 400GB 14 Spindle DAS Array dedicated to community use. I've attached a text file with dmesg and /proc/cpuinfo output. I'm working on how this will be setup and am open to suggestions on how to structure access. I'm currently in the process of having Gentoo linux reinstalled on the box since that is what I am most comfortable administering from a security perspective. If this will be a blocker for developers who would actually work on it, please let me know. If you're interested in access, my only requirement is that you're a current PgSQL Hacker with a proven track-record of committing patches to the community. This is a resource we could be using for something else, and I'd like to see the community get direct benefit from it as opposed to it being a play sandbox for people who want to tinker. Please let me know thoughts, concerns or suggestions. Gavin M. Roy CTO myYearbook.com [EMAIL PROTECTED] ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Machine available for community use
Let us know when/if and we'll pay command prompt to install the base OS on the system. All that we're waiting on at this point is the final on the OS. Gavin On 7/31/07, Tom Lane [EMAIL PROTECTED] wrote: Josh Berkus [EMAIL PROTECTED] writes: Hey, this is looking like a serious case of Bike Shedding. That is, a dozen people are arguing about what color to paint the bike shed instead of getting it built.[1] FWIW, it's looking like Red Hat will donate a RHEL/RHN subscription if we want one, though I don't have final approval quite yet. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Machine available for community use
It's actually in Texas, and we have no intention to put a time limit on its availability. I think the availability will be there as long as there is use and we're in the Texas data center, which I don't see ending any time soon. On 7/31/07, Josh Berkus [EMAIL PROTECTED] wrote: Gavin, I'm actually in the middle of assembling a general performance test lab for the PostgreSQL hackers, using equipment donated by Sun, Hi5, and (hopefully) Unisys and Intel. While your machine would obviously stay in Pennsylvania, it would be cool if we could somehow arrange a unified authentication booking system. I'm pretty sure I can even raise money to get one created. How long will this system remain available to us? -- Josh Berkus PostgreSQL @ Sun San Francisco ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Machine available for community use
Let me look at what makes sense there, I am open to it. On 7/26/07, Tom Lane [EMAIL PROTECTED] wrote: Greg Smith [EMAIL PROTECTED] writes: But this is pushing forward PostgreSQL development you're doing here. If you've got a problem such that something works differently based on the order in which you built the packages, which is going to be unique to every Linux distribution already, that is itself noteworthy and deserves engineering out. You might think of this high-end machine being a little different as usefully adding diversity robustness in a similar way to how the buildfarm helps improve the core right now. Actually, the thing that's concerning me is *exactly* lack of diversity. If we have just one of these things then there's a significant risk of unconsciously tuning PG towards that specific platform. I'd rather we take that risk with a well-standardized, widely used platform than with something no one else can reproduce. Really there's a pretty good argument for having several different OS'es available on the box --- I wonder whether Gavin is up to managing some sort of VM or multiboot setup. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] Machine available for community use
Recently I've been involved in or overheard discussions about SMP scalability at both the PA PgSQL get together and in some list traffic. myYearbook.com would ike to make one of our previous production machines available to established PgSQL Hackers who don't have access to this level of hardware for testing, benchmarking and development to work at improving SMP scalability and related projects. The machine is a HP 585 G1, 8 Core AMD, 32GB RAM with one 400GB 14 Spindle DAS Array dedicated to community use. I've attached a text file with dmesg and /proc/cpuinfo output. I'm working on how this will be setup and am open to suggestions on how to structure access. I'm currently in the process of having Gentoo linux reinstalled on the box since that is what I am most comfortable administering from a security perspective. If this will be a blocker for developers who would actually work on it, please let me know. If you're interested in access, my only requirement is that you're a current PgSQL Hacker with a proven track-record of committing patches to the community. This is a resource we could be using for something else, and I'd like to see the community get direct benefit from it as opposed to it being a play sandbox for people who want to tinker. Please let me know thoughts, concerns or suggestions. Gavin M. Roy CTO myYearbook.com [EMAIL PROTECTED] /proc/cpuinfo: processor : 0 vendor_id : AuthenticAMD cpu family : 15 model : 33 model name : AMD Opteron (tm) Processor 885 stepping: 2 cpu MHz : 1999.953 cache size : 1024 KB physical id : 0 siblings: 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext lm 3dnowext 3dnow pni bogomips: 3599.40 TLB size: 1088 4K pages clflush size: 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: ts fid vid ttp processor : 1 vendor_id : AuthenticAMD cpu family : 15 model : 33 model name : AMD Opteron (tm) Processor 885 stepping: 2 cpu MHz : 1999.953 cache size : 1024 KB physical id : 1 siblings: 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext lm 3dnowext 3dnow pni bogomips: 3599.40 TLB size: 1088 4K pages clflush size: 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: ts fid vid ttp processor : 2 vendor_id : AuthenticAMD cpu family : 15 model : 33 model name : AMD Opteron (tm) Processor 885 stepping: 2 cpu MHz : 1999.953 cache size : 1024 KB physical id : 2 siblings: 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext lm 3dnowext 3dnow pni bogomips: 3599.40 TLB size: 1088 4K pages clflush size: 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: ts fid vid ttp processor : 3 vendor_id : AuthenticAMD cpu family : 15 model : 33 model name : AMD Opteron (tm) Processor 885 stepping: 2 cpu MHz : 1999.953 cache size : 1024 KB physical id : 3 siblings: 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext lm 3dnowext 3dnow pni bogomips: 3599.40 TLB size: 1088 4K pages clflush size: 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: ts fid vid ttp processor : 4 vendor_id : AuthenticAMD cpu family : 15 model : 33 model name : AMD Opteron (tm) Processor 885 stepping: 2 cpu MHz : 1999.953 cache size : 1024 KB physical id : 0 siblings: 2 core id : 1 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext lm 3dnowext 3dnow pni bogomips: 3599.40 TLB size: 1088 4K pages clflush size: 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual
Re: [HACKERS] Machine available for community use
If you're interested in using the box, name what you want installed. On 7/25/07, Tom Lane [EMAIL PROTECTED] wrote: Gavin M. Roy [EMAIL PROTECTED] writes: I'm currently in the process of having Gentoo linux reinstalled on the box since that is what I am most comfortable administering from a security perspective. If this will be a blocker for developers who would actually work on it, please let me know. Personally I'd prefer almost any of the other Linux distros. Gentoo always leaves me wondering exactly what I'm running today, and I think reproducibility is an important attribute for a benchmarking machine. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Machine available for community use
Note it's a 28 disk system, and I can allocate more if needed, but I was going to use one MSA for internal use. On 7/25/07, Mark Wong [EMAIL PROTECTED] wrote: On 7/25/07, Tom Lane [EMAIL PROTECTED] wrote: Gavin M. Roy [EMAIL PROTECTED] writes: I'm currently in the process of having Gentoo linux reinstalled on the box since that is what I am most comfortable administering from a security perspective. If this will be a blocker for developers who would actually work on it, please let me know. Personally I'd prefer almost any of the other Linux distros. Gentoo always leaves me wondering exactly what I'm running today, and I think reproducibility is an important attribute for a benchmarking machine. Tom, have any specific ideas in mind for using the system? While I'm used to having more disks it could be useful nonetheless for the tests I used to run if there are no other ideas. Rats, I've always liked Gentoo. ;) Regards, Mark ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Machine available for community use
Ubuntu server? Slackware? Not a fan of Centos, RHEL or Fedora... What about on the BSD side of things? On 7/25/07, Tom Lane [EMAIL PROTECTED] wrote: Gavin M. Roy [EMAIL PROTECTED] writes: If you're interested in using the box, name what you want installed. Personally I use Fedora, but that's because of where I work ;-). I have no objection to some other distro so long as it's one where other people can duplicate your environment easily (no locally compiled stuff). A disadvantage of Fedora is its relatively short support lifetime --- if you don't want to have to reinstall at least once a year, something else would be better. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Machine available for community use
If RH can sponsor a license of RHEL I'm inclined to go there. Not that it was offered, but I think Dave's suggestion was Tom could field that for the box if inclined. If I'm wrong, let me know. If that can't happen, would people prefer CentOS or Ubuntu Server? The people I'm most concerned with are the people who will actually use it. If you consider yourself one of those people, pipe in now, I will tally votes and go from there. From a Gentoo side, I would have kept things pretty stable, but I'd rather developers be comfortable with the environment which will encourage you to use it. I'm not interested in running Debian, which I'm happy to talk about off topic, in private, if anyone cares enough to want to discuss it. What I'm most interested in to touch on Simon's request is SMP scaling. From another Hackers thread this month, which I can dig up, I've walked away with the impression that after 4 cores, we don't see the same level of per-processor performance improvement that we see = 4 cores. What you actually do is up to you, we want to provide this to the hacker community to use as they see fit to continue to improve PostgreSQL which is integral to our operation. Any performance, scalability or even advocacy efforts (read benchmarking) will benefit myYearbook. Gavin On 7/25/07, Simon Riggs [EMAIL PROTECTED] wrote: On Wed, 2007-07-25 at 14:32 -0400, Tom Lane wrote: Dave Page [EMAIL PROTECTED] writes: Perhaps RH could donate us a RHEL/RHN licence for this? I could ask, if there's consensus we want it. Please. It sounded like more people like Debian, though. Well, if you don't we probably will go Debian. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 6: explain analyze is your friend ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Machine available for community use
One thing to take into account is I dont have physical access to the box (It is in TX, I am in PA). All installs but Gentoo will be performed by a well trained NOC monkey. *cough* On 7/25/07, Dave Page [EMAIL PROTECTED] wrote: --- Original Message --- From: Tom Lane [EMAIL PROTECTED] To: Greg Smith [EMAIL PROTECTED] Sent: 25/07/07, 18:54:50 Subject: Re: [HACKERS] Machine available for community use Another fairly big issue is that we need to know whether measurements we take in August are comparable to measurements we take in October, so a fairly stable platform is important. As you say, a fast-changing kernel would make it difficult to have any confidence about comparability over time. That would tend to make me vote for RHEL/Centos, where long-term stability is an explicit development goal. Debian stable might do too, though I'm not as clear about their update criteria as I am about Red Hat's. Perhaps RH could donate us a RHEL/RHN licence for this? /D ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] pg_dump ignore tablespaces
Recently I ran into an issue where restoring from pg_dump from one machine to another with non-matching tablespaces. The primary issue is that index creation will fail if the tablespace does not exist from the dump. I was thinking to best solution for this would be a pg_dump option such as --ignore-tablespaces which would not dump any tablespace related data. This would benefit restoring a dump from one machine to another where there are different disk or tablespace topologies. If such a patch were presented and found to be acceptable code wise, would it be a generally useful enough option to be included? Gavin
Re: [HACKERS] [ANNOUNCE] Welcome Core Team member Dave Page
Congrats Dave! On Aug 25, 2005, at 5:59 PM, Josh Berkus wrote: Project members: On behalf of the PostgreSQL Core Team, I welcome Dave Page. Dave has been the head of the pgODBC project for a couple of years, started the pgAdmin project in 1998, has been our lead webmaster for three years, and is now (split with Magnus) in charge of the Win32 packaging. In these latter two roles, Dave's help is indispensable to the PostgreSQL release process, so we decided it was time for him to have the title for the work he's already doing. We believe that anyone who's been around the project for a few years will agree. So, welcome Dave Page as the newest member of Core! -- --Josh Berkus Josh Berkus PostgreSQL Project Core Team www.postgresql.org ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster Gavin M. Roy 800 Pound Gorilla [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] US Census database (Tiger 2004FE) - 4.4G
You can send it to me, and ehpg will host it. I'll send you a private email with my info. Gavin On Aug 4, 2005, at 8:26 AM, Mark Woodward wrote: It is 4.4G in space in a gzip package. I'll mail a DVD to two people who promise to host it for Hackers. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org Gavin M. Roy 800 Pound Gorilla [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] inclusions WAS: Increased company involvement
Joshua D. Drake wrote: PgFoundry is coming along in its own right. I see three main problems with it at current: 1. It looks like a separate project from PostgreSQL (website, name, etc...) I've been working on porting the site to use a derived theme from the main PostgreSQL site. My main issue right now is finding the right font for the header as Omar and Emily don't have access to it from what I've been told. Regards, Gavin ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [GENERAL] PHP/PDO Database Abstraction Layer
PDO just went into beta, and can be downloaded from http://pecl.php.net/package/pdo to be compiled into previous versions of PHP. We really should get some PHP and PgSQL people onto making sure the PgSQL driver is top notch (if it isn't already). Gavin On Mon, 2005-02-07 at 20:21 -0800, Joshua D. Drake wrote: Robby Russell wrote: Sincerely, Joshua D. Drake Command Prompt, Inc. 503-667-4564 It hasn't been updated since May 2004 though. :-/ Hmm... Well there must be another home for it then because it is set to be the default database api for 5.1. Ahh now I see it is already been pushed into the PHP source... http://php.oregonstate.edu/manual/en/ref.pdo.php I apologize for the confusion. Sincerely, Joshua D. Drake ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Beta5 now Available
No you can not, but the tracker isn't very resource intesive from my past experience. I can host it if needed. Gavin Marc G. Fournier wrote: On Wed, 24 Nov 2004, Thomas Hallgren wrote: Gaetano Mendola wrote: ...so the very first client is the real server that must be run 24/24. I don't think this is correct. You need a tracker for downloaders to be able to find each other but no client is more important than the others. can there be multiple trackers? for instance, if we ran bt.postgresql.org on two different servers, could they both run trackers at the same time? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Beta5 now Available
To a degree you are correct. AFAIK new downloads could not start if the tracker crashed. The tracker is the traffic cop that tells peer nodes about each other. I dont believe the tracker that comes from the main bit torrent author allows for multiple trackers with a common data repository, but if we're really interested, maybe we could hack up the code to talk to a central pgsql database allowing multiple trackers on a dns rr. Gavin Marc G. Fournier wrote: On Wed, 24 Nov 2004, Joshua D. Drake wrote: Marc G. Fournier wrote: On Wed, 24 Nov 2004, Gavin M. Roy wrote: No you can not, but the tracker isn't very resource intesive from my past experience. I can host it if needed. It wasn't that that I was thinking of ... just wondering if there was some way of having it redundant, instead of centralized ... nice thing about ftp mirrors, we have about 60 of them, so if one goes down, it doesn't really affect anything ... from what everyone is saying, if the tracker goes down, it affects everything ... seems odd to have new technology still having single points of failure :( O.k. I know nothing of bittorrent but couldn't we just have to machines that are identically configured that have a round robin DNS thing going on? we're not talking load issues this time ... the way I understand it, bittorrent has a 'tracker' process that only one can be running on the BT Distributed Network at once ... so, if the bt central server goes down, the whole bt network goes down with it ... At least, this is my understanding, someone please correct me if I'm wrong ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Beta5 now Available
The problem is it requires a box with X on it. (ie it's not console Java, it's gui java) I don't have a server to run it on right now, but will be readdressing server allocations shortly and may be able to set something up with x/vnc and would be happy to use that as a primary bt seeding site. Gavin Thomas Hallgren wrote: Marc G. Fournier wrote: What about the Java version that Gavin had mentioned? Aegus or something like that? http://azureus.sourceforge.net/ Regards, Thomas Hallgren ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Beta5 now Available
It's all peer to peer client type stuff with the exception of the tracker server. Gavin Marc G. Fournier wrote: On Mon, 22 Nov 2004, Thomas Hallgren wrote: Marc G. Fournier wrote: What about the Java version that Gavin had mentioned? Aegus or something like that? http://azureus.sourceforge.net/ There is a FreeBSD port of it also but it says A BitTorrent client written in Java ... does it work as server too, or, by its nature, are servers == clients in Bittorrent? :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] 8.0 Beta Thank You
Let me be one of the first to say thank you to all of you guys for yet another awesome version of PgSQL, beta or not. My company and I appreciate all the hard work and such that has gone into making this version happen. Keep up the good work and let me know where I can help. Sincerely, Gavin ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] localhost redux
Yeah, I was talking about it on freenode yesterday. If the stats collector doesn't start, the internal variable is set to off. Gavin Gaetano Mendola wrote: Gavin M. Roy wrote: I have no problem resolving localhost anywhere else on the box, do you have any suggestions on finding out if it's a misconfiguration? Gavin Are you the users on IRC yesterday night ? He had the same problem, however he/she had the GUC variable stats_start_collector equal to on in the configuration file but was shown as off with a SHOW stats_start_collector inside a psql section. That GUC variable is resetted to off is the collector doesn't start ? Regards Gaetano Mendola ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] localhost redux
I've been having a problem with the pgstat.c localhost change since 7.4, in that statistics gathering stopped working on my gentoo boxes. I've not traced it back beyond getaddrinfo called in getaddrinfo_all, and decided the quick fix was to go back and change pgstat to use 127.0.0.1 instead of localhost. I don't know if the problem is isolated to just me, but I wanted to suggest that we use a parameter for that, which can be configured in the postgresql.conf, with a default value if it's not set, set to localhost. I'd be happy to write a patch that follows this behavior but wanted to see if everyone agrees with that line of thinking before doing so. Gavin ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] localhost redux
I have no problem resolving localhost anywhere else on the box, do you have any suggestions on finding out if it's a misconfiguration? Gavin Tom Lane wrote: Gavin M. Roy [EMAIL PROTECTED] writes: I've been having a problem with the pgstat.c localhost change since 7.4, in that statistics gathering stopped working on my gentoo boxes. I've not traced it back beyond getaddrinfo called in getaddrinfo_all, and decided the quick fix was to go back and change pgstat to use 127.0.0.1 instead of localhost. I don't know if the problem is isolated to just me, but I wanted to suggest that we use a parameter for that, which can be configured in the postgresql.conf, with a default value if it's not set, set to localhost. I think you should first trace down what the problem really is --- is your system just misconfigured or is there some fundamental issue that we really ought to answer to? regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [ADMIN] Any Gentoo users interested in a slotted PostgreSQL
I dont see a real need for slots, but I do question why the 7.4 branch isnt marked as stable yet. :) Gavin Jyry Kuukkanen wrote: Hello Are there any Gentoo users here? Yes, there are some :) Do you wish the PostgreSQL ebuild made use of SLOTS? - to allow installing of major versions together (eg. 7.3 and 7.4 on the same host) - to ease migration of databases to new major versions (where a dump/reload is required) I've started a thread at: http://bugs.gentoo.org/show_bug.cgi?id=42894 With a initial plan for relocation of files to support slotting. Comments, suggestions, corrections, or messages to say it's a silly idea (and why) would be very much appreciated before I attempt this next week. I only have Gentoo installed on my old laptop that is not in very heavy and active use. However, PostgreSQL is installed on it and used every now and then. PostgreSQL migration has always worked fine for me, so it has been adequate to install fresher version on some sundry hardware and test the migration with it. This, quite obviously, is only my humble opinion and based on my limited experience :) Cheers, --Jyry C|:-(C|:-/C|8-OC|8-/C|:-( ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
