[HACKERS] timezone buglet?

2011-10-04 Thread daveg
Postgresql 9.0.4 has the timezone: America/Blanc-Sablon However other sources seem to spell this with an underscore instead of dash: America/Blanc_Sablon It appears that beside 'America/Blanc_Sablon', other multi-word timezones are spelled with underscore. For example: 'Australia/Broken_Hi

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 09:02:04PM -0400, Tom Lane wrote: > daveg writes: > > On Wed, Sep 07, 2011 at 07:39:15PM -0400, Tom Lane wrote: > >> BTW ... what were the last versions you were running on which you had > >> *not* seen the problem? (Just wondering about the po

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 07:39:15PM -0400, Tom Lane wrote: > daveg writes: > > Also, this is very intermittant, we have seen it only in recent months > > on both 8.4.7 and 9.0.4 after years of no problems. Lately we see it what > > feels like a few times a month. Possibly

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 06:25:23PM -0400, Tom Lane wrote: > Robert Haas writes: > > I thought about an error exit from client authentication, and that's a > > somewhat appealing explanation, but I can't quite see why we wouldn't > > clean up there the same as anywhere else. The whole mechanism fe

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 06:35:08PM -0400, Tom Lane wrote: > daveg writes: > > It does not seem restricted to pg_authid: > > 2011-08-24 18:35:57.445 24987 c23 apps ERROR: lock AccessShareLock on > > object 16403/2615/0 > > And I think I've seen it on o

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 04:55:24PM -0400, Tom Lane wrote: > Robert Haas writes: > > Tom's right to be skeptical of my theory, because it would require a > > CHECK_FOR_INTERRUPTS() outside of a transaction block in one of the > > pathways that use session-level locks, and I can't find one. > > Mor

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 10:20:24AM -0400, Tom Lane wrote: > Robert Haas writes: > > After spending some time staring at the code, I do have one idea as to > > what might be going on here. When a backend is terminated, > > ShutdownPostgres() calls AbortOutOfAnyTransaction() and then > > LockReleas

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Tue, Aug 23, 2011 at 12:15:23PM -0400, Robert Haas wrote: > On Mon, Aug 22, 2011 at 3:31 AM, daveg wrote: > > So far I've got: > > > >  - affects system tables > >  - happens very soon after process startup > >  - in 8.4.7 and 9.0.4 > >  - not likely

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-09-05 Thread daveg
On Mon, Sep 05, 2011 at 08:19:21PM -0400, Bruce Momjian wrote: > daveg wrote: > > > Can you tell me what table is showing this error? Does it happen during > > > vacuum? Can you run a vacuum verbose to see what it is throwing the > > > error on? Thanks. > >

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-09-05 Thread daveg
Sorry I missed your reply, catching up now. On Wed, Aug 31, 2011 at 09:56:59PM -0400, Bruce Momjian wrote: > daveg wrote: > > On Mon, Aug 29, 2011 at 07:49:24PM +0200, hubert depesz lubaczewski wrote: > > > On Mon, Aug 29, 2011 at 06:54:41PM +0200, hubert depesz lubaczewski wro

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-29 Thread daveg
On Mon, Aug 29, 2011 at 07:49:24PM +0200, hubert depesz lubaczewski wrote: > On Mon, Aug 29, 2011 at 06:54:41PM +0200, hubert depesz lubaczewski wrote: > vacuumdb: vacuuming of database "etsy_v2" failed: ERROR: could not access > status of transaction 3429738606 > DETAIL: Could not open file "pg

Re: [HACKERS] OperationalError: FATAL: lock AccessShareLock on object 0/1260/0 is already

2011-08-23 Thread daveg
On Sun, Aug 14, 2011 at 12:16:39AM -0400, Robert Haas wrote: > On Fri, Aug 12, 2011 at 7:19 PM, daveg wrote: > > This seems to be bug month for my client. Now there are seeing periods > > where all new connections fail immediately with the error: > > > >   FATAL:  lock

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-08-22 Thread daveg
On Fri, Aug 12, 2011 at 04:19:37PM -0700, daveg wrote: > > This seems to be bug month for my client. Now there are seeing periods > where all new connections fail immediately with the error: > >FATAL: lock AccessShareLock on object 0/1260/0 is already held > > This

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-15 Thread daveg
[adding back hackers so the thread shows the resolution] On Sun, Aug 14, 2011 at 07:02:55PM -0400, Tom Lane wrote: > Sounds good. Based on my own testing so far, I think that patch will > probably make things measurably better for you, though it won't resolve > every corner case. The most recent

Re: [HACKERS] OperationalError: FATAL: lock AccessShareLock on object 0/1260/0 is already

2011-08-13 Thread daveg
On Sun, Aug 14, 2011 at 12:16:39AM -0400, Robert Haas wrote: > On Fri, Aug 12, 2011 at 7:19 PM, daveg wrote: > > This seems to be bug month for my client. Now there are seeing periods > > where all new connections fail immediately with the error: > > > >   FATAL:  lock

[HACKERS] OperationalError: FATAL: lock AccessShareLock on object 0/1260/0 is already

2011-08-12 Thread daveg
This seems to be bug month for my client. Now there are seeing periods where all new connections fail immediately with the error: FATAL: lock AccessShareLock on object 0/1260/0 is already held This happens on postgresql 8.4.7 on a large (512GB, 32 core) system that has been up for months. I

Re: [HACKERS] our buffer replacement strategy is kind of lame

2011-08-12 Thread daveg
On Fri, Aug 12, 2011 at 01:28:49PM +0100, Simon Riggs wrote: > I think there are reasonable arguments to make > > * prefer_cache = off (default) | on a table level storage parameter, > =on will disable the use of BufferAccessStrategy > > * make cache_spoil_threshold a parameter, with default 0.25

Re: [HACKERS] VACUUM FULL versus system catalog cache invalidation

2011-08-12 Thread daveg
On Fri, Aug 12, 2011 at 09:26:02PM +0100, Simon Riggs wrote: > With HOT, there is very little need to perform a VACUUM FULL on any > shared catalog table. Look at the indexes... > > I would a suggest that VACUUM FULL perform only a normal VACUUM on > shared catalog tables, then perform an actual V

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-05 Thread daveg
On Fri, Aug 05, 2011 at 12:10:31PM -0400, Tom Lane wrote: > I wrote: > > Ahh ... you know what, never mind about stack traces, let's just see if > > the attached patch doesn't fix it. > > On reflection, that patch would only fix the issue for pg_class, and > that's not the only catalog that gets c

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Thu, Aug 04, 2011 at 04:16:08PM -0400, Tom Lane wrote: > daveg writes: > > We are seeing "cannot read' and 'cannot open' errors too that would be > > consistant with trying to use a vanished file. > > Yeah, these all seem consistent with the idea tha

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Thu, Aug 04, 2011 at 12:28:31PM -0400, Tom Lane wrote: > daveg writes: > > Summary: the failing process reads 0 rows from 0 blocks from the OLD > > relfilenode. > > Hmm. This seems to mean that we're somehow missing a relation mapping > invalidation message,

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Thu, Aug 04, 2011 at 12:28:31PM -0400, Tom Lane wrote: > daveg writes: > > Summary: the failing process reads 0 rows from 0 blocks from the OLD > > relfilenode. > > Hmm. This seems to mean that we're somehow missing a relation mapping > invalidation message,

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Wed, Aug 03, 2011 at 11:18:20AM -0400, Tom Lane wrote: > Evidently not, if it's not logging anything, but now the question is > why. One possibility is that for some reason RelationGetNumberOfBlocks > is persistently lying about the file size. (We've seen kernel bugs > before that resulted in

Re: [HACKERS] Further news on Clang - spurious warnings

2011-08-03 Thread daveg
On Wed, Aug 03, 2011 at 04:03:39PM -0400, Tom Lane wrote: > The C standard specifies that signed-to-unsigned conversions must work > like that; and even if the standard didn't, it would surely work like > that on any machine with two's-complement representation, which is to > say every computer bui

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-03 Thread daveg
On Mon, Aug 01, 2011 at 01:23:49PM -0400, Tom Lane wrote: > daveg writes: > > On Sun, Jul 31, 2011 at 11:44:39AM -0400, Tom Lane wrote: > >> I think we need to start adding some instrumentation so we can get a > >> better handle on what's going on in your databas

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-31 Thread daveg
On Sun, Jul 31, 2011 at 11:44:39AM -0400, Tom Lane wrote: > daveg writes: > > Here is the update: the problem happens with vacuum full alone, no reindex > > is needed to trigger it. I updated the script to avoid reindexing after > > vacuum. Over the past two days there are

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-31 Thread daveg
On Thu, Jul 28, 2011 at 11:31:31PM -0700, daveg wrote: > On Thu, Jul 28, 2011 at 07:45:01PM -0400, Robert Haas wrote: > > REINDEX. My guess is that this is happening either right around the > > time the VACUUM FULL commits or right around the time the REINDEX > > commits. It

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-29 Thread daveg
On Fri, Jul 29, 2011 at 09:55:46AM -0400, Tom Lane wrote: > The thing that was bizarre about the one instance in the buildfarm was > that the error was persistent, ie, once a session had failed all its > subsequent attempts to access pg_class failed too. I gather from Dave's > description that it'

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-28 Thread daveg
On Thu, Jul 28, 2011 at 07:45:01PM -0400, Robert Haas wrote: > On Thu, Jul 28, 2011 at 5:46 PM, daveg wrote: > > On Thu, Jul 28, 2011 at 09:46:41AM -0400, Robert Haas wrote: > >> On Wed, Jul 27, 2011 at 8:28 PM, daveg wrote: > >> > My client has been seeing regu

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-28 Thread daveg
On Thu, Jul 28, 2011 at 09:46:41AM -0400, Robert Haas wrote: > On Wed, Jul 27, 2011 at 8:28 PM, daveg wrote: > > My client has been seeing regular instances of the following sort of > > problem: > On what version of PostgreSQL? 9.0.4. I previously said: > > This occurs

[HACKERS] error: could not find pg_class tuple for index 2662

2011-07-27 Thread daveg
My client has been seeing regular instances of the following sort of problem: ... 03:06:09.453 exec_simple_query, postgres.c:900 03:06:12.042 XX000: could not find pg_class tuple for index 2662 at character 13 03:06:12.042 RelationReloadIndexInfo, relcache.c:1740 03:06:12.042 INSERT INTO zzz

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-08 Thread daveg
On Tue, Mar 08, 2011 at 10:37:24AM +0200, Heikki Linnakangas wrote: > On 08.03.2011 10:00, Heikki Linnakangas wrote: > >Another idea is to give up on the warning when it appears that > >oldestxmin has moved backwards, and assume that it's actually fine. We > >could still warn in other cases where t

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-08 Thread daveg
On Tue, Mar 08, 2011 at 10:00:01AM +0200, Heikki Linnakangas wrote: > On 08.03.2011 04:07, Greg Stark wrote: > >Well from that log you definitely have OldestXmin going backwards. And > >not by a little bit either. at 6:33 it set the all_visible flag and > >then at 7:01 it was almost 1.3 million tra

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-07 Thread daveg
On Fri, Mar 04, 2011 at 05:52:29PM +0200, Heikki Linnakangas wrote: > Hmm, if these all came from the same database, then it looks OldestXmin > has moved backwards. That would explain the warnings. First one vacuum > determines that all the tuples are visible to everyone and sets the > flag. The

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-04 Thread daveg
On Thu, Mar 03, 2011 at 09:04:04AM -0600, Merlin Moncure wrote: > On Thu, Mar 3, 2011 at 2:16 AM, Heikki Linnakangas > wrote: > > On 03.03.2011 09:12, daveg wrote: > >> > >> Question: what would be the consequence of simply patching out the setting > >> of

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-03 Thread daveg
On Thu, Mar 03, 2011 at 10:16:29AM +0200, Heikki Linnakangas wrote: > On 03.03.2011 09:12, daveg wrote: > >Question: what would be the consequence of simply patching out the setting > >of this flag? Assuming that the incorrect PD_ALL_VISIBLE flag is the only > >problem (b

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-02 Thread daveg
On Tue, Mar 01, 2011 at 08:40:37AM -0500, Robert Haas wrote: > On Mon, Feb 28, 2011 at 10:32 PM, Greg Stark wrote: > > On Tue, Mar 1, 2011 at 1:43 AM, David Christensen > > wrote: > >> Was this cluster upgraded to 8.4.4 from 8.4.0?  It sounds to me like a > >> known bug in 8.4.0 which was fixed

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-02 Thread daveg
On Wed, Mar 02, 2011 at 04:20:24PM -0800, bricklen wrote: > On Wed, Mar 2, 2011 at 3:53 PM, daveg wrote: > >> > Postgresql version is 8.4.4. > >> > >> I don't see how this could be related, but since you're running on NFS, > >> maybe it is, som

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-02 Thread daveg
On Wed, Mar 02, 2011 at 06:45:13PM -0300, Alvaro Herrera wrote: > Excerpts from daveg's message of mié mar 02 18:30:34 -0300 2011: > > > After a restart and vacuum of all dbs with no other activity things were > > quiet for a couple hours and then we started seeing these PD_ALL_VISIBLE > > message

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-02 Thread daveg
On Tue, Mar 01, 2011 at 01:20:43PM -0800, daveg wrote: > On Tue, Mar 01, 2011 at 12:00:54AM +0200, Heikki Linnakangas wrote: > > On 28.02.2011 23:28, daveg wrote: > > >On Wed, Jan 12, 2011 at 10:46:14AM +0200, Heikki Linnakangas wrote: > > >>We'll likely need to

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-01 Thread daveg
On Tue, Mar 01, 2011 at 12:00:54AM +0200, Heikki Linnakangas wrote: > On 28.02.2011 23:28, daveg wrote: > >On Wed, Jan 12, 2011 at 10:46:14AM +0200, Heikki Linnakangas wrote: > >>We'll likely need to go back and forth a few times with various > >>debugging patches u

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-01 Thread daveg
On Mon, Feb 28, 2011 at 07:43:39PM -0600, David Christensen wrote: > > On Feb 28, 2011, at 3:28 PM, daveg wrote: > > > Anything new on this? I'm seeing at on one of my clients production boxes. > > Also, what is the significance, ie what is the risk or damage potenti

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-02-28 Thread daveg
On Wed, Jan 12, 2011 at 10:46:14AM +0200, Heikki Linnakangas wrote: > On 12.01.2011 06:21, Fujii Masao wrote: > >On Sat, Dec 25, 2010 at 2:09 PM, Maxim Boguk wrote: > >>While I trying create reproducible test case for BUG #5798 I > >>encountered very strange effect on two of my servers (both serve

Re: [HACKERS] PostgreSQL and HugePage

2010-10-21 Thread daveg
On Thu, Oct 21, 2010 at 08:16:27AM -0700, Mark Wong wrote: > On Tue, Oct 19, 2010 at 8:30 PM, daveg wrote: > > On Wed, Oct 20, 2010 at 04:08:37PM +1300, Mark Kirkwood wrote: > >> On 20/10/10 16:05, Mark Kirkwood wrote: > >> > > >> > > >> >shm

Re: [HACKERS] PostgreSQL and HugePage

2010-10-20 Thread daveg
On Wed, Oct 20, 2010 at 12:28:25PM -0700, Greg Stark wrote: > On Wed, Oct 20, 2010 at 12:17 PM, Greg Stark wrote: > > I don't think it's a big cost once all the processes > > have been forked if you're reusing them beyond perhaps slightly more > > efficient cache usage. > > Hm, this site claims t

Re: [HACKERS] PostgreSQL and HugePage

2010-10-19 Thread daveg
On Wed, Oct 20, 2010 at 04:08:37PM +1300, Mark Kirkwood wrote: > On 20/10/10 16:05, Mark Kirkwood wrote: > > > > > >shmget and friends are hugetlbpage aware, so it seems it should 'just > >work'. > > > > Heh - provided you specify > > SHM_HUGETLB > > > in the relevant call that is :-) I had

Re: [HACKERS] a faster compression algorithm for pg_dump

2010-04-14 Thread daveg
On Tue, Apr 13, 2010 at 03:03:58PM -0400, Tom Lane wrote: > Joachim Wieland writes: > > If we still cannot do this, then what I am asking is: What does the > > project need to be able to at least link against such a compression > > algorithm? > > Well, what we *really* need is a convincing argume

Re: [HACKERS] TCP keepalive support for libpq

2010-02-10 Thread daveg
On Tue, Feb 09, 2010 at 09:34:10AM -0500, Andrew Chernow wrote: > Tollef Fog Heen wrote: > >(please Cc me on replies, I am not subscribed) > > > >Hi, > > > >libpq currently does not use TCP keepalives. This is a problem in our > >case where we have some clients waiting for notifies and then the >

Re: [HACKERS] pg_dump enhancement proposal

2009-12-13 Thread daveg
On Thu, Nov 12, 2009 at 04:31:37PM -0500, Tom Lane wrote: > Mark Hammonds writes: > > 2. Custom Query Exports > > > In my use of mysqldump, I found one feature very useful: the ability > > to execute a custom SELECT. . .WHERE statement and then dump only the > > results. This feature current

Re: [HACKERS] EOL for 7.4?

2009-11-12 Thread daveg
On Fri, Nov 13, 2009 at 02:47:56AM +, Greg Stark wrote: > On Fri, Nov 13, 2009 at 2:35 AM, daveg wrote: > > I suggest we announce now that both 7.4 and 8.0 will EOL when 8.5 is > > expected > > to ship, or to comfort those who never use .0 versions when 8.5.1 ships. >

Re: [HACKERS] EOL for 7.4?

2009-11-12 Thread daveg
On Fri, Nov 13, 2009 at 02:22:01AM +, Greg Stark wrote: > > Really I think you guys are on the wrong track trying to map Postgres > releases to commercial support terms. None of the Postgres releases > are "supported" in the sense that there's no warranty and no promises, > it's all best effor

Re: [HACKERS] EOL for 7.4?

2009-11-03 Thread daveg
On Tue, Nov 03, 2009 at 10:32:17AM -0800, Josh Berkus wrote: > So I'm going to make a case in favor of EOL'ing 7.4. In fact, I'd be in > favor of doing so in, say, February after an announcement this month. > > The main reason I'm in favor of this is that we have a lot of users > using 7.4 out of

Re: [HACKERS] Postgres server goes in recovery mode repeteadly

2009-10-20 Thread daveg
On Fri, Oct 02, 2009 at 07:57:13PM -0700, daveg wrote: > On Fri, Oct 02, 2009 at 10:41:07AM -0400, Alvaro Herrera wrote: > > daveg escribió: > > > > > I work with Kunal and have been looking into this. It appears to be the > > > same > > > as

Re: [HACKERS] Application name patch - v2

2009-10-20 Thread daveg
On Tue, Oct 20, 2009 at 12:16:42PM -0400, Tom Lane wrote: > Magnus Hagander writes: > > Also, how many platforms can't we do this on? If we have BSD and > > Windows covered already. on linux, I believe you can easily read it > > out of /proc/self/cmdline, no? > > Writing a pile of platform-specif

Re: [HACKERS] Application name patch - v2

2009-10-19 Thread daveg
On Mon, Oct 19, 2009 at 01:00:28PM +0100, Dave Page wrote: > On Mon, Oct 19, 2009 at 12:57 PM, Pavel Stehule > wrote: > > It is not practical. I'll log errors. Usually SQL injection generates > > lot of errors. Loging all statements has not sense. What is difference > > bad and good SQL statement

Re: [HACKERS] Deprecation

2009-10-19 Thread daveg
On Sat, Oct 17, 2009 at 03:01:27PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > Sounds like a good reason to remove add_missing_from in 8.5. > > Seems like the general consensus is that it's okay to do that. > I will go make it happen unless somebody squawks pretty soon... > >

Re: [HACKERS] Postgres server goes in recovery mode repeteadly

2009-10-02 Thread daveg
On Fri, Oct 02, 2009 at 10:41:07AM -0400, Alvaro Herrera wrote: > daveg escribió: > > > I work with Kunal and have been looking into this. It appears to be the same > > as the bug described in: > > > > http://archives.postgresql.org/pgsql-bugs/2009-09/msg00355.ph

Re: [HACKERS] Postgres server goes in recovery mode repeteadly

2009-10-01 Thread daveg
On Tue, Sep 29, 2009 at 09:52:06PM +0530, kunal sharma wrote: > Hi , > We are using Postgres 8.4 and its been found going into recovery > mode couple of times. The server process seems to fork another child process > which is another postgres server running under same data directory and aft

Re: [HACKERS] Limit allocated memory per session

2009-10-01 Thread daveg
On Thu, Oct 01, 2009 at 11:47:43AM -0400, Tom Lane wrote: > Euler Taveira de Oliveira writes: > > Tom Lane escreveu: > >> daveg writes: > >>> I'd like to propose adding a new GUC to limit the amount of memory a > >>> backend > >&

Re: [HACKERS] Limit allocated memory per session

2009-10-01 Thread daveg
On Thu, Oct 01, 2009 at 10:35:55AM -0400, Tom Lane wrote: > daveg writes: > > I'd like to propose adding a new GUC to limit the amount of memory a backend > > can allocate for its own use. > > Use ulimit. That was my initial thought too. However, ulimit() is d

[HACKERS] Limit allocated memory per session

2009-10-01 Thread daveg
I'd like to propose adding a new GUC to limit the amount of memory a backend can allocate for its own use. The problem this addresses is that sometimes one needs to set work_mem fairly high to get good query plans for large joins. However, some complex queries will then use huge amounts of memory

Re: [HACKERS] "BEGIN TRANSACTION" and "START TRANSACTION": different error handling

2009-09-24 Thread daveg
On Thu, Sep 24, 2009 at 12:16:43PM +0300, Hannu Krosing wrote: > "I expect the transaction is aborted and rollback is executed > automatically." - this is not how postgreSQL behaves. PostgreSQL needs > an explicit end of transaction from client, either COMMIT; or ROLLBACK; > > when run from psql,

Re: [HACKERS] Adding \ev view editor?

2009-09-23 Thread daveg
On Mon, Sep 21, 2009 at 02:26:05PM -0400, Andrew Dunstan wrote: >andrew=# select pg_get_viewdef('foo',true); >pg_get_viewdef >-- > SELECT 'a'::text AS b, > ( SELECT 1 >FROM dual) AS x, > random() AS

Re: [HACKERS] numeric_to_number() function skipping some digits

2009-09-23 Thread daveg
On Tue, Sep 22, 2009 at 10:27:19AM +0530, Jeevan Chalke wrote: > > It seems that Oracle reads formatting string from right-to-left. Here are > few results: > ('number','format') ==> Oracle PG > > ('34,50','999,99') ==> 3450340 > ('

Re: [HACKERS] happy birthday Tom Lane ...

2009-09-18 Thread daveg
On Fri, Sep 18, 2009 at 01:04:23PM +0200, Hans-Juergen Schoenig -- PostgreSQL wrote: > Tom, > > On behalf of the entire PostgreSQL team here in Austria I want to wish > you a happy birthday. > We hope that you fill be a vital part of PostgreSQL for many years to come. > > Best regards, > >

Re: [HACKERS] RfD: more powerful "any" types

2009-09-17 Thread daveg
On Tue, Sep 15, 2009 at 07:38:18AM +0200, Pavel Stehule wrote: > it isn't fair :) why you use $$ without single quote? And still this > case should be vulnerable on SQL injection. Maybe you or me knows, > what SQL injection means, but beginners knows nothing and this people > use following bad code

Re: [HACKERS] Ragged CSV import

2009-09-12 Thread daveg
On Fri, Sep 11, 2009 at 10:27:06AM +0200, Dimitri Fontaine wrote: > Maybe instead of opening FROM for COPY, having it accepted in WITH would > be better, the same way (from the user point of view) that DML returning > are worked on. ... > WITH csv AS ( >COPY t FROM stdin CSV > ) > INSERT IN

Re: [HACKERS] RfD: more powerful "any" types

2009-09-12 Thread daveg
On Fri, Sep 11, 2009 at 11:43:32AM -0400, Merlin Moncure wrote: > > If you are going to use printf format codes, which is good and useful > being something of a standard, I'd call routine printf (not format) > and actually wrap vsnprintf. The format codes in printf have a very > specific meaning:

Re: [HACKERS] remove flatfiles.c

2009-09-03 Thread daveg
On Thu, Sep 03, 2009 at 07:57:25PM -0400, Andrew Dunstan wrote: > daveg wrote: > >On Tue, Sep 01, 2009 at 07:42:56PM -0400, Tom Lane wrote: > >>I'm having a hard time believing that VACUUM FULL really has any > >>interesting use-case anymore. > > > >

Re: [HACKERS] remove flatfiles.c

2009-09-03 Thread daveg
On Tue, Sep 01, 2009 at 07:42:56PM -0400, Tom Lane wrote: > Greg Stark writes: > > On Wed, Sep 2, 2009 at 12:01 AM, Alvaro > > Herrera wrote: > >>> The use cases where VACUUM FULL wins currently are where storing two > >>> copies of the table and its indexes concurrently just isn't practical. > >>

Re: [HACKERS] Add YAML option to explain

2009-08-31 Thread daveg
On Mon, Aug 31, 2009 at 02:15:08PM -, Greg Sabino Mullane wrote: > > Greg, can we see a few examples of the YAML output > > compared to both json and text? ... > greg=# explain (format json, analyze on) select * from pg_class where relname > ~ 'x' order by 1,2,3; > QUER

Re: [HACKERS] Add YAML option to explain

2009-08-28 Thread daveg
On Fri, Aug 28, 2009 at 04:37:41PM -0700, David E. Wheeler wrote: > On Aug 28, 2009, at 3:45 PM, Stephen Frost wrote: > > >+1 from me. I've read the other comments and just plain don't agree > >with them. It's a small patch, adds a useful format for EXPLAIN, and > >would be used. > > > >One of t

Re: [HACKERS] 8.5 release timetable, again

2009-08-28 Thread daveg
On Thu, Aug 27, 2009 at 08:02:03PM -0700, Ron Mayer wrote: > Andrew Dunstan wrote: > > I don't know of anyone who is likely to want to try out alphas in their > > normal development environments. The client I approached was > > specifically prepared to test beta releases that way. > > Perhaps end-

Re: [HACKERS] 8.5 release timetable, again

2009-08-28 Thread daveg
On Thu, Aug 27, 2009 at 09:38:15PM +0200, Dimitri Fontaine wrote: > Exactly, and I think that what we're missing here is a simple tool for > our users to check a new PostgreSQL release against their existing > application. > > We already know how to either log all queries and analyze the log files

Re: [HACKERS] GRANT ON ALL IN schema

2009-08-16 Thread daveg
On Sun, Aug 16, 2009 at 02:59:53PM +0200, Pavel Stehule wrote: > 2009/8/16 Peter Eisentraut : > > On sön, 2009-08-16 at 00:04 -0400, Andrew Dunstan wrote: > >> SQL is not Lisp. Simple is  good. I didn't think Peter was really very > >> serious. > > > > Well, I don't know if we really need to call i

Re: [HACKERS] GRANT ON ALL IN schema

2009-08-16 Thread daveg
On Sat, Aug 15, 2009 at 11:34:04PM +0200, Dimitri Fontaine wrote: > Nitpicking dept, I think I prefer: > > DO [ [LANGUAGE] language] $$ ... $$; > DO plperl $$ ... $$; > DO language plpython $$ ... $$; > > language is optional and defaults to plpgsql. +1 -dg -- David Gould da...@soni

Re: [HACKERS] Alpha releases: How to tag

2009-08-08 Thread daveg
On Fri, Aug 07, 2009 at 06:28:34PM -0400, Tom Lane wrote: > David Fetter writes: > > I am not suggesting that this change be immediate, and it's not ivory > > tower. It's just how everybody else does it. > > You keep saying that, and it's completely meaningless. What do you know > about the dev

Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic

2009-08-03 Thread daveg
ging overhead. Now the patch could be slower than unpatched > as a result of different job-scheduling behavior ... but there's no > evidence here of a consistently measurable benefit or loss from that. > > IIRC daveg was volunteering to do some tests with his own data; maybe > we shou

Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic

2009-07-31 Thread daveg
On Thu, Jul 30, 2009 at 12:29:34PM -0500, Kevin Grittner wrote: > Tom Lane wrote: > > > I think we've pretty much established that it doesn't make things > > *worse*, so I'm sort of inclined to go ahead and apply it. The > > theoretical advantage of eliminating O(N^2) search behavior seems > >

Re: [HACKERS] display previous query string of idle-in-transaction

2009-07-23 Thread daveg
On Thu, Jun 04, 2009 at 10:22:41PM -0400, Robert Haas wrote: > The only thing I don't like about this is that I think it's kind of a > hack to shove the in transaction designation and the query > string into the same database column. I've never liked having to > write: > > select sum(1) from pg_

Re: [HACKERS] Is a plan for lmza commpression in pg_dump

2009-02-07 Thread daveg
On Sat, Feb 07, 2009 at 08:49:29PM -0500, Robert Haas wrote: > "Proprietary compression algorithms, even with Postgresql-specific > license exceptions"? To be fair, lzo is GPL, which is a stretch to consider proprietary. -dg -- David Gould da...@sonic.net 510 536 1443510 282

Re: [HACKERS] Is a plan for lmza commpression in pg_dump

2009-02-07 Thread daveg
On Sat, Feb 07, 2009 at 02:47:05PM -0500, Bruce Momjian wrote: > daveg wrote: > > On Wed, Feb 04, 2009 at 10:23:17PM -0500, Andrew Chernow wrote: > > > Dann Corbit wrote: > > > > > > > >The LZMA SDK is granted to the public domain: > > > >http:

Re: [HACKERS] Is a plan for lmza commpression in pg_dump

2009-02-07 Thread daveg
On Wed, Feb 04, 2009 at 10:23:17PM -0500, Andrew Chernow wrote: > Dann Corbit wrote: > > > >The LZMA SDK is granted to the public domain: > >http://www.7-zip.org/sdk.html > > > > I played with this but found the SDK extremely confusing and flat out > horrible. One personal dislike was the unneces

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread daveg
On Tue, Sep 09, 2008 at 11:03:56PM +0300, Hannu Krosing wrote: > On Tue, 2008-09-09 at 12:40 -0700, daveg wrote: > > > I'd be very interested in seeing a last schema modification time for > > pg_class > > objects. I don't care about it being preserved over dump

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread daveg
On Tue, Sep 09, 2008 at 03:36:19PM -0400, Tom Lane wrote: > Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= <[EMAIL PROTECTED]> writes: > > AFAICS, PostgreSQL is not keeping info about when a table, database, > > sequence, etc was created. We cannot get that info even from OS, since > > CLUSTER or VACUUM FULL m

Re: [HACKERS] IN vs EXISTS equivalence

2008-09-02 Thread daveg
On Thu, Aug 14, 2008 at 06:50:09PM +0100, Simon Riggs wrote: > > On Fri, 2008-08-08 at 16:23 -0400, Tom Lane wrote: > > > NOT IN is a lot trickier, > > condition: you must also assume that the comparison operator involved > > never yields NULL for non-null inputs. That might be okay for btree >

Re: [HACKERS] proposal sql: labeled function params

2008-08-23 Thread daveg
On Sat, Aug 23, 2008 at 05:08:25PM +0100, Gregory Stark wrote: > "Pavel Stehule" <[EMAIL PROTECTED]> writes: > > > Hello > > > > 2008/8/23 Peter Eisentraut <[EMAIL PROTECTED]>: > >> On Friday 22 August 2008 07:41:30 Decibel! wrote: > >>> If we're really worried about it we can have a GUC for a few

Re: [HACKERS] A smaller default postgresql.conf

2008-08-19 Thread daveg
On Tue, Aug 19, 2008 at 09:39:39PM +0300, Peter Eisentraut wrote: > On Tuesday 19 August 2008 19:12:16 Tom Lane wrote: > > Well, why not just make a one-eighty and say that the default > > postgresql.conf is *empty* (except for whatever initdb puts into it)? > > Well, my original implementation of

Re: [HACKERS] Adjusting debug_print_plan to be more useful by default

2008-08-19 Thread daveg
On Tue, Aug 19, 2008 at 06:33:33PM +0100, Simon Riggs wrote: > > On Tue, 2008-08-19 at 12:40 -0400, Tom Lane wrote: > > Back in April we changed EXPLAIN VERBOSE to not dump the internal plan > > tree anymore, on the grounds that non-hackers didn't want that info and > > hackers could get it with d

Re: [HACKERS] Visibility Groups

2008-08-08 Thread daveg
On Thu, Aug 07, 2008 at 01:30:27PM +0100, Gregory Stark wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > > Currently, we calculate a single OldestXmin across all snapshots on the > > assumption that any transaction might access any table. > > > > I propose creating "Visibility Groups" that *

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 05:19:50PM -0400, Robert Treat wrote: > See, this is what we ended up talking about before. Someone will say "I'd > like > to prevent my devs from accidentally doing queries with cartesian products" > and they will use this to do it... but that will only work in some case

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 05:17:59PM -0400, Jonah H. Harris wrote: > On Mon, Aug 4, 2008 at 5:08 PM, Simon Riggs <[EMAIL PROTECTED]> wrote: > > When primary server fails, it would be good if the clients connected to > > the primary knew to reconnect to the standby servers automatically. > > This wou

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 11:59:03AM -0700, Josh Berkus wrote: > Greg, > > >Well that's going to depend on the application But I suppose there's > >nothing wrong with having options which aren't always a good idea to use. > >The > >real question I guess is whether there's ever a situation where

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 02:35:07PM -0400, Robert Treat wrote: > On Monday 04 August 2008 03:50:40 daveg wrote: > > That's great for you, I am talking in the scope of a general solution. (Note > I'd also bet that even given the same hardware, different production loads &

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 03:09:34PM -0400, Greg Smith wrote: > On Mon, 4 Aug 2008, daveg wrote: > >We load the production dumps into our dev environment, which are the same > >hardware spec, so the costs should be identical. > > Not identical, just close. ANALYZE samples

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Sun, Aug 03, 2008 at 10:57:55PM -0400, Robert Treat wrote: > > ISTR that what ended up killing the enthusiasm for this was that most people > realized that this GUC was just a poor tool to take a stab at solving other > problems (ie. rate limiting cpu for queries). I'm not concerned with th

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-03 Thread daveg
On Sat, Aug 02, 2008 at 09:30:08PM +0200, Hans-Jürgen Schönig wrote: > On Aug 2, 2008, at 8:38 PM, Tom Lane wrote: > > >Andrew Dunstan <[EMAIL PROTECTED]> writes: > >>Hans-Jürgen Schönig wrote: > >>>i introduced a GUC called statement_cost_limit which can be used to > >>>error out if a statement i

Re: [HACKERS] Copy storage parameters on CREATE TABLE LIKE/INHERITS

2008-07-30 Thread daveg
On Wed, Jul 30, 2008 at 04:45:47PM +0900, ITAGAKI Takahiro wrote: > Here is a patch to copy column storage parameters and reloptions on > CREATE TABLE LIKE, which I proposed at: > [HACKERS] Uncopied parameters on CREATE TABLE LIKE > http://archives.postgresql.org/message-id/[EMAIL PROTECTED] >

Re: [HACKERS] [PATCHES] pg_dump additional options for performance

2008-07-27 Thread daveg
On Sun, Jul 27, 2008 at 10:37:34AM +0100, Simon Riggs wrote: > > On Sat, 2008-07-26 at 11:03 -0700, Joshua D. Drake wrote: > > > 2. We have no concurrency which means, anyone with any database over 50G > > has unacceptable restore times. > > Agreed. > > Also the core reason for wanting -w > >

Re: [HACKERS] Adding WHERE clause to pg_dump

2008-07-25 Thread daveg
On Fri, Jul 25, 2008 at 11:17:20PM +0100, Gregory Stark wrote: > "Joshua D. Drake" <[EMAIL PROTECTED]> writes: > > > How do we deal with this? > > > > pg_dump -w "last_update_timestamp < ..." -t 'table*' > > > > What I see is a recipe for inconsistent, un-restorable backups without a > > user real

  1   2   >