Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
> Surely the VM is already update-friendly. If you update a tuple in a > page with the visibility bit set, the bit must be unset or you will get > wrong results. > > > I was referring in the context of index only scans to skip visibility checks. I doubt, whether the visibility map feature to skip v

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Robert Haas
On Thu, Mar 18, 2010 at 7:54 PM, Greg Smith wrote: > If this is turning into a vote:  -1 from me for any work on this until > http://wiki.postgresql.org/wiki/PostgreSQL_9.0_Open_Items is cleared.  It > boggles my mind that anyone could have a different prioritization right now. This isn't about p

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Josh Berkus
On 3/18/10 4:54 PM, Greg Smith wrote: > > If this is turning into a vote: -1 from me for any work on this until > http://wiki.postgresql.org/wiki/PostgreSQL_9.0_Open_Items is cleared. > It boggles my mind that anyone could have a different prioritization > right now. Yes. I wasn't suggesting t

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Greg Smith
Robert Haas wrote: On Thu, Mar 18, 2010 at 4:28 PM, Marc G. Fournier wrote: On Thu, 18 Mar 2010, Joshua D. Drake wrote: On Thu, 2010-03-18 at 10:18 -0700, Josh Berkus wrote: Or, let's put it another way: I've made my opinion clear in the past that I think that we ought to shi

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Joshua D. Drake
On Thu, 2010-03-18 at 19:10 -0400, Robert Haas wrote: > On Thu, Mar 18, 2010 at 4:28 PM, Marc G. Fournier wrote: > > On Thu, 18 Mar 2010, Joshua D. Drake wrote: > > > >> On Thu, 2010-03-18 at 10:18 -0700, Josh Berkus wrote: > > > > It's already in the docs, so if they read it and understan

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Robert Haas
On Thu, Mar 18, 2010 at 4:28 PM, Marc G. Fournier wrote: > On Thu, 18 Mar 2010, Joshua D. Drake wrote: > >> On Thu, 2010-03-18 at 10:18 -0700, Josh Berkus wrote: > > It's already in the docs, so if they read it and understand it they can > add it to the postgresql.conf if they so choos

Re: [HACKERS] An idle thought

2010-03-18 Thread Tom Lane
Jeff Davis writes: > I tested in gdb, and it calls HeapTupleSatisfiesMVCC, until I VACUUM a > few times, and then it doesn't call it any more. So, apparently the seq > scan optimization _is_ there. And that means it is correctness-critical. The page header bit is critical. Not the VM.

Re: [HACKERS] An idle thought

2010-03-18 Thread Jeff Davis
On Thu, 2010-03-18 at 14:48 -0700, Jeff Davis wrote: > On Thu, 2010-03-18 at 17:17 -0400, Tom Lane wrote: > > > The VM cause wrong results if a bit is set that's not supposed to be -- > > > right? Am I missing something? How does a seq scan skip visibility > > > checks and still produce right resul

Re: [HACKERS] An idle thought

2010-03-18 Thread Jeff Davis
On Thu, 2010-03-18 at 17:17 -0400, Tom Lane wrote: > > The VM cause wrong results if a bit is set that's not supposed to be -- > > right? Am I missing something? How does a seq scan skip visibility > > checks and still produce right results, if it doesn't rely on the bit? > > It doesn't. The only

Re: [HACKERS] An idle thought

2010-03-18 Thread Greg Stark
On Thu, Mar 18, 2010 at 9:07 PM, Jeff Davis wrote: > The VM cause wrong results if a bit is set that's not supposed to be -- > right? Am I missing something? How does a seq scan skip visibility > checks and still produce right results, if it doesn't rely on the bit? > There's also a PD_ALL_VISIBL

Re: [HACKERS] An idle thought

2010-03-18 Thread Tom Lane
Jeff Davis writes: > On Thu, 2010-03-18 at 16:50 -0400, Tom Lane wrote: >> The VM is (a) not compressed and (b) not correctness-critical. >> Wrong bit values don't do any serious damage. > The VM cause wrong results if a bit is set that's not supposed to be -- > right? Am I missing something? How

Re: [HACKERS] An idle thought

2010-03-18 Thread Alex Hunsaker
On Thu, Mar 18, 2010 at 15:07, Jeff Davis wrote: > On Thu, 2010-03-18 at 16:50 -0400, Tom Lane wrote: >> The VM is (a) not compressed and (b) not correctness-critical. >> Wrong bit values don't do any serious damage. > > The VM cause wrong results if a bit is set that's not supposed to be -- > rig

Re: [HACKERS] An idle thought

2010-03-18 Thread Jeff Davis
On Thu, 2010-03-18 at 16:50 -0400, Tom Lane wrote: > The VM is (a) not compressed and (b) not correctness-critical. > Wrong bit values don't do any serious damage. The VM cause wrong results if a bit is set that's not supposed to be -- right? Am I missing something? How does a seq scan skip visibi

Re: [HACKERS] An idle thought

2010-03-18 Thread Tom Lane
Jeff Davis writes: > On Fri, 2010-03-19 at 01:59 +0530, Gokulakannan Somasundaram wrote: >> I believe it is very difficult to make visibility map update friendly >> without compromising durability. But such a functionality is very >> much wanted in PG still. > Surely the VM is already update-fri

Re: [HACKERS] An idle thought

2010-03-18 Thread Jeff Davis
On Fri, 2010-03-19 at 01:59 +0530, Gokulakannan Somasundaram wrote: > > The visibility map itself is already an example of > compression. If > visibility information were randomly distributed among tuples, > the > visibility map would be nearly useless. >

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
> The visibility map itself is already an example of compression. If > visibility information were randomly distributed among tuples, the > visibility map would be nearly useless. > > > I believe it is very difficult to make visibility map update friendly without compromising durability. But such

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Marc G. Fournier
On Thu, 18 Mar 2010, Joshua D. Drake wrote: On Thu, 2010-03-18 at 10:18 -0700, Josh Berkus wrote: It's already in the docs, so if they read it and understand it they can add it to the postgresql.conf if they so choose. I agree with Josh Berkus that vacuum_defer_cleanup_age should be in postgr

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Greg Smith
Joshua D. Drake wrote: As usual, the postgresql.conf is entirely too full. We should ship with the top 15. Maybe, but what we should do is ship, and then talk about this again when it's appropriate--earlier in the release cycle. Let me try and cut this one off before it generates a bunch of

Re: [HACKERS] Ragged latency log data in multi-threaded pgbench

2010-03-18 Thread Greg Smith
Takahiro Itagaki wrote: The log filenames are "pgbench_log.." for each thread, but the first thread (including single-threaded) still uses "pgbench_log." for the name because of compatibility. Attached is an updated version that I think is ready to commit. Only changes are docs--I rewrote

Re: [HACKERS] An idle thought

2010-03-18 Thread Jeff Davis
On Thu, 2010-03-18 at 14:29 +0530, Gokulakannan Somasundaram wrote: > If you want it to be cheaply updatable (or even cheaply > readable), > compression is not what you're going to do. > >regards, tom lane > > >

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Joshua D. Drake
On Thu, 2010-03-18 at 10:18 -0700, Josh Berkus wrote: > >> It's already in the docs, so if they read it and understand it they can > >> add it to the postgresql.conf if they so choose. > > > > I agree with Josh Berkus that vacuum_defer_cleanup_age should be in > > postgresql.conf. We don't stop l

Re: [HACKERS] C libpq frontend library fetchsize

2010-03-18 Thread Yeb Havinga
Tom Lane wrote: Yeb Havinga writes: What if the default operation of e.g. php using libpq would be as follows: set some default fetchsize (e.g. 1000 rows), then just issue getrow. In the php pg handling, a function like getnextrow would wait for the first pgresult with 1000 rows. Then if t

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Josh Berkus
>> It's already in the docs, so if they read it and understand it they can >> add it to the postgresql.conf if they so choose. > > I agree with Josh Berkus that vacuum_defer_cleanup_age should be in > postgresql.conf. We don't stop listing items just because they are > dangerous, e.g. fsync, or

Re: [HACKERS] C libpq frontend library fetchsize

2010-03-18 Thread Tom Lane
Yeb Havinga writes: > What if the default operation of e.g. php using libpq would be as > follows: set some default fetchsize (e.g. 1000 rows), then just issue > getrow. In the php pg handling, a function like getnextrow would wait > for the first pgresult with 1000 rows. Then if the pgresult i

Re: [HACKERS] C libpq frontend library fetchsize

2010-03-18 Thread Yeb Havinga
Robert Haas wrote: On Fri, Feb 26, 2010 at 3:28 PM, Yeb Havinga wrote: I'm wondering if there would be community support for adding using the execute message with a rownum > 0 in the c libpq client library, as it is used by the jdbc driver with setFetchSize. Not sure I follow what you

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Pavel Stehule
2010/3/18 Tom Lane : > Pavel Stehule writes: >> I know so Tom worries about using of share memory. > > You're right, and if I have any say in the matter no patch like this > will ever go in. I wrote second patch based on preloading. For real using it needs to design parametrisation. It working we

Re: [HACKERS] ALTER ROLE/DATABASE RESET ALL versus security

2010-03-18 Thread Tom Lane
Alvaro Herrera writes: > I have come up with the attached patch. I haven't tested it fully yet, > and I need to backport it. The gist of it is: we can't simply remove > the pg_db_role_setting tuple, we need to ask GUC to reset the settings > array, for which it checks superuser-ness on each sett

Re: [HACKERS] An idle thought

2010-03-18 Thread Chris Browne
si...@2ndquadrant.com (Simon Riggs) writes: > On Tue, 2010-03-16 at 15:29 +, Greg Stark wrote: > >> big batch delete > > Is one of the reasons for partitioning, allowing the use of truncate. Sure, but it would be even nicer if DELETE could be thus made cheaper without needing to interfere with

Re: [HACKERS] ALTER ROLE/DATABASE RESET ALL versus security

2010-03-18 Thread Alvaro Herrera
Bruce Momjian wrote: > Alvaro Herrera wrote: > > Tom Lane wrote: > > > Alvaro Herrera writes: > > > > Tom Lane wrote: > > > >> It looks to me like the code in AlterSetting() will allow an ordinary > > > >> user to blow away all settings for himself. Even those that are for > > > >> SUSET variable

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Pavel Stehule
2010/3/18 Pavel Stehule : > 2010/3/18 Tom Lane : >> Pavel Stehule writes: >>> I know so Tom worries about using of share memory. >> >> You're right, and if I have any say in the matter no patch like this >> will ever go in. >> >> What I would suggest looking into is some way of preprocessing the r

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Pavel Stehule
2010/3/18 Tom Lane : > Pavel Stehule writes: >> I know so Tom worries about using of share memory. > > You're right, and if I have any say in the matter no patch like this > will ever go in. > > What I would suggest looking into is some way of preprocessing the raw > text dictionary file into a fo

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Tom Lane
Pavel Stehule writes: > I know so Tom worries about using of share memory. You're right, and if I have any say in the matter no patch like this will ever go in. What I would suggest looking into is some way of preprocessing the raw text dictionary file into a format that can be slurped into memo

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make standby server continuously retry restoring the next WAL

2010-03-18 Thread Fujii Masao
On Wed, Mar 17, 2010 at 7:35 PM, Heikki Linnakangas wrote: > Fujii Masao wrote: >> I found another missing feature in new file-based log shipping (i.e., >> standby_mode is enabled and 'cp' is used as restore_command). >> >> After the trigger file is found, the startup process with pg_standby >> tr

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Pavel Stehule
2010/3/18 Heikki Linnakangas : > Pavel Stehule wrote: >> attached patch add possibility to share ispell dictionary between >> processes. The reason for this is the slowness of first tsearch query >> and size of allocated memory per process. When I tested loading of >> ispell dictionary (for Czech l

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Heikki Linnakangas
Pavel Stehule wrote: > attached patch add possibility to share ispell dictionary between > processes. The reason for this is the slowness of first tsearch query > and size of allocated memory per process. When I tested loading of > ispell dictionary (for Czech language) I got about 500 ms and 48MB.

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
> > > I didn't mean that we'd want to compress it to the absolute minimum > size. I had envisioned that it would be a simple scheme designed only to > eliminate long runs of identical visibility information (perhaps only > the frozen and always visible regions would be compressed). > > The extra le

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
> > Secondly there's the whole retail vacuum problem -- any > index entries referring to this page would be left dangling unless > there's some kind of retail vacuum or perhaps a page version number. > > The issue, we can divide into two a)volatile functions b)broken datatypes For a) I think vola

Re: [HACKERS] Command to prune archive at restartpoints

2010-03-18 Thread Simon Riggs
On Wed, 2010-03-17 at 11:37 +0200, Heikki Linnakangas wrote: > One awkward omission in the new built-in standby mode, mainly used for > streaming replication, is that there is no easy way to delete old > archived files like you do with the %r parameter to restore_command. > This was discussed at >

Re: [HACKERS] Command to prune archive at restartpoints

2010-03-18 Thread Heikki Linnakangas
Committed. Heikki Linnakangas wrote: > One awkward omission in the new built-in standby mode, mainly used for > streaming replication, is that there is no easy way to delete old > archived files like you do with the %r parameter to restore_command. > This was discussed at > http://archives.postgre

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
On Thu, Mar 18, 2010 at 2:50 AM, Tom Lane wrote: > Jeff Davis writes: > > There are all kinds of challenges there, but it might be worth thinking > > about. Visibility information is highly compressible, and requires > > constant maintenance (updates, deletes, freezing, etc.). It also might > >

Re: [HACKERS] PQftype implementation

2010-03-18 Thread Pavel Golub
Hello, Tom. Yes, you are absolutely right. My bad! Sorry guys! :) You wrote: TL> Pavel Golub writes: >> Here I created user-defined type "my_varchar" for internal tests. But >> PQftype returns 1043 (varchar oid) for the "info" column. TL> Really? I tried it and got 172069, which is about rig

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Simon Riggs
On Wed, 2010-03-17 at 23:29 -0400, Bruce Momjian wrote: > Simon Riggs wrote: > > On Sat, 2010-03-13 at 11:26 -0800, Josh Berkus wrote: > > > > The list has been reduced greatly in the past week. What about HS/SR > > > > open items? > > > > > > I'd like to see vacuum_defer_cleanup_age added to the