Re: [HACKERS] Getting to beta1

2010-03-18 Thread Simon Riggs
On Wed, 2010-03-17 at 23:29 -0400, Bruce Momjian wrote: Simon Riggs wrote: On Sat, 2010-03-13 at 11:26 -0800, Josh Berkus wrote: The list has been reduced greatly in the past week. What about HS/SR open items? I'd like to see vacuum_defer_cleanup_age added to the Archive section

Re: [HACKERS] PQftype implementation

2010-03-18 Thread Pavel Golub
Hello, Tom. Yes, you are absolutely right. My bad! Sorry guys! :) You wrote: TL Pavel Golub pa...@microolap.com writes: Here I created user-defined type my_varchar for internal tests. But PQftype returns 1043 (varchar oid) for the info column. TL Really? I tried it and got 172069, which is

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
On Thu, Mar 18, 2010 at 2:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Davis pg...@j-davis.com writes: There are all kinds of challenges there, but it might be worth thinking about. Visibility information is highly compressible, and requires constant maintenance (updates, deletes,

Re: [HACKERS] Command to prune archive at restartpoints

2010-03-18 Thread Heikki Linnakangas
Committed. Heikki Linnakangas wrote: One awkward omission in the new built-in standby mode, mainly used for streaming replication, is that there is no easy way to delete old archived files like you do with the %r parameter to restore_command. This was discussed at

Re: [HACKERS] Command to prune archive at restartpoints

2010-03-18 Thread Simon Riggs
On Wed, 2010-03-17 at 11:37 +0200, Heikki Linnakangas wrote: One awkward omission in the new built-in standby mode, mainly used for streaming replication, is that there is no easy way to delete old archived files like you do with the %r parameter to restore_command. This was discussed at

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
Secondly there's the whole retail vacuum problem -- any index entries referring to this page would be left dangling unless there's some kind of retail vacuum or perhaps a page version number. The issue, we can divide into two a)volatile functions b)broken datatypes For a) I think volatile

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
I didn't mean that we'd want to compress it to the absolute minimum size. I had envisioned that it would be a simple scheme designed only to eliminate long runs of identical visibility information (perhaps only the frozen and always visible regions would be compressed). The extra level of

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Heikki Linnakangas
Pavel Stehule wrote: attached patch add possibility to share ispell dictionary between processes. The reason for this is the slowness of first tsearch query and size of allocated memory per process. When I tested loading of ispell dictionary (for Czech language) I got about 500 ms and 48MB.

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Pavel Stehule
2010/3/18 Heikki Linnakangas heikki.linnakan...@enterprisedb.com: Pavel Stehule wrote: attached patch add possibility to share ispell dictionary between processes. The reason for this is the slowness of first tsearch query and size of allocated memory per process. When I tested loading of

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make standby server continuously retry restoring the next WAL

2010-03-18 Thread Fujii Masao
On Wed, Mar 17, 2010 at 7:35 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Fujii Masao wrote: I found another missing feature in new file-based log shipping (i.e., standby_mode is enabled and 'cp' is used as restore_command). After the trigger file is found, the startup

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: I know so Tom worries about using of share memory. You're right, and if I have any say in the matter no patch like this will ever go in. What I would suggest looking into is some way of preprocessing the raw text dictionary file into a format that

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Pavel Stehule
2010/3/18 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: I know so Tom worries about using of share memory. You're right, and if I have any say in the matter no patch like this will ever go in. What I would suggest looking into is some way of preprocessing the

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Pavel Stehule
2010/3/18 Pavel Stehule pavel.steh...@gmail.com: 2010/3/18 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: I know so Tom worries about using of share memory. You're right, and if I have any say in the matter no patch like this will ever go in. What I would

Re: [HACKERS] ALTER ROLE/DATABASE RESET ALL versus security

2010-03-18 Thread Alvaro Herrera
Bruce Momjian wrote: Alvaro Herrera wrote: Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Tom Lane wrote: It looks to me like the code in AlterSetting() will allow an ordinary user to blow away all settings for himself. Even those that are for SUSET

Re: [HACKERS] An idle thought

2010-03-18 Thread Chris Browne
si...@2ndquadrant.com (Simon Riggs) writes: On Tue, 2010-03-16 at 15:29 +, Greg Stark wrote: big batch delete Is one of the reasons for partitioning, allowing the use of truncate. Sure, but it would be even nicer if DELETE could be thus made cheaper without needing to interfere with the

Re: [HACKERS] ALTER ROLE/DATABASE RESET ALL versus security

2010-03-18 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: I have come up with the attached patch. I haven't tested it fully yet, and I need to backport it. The gist of it is: we can't simply remove the pg_db_role_setting tuple, we need to ask GUC to reset the settings array, for which it checks

Re: [HACKERS] WIP: shared ispell dictionary

2010-03-18 Thread Pavel Stehule
2010/3/18 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: I know so Tom worries about using of share memory. You're right, and if I have any say in the matter no patch like this will ever go in. I wrote second patch based on preloading. For real using it needs to

Re: [HACKERS] C libpq frontend library fetchsize

2010-03-18 Thread Yeb Havinga
Robert Haas wrote: On Fri, Feb 26, 2010 at 3:28 PM, Yeb Havinga yebhavi...@gmail.com wrote: I'm wondering if there would be community support for adding using the execute message with a rownum 0 in the c libpq client library, as it is used by the jdbc driver with setFetchSize. Not

Re: [HACKERS] C libpq frontend library fetchsize

2010-03-18 Thread Tom Lane
Yeb Havinga yebhavi...@gmail.com writes: What if the default operation of e.g. php using libpq would be as follows: set some default fetchsize (e.g. 1000 rows), then just issue getrow. In the php pg handling, a function like getnextrow would wait for the first pgresult with 1000 rows. Then

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Josh Berkus
It's already in the docs, so if they read it and understand it they can add it to the postgresql.conf if they so choose. I agree with Josh Berkus that vacuum_defer_cleanup_age should be in postgresql.conf. We don't stop listing items just because they are dangerous, e.g. fsync, or to

Re: [HACKERS] C libpq frontend library fetchsize

2010-03-18 Thread Yeb Havinga
Tom Lane wrote: Yeb Havinga yebhavi...@gmail.com writes: What if the default operation of e.g. php using libpq would be as follows: set some default fetchsize (e.g. 1000 rows), then just issue getrow. In the php pg handling, a function like getnextrow would wait for the first pgresult with

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Joshua D. Drake
On Thu, 2010-03-18 at 10:18 -0700, Josh Berkus wrote: It's already in the docs, so if they read it and understand it they can add it to the postgresql.conf if they so choose. I agree with Josh Berkus that vacuum_defer_cleanup_age should be in postgresql.conf. We don't stop listing

Re: [HACKERS] An idle thought

2010-03-18 Thread Jeff Davis
On Thu, 2010-03-18 at 14:29 +0530, Gokulakannan Somasundaram wrote: If you want it to be cheaply updatable (or even cheaply readable), compression is not what you're going to do. regards, tom lane

Re: [HACKERS] Ragged latency log data in multi-threaded pgbench

2010-03-18 Thread Greg Smith
Takahiro Itagaki wrote: The log filenames are pgbench_log.main-process-id.thread-serial-number for each thread, but the first thread (including single-threaded) still uses pgbench_log.main-process-id for the name because of compatibility. Attached is an updated version that I think is ready

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Greg Smith
Joshua D. Drake wrote: As usual, the postgresql.conf is entirely too full. We should ship with the top 15. Maybe, but what we should do is ship, and then talk about this again when it's appropriate--earlier in the release cycle. Let me try and cut this one off before it generates a bunch of

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Marc G. Fournier
On Thu, 18 Mar 2010, Joshua D. Drake wrote: On Thu, 2010-03-18 at 10:18 -0700, Josh Berkus wrote: It's already in the docs, so if they read it and understand it they can add it to the postgresql.conf if they so choose. I agree with Josh Berkus that vacuum_defer_cleanup_age should be in

Re: [HACKERS] An idle thought

2010-03-18 Thread Gokulakannan Somasundaram
The visibility map itself is already an example of compression. If visibility information were randomly distributed among tuples, the visibility map would be nearly useless. I believe it is very difficult to make visibility map update friendly without compromising durability. But such a

Re: [HACKERS] An idle thought

2010-03-18 Thread Jeff Davis
On Fri, 2010-03-19 at 01:59 +0530, Gokulakannan Somasundaram wrote: The visibility map itself is already an example of compression. If visibility information were randomly distributed among tuples, the visibility map would be nearly useless.

Re: [HACKERS] An idle thought

2010-03-18 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Fri, 2010-03-19 at 01:59 +0530, Gokulakannan Somasundaram wrote: I believe it is very difficult to make visibility map update friendly without compromising durability. But such a functionality is very much wanted in PG still. Surely the VM is already

Re: [HACKERS] An idle thought

2010-03-18 Thread Jeff Davis
On Thu, 2010-03-18 at 16:50 -0400, Tom Lane wrote: The VM is (a) not compressed and (b) not correctness-critical. Wrong bit values don't do any serious damage. The VM cause wrong results if a bit is set that's not supposed to be -- right? Am I missing something? How does a seq scan skip

Re: [HACKERS] An idle thought

2010-03-18 Thread Alex Hunsaker
On Thu, Mar 18, 2010 at 15:07, Jeff Davis pg...@j-davis.com wrote: On Thu, 2010-03-18 at 16:50 -0400, Tom Lane wrote: The VM is (a) not compressed and (b) not correctness-critical. Wrong bit values don't do any serious damage. The VM cause wrong results if a bit is set that's not supposed to

Re: [HACKERS] An idle thought

2010-03-18 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Thu, 2010-03-18 at 16:50 -0400, Tom Lane wrote: The VM is (a) not compressed and (b) not correctness-critical. Wrong bit values don't do any serious damage. The VM cause wrong results if a bit is set that's not supposed to be -- right? Am I missing

Re: [HACKERS] An idle thought

2010-03-18 Thread Greg Stark
On Thu, Mar 18, 2010 at 9:07 PM, Jeff Davis pg...@j-davis.com wrote: The VM cause wrong results if a bit is set that's not supposed to be -- right? Am I missing something? How does a seq scan skip visibility checks and still produce right results, if it doesn't rely on the bit? There's also a

Re: [HACKERS] An idle thought

2010-03-18 Thread Jeff Davis
On Thu, 2010-03-18 at 17:17 -0400, Tom Lane wrote: The VM cause wrong results if a bit is set that's not supposed to be -- right? Am I missing something? How does a seq scan skip visibility checks and still produce right results, if it doesn't rely on the bit? It doesn't. The only thing

Re: [HACKERS] An idle thought

2010-03-18 Thread Jeff Davis
On Thu, 2010-03-18 at 14:48 -0700, Jeff Davis wrote: On Thu, 2010-03-18 at 17:17 -0400, Tom Lane wrote: The VM cause wrong results if a bit is set that's not supposed to be -- right? Am I missing something? How does a seq scan skip visibility checks and still produce right results, if it

Re: [HACKERS] An idle thought

2010-03-18 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: I tested in gdb, and it calls HeapTupleSatisfiesMVCC, until I VACUUM a few times, and then it doesn't call it any more. So, apparently the seq scan optimization _is_ there. And that means it is correctness-critical. The page header bit is critical. Not the

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Robert Haas
On Thu, Mar 18, 2010 at 4:28 PM, Marc G. Fournier scra...@hub.org wrote: On Thu, 18 Mar 2010, Joshua D. Drake wrote: On Thu, 2010-03-18 at 10:18 -0700, Josh Berkus wrote: It's already in the docs, so if they read it and understand it they can add it to the postgresql.conf if they so choose.

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Joshua D. Drake
On Thu, 2010-03-18 at 19:10 -0400, Robert Haas wrote: On Thu, Mar 18, 2010 at 4:28 PM, Marc G. Fournier scra...@hub.org wrote: On Thu, 18 Mar 2010, Joshua D. Drake wrote: On Thu, 2010-03-18 at 10:18 -0700, Josh Berkus wrote: It's already in the docs, so if they read it and understand it

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Greg Smith
Robert Haas wrote: On Thu, Mar 18, 2010 at 4:28 PM, Marc G. Fournier scra...@hub.org wrote: On Thu, 18 Mar 2010, Joshua D. Drake wrote: On Thu, 2010-03-18 at 10:18 -0700, Josh Berkus wrote: Or, let's put it another way: I've made my opinion clear in the past that I think that

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Josh Berkus
On 3/18/10 4:54 PM, Greg Smith wrote: If this is turning into a vote: -1 from me for any work on this until http://wiki.postgresql.org/wiki/PostgreSQL_9.0_Open_Items is cleared. It boggles my mind that anyone could have a different prioritization right now. Yes. I wasn't suggesting that

Re: [HACKERS] Getting to beta1

2010-03-18 Thread Robert Haas
On Thu, Mar 18, 2010 at 7:54 PM, Greg Smith g...@2ndquadrant.com wrote: If this is turning into a vote:  -1 from me for any work on this until http://wiki.postgresql.org/wiki/PostgreSQL_9.0_Open_Items is cleared.  It boggles my mind that anyone could have a different prioritization right now.