Re: [HACKERS] Foreign table permissions and cloning

2011-04-15 Thread Thom Brown
On 15 April 2011 04:26, Tom Lane t...@sss.pgh.pa.us wrote: Why is this a documentation issue and not a code issue?  IMO we should flat out reject both NOT NULL and DEFAULT declarations on foreign tables, until such time as we're prepared to do something useful with them.  Reasons: If the

[HACKERS] MMAP Buffers

2011-04-15 Thread Radosław Smogura
Hello, If I may, I want to share some concept to use mmap in PG. It's far, far away from perfect, but it's keeps WAL before data. As well I crated table, with index, inserted few values, and I done vacuum full on this table. Db inits welcome from orginal sources. Performance of read (if

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Heikki Linnakangas
On 15.04.2011 13:32, Radosław Smogura wrote: If I may, I want to share some concept to use mmap in PG. It's far, far away from perfect, but it's keeps WAL before data. As well I crated table, with index, inserted few values, and I done vacuum full on this table. Db inits welcome from orginal

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Heikki Linnakangas
On 14.04.2011 23:02, Tom Lane wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes: There's one very low-hanging fruit here, though. I profiled the pgbench case, with -M prepared, and found that like in Greg Smith's profile, hash_seq_search pops up quite high in the list. Those

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Radosław Smogura
On Fri, 15 Apr 2011 14:33:37 +0300, Heikki Linnakangas wrote: On 15.04.2011 13:32, Radosław Smogura wrote: If I may, I want to share some concept to use mmap in PG. It's far, far away from perfect, but it's keeps WAL before data. As well I crated table, with index, inserted few values, and I

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Andrew Dunstan
On 04/15/2011 08:12 AM, Radosław Smogura wrote: The patch is quite hard to read because of random whitespace changes and other stylistic issues, but I have a couple of high-level questions on the design: Yes, but, hmm... in Netbeans I had really long gaps (probably 8 spaces, from tabs), so

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Noah Misch
On Thu, Apr 14, 2011 at 05:10:41PM -0400, Tom Lane wrote: Noah Misch n...@leadboat.com writes: On Thu, Apr 14, 2011 at 11:15:00AM -0700, Robert Haas wrote: It shouldn't be terribly difficult to come up with some kind of hash function based on, say, the first two characters of the keyword

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread David Fetter
On Thu, Apr 14, 2011 at 11:16:03PM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: On Thu, Apr 14, 2011 at 10:43:16AM -0400, Tom Lane wrote: ... I think a lot of this ultimately traces to the extensible, data-type-agnostic design philosophy. The fact that we don't know what

Re: [HACKERS] Windows build issues

2011-04-15 Thread Magnus Hagander
On Thu, Apr 14, 2011 at 00:14, Brar Piening b...@gmx.de wrote: On Wed, 13 Apr 2011 10:45:25 +0200, Magnus Hagander mag...@hagander.net wrote: So per your experience, all we really need to do is to define what the *max* level of the Windows SDK we can use is, to make sure people don't get the

Re: [HACKERS] Windows build issues

2011-04-15 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of vie abr 15 10:35:44 -0300 2011: On Thu, Apr 14, 2011 at 00:14, Brar Piening b...@gmx.de wrote: The problem with VS 2010 (and the associated Windows SDK's) is the fact that MS dropped VCBuild.exe and now uses MSBuild.exe to build Visual C++ Projects

Re: [HACKERS] Windows build issues

2011-04-15 Thread Magnus Hagander
On Fri, Apr 15, 2011 at 15:57, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Magnus Hagander's message of vie abr 15 10:35:44 -0300 2011: On Thu, Apr 14, 2011 at 00:14, Brar Piening b...@gmx.de wrote: The problem with VS 2010 (and the associated Windows SDK's) is the fact

Re: [HACKERS] WAL, xl_heap_insert and tuple oid mystry

2011-04-15 Thread Alvaro Herrera
Excerpts from jagan's message of jue abr 14 17:30:17 -0300 2011: In general, what is the best way to unpack buffers containing tuples? I came across heap_form_tuple but not sure if that is most optimal way to go about. Is there some documentation on how tuples are internally stored? Unpack

Re: [HACKERS] Foreign table permissions and cloning

2011-04-15 Thread Robert Haas
On Thu, Apr 14, 2011 at 8:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: 1. Accepting non-functional constraint declarations is something we've been heard to ridicule mysql for.  With good reason. 2. It probably won't be too long before the planner makes optimization decisions that assume NOT NULL

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Thu, Apr 14, 2011 at 05:10:41PM -0400, Tom Lane wrote: FWIW, mysql used to use gperf for this purpose, but they've abandoned it in favor of some homegrown hashing scheme. Do you have any details, like when mysql did this? With a quick look, I'm

Re: [HACKERS] [COMMITTERS] pgsql: Rename pg_regress option --multibyte to --encoding

2011-04-15 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: Rename pg_regress option --multibyte to --encoding Also refactor things a little bit so that the same methods for setting test locale and encoding can be used everywhere. The buildfarm shows that this patch fails on Windows --- the contrib/unaccent

Re: [HACKERS] pg_dump --binary-upgrade vs. ALTER TYPE ... DROP ATTRIBUTE

2011-04-15 Thread Noah Misch
Robert, Thanks for the review. On Wed, Apr 13, 2011 at 08:01:17PM -0700, Robert Haas wrote: I think you only need an AccessShareLock on InheritsRelationId, since you are only selecting from it. True; fixed. If we adopt the elsewhere-proposed approach of forbidding the use of rowtypes to

Re: [HACKERS] [COMMITTERS] pgsql: Rename pg_regress option --multibyte to --encoding

2011-04-15 Thread Andrew Dunstan
On 04/15/2011 11:18 AM, Tom Lane wrote: Peter Eisentrautpete...@gmx.net writes: Rename pg_regress option --multibyte to --encoding Also refactor things a little bit so that the same methods for setting test locale and encoding can be used everywhere. The buildfarm shows that this patch

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Joshua Berkus
Radoslaw, 10% improvement isn't very impressive from a switch to mmap. What workload did you test with? What I'd really like to see is testing with databases which are 50%, 90% and 200% the size of RAM ... that's where I'd expect the greatest gain from limiting copying. Netbeans is

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Joshua Berkus
All, While it would be nice to improve our performance on this workload, let me point out that it's not a very important workload from the point of view of real performance challenges. Yes, there are folks out there with 100MB databases who only run one-liner select queries. But frankly,

Re: [HACKERS] WAL, xl_heap_insert and tuple oid mystry

2011-04-15 Thread jagan
Hi, In general, what is the best way to unpack buffers containing tuples? Unpack buffers containing tuples?  You don't do that.  If you have a page that comes from a table, you grab the tuple offset from the line pointer, and start decoding there (probably heap_deform_tuple is what you

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Joshua D. Drake
On 04/15/2011 09:59 AM, Joshua Berkus wrote: All, While it would be nice to improve our performance on this workload, let me point out that it's not a very important workload from the point of view of real performance challenges. Yes, there are folks out there with 100MB databases who only

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Radosław Smogura
Joshua Berkus j...@agliodbs.com Friday 15 April 2011 18:55:04 Radoslaw, 10% improvement isn't very impressive from a switch to mmap. What workload did you test with? What I'd really like to see is testing with databases which are 50%, 90% and 200% the size of RAM ... that's where I'd

[HACKERS] Broken HOT chains in system catalogs

2011-04-15 Thread Tom Lane
I've been able to reproduce the syndrome reported here and here: http://archives.postgresql.org/pgsql-performance/2011-02/msg00497.php http://archives.postgresql.org/pgsql-general/2011-04/msg00487.php It's a bit tricky to do it reliably, but once you get into the right state it's reproducible.

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Tom Lane
Joshua Berkus j...@agliodbs.com writes: Certainly any sacrifice of functionality in order to be faster at that kind of trivial workload would be foolhardy. Yeah, and I'd further say that any sacrifice of maintainability would be equally foolhardy. In particular I'm repelled by David's

Re: [HACKERS] Broken HOT chains in system catalogs

2011-04-15 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: This amounts to assuming that no new indexes get added to system catalogs after initdb, or at least not during concurrent operations wherein indcheckxmin would be important. Sounds reasonable, but can we enforce it through locks rather than assuming, or

Re: [HACKERS] Broken HOT chains in system catalogs

2011-04-15 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Tom Lane t...@sss.pgh.pa.us wrote: This amounts to assuming that no new indexes get added to system catalogs after initdb, or at least not during concurrent operations wherein indcheckxmin would be important. Sounds reasonable, but can we

Re: [HACKERS] Broken HOT chains in system catalogs

2011-04-15 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: can we enforce it through locks [?] Well, we already enforce it through the allow_system_table_mods mechanism: Good enough for me. -Kevin -- Sent via pgsql-hackers mailing list

[HACKERS] regression test client encoding

2011-04-15 Thread Peter Eisentraut
Using pg_regress --encoding sets both the server encoding of the test database and the client encoding. (The choice of server encoding is further constrained by locale, but that's a different issue.) Looking at the expected variants of the pesky plpython_unicode test plpython_unicode.out

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Greg Smith
I did some more research into the memory allocation hotspots found in the profile, in regards to what MySQL has done in those areas. (And by research I mean went to the bar last night and asked Baron about it) The issue of memory allocation being a limiter on performance has been nagging

Re: [HACKERS] regression test client encoding

2011-04-15 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: What I'd suggest is that we take out the bit of code in pg_regress.c that overrides the client encoding. That doesn't seem like a particularly good idea in view of the recent changes in psql to try to intuit a default encoding from its locale

Re: [HACKERS] [COMMITTERS] pgsql: Rename pg_regress option --multibyte to --encoding

2011-04-15 Thread Peter Eisentraut
On Fri, 2011-04-15 at 12:28 -0400, Andrew Dunstan wrote: Yeah, what is the point of this?: -# Adjust REGRESS_OPTS because we need a UTF8 database -REGRESS_OPTS = --dbname=$(CONTRIB_TESTDB) --multibyte=UTF8 --no-locale +# We need a UTF8 database +ENCODING = UTF8 +NO_LOCALE = 1

Re: [HACKERS] regression test client encoding

2011-04-15 Thread Peter Eisentraut
On Fri, 2011-04-15 at 16:09 -0400, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: What I'd suggest is that we take out the bit of code in pg_regress.c that overrides the client encoding. That doesn't seem like a particularly good idea in view of the recent changes in psql to try

Re: [HACKERS] regression test client encoding

2011-04-15 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On Fri, 2011-04-15 at 16:09 -0400, Tom Lane wrote: That doesn't seem like a particularly good idea in view of the recent changes in psql to try to intuit a default encoding from its locale environment. If I say --encoding in the command line, that

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Kevin Grittner
Greg Smith g...@2ndquadrant.com wrote: I'm not sure exactly what is useful to be learned from that specific work. But it does suggest two things: one, this is far from an easy thing to fix. Two, the only reason MySQL does so well on it is because there was some focused work on it, taking

Re: [HACKERS] WAL, xl_heap_insert and tuple oid mystry

2011-04-15 Thread Greg Stark
On Fri, Apr 15, 2011 at 6:29 PM, jagan jaganr...@yahoo.com wrote: Thanks for the pointer. I will try that route of recreating tuple descriptors and using heap_deform_tuple, although I suspect that heap_deform_tuple cannot be used by a stand-alone application (as it is not externally visible).

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Greg Smith
Joshua Berkus wrote: Guys, can we *please* focus on the patch for now, rather than the formatting, which is fixable with sed? Never, and that's not true. Heikki was being nice; I wouldn't have even slogged through it long enough to ask the questions he did before kicking it back as