Re: [HACKERS] pgbench: new feature allowing to launch shell commands
Michael Paquier michael.paqu...@gmail.com wrote: Please find attached the latest version of the patch, with the threading bug corrected and the documentation updated as well. Please don't describe your-specific usage of shell command in the documentation Why do you use BUFFER_SIZE-1 for snprintf? snprintf(commandShell, SHELL_COMMAND_SIZE-1, ...) Trailing nulls are also included in the length, so snprintf(commandShell, SHELL_COMMAND_SIZE, ...) would be ok. (removed -1) I found several bugs and matters. * The calcuration of the shell command length is wrong. * You cannot assign 0 to variables with \setshell meta command. * Comparison with == true is a bad manner. * You called \shell shell command and \setsell shell script, but we don't need the difference. I think shell command is enough. Heavily cleaned up patch attached. Please review it. Regards, --- Takahiro Itagaki NTT Open Source Software Center pgbenchshell_20091210.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug: fuzzystrmatch levenshtein is wrong
On Thu, Dec 10, 2009 at 3:00 AM, Robert Haas robertmh...@gmail.com wrote: Done. Yeah, my first commit! Great! Also, thanks for getting this in 8.4.2. Otherwise I would have to compile this on Windows myself, which is no fun. About the tabs vs spaces problem - I`ve decided that copying the patch from a remote machine is best done by selecting it in the terminal and pasting into a text file. Don`t do that :) Greetings Marcin Mańk -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Installing PL/pgSQL by default
Tom Lane t...@sss.pgh.pa.us writes: It's not impossible that we'll have to tweak pg_dump a bit; it's never had to deal with languages that shouldn't be dumped ... Ah, the best would be to have extensions maybe. Then you could do this in initdb, filling in template0: CREATE EXTENSION plpgsql ...; Then at createdb time, what would become automatic is: INSTALL EXTENSION plpgsql; And that's it. pg_dump would now about extensions and only issues this latter statement in its dump. Bruce, there are some mails in the archive with quite advanced design proposal that has been discussed and not objected to, and I even provided a rough sketch of how I wanted to attack the problem. http://archives.postgresql.org/pgsql-hackers/2009-06/msg01281.php http://archives.postgresql.org/pgsql-hackers/2009-07/msg01425.php http://archives.postgresql.org/pgsql-hackers/2009-07/msg01468.php The major version dependant SQL code is now much less of a problem than before because we have inline DO statements. So you don't need to create a function for this anymore. Real life kept me away from having the time to prepare the code patch, and I don't think that will change a bit in the 8.5 release cycle, whatever my hopes were earlier this year. But having everyone talk about the feature and come to an agreement as to what it should provide and how was the hard part of it, I think, and is now about done. Would you be up for writing the extension facility? Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] 答复: questions about concurrency control in Postgresql
I think I know why we need tuple lock. Though we have tuple's infomask shows whether the tuple is being updated, before we set the tuple's infomask, there may be two transaction coming and updating the tuple. They both think the tuple is ok to be updated, and then it's wrong. In PostgreSQL, we can use buffer lock to solve the problem , but its granularity is not proper. So we must use tuple lock to solve the problem. Thank you, Greg. You prompt me to think clearly about it. Happy communicating with you, and thanks again. --Huang Xiaocheng --Database Information System Lab, Nankai University -邮件原件- 发件人: gsst...@gmail.com [mailto:gsst...@gmail.com] 代表 Greg Stark 发送时间: 2009年12月8日 20:16 收件人: 黄晓骋 抄送: pgsql-hackers@postgresql.org 主题: Re: questions about concurrency control in Postgresql 2009/12/8 黄晓骋 huangxcl...@gmail.com: From the above, I think the tuple lock is unnecessary, because it uses transaction lock. Besides, tuple lock is unlocked after the tuple is updated but not after the transaction commits. I mean it's not 2PL. It's a two step process. An update marks the tuple locked. Another transaction which comes along and wants to lock the tuple waits on the transaction marked on the tuple. When the first transaction commits or aborts then the second transaction can proceed and lock the tuple itself. The reason we need both locks is because the first transaction cannot go around the whole database finding every tuple it ever locked to unlock it, firstly that could be a very large list and secondly there would be no way to do that atomically. Tuple locks and all user-visible locks are indeed held until the end of the transaction. -- greg __ Information from ESET NOD32 Antivirus, version of virus signature database 4671 (20091208) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __ Information from ESET NOD32 Antivirus, version of virus signature database 4674 (20091209) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Robert Haas escribió: On first blush, I'm inclined to suggest that the addition of + signs to mark continuation lines is a misfeature. -1 EXPLAIN is a special case. IMHO it should be dealt with accordingly. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
On Thu, Dec 10, 2009 at 10:12:32AM -0300, Alvaro Herrera wrote: Robert Haas escribió: On first blush, I'm inclined to suggest that the addition of + signs to mark continuation lines is a misfeature. -1 EXPLAIN is a special case. IMHO it should be dealt with accordingly. If the formatting code can be taught that it's outputting for explain, we can skip the wrap/newline markup easily. I don't think we necessarily need to fall back to the old-ascii format, we just conditionally disable that specific part. Alternatively, would it make more sense just to add a boolean pset parameter to enable/disable the use of wrap/newline marks? It may be that people may wish to disable it for use cases in addition to EXPLAIN. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: [HACKERS] strange segfault with 8.3.8
W dniu 7 grudnia 2009 17:43 użytkownik Greg Smith g...@2ndquadrant.comnapisał: Filip Rembiałkowski wrote: Dec 7 07:24:45 nsXX kernel: postgres[22481]: segfault at 7fda5e1d5000 ip 7fda604553c3 sp 7fffe41faf28 error 4 in libc-2.9.so [7fda603d1000+168000] Dec 7 07:24:45 nsXX kernel: grsec: From XX.YY.ZZ.51: Segmentation fault occurred at 7fda5e1d5000 in /usr/lib/postgresql/8.3/bin/postgres[postgres:22481] uid/euid:103/103 gid/egid:114/114, parent /usr/lib/postgresql/8.3/bin/postgres[postgres:29857] uid/euid:103/103 gid/egid:114/114 This is on 8.3.8, latest Ubuntu distribution, server under quite heavy load. afterwards we had some corrupted pages (*ERROR: compressed data is corrupt*). Random segfaults only under heavy load with corrupted pages afterwards normally means bad RAM. I'd suggest a round of memtest86+ on the system if it's PC hardware. Just a note for all other who may encouter this problem: This was not a vanilla Ubuntu - this was vendor-modified version, witch custom grsecurity patched kernel and probably with custom patched libc. # uname -a Linux nsXX.ovh.net 2.6.31.5-grsec--grs-ipv4-64 #2 SMP Thu Nov 5 12:36:20 UTC 2009 x86_64 GNU/Linux -- Filip Rembiałkowski JID,mailto:filip.rembialkow...@gmail.com http://filip.rembialkowski.net/
Re: [HACKERS] explain output infelicity in psql
Roger Leigh wrote: On Thu, Dec 10, 2009 at 10:12:32AM -0300, Alvaro Herrera wrote: Robert Haas escribió: On first blush, I'm inclined to suggest that the addition of + signs to mark continuation lines is a misfeature. -1 EXPLAIN is a special case. IMHO it should be dealt with accordingly. If the formatting code can be taught that it's outputting for explain, we can skip the wrap/newline markup easily. I don't think we necessarily need to fall back to the old-ascii format, we just conditionally disable that specific part. Alternatively, would it make more sense just to add a boolean pset parameter to enable/disable the use of wrap/newline marks? It may be that people may wish to disable it for use cases in addition to EXPLAIN. We have already added a lot of knobs to twist, and I don't want to add any more. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
Robert Haas escreveu: I'm not sure whether this is a good idea or not. Let me read the patch. I'm not sure an EXPLAIN option is really an adequate substitute for log_statement_stats - the latter will let you get stats for all of your queries automatically, I believe, and might still be useful as a quick and dirty tool. Why? If you want this information for all of your queries, you can always set auto_explain.log_min_duration to 0. But if you're suggesting that we should maintain log_statement_stats (that was not I understand from Tom's email [1]), it's not that difficult to a change ShowBufferUsage(). We certainly should NOT count on dtrace as a substitute for anything. It's not available on Windows, or all other platforms either. But we can always count on EXPLAIN BUFFERS. Remember that some monitoring tasks are _only_ available via DTrace. I still think this is a bad format. Instead of putting ( and ) around each phrase, can't we just separate them with a , or ;? We already use ( and ) to group things. I don't remember us using , or ; in any output node. The suggested output is intuitive and similar to other nodes patterns. [1] http://archives.postgresql.org/pgsql-hackers/2009-10/msg00718.php -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
On Thu, Dec 10, 2009 at 9:03 AM, Euler Taveira de Oliveira eu...@timbira.com wrote: Robert Haas escreveu: I'm not sure whether this is a good idea or not. Let me read the patch. I'm not sure an EXPLAIN option is really an adequate substitute for log_statement_stats - the latter will let you get stats for all of your queries automatically, I believe, and might still be useful as a quick and dirty tool. Why? If you want this information for all of your queries, you can always set auto_explain.log_min_duration to 0. But if you're suggesting that we should maintain log_statement_stats (that was not I understand from Tom's email [1]), it's not that difficult to a change ShowBufferUsage(). Mmm, OK, if Tom thinks we should rip it out, I'm not going to second-guess him. I still think this is a bad format. Instead of putting ( and ) around each phrase, can't we just separate them with a , or ;? We already use ( and ) to group things. I don't remember us using , or ; in any output node. The suggested output is intuitive and similar to other nodes patterns. It isn't. In the other cases where we output multiple distinct values on the same output row - like the sort instrumentation when ANALYZE is turned on - they are separated with copious amounts of whitespace. Costs are an exception, but those aren't done the same way as this either. The only reason anyone is even thinking that they need parentheses here is because they're trying to put three separate groups of buffer-related statistics - a total of 8 values - on the same output line. If this were split up over three output lines, no one would even be suggesting parentheses. Maybe that's a saner way to go. If not, fine, but I don't believe for a minute that the suggested format is either correct or parallel to what has been done elsewhere. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
On Wed, Dec 9, 2009 at 6:41 PM, Andrew Dunstan and...@dunslane.net wrote: Oh, dear. I think that line continuation syntax was recently added - subsequent to the machine-readable EXPLAIN patch. The reason why it's coded to emit everything as a single row is because that will be most convenient for programs that are sucking down this data programatically. Otherwise, they'll have to concatenate all the lines that are returned. And in fact for XML format, it's even worse: the data is returned as type xml, but that obviously won't fly if we return each line as a separate tuple. On first blush, I'm inclined to suggest that the addition of + signs to mark continuation lines is a misfeature. I certainly agree we don't want to break up the non-text formats. A simple if ugly hack would make psql use old-ascii print style (which doesn't use these contionuation chars) if the first attribute in the resultset was named 'QUERY PLAN' If someone has a better suggestion I'm all ears. I don't believe that machine-readable EXPLAIN output is the only multi-line output value that anyone would ever wish to cut and paste into an editor without picking up a lot of stray garbage, so I don't think this is a solution. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Robert Haas wrote: I don't believe that machine-readable EXPLAIN output is the only multi-line output value that anyone would ever wish to cut and paste into an editor without picking up a lot of stray garbage, so I don't think this is a solution. Well, yes, another example that has just occurred to me is pg_proc.prosrc. So maybe this really is a misfeature. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] executor and slru dtrace probes
--On 8. Dezember 2009 11:10:44 +0100 Zdenek Kotala zdenek.kot...@sun.com wrote: If you think that it is better I could split patch into two separate patches and both can be reviewed separately. I split up this patch into two separate patches: one for SLRU and one for the executor probes. I've done some documentation on the SLRU part, maybe you can look over it (to make sure i didn't break anything). I left the executor probes out of the documentation for now, since it seems not to be clear how they would manifest. Out of curiosity: Why do we want to pass the SlruCtl pointer down to the probes? I don't understand what those probes are going to do with those pointers, can you explain? -- Thanks Bernd dtrace_executor_probes.patch Description: application/text dtrace_slru_probes.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
Takahiro Itagaki escribió: =# EXPLAIN (BUFFERS, ANALYZE) SELECT * FROM pgbench_accounts a, pgbench_branches b WHERE a.bid = b.bid AND abalance 0 ORDER BY abalance; QUERY PLAN -- Sort (cost=2891.03..2891.04 rows=1 width=461) (actual time=22.494..22.494 rows=0 loops=1) Sort Key: a.abalance Sort Method: quicksort Memory: 25kB Blocks: (shared hit=96 read=1544 written=0) (local hit=0 read=0 written=0) (temp read=0 written=0) Maybe I missed part of this discussion, but it seems a bit weird to have an option named buffers turn on a line that specifies numbers of blocks. I kept looking for where you were specifying the BLOCKS option to EXPLAIN in the command ... -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
Alvaro Herrera alvhe...@commandprompt.com writes: Takahiro Itagaki escribió: Blocks: (shared hit=96 read=1544 written=0) (local hit=0 read=0 written=0) (temp read=0 written=0) Maybe I missed part of this discussion, but it seems a bit weird to have an option named buffers turn on a line that specifies numbers of blocks. Agreed, and I have to agree also with the people who have been criticizing the output format. If we were trying to put the block counts onto the same line as everything else then maybe parentheses would be helpful, but here they're just clutter. Perhaps I/O: shared hit=96 read=1544 written=0 local hit=0 read=0 written=0 temp read=0 written=0 (although this would suggest making the option name io which is probably a poor choice) I also suggest that dropping out zeroes might help --- a large fraction of EXPLAIN work is done with SELECTs that aren't ever going to write anything. Then the above becomes I/O: shared hit=96 read=1544 which is vastly more readable. You wouldn't want that to happen in machine-readable formats of course, but I think we no longer care about whether the text format is easy for programs to parse. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] executor and slru dtrace probes
Dne 10.12.09 15:51, Bernd Helmle napsal(a): --On 8. Dezember 2009 11:10:44 +0100 Zdenek Kotala zdenek.kot...@sun.com wrote: If you think that it is better I could split patch into two separate patches and both can be reviewed separately. I split up this patch into two separate patches: one for SLRU and one for the executor probes. I've done some documentation on the SLRU part, maybe you can look over it (to make sure i didn't break anything). I left the executor probes out of the documentation for now, since it seems not to be clear how they would manifest. Out of curiosity: Why do we want to pass the SlruCtl pointer down to the probes? I don't understand what those probes are going to do with those pointers, can you explain? You need to determine which SLRU is used. Because SLRUs are initialized during startup pointer should be same in all backends. If I think more about it. Maybe it could be goot to add probe also into SimpleLruInit to catch name of SLRUs. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
On Thu, Dec 10, 2009 at 10:44 AM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Takahiro Itagaki escribió: Blocks: (shared hit=96 read=1544 written=0) (local hit=0 read=0 written=0) (temp read=0 written=0) Maybe I missed part of this discussion, but it seems a bit weird to have an option named buffers turn on a line that specifies numbers of blocks. Agreed, and I have to agree also with the people who have been criticizing the output format. If we were trying to put the block counts onto the same line as everything else then maybe parentheses would be helpful, but here they're just clutter. Perhaps I/O: shared hit=96 read=1544 written=0 local hit=0 read=0 written=0 temp read=0 written=0 (although this would suggest making the option name io which is probably a poor choice) I also suggest that dropping out zeroes might help --- a large fraction of EXPLAIN work is done with SELECTs that aren't ever going to write anything. Then the above becomes I/O: shared hit=96 read=1544 which is vastly more readable. You wouldn't want that to happen in machine-readable formats of course, but I think we no longer care about whether the text format is easy for programs to parse. Oooh, that's a nice idea, though I think you should throw in some commas if there is, say, both shared and local stuff: shared hit=96 read=1544, local read=19 I don't think IO is a terrible name for an option but I like BUFFERS better. I don't think the BUFFERS/BLOCKS confusion is too bad, but perhaps we could use BUFFERS in both places. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
Robert Haas robertmh...@gmail.com writes: On Thu, Dec 10, 2009 at 9:03 AM, Euler Taveira de Oliveira eu...@timbira.com wrote: Why? If you want this information for all of your queries, you can always set auto_explain.log_min_duration to 0. But if you're suggesting that we should maintain log_statement_stats (that was not I understand from Tom's email [1]), it's not that difficult to a change ShowBufferUsage(). Mmm, OK, if Tom thinks we should rip it out, I'm not going to second-guess him. Feel free to question that. But it's ancient code and I'm not convinced it still has a reason to live. If you want to investigate the I/O behavior of a particular query, you'll use EXPLAIN. If you want to get an idea of the system-wide behavior, you'll use the stats collector. What use case is left for the backend-local counters? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
Robert Haas wrote: I don't think IO is a terrible name for an option but I like BUFFERS better. I don't think the BUFFERS/BLOCKS confusion is too bad, but perhaps we could use BUFFERS in both places. I don't know how blocks got into here in the first place--this concept is buffers just about everywhere else already, right? -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
On Thu, Dec 10, 2009 at 10:52 AM, Greg Smith g...@2ndquadrant.com wrote: Robert Haas wrote: I don't think IO is a terrible name for an option but I like BUFFERS better. I don't think the BUFFERS/BLOCKS confusion is too bad, but perhaps we could use BUFFERS in both places. I don't know how blocks got into here in the first place--this concept is buffers just about everywhere else already, right? I think we have some places already in the system where we bounce back and forth between those terms. I expect that's the reason. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
On Thu, Dec 10, 2009 at 10:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Dec 10, 2009 at 9:03 AM, Euler Taveira de Oliveira eu...@timbira.com wrote: Why? If you want this information for all of your queries, you can always set auto_explain.log_min_duration to 0. But if you're suggesting that we should maintain log_statement_stats (that was not I understand from Tom's email [1]), it's not that difficult to a change ShowBufferUsage(). Mmm, OK, if Tom thinks we should rip it out, I'm not going to second-guess him. Feel free to question that. But it's ancient code and I'm not convinced it still has a reason to live. If you want to investigate the I/O behavior of a particular query, you'll use EXPLAIN. If you want to get an idea of the system-wide behavior, you'll use the stats collector. What use case is left for the backend-local counters? Beats me. Tracing just your session without having to EXPLAIN each query (and therefore not get the output rows)? OK, I'm reaching. I tend to be very conservative about ripping things out that someone might want unless they're actually getting in the way of doing some new thing that we want to do - but so are you, and you know the history of this code better than I do. I'm happy to save my questioning for a more important issue. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
Robert Haas escreveu: The only reason anyone is even thinking that they need parentheses here is because they're trying to put three separate groups of buffer-related statistics - a total of 8 values - on the same output line. If this were split up over three output lines, no one would even be suggesting parentheses. That's the point. I'm afraid 3 new lines per node is too verbose. -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] executor and slru dtrace probes
--On 9. Dezember 2009 19:08:07 -0500 Theo Schlossnagle je...@omniti.com wrote: Now, there was some indication that there was a better place to probe that would be more comprehensive -- that should be addressed. For now there exists no consensus where they should go in. Tom pointed out various issues with ExecProcNode() and he's worried about the performance penalty those probes might introduce. I admit I'm not very experienced with dtrace, but maybe some worries exists because an expensive instrumented executor probe can cause forged results? -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Alvaro Herrera wrote: Robert Haas escribió: On first blush, I'm inclined to suggest that the addition of + signs to mark continuation lines is a misfeature. EXPLAIN is a special case. IMHO it should be dealt with accordingly. Is it? Wouldn't it affect anyone who stuck XML in a txt column and wanted to copy and paste it into a parser? Perhaps single column output usually won't want the + signs (because it's copypasteable) but multi-column output could? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] tsearch parser inefficiency if text includes urls or emails - new version
Andres Freund and...@anarazel.de wrote: I think you see no real benefit, because your strings are rather short - the documents I scanned when noticing the issue where rather long. The document I used in the test which showed the regression was 672,585 characters, containing 10,000 URLs. A rather extreme/contrived example: postgres=# SELECT 1 FROM to_tsvector(array_to_string(ARRAY(SELECT 'and...@anarazel.de http://www.postgresql.org/'::text FROM generate_series(1, 2) g(i)), ' - ')); The most extreme of your examples uses a 979,996 character string, which is less than 50% larger than my test. I am, however, able to see the performance difference for this particular example, so I now have something to work with. I'm seeing some odd behavior in terms of when there is what sort of difference. Once I can categorize it better, I'll follow up. Thanks for the sample which shows the difference. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Ron Mayer wrote: Alvaro Herrera wrote: Robert Haas escribió: On first blush, I'm inclined to suggest that the addition of + signs to mark continuation lines is a misfeature. EXPLAIN is a special case. IMHO it should be dealt with accordingly. Is it? Wouldn't it affect anyone who stuck XML in a txt column and wanted to copy and paste it into a parser? Perhaps single column output usually won't want the + signs (because it's copypasteable) but multi-column output could? Yeah, I'm thinking we should probably suppress output of format.nl_right (no matter what the format) where there is only one column. (This is even uglier with unicode linestyle, btw). That's a sane rule and it's not an ugly hack. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
On Thu, Dec 10, 2009 at 11:44 AM, Ron Mayer rm...@cheapcomplexdevices.com wrote: Alvaro Herrera wrote: Robert Haas escribió: On first blush, I'm inclined to suggest that the addition of + signs to mark continuation lines is a misfeature. EXPLAIN is a special case. IMHO it should be dealt with accordingly. Is it? Wouldn't it affect anyone who stuck XML in a txt column and wanted to copy and paste it into a parser? Perhaps single column output usually won't want the + signs (because it's copypasteable) but multi-column output could? I don't think inconsistency is a good thing here. Apparently the old format is available via \pset linestyle old-ascii. We can either decide that people should use that format if they want that behavior, or we can decide that changing the default was a mistake and revert it. I don't think a half-way-in-between solution is a good option. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Robert Haas robertmh...@gmail.com wrote: Andrew Dunstan and...@dunslane.net wrote: A simple if ugly hack would make psql use old-ascii print style (which doesn't use these contionuation chars) if the first attribute in the resultset was named 'QUERY PLAN' I don't believe that machine-readable EXPLAIN output is the only multi-line output value that anyone would ever wish to cut and paste into an editor without picking up a lot of stray garbage, so I don't think this is a solution. Agreed. This would be a significant annoyance for me on a regular basis. If I can't turn it off, it would probably cause me to create my own locally patched version of psql. Another alternative would be to use some other tool to run queries where I wanted long values without this, but psql has so many nice features that I'd be switching back and forth, so the patch would probably be easier. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Andrew Dunstan and...@dunslane.net writes: Yeah, I'm thinking we should probably suppress output of format.nl_right (no matter what the format) where there is only one column. (This is even uglier with unicode linestyle, btw). That's a sane rule and it's not an ugly hack. Yes it is. The real problem here is expecting the tabular format to be copy and pasteable, which is not a design goal it's ever had, and not one that we can start imposing on it at this late date. Why don't you just do \pset format unaligned (or \a if you're lazy)? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Yeah, I'm thinking we should probably suppress output of format.nl_right (no matter what the format) where there is only one column. (This is even uglier with unicode linestyle, btw). That's a sane rule and it's not an ugly hack. Yes it is. I don't see much virtue of having these characters when there is only one column. The real problem here is expecting the tabular format to be copy and pasteable, which is not a design goal it's ever had, and not one that we can start imposing on it at this late date. Why don't you just do \pset format unaligned (or \a if you're lazy)? Well, I haven't had to up to now. I'm not sure what we have is exactly an advance. I guess I can set the linestyle to old-ascii in my .psqlrc, but having to do that all over the place is annoying. And clearly I'm not the only person who doesn't like this behaviour. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Andrew Dunstan and...@dunslane.net writes: I don't see much virtue of having these characters when there is only one column. So you can tell a newline in the data from a wrap due to line length. The need to be able to do that is not dependent on how many columns there are. And clearly I'm not the only person who doesn't like this behaviour. It's just our usual negative reaction to any change whatsoever ;-). I was unimpressed with Leigh's changes too at the start, but I can see that there is value in it. I think that changing the behavior depending on how many columns there are is an incredibly ugly hack, and your assertions to the contrary aren't going to change my mind. If we think that this is such a bad idea it should be reverted, then let's revert it altogether. Another possibility, which I don't understand why it was dismissed so cavalierly, is to have EXPLAIN put out one row per logical line instead of using embedded newlines. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
On Thu, Dec 10, 2009 at 12:43 PM, Andrew Dunstan and...@dunslane.net wrote: Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Yeah, I'm thinking we should probably suppress output of format.nl_right (no matter what the format) where there is only one column. (This is even uglier with unicode linestyle, btw). That's a sane rule and it's not an ugly hack. Yes it is. I don't see much virtue of having these characters when there is only one column. I don't see much virtue in having these characters, period. The reason for having them is presumably to avoid confusion between two rows and one row with an embedded newline. If anything, this confusion is more likely with a single column than it is with multiple columns, since you don't have the context of the surrounding output. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Need --without-docs build switch
Hackers, Testing Alpha2 with the Exclusion Constraints patch on Mac recently forced me to edit the makefiles to remove the make docs statements. (The Mac Openjade toolchain is unalterably broken, AFAICT) Without editing the makefiles, it wasn't possible to build PostgreSQL on my OSX 10.4 laptop. It strikes me that forcing people to build docs even if they don't want to is not a good thing, both because docs won't build on all platforms which PostgreSQL will build on, and because sometimes people just don't want to spend the extra 50% of their build time to build the docs (updates, headless machines, test machines). Given that, we should have a --without-docs build switch which allows us to disable the doc build. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need --without-docs build switch
Josh Berkus j...@agliodbs.com writes: Testing Alpha2 with the Exclusion Constraints patch on Mac recently forced me to edit the makefiles to remove the make docs statements. How so? The make process does not try to build the docs unless you specifically tell it to. Given that, we should have a --without-docs build switch which allows us to disable the doc build. This is already the default behavior... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] tsearch parser inefficiency if text includes urls or emails - new version
I wrote: Thanks for the sample which shows the difference. Ah, once I got on the right track, there is no problem seeing dramatic improvements with the patch. It changes some nasty O(N^2) cases to O(N). In particular, the fixes affect parsing of large strings encoded with multi-byte character encodings and containing email addresses or URLs with a non-IP-address host component. It strikes me as odd that URLs without a slash following the host portion, or with an IP address, are treated so differently in the parser, but if we want to address that, it's a matter for another patch. I'm inclined to think that the minimal differences found in some of my tests probably have more to do with happenstance of code alignment than the particulars of the patch. I did find one significant (although easily solved) problem. In the patch, the recursive setup of usewide, pgwstr, and wstr are not conditioned by #ifdef USE_WIDE_UPPER_LOWER in the non-patched version. Unless there's a good reason for that, the #ifdef should be added. Less critical, but worth fixing one way or the other, TParserClose does not drop breadcrumbs conditioned on #ifdef WPARSER_TRACE, but TParserCopyClose does. I think this should be consistent. Finally, there's that spelling error in the comment for TParserCopyInit. Please fix. If a patch is produced with fixes for these three things, I'd say it'll be ready for committer. I'm marking it as Waiting on Author for fixes to these three items. Sorry for the delay in review. I hope there's still time to get this committed in this CF. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I don't see much virtue of having these characters when there is only one column. So you can tell a newline in the data from a wrap due to line length. The need to be able to do that is not dependent on how many columns there are. If that's really what we want then I think we're doing a terrible job of it. Have a look at the output of: select E'xxx\nxx\n' as a, 1 as b; How do we know from that where the linefeeds are, exactly? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: So you can tell a newline in the data from a wrap due to line length. The need to be able to do that is not dependent on how many columns there are. If that's really what we want then I think we're doing a terrible job of it. Have a look at the output of: select E'xxx\nxx\n' as a, 1 as b; How do we know from that where the linefeeds are, exactly? It works quite nicely for me ... in HEAD that is: regression=# select E'xxx\nxx\n' as a, 1 as b; a | b --+--- xxx +| 1 xx +| | (1 row) regression=# The point here is exactly that previous versions didn't show the distinction well. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need --without-docs build switch
On 12/10/09 10:03 AM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Testing Alpha2 with the Exclusion Constraints patch on Mac recently forced me to edit the makefiles to remove the make docs statements. How so? The make process does not try to build the docs unless you specifically tell it to. It most certainly did with Alpha2. There was no way to get it NOT to build the docs, except editing the makefile. I haven't encountered this behaviour before, though ... --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Tom Lane wrote: Why don't you just do \pset format unaligned (or \a if you're lazy)? That's fair. Now that I see it, I guess I should have been doing that for copypaste work anyway. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need --without-docs build switch
Josh Berkus j...@agliodbs.com writes: On 12/10/09 10:03 AM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Testing Alpha2 with the Exclusion Constraints patch on Mac recently forced me to edit the makefiles to remove the make docs statements. How so? The make process does not try to build the docs unless you specifically tell it to. It most certainly did with Alpha2. Hm, maybe the alpha2 tarball had docs in it already? And they weren't up to date? You might be hitting the same thing I've complained of: since make distclean no longer removes built docs, you can find yourself running an update cycle when you didn't want to. Peter rejected my opinion that we should go back to the old behavior of make distclean, but I'm still not happy about it. http://archives.postgresql.org/pgsql-hackers/2009-08/msg01336.php regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need --without-docs build switch
Hm, maybe the alpha2 tarball had docs in it already? And they weren't up to date? You might be hitting the same thing I've complained of: since make distclean no longer removes built docs, you can find yourself running an update cycle when you didn't want to. Peter rejected my opinion that we should go back to the old behavior of make distclean, but I'm still not happy about it. http://archives.postgresql.org/pgsql-hackers/2009-08/msg01336.php Oh, now *that's* possible. I copied it right over the directory for Alpha1. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need --without-docs build switch
--On 10. Dezember 2009 09:59:44 -0800 Josh Berkus j...@agliodbs.com wrote: (The Mac Openjade toolchain is unalterably broken, AFAICT) Without editing the makefiles, it wasn't possible to build PostgreSQL on my OSX 10.4 laptop. You might get an idea how i fixed that here: http://psoos.blogspot.com/2009/09/building-postgresql-documentation-on.html However, this was on 10.5 (works on 10.6, too), so i'm not sure this applies to 10.4 the same way (however, at least macports is still available for 10.4). -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] tsearch parser inefficiency if text includes urls or emails - new version
I wrote: I did find one significant (although easily solved) problem. In the patch, the recursive setup of usewide, pgwstr, and wstr are not conditioned by #ifdef USE_WIDE_UPPER_LOWER in the non-patched version. Unless there's a good reason for that, the #ifdef should be added. That should read: I did find one significant (although easily solved) problem. In the patch, the recursive setup of usewide, pgwstr, and wstr are not conditioned by #ifdef USE_WIDE_UPPER_LOWER as they are in the non-patched version. Unless there's a good reason for that, the #ifdef should be added. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] tsearch parser inefficiency if text includes urls or emails - new version
On Thursday 10 December 2009 19:10:24 Kevin Grittner wrote:
I wrote:
Thanks for the sample which shows the difference.
Ah, once I got on the right track, there is no problem seeing
dramatic improvements with the patch. It changes some nasty O(N^2)
cases to O(N). In particular, the fixes affect parsing of large
strings encoded with multi-byte character encodings and containing
email addresses or URLs with a non-IP-address host component. It
strikes me as odd that URLs without a slash following the host
portion, or with an IP address, are treated so differently in the
parser, but if we want to address that, it's a matter for another
patch.
Same here. Generally I do have to say that I dont find that parser very nice -
and actually I think its not very efficient as well because it searches a big
part of the search space all the time.
I think a generated parser would be more efficient and way much easier to
read...
I'm inclined to think that the minimal differences found in some of
my tests probably have more to do with happenstance of code
alignment than the particulars of the patch.
Yes, I think that as well.
I did find one significant (although easily solved) problem. In the
patch, the recursive setup of usewide, pgwstr, and wstr are not
conditioned by #ifdef USE_WIDE_UPPER_LOWER in the non-patched
version. Unless there's a good reason for that, the #ifdef should
be added.
Uh. Sorry. Fixed.
Less critical, but worth fixing one way or the other, TParserClose
does not drop breadcrumbs conditioned on #ifdef WPARSER_TRACE, but
TParserCopyClose does. I think this should be consistent.
Actually there was *some* thinking behind that: The end of parsing is quite
obvious (the call returns, and its so verbose you will never do more than one
call anyway) - but knowing where the copy ends is quite important to
understand the parse flow.
I added a log there because in the end that line is not going to make any
difference ;-)
Sorry for the delay in review. I hope there's still time to get
this committed in this CF.
Thanks for your reviewing!
Actually I dont mind very much if it gets delayed or not. Its trivial enough
that it shouldnt cause much work/conflicts/whatever next round and I am running
patched versions anyway, so ...
Andres
From e01bac641f318b378c4353aa6ccebc76b3071166 Mon Sep 17 00:00:00 2001
From: Andres Freund and...@anarazel.de
Date: Thu, 10 Dec 2009 19:54:22 +0100
Subject: [PATCH] Fix TSearch inefficiency because of repeated copying of strings
---
src/backend/tsearch/wparser_def.c | 69 ++--
1 files changed, 65 insertions(+), 4 deletions(-)
diff --git a/src/backend/tsearch/wparser_def.c b/src/backend/tsearch/wparser_def.c
index 72b435c..46e86ee 100644
*** a/src/backend/tsearch/wparser_def.c
--- b/src/backend/tsearch/wparser_def.c
*** TParserInit(char *str, int len)
*** 328,333
--- 328,371
return prs;
}
+ /*
+ * As an alternative to a full TParserInit one can create a
+ * TParserCopy which basically is a normally TParser without a private
+ * copy of the string - instead it uses the one from another TParser.
+ * This is useful because at some places TParsers are created
+ * recursively and the repeated copying around of the strings can
+ * cause major inefficiency.
+ * Obviously one may not close the original TParser before the copy.
+ */
+ static TParser *
+ TParserCopyInit(const TParser const* orig)
+ {
+ TParser*prs = (TParser *) palloc0(sizeof(TParser));
+
+ prs-charmaxlen = orig-charmaxlen;
+ prs-str = orig-str + orig-state-posbyte;
+ prs-lenstr = orig-lenstr - orig-state-posbyte;
+
+ #ifdef USE_WIDE_UPPER_LOWER
+ prs-usewide = orig-usewide;
+
+ if(orig-pgwstr)
+ prs-pgwstr = orig-pgwstr + orig-state-poschar;
+ if(orig-wstr)
+ prs-wstr = orig-wstr + orig-state-poschar;
+ #endif
+
+ prs-state = newTParserPosition(NULL);
+ prs-state-state = TPS_Base;
+
+ #ifdef WPARSER_TRACE
+ fprintf(stderr, parsing copy of \%.*s\\n, len, str);
+ #endif
+
+ return prs;
+ }
+
+
static void
TParserClose(TParser *prs)
{
*** TParserClose(TParser *prs)
*** 346,354
--- 384,415
pfree(prs-pgwstr);
#endif
+ #ifdef WPARSER_TRACE
+ fprintf(stderr, closing parser);
+ #endif
+ pfree(prs);
+ }
+
+ /*
+ * See TParserCopyInit
+ */
+ static void
+ TParserCopyClose(TParser *prs)
+ {
+ while (prs-state)
+ {
+ TParserPosition *ptr = prs-state-prev;
+
+ pfree(prs-state);
+ prs-state = ptr;
+ }
+ #ifdef WPARSER_TRACE
+ fprintf(stderr, closing parser copy);
+ #endif
pfree(prs);
}
+
/*
* Character-type support functions, equivalent to is* macros, but
* working with any possible encodings and locales. Notes:
*** p_isignore(TParser *prs)
*** 617,623
static int
p_ishost(TParser *prs)
{
! TParser*tmpprs = TParserInit(prs-str + prs-state-posbyte, prs-lenstr - prs-state-posbyte);
int res = 0;
Re: [HACKERS] explain output infelicity in psql
Ron Mayer rm...@cheapcomplexdevices.com wrote: Tom Lane wrote: Why don't you just do \pset format unaligned (or \a if you're lazy)? That's fair. Now that I see it, I guess I should have been doing that for copypaste work anyway. That'll cover my use cases. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Tom Lane wrote: regression=# select E'xxx\nxx\n' as a, 1 as b; a | b --+--- xxx +| 1 xx +| | (1 row) regression=# The point here is exactly that previous versions didn't show the distinction well. If we really want to make linefeeds visible, I think we should place the indicators immediately after the character preceding the line feed, not next to the column separator. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] tsearch parser inefficiency if text includes urls or emails - new version
Andres Freund and...@anarazel.de wrote: [concerns addressed in new patch version] Looks good to me. I'm marking this Ready for Committer. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need --without-docs build switch
Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: On 12/10/09 10:03 AM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Testing Alpha2 with the Exclusion Constraints patch on Mac recently forced me to edit the makefiles to remove the make docs statements. How so? The make process does not try to build the docs unless you specifically tell it to. It most certainly did with Alpha2. Hm, maybe the alpha2 tarball had docs in it already? And they weren't up to date? You might be hitting the same thing I've complained of: since make distclean no longer removes built docs, you can find yourself running an update cycle when you didn't want to. Peter rejected my opinion that we should go back to the old behavior of make distclean, but I'm still not happy about it. http://archives.postgresql.org/pgsql-hackers/2009-08/msg01336.php If the patch added new docs, then a doc build would be invoked by make. A make -C doc maintainer-clean would remove the trigger files and thus the docs would not be built after applying the patch. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Zdenek Kotala zdenek.kot...@sun.com writes: [...] + header = (StandardChunkHeader *) + ((char *) ret - STANDARDCHUNKHEADERSIZE); + +// TRACE_POSTGRESQL_MCXT_ALLOC(context-name, context, size, header-size, true); + [...] If the dormant overhead of these probes is measured or suspected to be excessive, consider using the dtrace-generated per-probe foo_ENABLED() conditional, or a postgres configuration global thusly: if (__builtin_expect(TRACE_POSTGRESQL_MCXT_ALLOC_ENABLED(), 0)) TRACE_POSTGRESQL_MCXT_ALLOC(...); so that the whole instrumentation parameter setup/call can be placed out of the hot line with gcc -freorder-blocks. - FChE -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Andrew Dunstan and...@dunslane.net writes: If we really want to make linefeeds visible, I think we should place the indicators immediately after the character preceding the line feed, not next to the column separator. Then it's hard to tell it apart from an instance of that character in the data. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: If we really want to make linefeeds visible, I think we should place the indicators immediately after the character preceding the line feed, not next to the column separator. Then it's hard to tell it apart from an instance of that character in the data. Yeah, I just thought of that. Oh, well, old-ascii for me ;-) cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Frank Ch. Eigler píše v čt 10. 12. 2009 v 14:11 -0500: Zdenek Kotala zdenek.kot...@sun.com writes: [...] + header = (StandardChunkHeader *) + ((char *) ret - STANDARDCHUNKHEADERSIZE); + +// TRACE_POSTGRESQL_MCXT_ALLOC(context-name, context, size, header-size, true); + [...] If the dormant overhead of these probes is measured or suspected to be excessive, consider using the dtrace-generated per-probe foo_ENABLED() conditional, or a postgres configuration global thusly: TRACE_POSTGRESQL_MCXT_ALLOC and TRACE_POSTGRESQL_ASET_ALLOC are duplicated probes. Have them both make sense but from performance point of view to have one of them is acceptable. foo_enable() is good to use when number of argument and their evaluation cost too much. In this case it does no seem to be much useful. See ASM code: AllocSetAlloc+0x17: xorq %rax,%rax AllocSetAlloc+0x1a: nop AllocSetAlloc+0x1b: nop AllocSetAlloc+0x1c: testl %eax,%eax AllocSetAlloc+0x1e: je +0xb AllocSetAlloc+0x2b AllocSetAlloc+0x20: movq %r13,%rdi AllocSetAlloc+0x23: movq %r14,%rsi AllocSetAlloc+0x26: nop AllocSetAlloc+0x27: nop AllocSetAlloc+0x28: nop AllocSetAlloc+0x29: nop AllocSetAlloc+0x2a: nop if (__builtin_expect(TRACE_POSTGRESQL_MCXT_ALLOC_ENABLED(), 0)) TRACE_POSTGRESQL_MCXT_ALLOC(...); so that the whole instrumentation parameter setup/call can be placed out of the hot line with gcc -freorder-blocks. compiler specific construct is not good way. Do not forget that also other compiler exists. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PL/Python array support
On tis, 2009-12-01 at 20:53 -0700, Joshua Tolley wrote: This patch doesn't include any documentation; my reading of the PL/Python docs suggests that's probably acceptable, as the existing docs don't talk about its array handling. That said, it might be useful to include an example, to show for instance that identical PL/Python code could create either an array of a type or a set of rows of that type: I added a bit of documentation like that. Thanks. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] XLogInsert
On Wed, Dec 9, 2009 at 9:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: so I'd like some independent confirmation that it does. what kind of tests could show that? or simply running pgbench several times for 15 minutes each run could show any benefit? -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil - Ecuador Cel. +59387171157 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need --without-docs build switch
On tor, 2009-12-10 at 09:59 -0800, Josh Berkus wrote: Given that, we should have a --without-docs build switch which allows us to disable the doc build. Btw., if you really, really, really want to avoid the issue (if there were any), go into the src directory and build there. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] XLogInsert
Jaime Casanova escribió: On Wed, Dec 9, 2009 at 9:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: so I'd like some independent confirmation that it does. what kind of tests could show that? or simply running pgbench several times for 15 minutes each run could show any benefit? Isn't the supposed performance improvement in this patch linked strongly to backup blocks? If that's really the case, I think it would show more prominently if you had very frequent checkpoints. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] tsearch parser overhaul
re: http://archives.postgresql.org/pgsql-hackers/2009-11/msg00754.php Alvaro Herrera alvhe...@commandprompt.com wrote: Kevin Grittner wrote: (Note: I personally would much rather see the performance penalty addressed this way, and a TODO added for the more invasive work, than to leave this alone for the next release if there's nobody willing to tackle the problem at a more fundamental level.) +1 I haven't added a TODO yet because I'm not sure how to frame it. I'm inclined that it would be no more work to replace the current recursively called state engine with something easier to read and understand than to try to fix the current oddities. Perhaps something along the lines of this?: http://vo.astronet.ru/arxiv/dict_regex.html I suspect we'd need to get it to use the same regexp code used elsewhere in PostgreSQL. Thoughts? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
On Wed, Dec 9, 2009 at 10:43 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Wed, Dec 9, 2009 at 5:38 PM, Bruce Momjian br...@momjian.us wrote: If you want to avoid all good reasons for this features and are looking for reasons why this patch is a bad idea, I am sure you can find them. You seem to be suggesting that our reactions are pure obstructionism, or that they have an ulterior motive. I am merely stating that this is the same as the Win32 port, and that there are many reasons to believe the SE-PostgreSQL patch will cause all sorts of problems --- this is not a surprise. I am giving a realistic analysis of the patch --- if people want to say that thinking of it as two separate patches that have to be maintained separately is a terrible idea, I have no reply except to say that realistically that is the only possible direction I see for this feature in the short term. Few Postgres people modifying the permissions system are going to understand how to modify SE-Linux support routines to match their changes. I got a similar reaction when I wanted to do the Win32 port, and the reasons not to do it were similar to the ones I am hearing now. Finally the agreement was that I could attempt the Win32 port as long as I didn't destabilize the rest of the code --- not exactly a resounding endorsement. Looking back I think everyone is glad we did the port, but at the time there wasn't much support. I got the same reaction to pg_migrator. I am having trouble figuring out when I should heed community concerns, and when the concerns are merely because the task is hard/messy/difficult. Frankly, we don't analyze hard/messy/difficult tasks very well. Now, I am not saying that the SE-PostgreSQL patch should be pursued, but I am saying that we shouldn't avoid it for these reasons, because sometimes hard/messy/difficult is necessary to accomplish dramatic software advances. I don't have any easy answers here. I'm actually trying not to make a value judgment about the feature and focus on the technical problems with the patch. If those problems are fixed, which as you say probably doable, then I don't mind seeing it committed. I think that the reason we don't analyze hard/messy/difficult problems very well is because on the one hand you have people saying this feature would be great. On the other hand you have people saying this feature will be a lot of work. But those things are not opposites. Unlike Tom (I think), I do believe that there is demand (possibly only from a limited number of people, but demand all the same) for this feature. And I also believe that most people in our community are generally supportive of the idea, but only a minority are willing to put in time to make it happen. So I have no problem saying to the people who want the feature - none of our committers feel like working on this. Sorry. On the other hand, I also have no problem telling them - good news, Bruce Momjian thinks this is a great feature and wants to help you get it done. I *do* have a problem with saying - we don't really know whether anyone will ever want to work on this with you or not. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Hi - On Thu, Dec 10, 2009 at 09:33:28PM +0100, Zdenek Kotala wrote: [...] If the dormant overhead of these probes is measured or suspected to be excessive, consider using the dtrace-generated per-probe foo_ENABLED() conditional, or a postgres configuration global thusly: [...] foo_enable() is good to use when number of argument and their evaluation cost too much. In this case it does no seem to be much useful. [...] Right, I just wanted to make the others aware of the option. if (__builtin_expect(TRACE_POSTGRESQL_MCXT_ALLOC_ENABLED(), 0)) TRACE_POSTGRESQL_MCXT_ALLOC(...); so that the whole instrumentation parameter setup/call can be placed out of the hot line with gcc -freorder-blocks. compiler specific construct is not good way. Do not forget that also other compiler exists. Certainly. Many projects -- but apparently not postgresql -- wrap such branch prediction hints in macros such as likely() and unlikely(), which are easily no-op'd for compilers that don't support this sort of thing. - FChE -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Robert Haas robertmh...@gmail.com writes: Unlike Tom (I think), I do believe that there is demand (possibly only from a limited number of people, but demand all the same) for this feature. Please note that I do not think there is *zero* demand for the feature. There is obviously some. What I find highly dubious is whether there is enough demand to justify the amount of effort, both short- and long-term, that the community would have to put into it. And I also believe that most people in our community are generally supportive of the idea, but only a minority are willing to put in time to make it happen. So I have no problem saying to the people who want the feature - none of our committers feel like working on this. Sorry. On the other hand, I also have no problem telling them - good news, Bruce Momjian thinks this is a great feature and wants to help you get it done. I *do* have a problem with saying - we don't really know whether anyone will ever want to work on this with you or not. If I thought that Bruce could go off in a corner and make this happen and it would create no demands on anybody but him and KaiGai-san, I would say fine, if that's where you want to spend your time, go for it. But even to state that implied claim is to see how false it is. Bruce is pointing to the Windows port, but he didn't make it happen by himself, or any close approximation of that. Everybody who works on this project has been affected by that, and we're *still* putting significant amounts of time into Windows compatibility, over five years later. My guess is that a credible SEPostgres offering will require a long-term amount of work at least equal to, and very possibly a good deal more than, what it took to make a native Windows port. If SEPostgres could bring us even 10% as many new users as the Windows port did, it'd probably be a worthwhile use of our resources. But again, that's an assumption that's difficult to type without bursting into laughter. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unprivileged user
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: The other day I returned idly to thinking about some work I did a few years ago on creating a totally unprivileged user, i.e. one with not even public permissions. And the point would be what exactly? Well, when I was looking at it originally it was in the context of a layered security setup, where we wanted to minimise the danger from a client machine (say a web server) being subverted. The reasoning was that if the subverted user had no access to the database layout, but had only access to a very tightly defined set of stored functions, it would be harder to devise attacks against the database. It might be argued that this is security by obscurity, but obscurity does have some uses, albeit never as a complete security mechanism. Some time later it came up again, this time when someone wanted to use a readonly database (hence no pg_dump required) with an application and wanted to keep the database layout and the source code of stored functions hidden as they regarded it as proprietary information. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
On Thu, 2009-12-10 at 17:08 -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Unlike Tom (I think), I do believe that there is demand (possibly only from a limited number of people, but demand all the same) for this feature. Please note that I do not think there is *zero* demand for the feature. There is obviously some. What I find highly dubious is whether there is enough demand to justify the amount of effort, both short- and long-term, that the community would have to put into it. And I also believe that most people in our community are generally supportive of the idea, but only a minority are willing to put in time to make it happen. So I have no problem saying to the people who want the feature - none of our committers feel like working on this. Sorry. On the other hand, I also have no problem telling them - good news, Bruce Momjian thinks this is a great feature and wants to help you get it done. I *do* have a problem with saying - we don't really know whether anyone will ever want to work on this with you or not. If I thought that Bruce could go off in a corner and make this happen and it would create no demands on anybody but him and KaiGai-san, I would say fine, if that's where you want to spend your time, go for it. But even to state that implied claim is to see how false it is. Bruce is pointing to the Windows port, but he didn't make it happen by himself, or any close approximation of that. Everybody who works on this project has been affected by that, and we're *still* putting significant amounts of time into Windows compatibility, over five years later. My guess is that a credible SEPostgres offering will require a long-term amount of work at least equal to, and very possibly a good deal more than, what it took to make a native Windows port. If SEPostgres could bring us even 10% as many new users as the Windows port did, it'd probably be a worthwhile use of our resources. But again, that's an assumption that's difficult to type without bursting into laughter. regards, tom lane So a couple of us in the Maryland/DC area went to the BWPUG meeting last night and we sat down for two hours and answered a bunch of questions from Greg Smith, Steve Frost, and a few others. Greg was taking notes during the entire meeting and I believe he will be starting a thread with the minutes from the meeting. Greg brought up 5 or 6 concerns that he has observed in the community about the work including the issue of who is going to use this. The minutes will give a much better account of the conversation but Josh Brindle and I have gave examples outside of DoD where the MAC framework without row based access controls can be useful. For our purposes in DoD we need the MAC Framework and the row based access controls but if a good starting point is to just do the access control over the database objects then it will be useful for some commercial cases and some limited military cases. Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Hi, On Thursday 10 December 2009 23:08:17 Tom Lane wrote: My guess is that a credible SEPostgres offering will require a long-term amount of work at least equal to, and very possibly a good deal more than, what it took to make a native Windows port. If SEPostgres could bring us even 10% as many new users as the Windows port did, it'd probably be a worthwhile use of our resources. But again, that's an assumption that's difficult to type without bursting into laughter. Sorry, could not resist: It could possibly bring us more interesting users... While mainly an humoristic remark, I think it might even have some truth to it... Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] SE-PostgreSQL/Lite Review
It's funny; we started out this CommitFest with me scrambling to find someone, anyone, willing to review the latest SE-PostgreSQL patch, knowing it was a big job and few were likely to volunteer. Then schedules lined up just right, and last night I managed to get a great group of people all together to do perhaps the biggest single patch review ever, to work on just that.I gathered up a list of the biggest concerns about this feature and its associated implementation, we got a number of regular PostgreSQL hackers and two of the security guys you've seen on this list all in the same room, and we talked about little but SEPostgreSQL for hours. Minutes are at http://wiki.postgresql.org/wiki/SEPostgreSQL_Review_at_the_BWPUG and I'd suggest anyone interested in this feature (or in rejecting this feature) to take a look at what we covered. There's comments there, with references for you [citation needed] types, to help answer four of the most common complaints I've seen on this list about this feature: -Is there really a user base for it? -Has it been audited by security professionals? -Is there a useful SELinux policty to support it? -Will this work with other security frameworks? I feel pretty good now that these are not really our community's problems--these have had at least modest, and in some cases extensive, due diligence applied to them. And we've confirmed there's access to expertise from the security community to help out with remaining concerns there, in person even if we plan it out right. I personally suspect they've been discouraged from getting involved more by the slow pace this has been proceeding within our community and the general disarray around it, which would be understandable. The parts I do believe we have reason to be concerned are with the code integration into the PostgreSQL core, and no one has any easy answers to things like isn't this going to increase CERT advisories? and who's going to maintain this besides KaiGai? I personally feel that Steven Frost's recent comments here about how the PostgreSQL code makes this harder than it should be really cuts to the core of a next step here. The problem facing us isn't is SEPostgreSQL the right solution for providing external security checks?; it's how can the PostgreSQL code be improved so that integrating external security is easier? Looking at SEPostgreSQL is great because it really highlights where the existing set of places are. This general idea matches where thinking on things like row-level security was already going too--implement this for the database in general, then link SEPostgres in as just one provider of a security restriction. I hope the review from the BWPUG helps everyone out, and that the suggestions on the wiki for the Follow-up plan are helpful. As CF Manager, I feel we've given this patch its fair chunk of time this last month. I don't really see any options except to mark it returned with feedback yet again for now, as this CF is nearing its close and there's still plenty of open issues. My hope is that we've made progress toward answering some of the fundamental concerns that keep popping up around this patch for good, and that a plan with community members who will act on it (in this country for once) is coming together. As always, we owe KaiGai a debt for offering his code contributions, sticking through an immense amount of criticism, and suggesting ways the rest of the database might become better overall through that interaction. I do hope a committer is able to give his Largeobject access controls patch proper attention and commit it if feasible to do so. It would be nice to see confirmed progress toward the larger goal of both feature and buzzword/checkbox complete PostgreSQL security is being made through his contributions. At this point, I think someone comfortable with hacking into the PostgreSQL core will need to work on this project from that angle before even SE/PostgreSQL Lite is likely to be something we can commit. Maybe we can get KaiGai thinking in those terms instead of how he's been approaching the problem. Maybe Bruce or Steven can champion that work. I have to be honest and say that I'm not optimistic that this is possible or even a good idea to accomplish in the time remaining during this release. A patch that touches the security model in fairly fundamental ways seems like it would be much better as an alpha-1 candidate, while there's still plenty of time to shake out issues, than as a beta-1 or even alpha-3 one. And I'm skeptical that this feature really fits the true use-cases for SEPostgres without row-level filtering anyway. After our talk last night, it's obvious we need to figure out how to get that back before including the code does what people really want here. But based on looking at the market for this sort of feature, providing this new form of security integrated into the
Re: [HACKERS] Adding support for SE-Linux security
My two cents - if it's desired - I invariably disable selinux from all of my production machines. Once upon a time I tried to work with it time and time again - but it was such a head ache to administer for what I considered to be marginal gains, that I eventually gave up. Every time I add a server, it needs to be setup. Or it runs in tolerant mode at which point I'm not sure what value I am really getting at all. Too many times people have come to me with weird problems of servers not starting, or not working properly, and I have now started with the question do you have selinux running? try turning it off... I'm sure some people somewhere love selinux - but I suspect most people find the most relief once they turn it off. I vote for PostgreSQL committers spending their time on things that bring value to the most number of people. Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] CommitFest 2009-11: Reviews complete
We have a couple of lingering re-reviews that are still ongoing (myself on pgbench setshell and the two DTrace ones), and the ECPG patches are still hanging around, but we're basically finished with reviews for all submitted patches at this point. So far 12 patches have been commited since the official CF started, and there's now 8 queued up as Ready for Committer--not counting the HS/SR pair or the ones still being worked on. That means the work on part of the committers part is at best around halfway done though, so please don't start throwing new material at the list just yet. This is particularly true given that I think there's still potential for HS or SR to make it in this round; I know I'd like to see one or both available in the easier to test alpha3 build. Thanks to all our reviewers for helping cut down on the amount of work our committers have to do though. It certainly feels to me like the average quality of patches making their way through to our committers keeps itching upwards, so they spend less time fighting things like patch bit-rot and stuff that just plain doesn't work, and more time doing serious review work. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] enable-thread-safety defaults?
On tis, 2009-12-01 at 18:03 -0500, Bruce Momjian wrote: Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: 2009/12/1 Bruce Momjian br...@momjian.us: What are we going to do for build farm members who don't support threading? Is someone going to manually update their configure flags? Yeah, I think so. Unless there's a whole lot of them, in which case we revert the patch. It would seem like we ought to try the one-liner patch Magnus proposed (ie flip the default) and see what the effects are, before we go with the much larger patch Bruce wrote. OK, done --- let the breakage begin. (I will be monitoring the build farm and will work with Andrew Dunstan on any issues.) You have removed all the configure code that defined ENABLE_THREAD_SAFETY, but there is still lots and lots of code that checks for #ifdef ENABLE_THREAD_SAFETY. So this is is probably quite broken at the moment. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] thread safety on clients
On ons, 2009-12-09 at 14:02 -0500, Jaime Casanova wrote: Hi, I compiled current HEAD and trying to use pgbench, i initialized a test database this way: bin/pgbench -i -F80 -s100 test and then run with this options: bin/pgbench -c 50 -j 5 -l -t 20 test and get this crash: starting vacuum...end. TRAP: FailedAssertion(!((data - start) == data_size), File: heaptuple.c, Line: 255) Client 0 aborted in state 8. Probably the backend died while processing. LOG: server process (PID 30713) was terminated by signal 6: Aborted TRAP: FailedAssertion(!((data - start) == data_size), File: heaptuple.c, Line: 255) Client 8 aborted in state 8. Probably the backend died while processing. LOG: terminating any other active server processes WARNING: terminating connection because of crash of another server process DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. if i remove the -j option then it runs without a problem Possibly related to the incomplete removal of the enable-thread-safety option that I just posted about. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Has anyone used CLANG yet?
age...@themactionfaction.com (A.M.) writes: [Much of interest elided... Cool to see that clang clearly *can* compile PostgreSQL...] You are probably running configure with gcc, no? I was *attempting* to run configure using clang: CC=/usr/bin/clang ./configure --prefix=/home/chris/dbs/postgresql-git-head I know it's using clang, as some of the early tests indicate that specifically. checking types of arguments for accept()... configure: error: could not determine argument types It's worth noting that the problem is NOT fundamentally any Pascal-parm-passing-style issue; that's a red herring. The trouble is that it's not finding a function signature for accept(), and a number of the attempts (well, half of them...) happen to try to use Pascal parm-passing conventions. Actually, there's a little more mystery to it... I pulled out the C code from config.log that corresponds with my favorite /usr/include/sys/socket.h accept() signature, and clang is happy to compile it, even though configure logs, in config.log, that there was a mismatch. So, for some reason, configure had no problem running clang a bunch of times against *other* C fragments, but somehow didn't like how it ran this one. Presumably there's some dang GNU magic going on ;-). Thanks for verifying that the notion of compiling PostgreSQL using clang is something that in principle ought to be able to work. Perhaps this first Debian packaging of it has some deficiency, or my workstation hates me! :-). -- output = (cbbrowne @ gmail.com) The real problem with the the year 2000 is that there are too many zero bits and that adversely affects the global bit density. -- Boyd Roberts b...@france3.fr -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] tsearch parser overhaul
Kevin Grittner kevin.gritt...@wicourts.gov writes: I'm inclined that it would be no more work to replace the current recursively called state engine with something easier to read and understand than to try to fix the current oddities. Perhaps something along the lines of this?: http://vo.astronet.ru/arxiv/dict_regex.html I suspect we'd need to get it to use the same regexp code used elsewhere in PostgreSQL. We're certainly not going to start carrying two regexp engines, so yeah ;-) I guess if this is proposed as a replacement for the existing parser, we'd need to see some speed comparisons. I have no idea at all which is faster. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On tor, 2009-11-12 at 16:06 -0500, Tom Lane wrote:
There was considerable debate earlier about whether we wanted to treat
Python 3 as a separate PL so it could be available in parallel with
plpython 2, because of the user-level coding incompatibilities. It
looks like this patch simply ignores that problem. What is going to
happen to plpython functions that depend on 2.x behavior?
I have a proposal for how to handle this, and a prototype patch
attached. This follows essentially what the CPython distribution itself
does, which will make this tolerably easy to follow for users.
We install plpython as plpython2.so or plpython3.so, depending on the
version used to build it. Then, plpython.so is a symlink to
plpython2.so.
We then create three language definition templates:
plpythonu - plpython.so
plpython2u - plpython2.so
plpython3u - plpython3.so
In the far future we flip the default symlink to plpython3.so, maybe in
about 5 years when Python 2.x expires.
This gives the users the following options and scenarios:
- Existing users don't have to do anything, until maybe in five years
they will notice that their OS doesn't ship Python 2 anymore and they
will have to act anyway. In practice, by then they might have adjusted
their coding style to Python 2.6/2.7 norms and their functions will
migrate to 3.x without change.
- Users who know that they have heavily Python 2.x dependent code and
don't want to ever change it can make use of the plpython2u language
name, just like they should probably change their scripts to use
something like #!/usr/bin/python2.
- Users who want to dive into Python 3 can use the plpython3u language
name, which will basically keep working forever by today's standards.
Those users would probably also use #!/usr/bin/python3 or the like in
their scripts. In the far future they might like to remove the 3.
- Daredevils can change symlink manually and make plpython3.so the
default plpythonu implementation. Those people would probably also
make /usr/bin/python be version 3.
Comments?
diff --git a/config/python.m4 b/config/python.m4
index 9160a2b..32fff43 100644
--- a/config/python.m4
+++ b/config/python.m4
@@ -30,10 +30,12 @@ else
AC_MSG_ERROR([distutils module not found])
fi
AC_MSG_CHECKING([Python configuration directory])
+python_majorversion=`${PYTHON} -c import sys; print(sys.version[[0]])`
python_version=`${PYTHON} -c import sys; print(sys.version[[:3]])`
python_configdir=`${PYTHON} -c from distutils.sysconfig import get_python_lib as f; import os; print(os.path.join(f(plat_specific=1,standard_lib=1),'config'))`
python_includespec=`${PYTHON} -c import distutils.sysconfig; print('-I'+distutils.sysconfig.get_python_inc())`
+AC_SUBST(python_majorversion)[]dnl
AC_SUBST(python_version)[]dnl
AC_SUBST(python_configdir)[]dnl
AC_SUBST(python_includespec)[]dnl
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index ca7f499..7a6e3a9 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -171,6 +171,7 @@ python_libdir = @python_libdir@
python_libspec = @python_libspec@
python_additional_libs = @python_additional_libs@
python_configdir = @python_configdir@
+python_majorversion = @python_majorversion@
python_version = @python_version@
krb_srvtab = @krb_srvtab@
diff --git a/src/include/catalog/pg_pltemplate.h b/src/include/catalog/pg_pltemplate.h
index 8cdedb4..cbb0a33 100644
--- a/src/include/catalog/pg_pltemplate.h
+++ b/src/include/catalog/pg_pltemplate.h
@@ -73,5 +73,7 @@ DATA(insert ( pltclu f f pltclu_call_handler _null_ _null_ $libdir/pltcl
DATA(insert ( plperl t t plperl_call_handler plperl_inline_handler plperl_validator $libdir/plperl _null_ ));
DATA(insert ( plperlu f f plperl_call_handler plperl_inline_handler plperl_validator $libdir/plperl _null_ ));
DATA(insert ( plpythonu f f plpython_call_handler _null_ _null_ $libdir/plpython _null_ ));
+DATA(insert ( plpython2u f f plpython_call_handler _null_ _null_ $libdir/plpython2 _null_ ));
+DATA(insert ( plpython3u f f plpython_call_handler _null_ _null_ $libdir/plpython3 _null_ ));
#endif /* PG_PLTEMPLATE_H */
diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile
index 840b874..d11525d 100644
--- a/src/pl/plpython/Makefile
+++ b/src/pl/plpython/Makefile
@@ -36,7 +36,7 @@ override CPPFLAGS := -I$(srcdir) $(python_includespec) $(CPPFLAGS)
rpathdir = $(python_libdir)
-NAME = plpython
+NAME = plpython$(python_majorversion)
OBJS = plpython.o
@@ -56,7 +56,10 @@ endif
SHLIB_LINK = $(python_libspec) $(python_additional_libs) $(filter -lintl,$(LIBS))
-REGRESS_OPTS = --dbname=$(PL_TESTDB) --load-language=plpythonu
+REGRESS_OPTS = --dbname=$(PL_TESTDB)
+ifeq ($(python_majorversion),2)
+REGRESS_OPTS += --load-language=plpythonu
+endif
REGRESS = \
plpython_schema \
plpython_populate \
@@ -83,10 +86,16 @@ include $(top_srcdir)/src/Makefile.shlib
all: all-lib
install: all installdirs install-lib
+ifeq ($(python_majorversion),2)
+ cd
Re: [HACKERS] unprivileged user
On tor, 2009-12-10 at 17:20 -0500, Andrew Dunstan wrote: Some time later it came up again, this time when someone wanted to use a readonly database (hence no pg_dump required) with an application and wanted to keep the database layout and the source code of stored functions hidden as they regarded it as proprietary information. Well, the information schema already implements a policy of this sort, because it only shows information about things you have some kind of access to. (I assume you are allowed to know about the things you have access to.) The problem in this sort of scheme is always that the system catalogs are world readable, and changing that would break about every tool and driver in existence. It's not clear how to fix that, at least not without row-level security. Or how did your old patch address this? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Has anyone used CLANG yet?
On ons, 2009-12-09 at 16:23 -0500, Chris Browne wrote:
This is a C front end for the LLVM compiler... I noticed that it
entered Debian/Unstable today:
http://packages.debian.org/sid/main/clang
I thought it would be interesting to see if PostgreSQL compiles with
this, as an alternative compiler that should presumably become more and
more available on Linux et al. (And I suppose that the randomly
selected .sig is supremely apropos!)
configure blows up here at the following:
conftest.c:75:28: error: invalid token after top level declarator
extern unsigned int PASCAL accept (unsigned int, void *, void *);
I suspect there's something about PASCAL that's a problem, as clang is
nominally supposed to be a C compiler ;-).
I haven't looked deeper, so haven't the remotest idea how deep the issue
lies.
The problem is the -D_GNU_SOURCE in src/template/linux. This bit
from /usr/include/sys/socket.h would appear to be the explanation.
Apparently CLANG claims to be GCC-something-recent but does not
implement this transparent-union hocus pocus in quite the same way. If
you don't use _GNU_SOURCE, then it uses the #define version and the
configure test passes.
/* This is the type we use for generic socket address arguments.
With GCC 2.7 and later, the funky union causes redeclarations or
uses with any of the listed types to be allowed without complaint.
G++ 2.7 does not support transparent unions so there we want the
old-style declaration, too. */
#if defined __cplusplus || !__GNUC_PREREQ (2, 7) || !defined __USE_GNU
# define __SOCKADDR_ARG struct sockaddr *__restrict
# define __CONST_SOCKADDR_ARG __const struct sockaddr *
#else
/* Add more `struct sockaddr_AF' types here as necessary.
These are all the ones I found on NetBSD and Linux. */
# define __SOCKADDR_ALLTYPES \
__SOCKADDR_ONETYPE (sockaddr) \
__SOCKADDR_ONETYPE (sockaddr_at) \
__SOCKADDR_ONETYPE (sockaddr_ax25) \
__SOCKADDR_ONETYPE (sockaddr_dl) \
__SOCKADDR_ONETYPE (sockaddr_eon) \
__SOCKADDR_ONETYPE (sockaddr_in) \
__SOCKADDR_ONETYPE (sockaddr_in6) \
__SOCKADDR_ONETYPE (sockaddr_inarp) \
__SOCKADDR_ONETYPE (sockaddr_ipx) \
__SOCKADDR_ONETYPE (sockaddr_iso) \
__SOCKADDR_ONETYPE (sockaddr_ns) \
__SOCKADDR_ONETYPE (sockaddr_un) \
__SOCKADDR_ONETYPE (sockaddr_x25)
# define __SOCKADDR_ONETYPE(type) struct type *__restrict __##type##__;
typedef union { __SOCKADDR_ALLTYPES
} __SOCKADDR_ARG __attribute__ ((__transparent_union__));
# undef __SOCKADDR_ONETYPE
# define __SOCKADDR_ONETYPE(type) __const struct type *__restrict
__##type##__;
typedef union { __SOCKADDR_ALLTYPES
} __CONST_SOCKADDR_ARG __attribute__
((__transparent_union__));
# undef __SOCKADDR_ONETYPE
#endif
Unfortunately, removing _GNU_SOURCE currently breaks the build because
it masks the definition of struct ucred from the headers. That could be
fixed with more autoconfigury. And it breaks PL/Perl, as the note there
says.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Viewing table data only from its corresponding oid-named file
Hi Guys I have some files that were part of a database, I mean they were under /*data/base/DB_OID/table_oid* postgres structure As I understand *table_oid *which is a numbered file corresponds to one table in the original database. Here is my question: Is there a tool/method to view the data in this file so I can get a clue of what the data in that table was? We had a major drive crash and we could recover only a few of those files and want to know if we have a chance to recover some data only from those kind of files. Thanks in advance. -- Heiner Vega
Re: [HACKERS] unprivileged user
Peter Eisentraut wrote: On tor, 2009-12-10 at 17:20 -0500, Andrew Dunstan wrote: Some time later it came up again, this time when someone wanted to use a readonly database (hence no pg_dump required) with an application and wanted to keep the database layout and the source code of stored functions hidden as they regarded it as proprietary information. Well, the information schema already implements a policy of this sort, because it only shows information about things you have some kind of access to. (I assume you are allowed to know about the things you have access to.) The problem in this sort of scheme is always that the system catalogs are world readable, and changing that would break about every tool and driver in existence. It's not clear how to fix that, at least not without row-level security. Or how did your old patch address this? Well, it will certainly break plenty of things. But it didn't prevent the client from doing the things we wanted to allow it to do, i.e. make a few function calls. Basically what I did was to revoke public privileges from just about everything I could lay my hands on and regrant the same privileges to a group called pseudopublic, which contained every user but the one I wanted to restrict. The client I tested with was psql, and I recall being surprised that I was still able to do what I wanted - I had thought I would break things completely. I haven't tried to repeat the experiment recently, though. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Installing PL/pgSQL by default
Dimitri Fontaine wrote: Tom Lane t...@sss.pgh.pa.us writes: It's not impossible that we'll have to tweak pg_dump a bit; it's never had to deal with languages that shouldn't be dumped ... Ah, the best would be to have extensions maybe. Then you could do this in initdb, filling in template0: CREATE EXTENSION plpgsql ...; Then at createdb time, what would become automatic is: INSTALL EXTENSION plpgsql; And that's it. pg_dump would now about extensions and only issues this latter statement in its dump. Bruce, there are some mails in the archive with quite advanced design proposal that has been discussed and not objected to, and I even provided a rough sketch of how I wanted to attack the problem. http://archives.postgresql.org/pgsql-hackers/2009-06/msg01281.php http://archives.postgresql.org/pgsql-hackers/2009-07/msg01425.php http://archives.postgresql.org/pgsql-hackers/2009-07/msg01468.php The major version dependant SQL code is now much less of a problem than before because we have inline DO statements. So you don't need to create a function for this anymore. Real life kept me away from having the time to prepare the code patch, and I don't think that will change a bit in the 8.5 release cycle, whatever my hopes were earlier this year. But having everyone talk about the feature and come to an agreement as to what it should provide and how was the hard part of it, I think, and is now about done. Would you be up for writing the extension facility? Uh, well, I need to help with the patch commit process at this point --- if I find I have extra time, I could do it. I will keep this in mind. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] thread safety on clients
Peter Eisentraut wrote: starting vacuum...end. TRAP: FailedAssertion(!((data - start) == data_size), File: heaptuple.c, Line: 255) Client 0 aborted in state 8. Probably the backend died while processing. LOG: server process (PID 30713) was terminated by signal 6: Aborted TRAP: FailedAssertion(!((data - start) == data_size), File: heaptuple.c, Line: 255) Client 8 aborted in state 8. Probably the backend died while processing. LOG: terminating any other active server processes WARNING: terminating connection because of crash of another server process DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. if i remove the -j option then it runs without a problem Possibly related to the incomplete removal of the enable-thread-safety option that I just posted about. I thought about that but I can't figure out how that would affect pgbench. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Viewing table data only from its corresponding oid-named file
Heiner Vega Thames escribió: Hi Guys I have some files that were part of a database, I mean they were under /*data/base/DB_OID/table_oid* postgres structure As I understand *table_oid *which is a numbered file corresponds to one table in the original database. Here is my question: Is there a tool/method to view the data in this file so I can get a clue of what the data in that table was? We had a major drive crash and we could recover only a few of those files and want to know if we have a chance to recover some data only from those kind of files. It's rather difficult. Try pgfsck and/or pg_filedump. Decoding the data in them may be very hard -- if you are able to reconstruct the tables with all columns, it might work, but it will still be very difficult. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Andrew Dunstan wrote:
Tom Lane wrote:
regression=# select
E'xxx\nxx\n'
as a, 1 as b;
a | b
--+---
xxx +| 1
xx +|
|
(1 row)
regression=#
The point here is exactly that previous versions didn't show the
distinction well.
If we really want to make linefeeds visible, I think we should place the
indicators immediately after the character preceding the line feed, not
next to the column separator.
One idea would be to change the column _separator_ for newlines --- that
way, they don't show up for single-column output.
--
Bruce Momjian br...@momjian.ushttp://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] thread safety on clients
Bruce Momjian wrote: Peter Eisentraut wrote: if i remove the -j option then it runs without a problem Possibly related to the incomplete removal of the enable-thread-safety option that I just posted about. I thought about that but I can't figure out how that would affect pgbench. The -j option is the recent addition to pgbench that causes it to launch multiple client threads when enabled, each handling a subset of the transactions. There's blocks of codes in pgbench.c now that depend on having sane values for thread safety in libpq. That it may be detecting the wrong thing and operating in an unsafe way after the recent change is what Peter's suggesting. This is good, actually, because I don't think we had many client-side thread-safety tests floating around to catch problems in this area before. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com
Re: [HACKERS] Largeobject Access Controls (r2460)
KaiGai Kohei kai...@ak.jp.nec.com wrote:
we still allow SELECT * FROM pg_largeobject ...right?
It can be solved with revoking any privileges from anybody in the initdb
phase. So, we should inject the following statement for setup_privileges().
REVOKE ALL ON pg_largeobject FROM PUBLIC;
OK, I'll add the following description in the documentation of pg_largeobject.
structnamepg_largeobject/structname should not be readable by the
public, since the catalog contains data in large objects of all users.
structnamepg_largeobject_metadata/ is a publicly readable catalog
that only contains identifiers of large objects.
{lo_compat_privileges, PGC_SUSET, COMPAT_OPTIONS_PREVIOUS,
gettext_noop(Turn on/off privilege checks on large objects.),
The description is true, but gives a confusion because
lo_compat_privileges = on means privilege checks are turned off.
short desc: Enables backward compatibility in privilege checks on large
objects
long desc: When turned on, privilege checks on large objects are disabled.
Are those descriptions appropriate?
Regards,
---
Takahiro Itagaki
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Tom Lane wrote: My guess is that a credible SEPostgres offering will require a long-term amount of work at least equal to, and very possibly a good deal more than, what it took to make a native Windows port. Wow, if I thought that was the case I'd be as negative about the whole thing as you obviously are. In my head, I've been mentally bounding the effort by thinking that its worst case work would be more like what it took to add the role-based security to the system. I'd think that adding a new feature to the existing security setup couldn't be more painful than adding security in the first place, right? I didn't carefully watch either play out , but I was under the impression that the Windows port was quite a bit more work than that. Since the current discussion keeps going around in circles, the way I was trying to tilt the other thread I started towards was asking the question what would need to change in the current PostgreSQL code to make the impact of adding the SEPostgreSQL code smaller? I'd be curious to hear any thoughts you had on that topic. We already sort of refactored out adding row-level security as one answer to that, I feel like there may be others in there too. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] thread safety on clients
Greg Smith wrote: Bruce Momjian wrote: Peter Eisentraut wrote: if i remove the -j option then it runs without a problem Possibly related to the incomplete removal of the enable-thread-safety option that I just posted about. I thought about that but I can't figure out how that would affect pgbench. The -j option is the recent addition to pgbench that causes it to launch multiple client threads when enabled, each handling a subset of the transactions. There's blocks of codes in pgbench.c now that depend on having sane values for thread safety in libpq. That it may be detecting the wrong thing and operating in an unsafe way after the recent change is what Peter's suggesting. This is good, actually, because I don't think we had many client-side thread-safety tests floating around to catch problems in this area before. I can reproduce the crash here so I can see if I can find the cause. However, the failure is happening in the _server_. Threading is unrelated to the server itself, only the client. I suppose the first test for me will be to test CVS before the thread change was made. The failure is in heap_fill_tuple(), and I am unclear how that assert could be getting triggered: CONTEXT: automatic analyze of table test.public.pgbench_accounts TRAP: FailedAssertion(!((data - start) == data_size), File: heaptuple.c, Line: 255) TRAP: FailedAssertion(!((data - start) == data_size), File: heaptuple.c, Line: 255) TRAP: FailedAssertion(!((data - start) == data_size), File: heaptuple.c, Line: 255) LOG: server process (PID 6076) was terminated by signal 6: Abort trap LOG: terminating any other active server processes -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
Tom Lane wrote: Greg Smith g...@2ndquadrant.com writes: To be clear about which version we're talking about: http://archives.postgresql.org/message-id/20091130123456.4a03.52131...@oss.ntt.co.jp is the candidate for commit that includes the cleanup you've already done. I suppose this is subject to the same auto_explain problem already noticed for JSON, but I would suggest that we commit this first and then fix both together, rather than create merge issues. OK, I've committed the YAML stuff, so who is working on the auto-explain bug? Robert? I'd like that fixed before I add in the query text to auto-explain output. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
On Thu, Dec 10, 2009 at 5:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: If I thought that Bruce could go off in a corner and make this happen and it would create no demands on anybody but him and KaiGai-san, I would say fine, if that's where you want to spend your time, go for it. But even to state that implied claim is to see how false it is. Bruce is pointing to the Windows port, but he didn't make it happen by himself, or any close approximation of that. Everybody who works on this project has been affected by that, and we're *still* putting significant amounts of time into Windows compatibility, over five years later. This is also one of my concerns. Bruce has been careful to say that he will either make this happen himself or find others to help. The thing is, who are the others, are they people we already trust, and how do we know whether they'll be around after this is committed? I'm excited to see Greg Smith getting more involved in dealing with this patch-set, and I know Stephen Frost did some reviewing as well, but overall the community support has been pretty limpid. It's probably impossible to completely eliminate the impact of this feature on the community, but having a core of involved people - preferably including several committers - who will maintain it would help a lot. We're not there yet. My guess is that a credible SEPostgres offering will require a long-term amount of work at least equal to, and very possibly a good deal more than, what it took to make a native Windows port. If SEPostgres could bring us even 10% as many new users as the Windows port did, it'd probably be a worthwhile use of our resources. But again, that's an assumption that's difficult to type without bursting into laughter. The SEPostgres community is surely a lot smaller than the Windows community, but I'm not sure whether the effort estimate is accurate or not. If credible includes row-level security, then I think I might agree, but right now we're just trying to get off the ground. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] enable-thread-safety defaults?
Peter Eisentraut wrote: On tis, 2009-12-01 at 18:03 -0500, Bruce Momjian wrote: Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: 2009/12/1 Bruce Momjian br...@momjian.us: What are we going to do for build farm members who don't support threading? Is someone going to manually update their configure flags? Yeah, I think so. Unless there's a whole lot of them, in which case we revert the patch. It would seem like we ought to try the one-liner patch Magnus proposed (ie flip the default) and see what the effects are, before we go with the much larger patch Bruce wrote. OK, done --- let the breakage begin. (I will be monitoring the build farm and will work with Andrew Dunstan on any issues.) You have removed all the configure code that defined ENABLE_THREAD_SAFETY, but there is still lots and lots of code that checks for #ifdef ENABLE_THREAD_SAFETY. So this is is probably quite broken at the moment. Yep, you have identified the bug, and I have applied the proper patch, thanks. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] thread safety on clients
Bruce Momjian wrote: Greg Smith wrote: Bruce Momjian wrote: Peter Eisentraut wrote: if i remove the -j option then it runs without a problem Possibly related to the incomplete removal of the enable-thread-safety option that I just posted about. I thought about that but I can't figure out how that would affect pgbench. The -j option is the recent addition to pgbench that causes it to launch multiple client threads when enabled, each handling a subset of the transactions. There's blocks of codes in pgbench.c now that depend on having sane values for thread safety in libpq. That it may be detecting the wrong thing and operating in an unsafe way after the recent change is what Peter's suggesting. This is good, actually, because I don't think we had many client-side thread-safety tests floating around to catch problems in this area before. I can reproduce the crash here so I can see if I can find the cause. However, the failure is happening in the _server_. Threading is unrelated to the server itself, only the client. I suppose the first test for me will be to test CVS before the thread change was made. The failure is in heap_fill_tuple(), and I am unclear how that assert could be getting triggered: CONTEXT: automatic analyze of table test.public.pgbench_accounts TRAP: FailedAssertion(!((data - start) == data_size), File: heaptuple.c, Line: 255) TRAP: FailedAssertion(!((data - start) == data_size), File: heaptuple.c, Line: 255) TRAP: FailedAssertion(!((data - start) == data_size), File: heaptuple.c, Line: 255) LOG: server process (PID 6076) was terminated by signal 6: Abort trap LOG: terminating any other active server processes OK, turns out I did break threading by not defining ENABLE_THREAD_SAFETY in configure. I have applied a patch to CVS to fix this. However, the larger question is how did I crash a backend by not defining this? I guess it is possible for a client to crash the server by having a misconfigured client --- I wasn't aware of that. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
Robert Haas robertmh...@gmail.com wrote:
On Thu, Dec 10, 2009 at 10:52 AM, Greg Smith g...@2ndquadrant.com wrote:
I don't think IO is a terrible name for an option but I like BUFFERS
better. ?I don't think the BUFFERS/BLOCKS confusion is too bad, but
perhaps we could use BUFFERS in both places.
I don't know how blocks got into here in the first place--this concept is
buffers just about everywhere else already, right?
I think we have some places already in the system where we bounce back
and forth between those terms. I expect that's the reason.
The blocks comes from pg_statio_all_tables.heap_blks_{read|hit},
but buffers might be easy to understand. One matter for concern
is that buffer read might not be clear whether it is a memory access
or a disk read.
Anyway, a revised patch according to the comments is attached.
The new text format is:
Buffers: shared hit=675 read=968, temp read=1443 written=1443
* Zero values are omitted. (Non-text formats could have zero values.)
* Rename Blocks: to Buffers:.
* Remove parentheses and add a comma between shared, local and temp.
=# EXPLAIN (BUFFERS, ANALYZE) SELECT *
FROM pgbench_accounts a, pgbench_branches b
WHERE a.bid = b.bid AND abalance = 0 ORDER BY abalance;
QUERY PLAN
---
Sort (cost=54151.83..54401.83 rows=10 width=461) (actual
time=92.551..109.646 rows=10 loops=1)
Sort Key: a.abalance
Sort Method: external sort Disk: 11544kB
Buffers: shared hit=675 read=968, temp read=1443 written=1443
- Nested Loop (cost=0.00..4141.01 rows=10 width=461) (actual
time=0.048..42.190 rows=10 loops=1)
Join Filter: (a.bid = b.bid)
Buffers: shared hit=673 read=968
- Seq Scan on pgbench_branches b (cost=0.00..1.01 rows=1 width=364)
(actual time=0.003..0.004 rows=1 loops=1)
Buffers: shared hit=1
- Seq Scan on pgbench_accounts a (cost=0.00..2890.00 rows=10
width=97) (actual time=0.038..22.912 rows=10 loops=1)
Filter: (a.abalance = 0)
Buffers: shared hit=672 read=968
Total runtime: 116.058 ms
(13 rows)
Regards,
---
Takahiro Itagaki
NTT Open Source Software Center
explain_buffers_20091211.patch
Description: Binary data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
On Thu, Dec 10, 2009 at 8:39 PM, Andrew Dunstan and...@dunslane.net wrote:
OK, I've committed the YAML stuff, so who is working on the auto-explain
bug? Robert? I'd like that fixed before I add in the query text to
auto-explain output.
I'm going to propose fixing this in what seems to me to be the
simplest possible way, per the attached. Anyone want to argue?
...Robert
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 75ac9ca..f0d907d 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -223,7 +223,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es.verbose = auto_explain_log_verbose;
es.format = auto_explain_log_format;
+ ExplainBeginOutput(es);
ExplainPrintPlan(es, queryDesc);
+ ExplainEndOutput(es);
/* Remove last line break */
if (es.str-len 0 es.str-data[es.str-len - 1] == '\n')
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 0437ffa..2067636 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -91,8 +91,6 @@ static void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
static void ExplainDummyGroup(const char *objtype, const char *labelname,
ExplainState *es);
-static void ExplainBeginOutput(ExplainState *es);
-static void ExplainEndOutput(ExplainState *es);
static void ExplainXMLTag(const char *tagname, int flags, ExplainState *es);
static void ExplainJSONLineEnding(ExplainState *es);
static void ExplainYAMLLineStarting(ExplainState *es);
@@ -1791,7 +1789,7 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
* This is just enough different from processing a subgroup that we need
* a separate pair of subroutines.
*/
-static void
+void
ExplainBeginOutput(ExplainState *es)
{
switch (es-format)
@@ -1822,7 +1820,7 @@ ExplainBeginOutput(ExplainState *es)
/*
* Emit the end-of-output boilerplate.
*/
-static void
+void
ExplainEndOutput(ExplainState *es)
{
switch (es-format)
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index ab48825..ba2ba08 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -65,6 +65,8 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, ExplainState *es,
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainBeginOutput(ExplainState *es);
+extern void ExplainEndOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
#endif /* EXPLAIN_H */
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL/Lite Review
Greg Smith wrote: It's funny; we started out this CommitFest with me scrambling to find someone, anyone, willing to review the latest SE-PostgreSQL patch, knowing it was a big job and few were likely to volunteer. Then schedules lined up just right, and last night I managed to get a great group of people all together to do perhaps the biggest single patch review ever, to work on just that.I gathered up a list of the biggest concerns about this feature and its associated implementation, we got a number of regular PostgreSQL hackers and two of the security guys you've seen on this list all in the same room, and we talked about little but SEPostgreSQL for hours. Minutes are at http://wiki.postgresql.org/wiki/SEPostgreSQL_Review_at_the_BWPUG and I'd suggest anyone interested in this feature (or in rejecting this feature) to take a look at what we covered. I repent that I cannot attend here. The wikipage is a good summarize. Thanks for your efforts. There's comments there, with references for you [citation needed] types, to help answer four of the most common complaints I've seen on this list about this feature: -Is there really a user base for it? -Has it been audited by security professionals? -Is there a useful SELinux policty to support it? -Will this work with other security frameworks? I feel pretty good now that these are not really our community's problems--these have had at least modest, and in some cases extensive, due diligence applied to them. And we've confirmed there's access to expertise from the security community to help out with remaining concerns there, in person even if we plan it out right. I personally suspect they've been discouraged from getting involved more by the slow pace this has been proceeding within our community and the general disarray around it, which would be understandable. IMO, the slow pace is not a primary reason. In fact, SELinux was released at 2000 Dec, but it gets integtated into the mainline kernel at 2003 Aug with various kind of improvements. It takes about 3 years from the first relase. IIRC, now we take 2.5 years from the first announce of SE-PgSQL in this list, and various kind of improvements and cleanups had been done. It is a bit long term, but not too long term. The reason of this gap is that people have individual consciousness about their security. I often represent it as security is a subjective term. Needless to say, we don't have magic-bullets for any threats. Any technology has its metirs and limitations. However, people tend to say it is nonsense, if the feature does not match with their recognizable demands or threats. Security-folks know MAC is not magic-bullets, while it is a significant piece of system security. But some of complaints might be pointless for security experts, so had been dismotivated. From the perspective of security folks, we have to introduce it doggedly. And, I'd like to understand database folks there are various kind of security models and viewpoints here. SELinux is one of them. The parts I do believe we have reason to be concerned are with the code integration into the PostgreSQL core, and no one has any easy answers to things like isn't this going to increase CERT advisories? and who's going to maintain this besides KaiGai? I personally feel that Steven Frost's recent comments here about how the PostgreSQL code makes this harder than it should be really cuts to the core of a next step here. The problem facing us isn't is SEPostgreSQL the right solution for providing external security checks?; it's how can the PostgreSQL code be improved so that integrating external security is easier? Looking at SEPostgreSQL is great because it really highlights where the existing set of places are. This general idea matches where thinking on things like row-level security was already going too--implement this for the database in general, then link SEPostgres in as just one provider of a security restriction. Right, it seems to me the security provider is a good phrase to represent this feature. It just provides additional access control decisions based on the different perspective of security model. Please note that the access control granularity is not an essential issue. We can also assume table-level mandatory access controls for instance. The latest patch provides table/column level controls without row-level, because the current PgSQL has facilities to know what tables and columns are referenced reasonably, so SE-PgSQL also can know what tables/columns are referenced without special tricks. Please remind the earlier SE-PgSQL in v8.2.x. It walked on the Query tree to pick up what columns were accessed. I hope the review from the BWPUG helps everyone out, and that the suggestions on the wiki for the Follow-up plan are helpful. As CF Manager, I feel we've given this patch its fair chunk of time this last month. I don't really
Re: [HACKERS] Adding support for SE-Linux security
David P. Quigley wrote: On Thu, 2009-12-10 at 17:08 -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Unlike Tom (I think), I do believe that there is demand (possibly only from a limited number of people, but demand all the same) for this feature. Please note that I do not think there is *zero* demand for the feature. There is obviously some. What I find highly dubious is whether there is enough demand to justify the amount of effort, both short- and long-term, that the community would have to put into it. And I also believe that most people in our community are generally supportive of the idea, but only a minority are willing to put in time to make it happen. So I have no problem saying to the people who want the feature - none of our committers feel like working on this. Sorry. On the other hand, I also have no problem telling them - good news, Bruce Momjian thinks this is a great feature and wants to help you get it done. I *do* have a problem with saying - we don't really know whether anyone will ever want to work on this with you or not. If I thought that Bruce could go off in a corner and make this happen and it would create no demands on anybody but him and KaiGai-san, I would say fine, if that's where you want to spend your time, go for it. But even to state that implied claim is to see how false it is. Bruce is pointing to the Windows port, but he didn't make it happen by himself, or any close approximation of that. Everybody who works on this project has been affected by that, and we're *still* putting significant amounts of time into Windows compatibility, over five years later. My guess is that a credible SEPostgres offering will require a long-term amount of work at least equal to, and very possibly a good deal more than, what it took to make a native Windows port. If SEPostgres could bring us even 10% as many new users as the Windows port did, it'd probably be a worthwhile use of our resources. But again, that's an assumption that's difficult to type without bursting into laughter. regards, tom lane So a couple of us in the Maryland/DC area went to the BWPUG meeting last night and we sat down for two hours and answered a bunch of questions from Greg Smith, Steve Frost, and a few others. Greg was taking notes during the entire meeting and I believe he will be starting a thread with the minutes from the meeting. Greg brought up 5 or 6 concerns that he has observed in the community about the work including the issue of who is going to use this. The minutes will give a much better account of the conversation but Josh Brindle and I have gave examples outside of DoD where the MAC framework without row based access controls can be useful. For our purposes in DoD we need the MAC Framework and the row based access controls but if a good starting point is to just do the access control over the database objects then it will be useful for some commercial cases and some limited military cases. I repent that I live in behind of the earth. :( I'd like to introduce a story related to Maryland/Baltimore where is the first city I've visited in US a bit. The SELinux symposium and developers summit had been held in Baltimore between 2005 and 2007. (It has been held with LinuxCon at Portland/OR in recent years.) I also had a short (works-in-progress) session in the symposium of 2007 to introduce an early concept and design of SE-PostgreSQL. http://selinux-symposium.org/2007/wipsbofs.php#sepostgresql After the 20 minutes talks, I was encircled by several stalwart-guys and pestered with questions about its behavior and so on. He also gave me a contact address in .mil domain. It was the first experience for me to see this domain actually. Maybe, we cannot see these people in PGcon. What I want to say in this story is that our domain of audiences depends on our standpoint. If eyesight of developers cannot catch their figures, we may misunderstand actual voice and demands from (potential) users. However, it is *never* easy job. Please remind how much cost our company have spent on marketing research annually. Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
Robert Haas robertmh...@gmail.com wrote: On Thu, Dec 10, 2009 at 8:39 PM, Andrew Dunstan and...@dunslane.net wrote: OK, I've committed the YAML stuff, so who is working on the auto-explain bug? Robert? I'm going to propose fixing this in what seems to me to be the simplest possible way, per the attached. Anyone want to argue? +1 to the fix. Typical usage of explain functions will be: 1. ExplainInitState() 2. (setup ExplainState) 3. ExplainBeginOutput() 4. ExplainXXX() except ExplainQuery() 5. ExplainEndOutput() Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
On Thu, Dec 10, 2009 at 8:11 PM, Bruce Momjian br...@momjian.us wrote: Andrew Dunstan wrote: Tom Lane wrote: regression=# select E'xxx\nxx\n' as a, 1 as b; a | b --+--- xxx +| 1 xx +| | (1 row) regression=# The point here is exactly that previous versions didn't show the distinction well. If we really want to make linefeeds visible, I think we should place the indicators immediately after the character preceding the line feed, not next to the column separator. One idea would be to change the column _separator_ for newlines --- that way, they don't show up for single-column output. Hilariously enough, that's exactly what we used to do. I am leaning toward the view that we should revert all the ASCII output format changes vs. 8.4 and let people use the new unicode mode if they want all the spiffy bells and whistles. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Largeobject Access Controls (r2460)
Takahiro Itagaki wrote:
KaiGai Kohei kai...@ak.jp.nec.com wrote:
we still allow SELECT * FROM pg_largeobject ...right?
It can be solved with revoking any privileges from anybody in the initdb
phase. So, we should inject the following statement for setup_privileges().
REVOKE ALL ON pg_largeobject FROM PUBLIC;
OK, I'll add the following description in the documentation of pg_largeobject.
structnamepg_largeobject/structname should not be readable by the
public, since the catalog contains data in large objects of all users.
structnamepg_largeobject_metadata/ is a publicly readable catalog
that only contains identifiers of large objects.
It makes sense to me.
{lo_compat_privileges, PGC_SUSET, COMPAT_OPTIONS_PREVIOUS,
gettext_noop(Turn on/off privilege checks on large objects.),
The description is true, but gives a confusion because
lo_compat_privileges = on means privilege checks are turned off.
short desc: Enables backward compatibility in privilege checks on large
objects
long desc: When turned on, privilege checks on large objects are disabled.
Are those descriptions appropriate?
The long description is a bit confusable, because it does not disable all the
privilege checks, such as lo_export/lo_import which already checks superuser
privilege on execution in the earlier releases.
What's your opinion about:
long desc: When turned on, privilege checks on large objects perform with
backward compatibility as 8.4.x or earlier releases.
Thanks,
--
OSS Platform Development Division, NEC
KaiGai Kohei kai...@ak.jp.nec.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain output infelicity in psql
Robert Haas robertmh...@gmail.com writes: Hilariously enough, that's exactly what we used to do. I am leaning toward the view that we should revert all the ASCII output format changes vs. 8.4 and let people use the new unicode mode if they want all the spiffy bells and whistles. There are clearly use-cases for this feature; I think arguing to revert it is an extreme overreaction. We could reconsider which behavior ought to be default, perhaps. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL/Lite Review
* Greg Smith (g...@2ndquadrant.com) wrote: I personally feel that Steven Frost's recent comments here about how the PostgreSQL code makes this harder than it should be really cuts to the core of a next step here. The problem facing us isn't is SEPostgreSQL the right solution for providing external security checks?; it's how can the PostgreSQL code be improved so that integrating external security is easier? Looking Thanks for that support, Greg. This was what I was principally trying to do with KaiGai and the commitfest patch I reviewed of his last round. Unfortunately, the committer comments I received on that patch didn't help us outline a path forward, just declared that the approach in the current patch wasn't workable. My, now much more optimistic thanks to our meeting, view is that the concept of abstracting the access controls is solid and a necessary first step; but we need to find a better way to implement it. Also thanks to our discussion, I've got a much better handle on how SELinux and the general secuirty community views PostgreSQL (and the Linux kernel for that matter)- it's an application which consists of a set of object managers. That then leads into an approach to address at least some of Tom's comments: Let's start by taking the patch I reviewed and splitting up security/access_control.c along object lines. Of course, the individual security/object_ac.c files would only include the .h's that are necessary. This would be a set of much smaller and much more managable files which only know about what they should know about- the object type they're responsible for. Clearly, code comments also need to be reviewed and issues with them addressed. I'm also not a fan of the skip permissions check arguments, which are there specifically to address cascade deletions, which is a requirment of our PG security model. I'd love to see a better solution and am open to suggestions or thoughts about what one would be. I know one of KaiGai's concerns in this area was performance, butas we often do for PG, perhaps we should consider that second and first consider what it would look like if we ignored performance concerns. This would change skip permissions check to what object is being deleted, and am I a sub-object of that?. I don't feel like I've explained that well enough, perhaps someone else could come up with another solution, or maybe figure out a better way to describe what I'm trying to get with this. Regarding the special-purpose shims- I feel there should be a way for us to handle them better. This might be a good use-case for column-level privileges in pg_catalog. That's pure speculation at the moment tho, I havn't re-looked at those shims lately to see if that even makes sense. I don't like them either though, for what it's worth. Regarding contrib modules- I view them as I do custom code which the user writes and loads through dlopen() into the backend process- there should be a way for it to do security right, but it ultimately has to be the responsibility of the contrib module. Admins can review what contrib modules have been made SELinux/etc aware and choose to install only those which they trust. Maybe Bruce or Steven can champion that work. See above? ;) I've had enough of a break from this and our discussion has revitalized me. I certainly welcome Bruce's comments, as well as anyone else. I have to be honest and say that I'm not optimistic that this is possible or even a good idea to accomplish in the time remaining during this release. While I agree with you, I wish you hadn't brought it up. :) Mostly because I feel like it may discourage people from wanting to spend time on it due to desire to focus on things which are likely to make it into the next release. That being said, I don't feel that'll be an issue for KaiGai or myself, but getting someone beyond us working on this would really be great, especially yourself and/or Bruce. On my side, in addition to helping coordinate everyone pushing in the same direction, I'll also continue trying to shake out some sponsorship funding for additional work out of the people in this country it would benefit. It seems I'm going to keep getting pulled into the middle of this area regularly anyway. Thank you for that. I'm planning to do the same and will certainly let people know, to the extent possible, of anything I'm able to dig up. Thanks! Stephen signature.asc Description: Digital signature
Re: [HACKERS] Largeobject Access Controls (r2460)
KaiGai Kohei kai...@ak.jp.nec.com wrote: What's your opinion about: long desc: When turned on, privilege checks on large objects perform with backward compatibility as 8.4.x or earlier releases. I updated the description as your suggest. Applied with minor editorialization, mainly around tab-completion support in psql. # This is my first commit :) Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL/Lite Review
Stephen Frost wrote: * Greg Smith (g...@2ndquadrant.com) wrote: I have to be honest and say that I'm not optimistic that this is possible or even a good idea to accomplish in the time remaining during this release. While I agree with you, I wish you hadn't brought it up. :) Mostly because I feel like it may discourage people from wanting to spend time on it due to desire to focus on things which are likely to make it into the next release. The reason I mentioned it is that I was getting a general feeling that the community at large was going to view not getting the patch into 8.5 as a final straw for working on it. I wanted to be clear that the scope of doing this right may extend beyond that, but that shouldn't be a reason to give up on it. I think KaiGai's comments about how it took 3 years to get SELinux similarly integrated into the Linux core rather then shipping as an add-on is encouraging, in that the way and scope of how we're struggling with this topic is not something we should consider extraordinary and therefore a failing. This is not an easy thing to do for anyone who tries it. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] thread safety on clients
Greg Smith g...@2ndquadrant.com writes: The -j option is the recent addition to pgbench that causes it to launch multiple client threads when enabled, each handling a subset of the transactions. There's blocks of codes in pgbench.c now that depend on having sane values for thread safety in libpq. That it may be detecting the wrong thing and operating in an unsafe way after the recent change is what Peter's suggesting. This is good, actually, because I don't think we had many client-side thread-safety tests floating around to catch problems in this area before. The report showed an assert inside the backend. It really doesn't matter *how* broken pgbench might be, it should not be able to cause that. My bet is that the real problem was a build inconsistency in the backend. Does make distclean and rebuild make it go away? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Largeobject Access Controls (r2460)
Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp writes: OK, I'll add the following description in the documentation of pg_largeobject. structnamepg_largeobject/structname should not be readable by the public, since the catalog contains data in large objects of all users. This is going to be a problem, because it will break applications that expect to be able to read pg_largeobject. Like, say, pg_dump. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
