[HACKERS] Query from VMWare re use of PostgreSQL in new TPC-V benchmark
Heya Just in case someone missed it on -perform and is interested, a couple of questions have come in from VMWare, who want to use PostgreSQL as the default reference implementation for a new TPC benchmark, TPC-V. http://archives.postgresql.org/message-id/66ce997fb523c04e9749452273184c6c137cb88...@exch-mbx-113.vmware.com http://archives.postgresql.org/message-id/66ce997fb523c04e9749452273184c6c137cb88...@exch-mbx-113.vmware.com Exciting stuff IMO; the TPC has always been a PITA due to its reliance on closed implementations and its difficult licensing, and this looks like a nice chance to help change that. -- Craig Ringer
Re: [HACKERS] Allow WAL information to recover corrupted pg_controldata
From: Robert Haas [mailto:robertmh...@gmail.com] Sent: Friday, June 22, 2012 8:59 PM On Fri, Jun 22, 2012 at 5:25 AM, Amit Kapila wrote: >> Based on the discussion and suggestions in this mail chain, following features can be implemented: >> >> 1. To compute the value of max LSN in data pages based on user input whether he wants it for an individual file, >> a particular directory or whole database. >> >> 2a. To search the available WAL files for the latest checkpoint record and prints the value. >> 2b. To search the available WAL files for the latest checkpoint record and recreates a pg_control file pointing at that checkpoint. >> >> I have kept both options to address different kind of corruption scenarios. > I think I can see all of those things being potentially useful. There > are a couple of pending patches that will revise the WAL format > slightly; not sure how much those are likely to interfere with any > development you might do on (2) in the meantime. Below is the details of Option-2, for Option-1, I will send mail separately New option for pg_resetxlog: - 1. Introduce option -r to restore the control file if possible and print those values. 3. User need to give option -f along with -r to rewrite the control file from WAL files. 2. If not able to get the control information from WAL files then the control data will be guessed and proceedes as normal reset xlog. 4. If the control information is restored, then the option -l is ignored. Design for new Option: -- 1. Validate the pg_xlog directory before proceeding of restoring control values. if the directory is invalid then the control values will be guessed. 2. Read the pg_xlog directory and read all the existing files. 3. If it is a valid xlog file then add it to a list in an increasing order, Otherwise the file is ignored and continue to the next file. 4. Try to find the last timestamp file from the list to start reading for a checkpoint record. 5. Read the first page from the file and validate it. if the validation fails the restore happens with guessed values. 6. Read the first record as start of the record from the identified first xlog file. 7. If the first record is a continuation record from a previous record then ignore the record and continue to the next record. 8. After getting the entire record then the record is validated, if it is not a valid record searching for the next record will be stopped and the control values will be guessed. 9. Search all the files to the end of the last file to get the latest checkpoint record. 10. While searching for the record, if it is not reaching the last file (there is missing file or invalid record) then treat this scenario as a failure of finding the checkpoint record and go for guessing the control values. 11. After finding the last checkpoint record, update the checkpoint record information in the control file. Implementation: 1. We need to use most of the functionality of functions mentioned below. One way is to duplicate the code of these functions related to functionality required by pg_resetxlog in pg_resetxlog module. I have checked other modules also but didn't find how we can use common functionality in server utility from backend code. Could you please point me for the appropriate way for doing it. The list of functions: 1. ValidateXLOGDirectoryStructure 2. XLogPageRead 3. ReadRecord 4. RecordIsValid 5. ValidXLOGPageHeader 6. ValidXLogRecordHeader Suggestions/Comments/Thoughts? With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] embedded list v2
On 28 June 2012 19:20, Andres Freund wrote: > <0001-Add-embedded-list-interface.patch> > > Looks good now? I have a few gripes. + * there isn't much we can test in a single linked list except that its There are numerous references to "single linked lists", where, I believe, "singly linked list" is intended (the same can be said for "double" and "doubly" respectively). /* Functions we want to be inlined if possible */ + #ifndef USE_INLINE ... + #endif /* USE_INLINE */ A minor quibble, but that last line probably be: #endif /* ! defined USE_INLINE */ Another minor quibble. We usually try and keep these in alphabetical order: *** a/src/backend/lib/Makefile --- b/src/backend/lib/Makefile *** subdir = src/backend/lib *** 12,17 top_builddir = ../../.. include $(top_builddir)/src/Makefile.global ! OBJS = dllist.o stringinfo.o include $(top_srcdir)/src/backend/common.mk --- 12,17 top_builddir = ../../.. include $(top_builddir)/src/Makefile.global ! OBJS = dllist.o stringinfo.o ilist.o include $(top_srcdir)/src/backend/common.mk New files generally don't require this: + * Portions Copyright (c) 1994, Regents of the University of California This needs to be altered: + /* + * enable for extra debugging. This is rather expensive so its not enabled by + * default even with --enable-cassert + */ + /* #define ILIST_DEBUG */ I'm not sure that this extra error-detection warrants its own macro. syncrep.c similarly has its own rather expensive error-detection, with entire function bodies only compiled when USE_ASSERT_CHECKING is defined. Any of the other dedicated "debugging macros", like LOCK_DEBUG or WAL_DEBUG seem to exist to dump LOG messages when binaries were built with the macros defined (this can happen via pg_config_manual.h. I note that what you have here lacks a corresponding entry). I think that it would be more idiomatic to just use USE_ASSERT_CHECKING, and rewrite the debugging functions such that they are used directly within an Assert, in the style of syncrep.c . Now, I know Tom wasn't too enthusiastic about having this sort of thing within --enable-cassert builds, but I'm inclined to think that there is little point in having this additional error checking if they're not at least available when that configuration is used. Maybe we should consider taking the sophisticated asserts out of --enable-cassert builds, or removing them entirely, if and when they prove to be annoying? There is slight misalignment within the comments at the top of ilist.h. Within ilist.c, the following block exists: + #ifndef USE_INLINE + #define ILIST_USE_DECLARATION + #endif + + #include "lib/ilist.h" + + #ifndef USE_INLINE + #undef ILIST_USE_DECLARATION + #endif I see little reason for the latter "#undef" block within ilist.c. It isn't that exactly the same code already exists at the end of ilist.h (though there is that too) - it's mostly that it's unnecessary, because there is no need or logical reason to #undef within ilist.c. + /* + * The following function declarations are only used if inlining is supported + * or when included from a file that explicitly declares USE_DECLARATION + */ + #ifdef ILIST_USE_DECLARATION Shouldn't that be "The following function definitions..." and ILIST_USE_DEFINITIONS? I think the fact that it's possible in principle for ILIST_USE_DECLARATION to not be exactly equivalent to ! USE_INLINE is strictly speaking dangerous, since USE_INLINE needs to be baked into the ABI targeted by third-party module developers. What if a module was built that called a function that didn't have an entry in the procedure linkage table, due to ad-hoc usage of ILIST_USE_DECLARATION? That'd result in a libdl error, if you're lucky. Now, that probably sounds stupid - I'm pretty sure that you didn't intend that ILIST_USE_DECLARATION be set by just any old client of this infrastructure. Rather, you intended that ILIST_USE_DECLARATION be set either when ilist.h was included while USE_INLINE, so that macro expansion would make those functions inline, or within ilist.c, when !USE_INLINE, so that the functions would not be inlined. This should be much more explicit though. You simply describe the mechanism rather than the intent at present. I rather suspect that INLINE_IF_POSSIBLE should be a general purpose utility, perhaps defined next to NON_EXEC_STATIC within c.h, with a brief explanation there (rather than in any new header that needs to do this). I think you'd be able to reasonably split singly and doubly linked lists into their own headers without much repetition between the two then. You could formalise the idea that where USE_INLINE, another macro, defined alongside INLINE_IF_POSSIBLE (functionally equivalent to ILIST_USE_DECLARATION in the extant code - say, USE_INLINING_DEFINITIONS) is going to be generally defined everywhere USE_INLINE is defined. You're then going to have to deal with the hack whereby USE_INLINING_DEFINITIONS is set just "to suck th
Re: [HACKERS] Re: [COMMITTERS] pgsql: Run pgindent on 9.2 source tree in preparation for first 9.3
On Tue, Jun 12, 2012 at 02:40:09PM -0400, Robert Haas wrote: > On Tue, Jun 12, 2012 at 2:02 PM, Bruce Momjian wrote: > > On Tue, Jun 12, 2012 at 01:50:48PM -0400, Noah Misch wrote: > >> On Mon, Jun 11, 2012 at 05:57:41PM -0400, Alvaro Herrera wrote: > >> > What about something like this in the root of the tree: > >> > find . -name \*.pl -o -name \*.pm | xargs perltidy -b -bl -nsfs -naws > >> > -l=100 -ole=unix > >> > > >> > There are files all over the place. The file that would most be > >> > affected with one run of this is the ECPG grammar generator. > >> > > >> > I checked the "-et=4" business (which is basically entab). We're pretty > >> > inconsistent about tabs in perl code it seems; some files use tabs > >> > others use spaces. Honestly I would just settle on what we use on C > >> > files, even if the Perl devs don't recommend it "because of > >> > maintainability and portability". I mean if it works well for us for C > >> > code, why would it be a problem in Perl code? However, I don't write > >> > much of that Perl code myself. > >> > >> +1 for formatting all our Perl scripts and for including -et=4. Since that > >> will rewrite currently-tidy files anyway, this is a good time to audit our > >> perltidy settings. > > > > OK, another open question is whether we should do any of these changes > > now for 9.2/9.3 or wait for 9.3/9.4? > > I don't think it matters very much - very few commits touch those perl > scripts. If we have a consensus, I think it's fine to do it now, or > even after we branch. Done. Run on HEAD and 9.2 --- sorry for the delay. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch: add conversion from pg_wchar to multibyte
> On Sun, Jul 1, 2012 at 5:11 AM, Alexander Korotkov > wrote: >> [ new patch ] > > With the improved comments in pg_wchar.h, it seemed clear what needed > to be done here, so I fixed up the MULE conversion and committed this. > I'd appreciate it if someone would check my work, but I think it's > right. For me your commit looks good. -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese: http://www.sraoss.co.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Schema version management
On Wed, Jul 4, 2012 at 9:02 AM, Joel Jacobson wrote: > On Tue, Jul 3, 2012 at 7:49 PM, Peter Eisentraut wrote: >> >> I think this idea has merit. Prepare a patch and put it into the next >> commit fest. > > Glad to hear, I'm on it! > >> >> I see the problem that since the dump order is in general not >> deterministic, this will cause random reordering in your master file >> that includes all the individual files. >> >> >> Then again, making the dump order deterministic is a problem that can be >> solved (I suppose), so maybe starting there would be a good step. But >> it will require a small amount of in-depth pg_dump hacking. > > > I just made a test, where I created objects in different order and compared > the dumps. > It appears pg_dump dumps objects in alphabetically sorted order. > This works fine for most objects, but not for overloaded functions, in which > case > they are dumped in oid order. > > Are there any other cases than overloaded functions, where the dump order > isn't deterministic? > > While waiting for your reply, I'll be working on fixing the problem with > overloaded functions. My vote is - when there's an overloaded function, put each version in its own file. And name the files something like functionname_something.sql. And just document that something may not be entirely stable. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Pg default's verbosity?
On Mon, Jul 2, 2012 at 3:47 PM, Robert Haas wrote: > On Fri, Jun 29, 2012 at 3:07 AM, Bruce Momjian wrote: >>> Personally, I'd have no problem with flat-out dropping (not demoting) >>> both of those two specific messages. I seem to recall that Bruce has >>> lobbied for them heavily in the past, though. >> >> I would like to see them gone or reduced as well. I think I wanted them >> when we changed the fact that SERIAL doesn't create unique indexes, but >> that was long ago --- I think everyone knows what happens now. > > Patch attached. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Oracle porting sample instr function
On Wed, Jul 4, 2012 at 3:50 AM, Albe Laurenz wrote: > Robert Haas wrote: >>> I can confirm that Oracle returns 0 if the third argument to >>> INSTR is 0. > >> Can someone provide a suitable doc patch? Thanks, committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch: add conversion from pg_wchar to multibyte
On Sun, Jul 1, 2012 at 5:11 AM, Alexander Korotkov wrote: > [ new patch ] With the improved comments in pg_wchar.h, it seemed clear what needed to be done here, so I fixed up the MULE conversion and committed this. I'd appreciate it if someone would check my work, but I think it's right. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] User-Id Tracking when Portal was started
2012/7/4 Robert Haas : > On Tue, Jul 3, 2012 at 12:46 PM, Kohei KaiGai wrote: >>> My point is that it seems like a bug that the secContext gets restored >>> in one case and not the other, depending on which user ID was specified >>> in SET SESSION AUTHORIZATION. >>> >> Sorry, the above description mention about a case when it does not use >> the marker to distinguish a case to switch user-id from a case not to switch. >> (I though I was asked the behavior if this logic always switches / >> restores ids.) >> >> The patch itself works correctly, no regression test failed even though >> several tests switches user-id using SET SESSION AUTHORIZATION. > > I don't believe that proves anything. There are lots of things that > aren't tested by the regression tests, and there's no guarantee that > any you've added cover all bases, either. We always treat user-ID and > security context as a unit; you haven't given any reason why this case > should be handled differently, and I bet it shouldn't. > This patch always handles user-id and security context as a unit. In case when it was switched, then it shall be always restored. And, in case when it was not switched, then it shall never be restored. Was my explanation confusing? -- KaiGai Kohei -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Posix Shared Mem patch
On Tue, Jul 3, 2012 at 1:46 PM, Josh Kupershmidt wrote: > On Tue, Jul 3, 2012 at 6:57 AM, Robert Haas wrote: >> Here's a patch that attempts to begin the work of adjusting the >> documentation for this brave new world. I am guessing that there may >> be other places in the documentation that also require updating, and >> this page probably needs more work, but it's a start. > > I think the boilerplate warnings in config.sgml about needing to raise > the SysV parameters can go away; patch attached. Thanks, committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] User-Id Tracking when Portal was started
On Tue, Jul 3, 2012 at 12:46 PM, Kohei KaiGai wrote: >> My point is that it seems like a bug that the secContext gets restored >> in one case and not the other, depending on which user ID was specified >> in SET SESSION AUTHORIZATION. >> > Sorry, the above description mention about a case when it does not use > the marker to distinguish a case to switch user-id from a case not to switch. > (I though I was asked the behavior if this logic always switches / > restores ids.) > > The patch itself works correctly, no regression test failed even though > several tests switches user-id using SET SESSION AUTHORIZATION. I don't believe that proves anything. There are lots of things that aren't tested by the regression tests, and there's no guarantee that any you've added cover all bases, either. We always treat user-ID and security context as a unit; you haven't given any reason why this case should be handled differently, and I bet it shouldn't. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch for Todo Item : Provide fallback_application_name in contrib/pgbench, oid2name, and dblink
On Tue, Jul 3, 2012 at 11:36 PM, Amit Kapila wrote: > Hi Shigeru/Robert, > >> The way fixing oid2name and pgbench seems reasonable, so applying it to >> vacuumlo (as Peter mentioned) would be enough for this issue. > > Shall I consider following 2 points to update the patch: > 1. Apply changes similar to pgbench and oid2name for vacuumlo > 2. Remove the modifications for dblink. I've done these two things and committed this. Along the way, I also fixed it to use a stack-allocated array instead of using malloc, since there's no need to malloc a fixed-size array with 7 elements. Thanks for the patch. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] enhanced error fields
2012/7/4 Tom Lane : > Alvaro Herrera writes: >> I think if you don't have a true constraint name to use here, you >> shouldn't use anything. > > Yeah, I agree. Don't invent a value, just omit the field. ok Pavel > > regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] waitpid in pg_basebackup
Fujii Masao writes: > waitpid() is used with "#ifdef HAVE_WAITPID" in reaper(), but NOT in > BaseBackup(). > Why not? We can ensure that all platforms which PostgreSQL supports > have waitpid()? > If so, can we get rid of "#ifdef HAVE_WAITPID" in reaper()? The Single Unix Spec V2 (1997) specifies both waitpid() and wait3(), but says the latter is "legacy" and new applications should use only waitpid(); furthermore it documents that platforms need not support wait3() (they can just return ENOSYS instead of making it work). I notice that pgbench also uses waitpid() unconditionally in its "#ifndef ENABLE_THREAD_SAFETY" section. That's been there for quite some time, making it even less likely that there are still any platforms where waitpid() isn't available. I agree, let's drop the support for waitpid() not being present. It looks to me like we could remove the macro maze in reaper() completely, if we fixed win32_waitpid() to not have an API randomly different from the real waitpid(). That would be a noticeable readability gain there. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] enhanced error fields
Alvaro Herrera writes: > I think if you don't have a true constraint name to use here, you > shouldn't use anything. Yeah, I agree. Don't invent a value, just omit the field. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] lock_timeout and common SIGALRM framework
2012-07-04 17:25 keltezéssel, Alvaro Herrera írta: Excerpts from Boszormenyi Zoltan's message of mié jul 04 07:03:44 -0400 2012: 2012-07-03 23:38 keltezéssel, Alvaro Herrera írta: I don't understand why PGSemaphoreTimedLock() is not broken. I mean surely you need a bool return to let the caller know whether the acquisition succeeded or failed? Well, this is the same interface PGSemaphoreTryLock() uses. By this reasoning, it's also broken. Eh, no -- as far as I can see, PGSemaphoreTryLock returns a boolean, which is precisely my point. You're right. I blame the heat for not being able to properly read my own code. -- -- Zoltán Böszörményi Cybertec Schönig & Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt, Austria Web: http://www.postgresql-support.de http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] waitpid in pg_basebackup
Hi, waitpid() is used with "#ifdef HAVE_WAITPID" in reaper(), but NOT in BaseBackup(). Why not? We can ensure that all platforms which PostgreSQL supports have waitpid()? If so, can we get rid of "#ifdef HAVE_WAITPID" in reaper()? Regards, -- Fujii Masao -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH] pg_dump: Sort overloaded functions in deterministic order
I renamed the new element to DumpableObject from "proargs" to the more general name "sortkey". This way this element can be used by any object types in the future, which might require sorting by additional information than type, namespace and name. Currently, it's only set for functions/aggregates though, its NULL for all other object types. I felt less ugly to add a new element with a general name than one specific for functions. I also moved the check to the last part of DOTypeNameCompare, just before sorting by OIDs as a last resort. Feedback on the implementation is welcomed. If this can be achieved without adding a new element to DumpableObject, it is of course much better, but I couldn't find a way of doing that. pg_dump_deterministic_order_v2.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] enhanced error fields
Excerpts from Pavel Stehule's message of mié jul 04 05:33:48 -0400 2012: > Hello > > 2012/7/3 Matthew Woodcraft : > > Peter Geoghegan writes: > >> So I took a look at the patch eelog-2012-05-09.diff today. All of the > >> following remarks apply to it alone. > > > > I've been trying out this patch for my own interest (I'm very pleased to > > see work on this feature), and I have a couple of suggestions from a > > user's point of view. > > > > > > First: if a not null constraint is violated, the error report includes > > CONSTRAINT NAME 'not_null_violation'. I think I would find it more > > useful if CONSTRAINT NAME were left unset rather than given a value that > > doesn't correspond to a real constraint. A client program can tell it's > > a null constraint violation from the SQLSTATE. > > > > I don't think so generation some special name is good idea. In this > case - important values are in COLUMN_NAME, TABLE_NAME, SCHEMA_NAME > > postgres=# create table ff(a int not null); > CREATE TABLE > postgres=# \set VERBOSITY verbose > postgres=# insert into ff values(null); > ERROR: 23502: null value in column "a" violates not-null constraint > DETAIL: Failing row contains (null). > LOCATION: ExecConstraints, execMain.c:1527 > COLUMN NAME: a > TABLE NAME: ff > SCHEMA NAME: public > CONSTRAINT NAME: not_null_violation > CONSTRAINT SCHEMA: public I think if you don't have a true constraint name to use here, you shouldn't use anything. When and if we get NOT NULL constraints catalogued, we can add a constraint name field as a new error field. In other words +1 for Matthew's opinion. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] lock_timeout and common SIGALRM framework
Excerpts from Boszormenyi Zoltan's message of mié jul 04 07:03:44 -0400 2012: > > 2012-07-03 23:38 keltezéssel, Alvaro Herrera írta: > > > > I don't understand why PGSemaphoreTimedLock() is not broken. I mean > > surely you need a bool return to let the caller know whether the > > acquisition succeeded or failed? > > Well, this is the same interface PGSemaphoreTryLock() uses. > By this reasoning, it's also broken. Eh, no -- as far as I can see, PGSemaphoreTryLock returns a boolean, which is precisely my point. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] lock_timeout and common SIGALRM framework
Excerpts from Boszormenyi Zoltan's message of mié jul 04 06:32:46 -0400 2012: > 2012-07-04 12:09 keltezéssel, Boszormenyi Zoltan írta: > > You just broke initdb with this cleanup. :-) Ouch. > > initdb starts postgres --single, that doesn't do BackendInitialize(), > > only PostgresMain(). So, you need InitializeTimeouts() before > > the RegisterTimeout() calls in PostgresMain and the elog(PANIC) > > must not be in InitializeTimeouts() if called twice. > > Attached is the fix for this problem. PostgresMain() has a new > argument: bool single_user. This way, InitializeTimeouts() can > keep its elog(PANIC) if called twice and "postgres --single" > doesn't fail its Assert() in RegisterTimeout(). Hmm. Maybe it's better to leave InitializeTimeouts to be called twice after all. The fix seems a lot uglier than the disease it's curing. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgfoundry references in docs
On Wed, Jul 4, 2012 at 4:21 PM, David E. Wheeler wrote: > On Jul 4, 2012, at 9:15 AM, Magnus Hagander wrote: > >> Not really. We have nowhere else to recommend, since we don't run a >> replacement for it. And we really don't want to get involved in >> listing all the different third party sites out there. (For example, >> we had a reference to sourceforge.net in the same paragraph. And while >> that was certainly "state of the art" when the docs were written, I >> don't think anybody sane would recommend that today. The reality keeps >> changing on those things, so it really doesn't belong in the docs). We >> could put a set of links on the wiki if we want something "more live". > > Ah, then perhaps a link to such a wiki page would suffice. I think that would > be a good compromise. That can really be said for all of Appendix H in that case... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH] pg_dump: Sort overloaded functions in deterministic order
I have received positive feedback on the pg_dump --split option I suggested, but it depends on pg_dump dumping objects in a deterministic order. I'm committed to fixing this. The first problem I've spotted is overloaded functions. This patch adds a new element to DumpableObject: char *proargs This is set to the output from pg_get_function_identity_arguments(oid) for all functions, and set to NULL for all other objects. sortDumpableObjectsByTypeName calls DOTypeNameCompare, which in addition to sorting objects by type, namespace and name, now also sorts by the function identity arguments. This makes overloaded functions being dumped in the same order, regardless of which order they were created. Are there any other object types where the order isn't deterministic? pg_dump_deterministic_order.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [WIP] Patch : Change pg_ident.conf parsing to be the same as pg_hba.conf
From: Robert Haas [mailto:robertmh...@gmail.com] Sent: Tuesday, July 03, 2012 9:43 PM On Mon, Jul 2, 2012 at 8:08 AM, Amit Kapila wrote: >> Suggestions? > I suggest you add this to the next CommitFest. :-) > https://commitfest.postgresql.org/action/commitfest_view?id=14 > Meanwhile, we have this CommitFest to get finished with... I shall add it to next CF. The testing done for the patch is as follows: Testing Done For the Patch is as follows: PRESETUP: 1. configure HBA file to use ident authentication method and mapname = omicron local all all ident map=omicron hostall all 127.0.0.1/32ident map=omicron 2. Configure IDENT file to have following configuration omicron kiran kumar omicron /^a(.*)$ a\\1 3. Start the server. Testcases: 1. Try connecting to kumar (pguser) from kiran system login. -- connection successful. 2. Try connecting to admin (pguser) from admin system login -- connection successful. 3. Try connecting to viswa (pguser) from viswa system login -- connection failed as mapname not found. Invalid Ident file testcases. modify the ident file as follows omicron kiran kumar omicron /^a.*)$ a\\1 4. Try reload the configuration files server will give error as follows LOG: received SIGHUP, reloading configuration files LOG: invalid regular expression "^r.*)$": parentheses () not balanced WARNING: pg_ident.conf not reloaded After that execute the previous success sceanrios 5. Try connecting to kumar (pguser) from kiran system login. -- connection successful. 6. Try connecting to admin (pguser) from admin system login -- connection successful. 7. Try connecting to viswa (pguser) from viswa system login -- connection failed as mapname not found. 8. Stop the server and start it again. LOG: invalid regular expression "^a.*)$": parentheses () not balanced FATAL: could not load pg_ident.conf With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgfoundry references in docs
On Jul 4, 2012, at 9:15 AM, Magnus Hagander wrote: > Not really. We have nowhere else to recommend, since we don't run a > replacement for it. And we really don't want to get involved in > listing all the different third party sites out there. (For example, > we had a reference to sourceforge.net in the same paragraph. And while > that was certainly "state of the art" when the docs were written, I > don't think anybody sane would recommend that today. The reality keeps > changing on those things, so it really doesn't belong in the docs). We > could put a set of links on the wiki if we want something "more live". Ah, then perhaps a link to such a wiki page would suffice. I think that would be a good compromise. David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_receivexlog and feedback message
On Mon, Jun 11, 2012 at 5:24 PM, Fujii Masao wrote:
> On Mon, Jun 11, 2012 at 10:04 PM, Magnus Hagander wrote:
>> On Sun, Jun 10, 2012 at 4:02 PM, Fujii Masao wrote:
>>> On Sun, Jun 10, 2012 at 7:55 PM, Magnus Hagander
>>> wrote:
How about this?
>>>
>>> + /*
>>> +* Set flushed position to the last byte in
>>> the previous
>>> +* file. Per above we know that
>>> xrecoff%XLOG_SEG_SIZE=0
>>> +*/
>>> + flushedpos = blockpos;
>>> + if (flushedpos.xrecoff == 0)
>>> + {
>>> + flushedpos.xlogid--;
>>> + flushedpos.xrecoff = XLogFileSize-1;
>>> + }
>>> + else
>>> + flushedpos.xrecoff--;
>>>
>>> flushedpos.xrecoff doesn't need to be decremented by one.
>>> If xrecoff % XLOG_SEG_SIZE = 0, the position should be the last
>>> byte of previous (i.e., flushed) WAL file.
>>
>> Hmm. I thikn I confused myself with "last byte written" vs "current
>> position". And we're dealing with current position here...
>>
>> So it should just be flushedpos = blockpos and be done with it, right?
>
> Yep.
>
>> Though before I commit anything with this, we need to decide what to
>> wrt syncrep on that, per the other thread.
>
> Yep.
Per the other thread, we decided to postpone this until 9.3. And also
figure out a better set of switches for pg_receivexlog to control it
with.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [ADMIN] pg_basebackup blocking all queries with horrible performance
On Mon, Jul 2, 2012 at 8:17 PM, Fujii Masao wrote: > On Mon, Jul 2, 2012 at 4:01 AM, Magnus Hagander wrote: >> On Sun, Jul 1, 2012 at 7:14 PM, Fujii Masao wrote: >>> >>> On Fri, Jun 29, 2012 at 7:22 PM, Magnus Hagander >>> wrote: On Wed, Jun 27, 2012 at 7:24 PM, Fujii Masao wrote: > On Thu, Jun 21, 2012 at 3:18 AM, Robert Haas > wrote: >> On Wed, Jun 20, 2012 at 7:18 AM, Magnus Hagander >> wrote: >> You agreed to add something like NOSYNC option into >> START_REPLICATION command? > > I'm on the fence. I was hoping somebody else would chime in with an > opinion as well. +1 >>> >>> Nobody else with any opinion on this? :( >> >> I don't think we really need a NOSYNC flag at this point. Just not >> setting the flush location in clients that make a point of flushing in >> a timely fashion seems fine. > > Okay, I'm in the minority, so I'm writing the patch that way. WIP > patch attached. > > In the patch, pg_basebackup background process and pg_receivexlog always > return invalid location as flush one, and will never become sync standby > even > if their name is in synchronous_standby_names. The timing of their sending That doesn't match with the patch, afaics. The patch always sets the correct write location, which means it can become a remote_write synchronous standby, no? It will only send it back when timeout expires, but it will be sent back. >>> >>> No. Though correct write location is sent back, they don't become sync >>> standby >>> because flush location is always invalid. While flush location is >>> invalid, the master >>> will never regard the remote server as sync one even if synchronous_commit >>> is >>> set to remote_write. >> >> Oh. I wasn't aware of that part. >> >> I wonder if that might actually be a more reasonable mode of operation in general: * always send back the write position, at the write interval * always send back the flush position, when we're flushing (meaning when we switch xlog) have an option that makes it possible to: * always send back the write position as soon as it changes (making for a reasonable remote_write sync standby) * actually flush the log after each write instead of end of file (making for a reasonable full sync standby) meaning you'd have something like "pg_receivexlog --sync=write" and "pg_receivexlog --sync=flush" controlling it instead. >>> >>> Yeah, in this way, pg_receivexlog can become sync even if >>> synchronous_commit is on, which seems more useful. But >>> I'm thinking that the synchronous pg_receivexlog stuff should >>> be postponed to 9.3 because its patch seems to become too >>> big to apply at this beta stage. So, in 9.2, to fix the problem, >>> what about just applying the simple patch which prevents >>> pg_basebackup background process and pg_receivexlog from >>> becoming sync standby whatever synchronous_standby_names >>> and synchronous_commit are set to? >> >> Agreed. >> >> With the addition that we should set the write location, because >> that's very useful and per what you said above should be perfectly >> safe. >> >> And deal with the "user put * in synchronous_standby_names and accidentally got pg_receivexlog as the sync standby" by more clearly warning people not to use * for that parameter... Since it's simply dangerous :) >>> >>> Yep. >> >> What would be good wording? Something along the line of "Using the * >> entry is not recommended since it can lead to unexpected results when >> new standbys are added" or something like that? >> >> > the reply depends on the standby_message_timeout specified in -s option. > So > the write position may lag behind the true position. > > pg_receivexlog accepts new option -S (better option character?). If this > option > is specified, pg_receivexlog returns true flush position, and can become > sync > standby. It sends back the reply to the master each time the write > position > changes or the timeout passes. If synchronous_commit is set to > remote_write, > synchronous replication to pg_receivexlog would work well. Yeah, I hadn't considered the remote_write mode, but I guess that's why you have to track the current write position across loads, which first confused me. >>> >>> The patch has to track the current write location to decide whether to send >>> back the reply to the master, IOW to know whether the write location >>> has changed, IOW to know whether we've already sent the reply about >>> the latest write location. >> >> Yeha, makes perfect sense. >> >> Looking at some other usecases for this, I wonder if we should also force a status message whenever we switch xlog files, even if we aren't running in sync mode, even if the timeout hasn't
Re: [HACKERS] Schema version management
On Tue, Jul 3, 2012 at 7:49 PM, Peter Eisentraut wrote: > I think this idea has merit. Prepare a patch and put it into the next > commit fest. Glad to hear, I'm on it! > I see the problem that since the dump order is in general not > deterministic, this will cause random reordering in your master file > that includes all the individual files. > Then again, making the dump order deterministic is a problem that can be > solved (I suppose), so maybe starting there would be a good step. But > it will require a small amount of in-depth pg_dump hacking. > I just made a test, where I created objects in different order and compared the dumps. It appears pg_dump dumps objects in alphabetically sorted order. This works fine for most objects, but not for overloaded functions, in which case they are dumped in oid order. Are there any other cases than overloaded functions, where the dump order isn't deterministic? While waiting for your reply, I'll be working on fixing the problem with overloaded functions.
Re: [HACKERS] plpython issue with Win64 (PG 9.2)
On 04/07/12 13:58, Asif Naeem wrote:
Patch attached. Asif, could you try a few things on a CP1252 database?
First verify if your original test case now works and then try this:
I have test the patch on Win64. postgres server is working fine now for
WIN1252. Thanks.
create function enctest() returns text as $$
return b'tr\xc3\xb3spido'.decode('**utf-8')
$$ language plpython3u;
select enctest(), encode(convert_to(enctest(), 'utf-8'), 'hex');
create function enctest() returns text as $$
return b'tr\xc3\xb3spido'.decode('utf-8')
$$ language plpython3u;
select enctest(), encode(convert_to(enctest(), 'utf-8'), 'hex');
enctest | encode
--+
tróspido | 7472c3b3737069646f
(1 row)
Please do let me know If you have any other query. Thanks.
Great, this looks correct.
Can we apply this to 9.2?
Cheers,
Jan
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] plpython issue with Win64 (PG 9.2)
> Patch attached. Asif, could you try a few things on a CP1252 database?
First verify if your original test case now works and then try this:
>
>
I have test the patch on Win64. postgres server is working fine now for
WIN1252. Thanks.
> create function enctest() returns text as $$
> return b'tr\xc3\xb3spido'.decode('**utf-8')
> $$ language plpython3u;
>
> select enctest(), encode(convert_to(enctest(), 'utf-8'), 'hex');
>
create function enctest() returns text as $$
return b'tr\xc3\xb3spido'.decode('utf-8')
$$ language plpython3u;
select enctest(), encode(convert_to(enctest(), 'utf-8'), 'hex');
enctest | encode
--+
tróspido | 7472c3b3737069646f
(1 row)
Please do let me know If you have any other query. Thanks.
Best Regards,
Muhammad Asif Naeem
Re: [HACKERS] [PATCH] lock_timeout and common SIGALRM framework
2012-07-03 23:38 keltezéssel, Alvaro Herrera írta: I don't understand why PGSemaphoreTimedLock() is not broken. I mean surely you need a bool return to let the caller know whether the acquisition succeeded or failed? Well, this is the same interface PGSemaphoreTryLock() uses. By this reasoning, it's also broken. AFAICS you are relying on get_timeout_indicator() but this seems to me the wrong thing to do ... (not to mention how ugly it is to percolate through two levels of abstraction) What other way do you suggest? EINTR may come from a different signal, which may also be ignored or not. Ctrl-C is handled and leads to elog(ERROR) but an ignored signal technically calls a NOP handler deep inside the OS runtime libraries but the signal *is* delivered to the backend which in turn interrupts semop() or whatever the platform equivalent is. I can add a flag to timeout.c that is set whenever SIGALRM is delivered but checking that would be another "abstraction violation" as calling get_timeout_indicator() in your opinion. The original coding of PGSemaphoreTryLock() used semtimedop(), sem_timedwait() and the timeout value applied to WaitForMultipleObjectsEx(). This was quickly shot down as using the SIGALRM signal and its behaviour to interrupt the locking operation is be better and fits the PostgreSQL portability features. Also, OS X at the time didn't support sem_timedwait(). I am not complaining, just recalling the different details. How about not hardcoding get_timeout_indicator(LOCK_TIMEOUT) into PGSemaphoreTimedLock()? Passing TimeoutName to it would make it more generic and usable for other timeout sources. -- -- Zoltán Böszörményi Cybertec Schönig & Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt, Austria Web: http://www.postgresql-support.de http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] lock_timeout and common SIGALRM framework
2012-07-04 12:09 keltezéssel, Boszormenyi Zoltan írta:
2012-07-03 23:31 keltezéssel, Alvaro Herrera írta:
Excerpts from Boszormenyi Zoltan's message of vie jun 29 14:30:28 -0400 2012:
Does anyone have a little time to look at the latest timeout framework
with the registration interface and the 2nd patch too? I am at work
until Friday next week, after that I will be on vacation for two weeks.
Just in case there is anything that needs tweaking to make it more
acceptable.
I cleaned up this a bit more and now I think it's ready to commit --
as soon as somebody tests that the standby bits still work.
You just broke initdb with this cleanup. :-)
---8<--8<--8<--8<--8<--8<--8<---
$ cat src/test/regress/log/initdb.log
Running in noclean mode. Mistakes will not be cleaned up.
The files belonging to this database system will be owned by user "zozo".
This user must also own the server process.
The database cluster will be initialized with locales
COLLATE: hu_HU.utf8
CTYPE:hu_HU.utf8
MESSAGES: C
MONETARY: hu_HU.utf8
NUMERIC: hu_HU.utf8
TIME: hu_HU.utf8
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "hungarian".
creating directory
/home/zozo/lock-timeout/9.1/1/postgresql.14/src/test/regress/./tmp_check/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 32MB
creating configuration files ... ok
creating template1 database in
/home/zozo/lock-timeout/9.1/1/postgresql.14/src/test/regress/./tmp_check/data/base/1 ... ok
initializing pg_authid ... TRAP: FailedAssertion("!(base_timeouts_initialized)", File:
"timeout.c", Line: 217)
sh: line 1: 29872 Aborted (core dumped)
"/home/zozo/lock-timeout/9.1/1/postgresql.14/src/test/regress/tmp_check/install/home/zozo/pgc92dev-locktimeout/bin/postgres"
--single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 > /dev/null
child process exited with exit code 134
initdb: data directory
"/home/zozo/lock-timeout/9.1/1/postgresql.14/src/test/regress/./tmp_check/data" not
removed at user's request
---8<--8<--8<--8<--8<--8<--8<---
initdb starts postgres --single, that doesn't do BackendInitialize(),
only PostgresMain(). So, you need InitializeTimeouts() before
the RegisterTimeout() calls in PostgresMain and the elog(PANIC)
must not be in InitializeTimeouts() if called twice.
Attached is the fix for this problem. PostgresMain() has a new
argument: bool single_user. This way, InitializeTimeouts() can
keep its elog(PANIC) if called twice and "postgres --single"
doesn't fail its Assert() in RegisterTimeout().
Comments?
Best regards,
Zoltán Böszörményi
--
--
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
diff -durpN postgresql.14.orig/src/backend/main/main.c postgresql.14/src/backend/main/main.c
--- postgresql.14.orig/src/backend/main/main.c 2012-06-26 09:10:21.272759354 +0200
+++ postgresql.14/src/backend/main/main.c 2012-07-04 12:21:58.869037014 +0200
@@ -192,7 +192,7 @@ main(int argc, char *argv[])
else if (argc > 1 && strcmp(argv[1], "--describe-config") == 0)
GucInfoMain(); /* does not return */
else if (argc > 1 && strcmp(argv[1], "--single") == 0)
- PostgresMain(argc, argv, get_current_username(progname)); /* does not return */
+ PostgresMain(argc, argv, get_current_username(progname), true); /* does not return */
else
PostmasterMain(argc, argv); /* does not return */
abort(); /* should not get here */
diff -durpN postgresql.14.orig/src/backend/postmaster/postmaster.c postgresql.14/src/backend/postmaster/postmaster.c
--- postgresql.14.orig/src/backend/postmaster/postmaster.c 2012-07-04 12:25:09.247183727 +0200
+++ postgresql.14/src/backend/postmaster/postmaster.c 2012-07-04 12:23:22.933543240 +0200
@@ -3626,7 +3626,7 @@ BackendRun(Port *port)
*/
MemoryContextSwitchTo(TopMemoryContext);
- PostgresMain(ac, av, port->user_name);
+ PostgresMain(ac, av, port->user_name, false);
}
diff -durpN postgresql.14.orig/src/backend/tcop/postgres.c postgresql.14/src/backend/tcop/postgres.c
--- postgresql.14.orig/src/backend/tcop/postgres.c 2012-07-04 12:25:09.255183775 +0200
+++ postgresql.14/src/backend/tcop/postgres.c 2012-07-04 12:24:17.685873058 +0200
@@ -3509,7 +3509,7 @@ process_postgres_switches(int argc, char
*
*/
void
-PostgresMain(int argc, char *argv[], const char *username)
+PostgresMain(int argc, char *argv[], const char *username, bool single_user)
{
const char *dbname;
int firstchar;
@@ -3603,8 +3603,11 @@ PostgresMain(int argc, char *argv[], con
{
/*
* Register timeout sources needed by backend operation. Note
- * that InitializeTimeout was already ca
Re: [HACKERS] [PATCH] lock_timeout and common SIGALRM framework
2012-07-03 23:31 keltezéssel, Alvaro Herrera írta:
Excerpts from Boszormenyi Zoltan's message of vie jun 29 14:30:28 -0400 2012:
Does anyone have a little time to look at the latest timeout framework
with the registration interface and the 2nd patch too? I am at work
until Friday next week, after that I will be on vacation for two weeks.
Just in case there is anything that needs tweaking to make it more
acceptable.
I cleaned up this a bit more and now I think it's ready to commit --
as soon as somebody tests that the standby bits still work.
You just broke initdb with this cleanup. :-)
---8<--8<--8<--8<--8<--8<--8<---
$ cat src/test/regress/log/initdb.log
Running in noclean mode. Mistakes will not be cleaned up.
The files belonging to this database system will be owned by user "zozo".
This user must also own the server process.
The database cluster will be initialized with locales
COLLATE: hu_HU.utf8
CTYPE:hu_HU.utf8
MESSAGES: C
MONETARY: hu_HU.utf8
NUMERIC: hu_HU.utf8
TIME: hu_HU.utf8
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "hungarian".
creating directory
/home/zozo/lock-timeout/9.1/1/postgresql.14/src/test/regress/./tmp_check/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 32MB
creating configuration files ... ok
creating template1 database in
/home/zozo/lock-timeout/9.1/1/postgresql.14/src/test/regress/./tmp_check/data/base/1 ... ok
initializing pg_authid ... TRAP: FailedAssertion("!(base_timeouts_initialized)", File:
"timeout.c", Line: 217)
sh: line 1: 29872 Aborted (core dumped)
"/home/zozo/lock-timeout/9.1/1/postgresql.14/src/test/regress/tmp_check/install/home/zozo/pgc92dev-locktimeout/bin/postgres"
--single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 > /dev/null
child process exited with exit code 134
initdb: data directory
"/home/zozo/lock-timeout/9.1/1/postgresql.14/src/test/regress/./tmp_check/data" not
removed at user's request
---8<--8<--8<--8<--8<--8<--8<---
initdb starts postgres --single, that doesn't do BackendInitialize(),
only PostgresMain(). So, you need InitializeTimeouts() before
the RegisterTimeout() calls in PostgresMain and the elog(PANIC)
must not be in InitializeTimeouts() if called twice.
--
--
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgsql_fdw in contrib
Hi Kaigai-san, Sorry for delayed reply. On Tue, Jun 26, 2012 at 10:50 PM, Kohei KaiGai wrote: > * It seems to me expected results of the regression test is not > attached, even though test cases were included. Please add it. AFAICS the patch I posted contains both test script and expected result. If you have created new git branch and applied patch, you might have forgotten to git-add expected/pgsql_fdw.out. Would you check that? > * cleanup_connection() does not close the connection in case > when this callback was invoked due to an error under sub- > transaction. It probably makes problematic behavior. > > See the following steps to reproduce the matter: Oops, good catch. I could reproduce this problem. > Once we got an error under the remote transaction, it eventually raises > an error on local side, then cleanup_connection() should be invoked. > But it ignores the error due to sub-transaction, thus, the remote > transaction > being already aborted is kept. > I'd like to suggest two remedy. > 1. connections are closed, even if errors on sub-transaction. > 2. close the connection if PQexecParams() returns an error, > on execute_query() prior to raise a local error. Background: the reason why pgsql_fdw doesn't disconnect at errors in sub-transaction is to allow FETCHing from cursor which uses foreign table and is opened before the local error. How about to close connection when released connection is in failed transaction? It can be determined with PQtransactionStatus() in ReleaseConnection(). > * Regarding to deparseSimpleSql(), it pulls attributes being referenced > from baserestrictinfo and reltargetlist using pull_var_clause(). > Is it unavailable to use local_conds instead of baserestrictinfo? > We can optimize references to the variable being consumed at the > remote side only. All we need to transfer is variables referenced > in targetlist and local-clauses. It sounds possible. > In addition, is pull_var_clause() reasonable to list up all the attribute > referenced at the both expression tree? It seems to be pull_varattnos() > is more useful API in this situation. Oh, I didn't know that. It seems more efficient, I'll check it out. > * Regarding to deparseRelation(), it scans simple_rte_array to fetch > RangeTblEntry with relation-id of the target foreign table. > It does not match correct entry if same foreign table is appeared in > this list twice or more, like a case of self-join. I'd like to recommend > to use simple_rte_array[baserel->relid] instead. > In addition, it checks whether relkind is RELKIND_FOREIGN_TABLE, > or not. It seems to me this check should be Assert(), if routines of > pgsql_fdw is called towards regular relations. > > * Regarding to deparseVar(), is it unnecessary to check relkind of > the relation being referenced by Var node, isn't it? > > * Regarding to deparseBoolExpr(), compiler raised a warning > because op can be used without initialized. > > * Even though it is harmless, sortConditions() is a misleading function > name. How about categolizeConditions() or screeningConditions()? I'll reply these comments in another post. Regards, -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] enhanced error fields
Hello 2012/7/3 Matthew Woodcraft : > Peter Geoghegan writes: >> So I took a look at the patch eelog-2012-05-09.diff today. All of the >> following remarks apply to it alone. > > I've been trying out this patch for my own interest (I'm very pleased to > see work on this feature), and I have a couple of suggestions from a > user's point of view. > > > First: if a not null constraint is violated, the error report includes > CONSTRAINT NAME 'not_null_violation'. I think I would find it more > useful if CONSTRAINT NAME were left unset rather than given a value that > doesn't correspond to a real constraint. A client program can tell it's > a null constraint violation from the SQLSTATE. > I don't think so generation some special name is good idea. In this case - important values are in COLUMN_NAME, TABLE_NAME, SCHEMA_NAME postgres=# create table ff(a int not null); CREATE TABLE postgres=# \set VERBOSITY verbose postgres=# insert into ff values(null); ERROR: 23502: null value in column "a" violates not-null constraint DETAIL: Failing row contains (null). LOCATION: ExecConstraints, execMain.c:1527 COLUMN NAME: a TABLE NAME: ff SCHEMA NAME: public CONSTRAINT NAME: not_null_violation CONSTRAINT SCHEMA: public > > Second: in the case where a foreign-key constraint is violated by a > change in the primary-key table, the error report gives the following > information: > > TABLE NAME: name of primary-key table > SCHEMA NAME: schema of primary-key table > CONSTRAINT NAME: name of foreign-key constraint > CONSTRAINT SCHEMA: schema of foreign-key table > postgres=# create table a1(a int primary key); NOTICE: 0: CREATE TABLE / PRIMARY KEY will create implicit index "a1_pkey" for table "a1" LOCATION: DefineIndex, indexcmds.c:600 CREATE TABLE postgres=# create table a2(a int references a1(a)); CREATE TABLE postgres=# insert into a2 values(10); ERROR: 23503: insert or update on table "a2" violates foreign key constraint "a2_a_fkey" DETAIL: Key (a)=(10) is not present in table "a1". LOCATION: ri_ReportViolation, ri_triggers.c:3228 TABLE NAME: a2 SCHEMA NAME: public CONSTRAINT NAME: a2_a_fkey CONSTRAINT SCHEMA: public postgres=# \d a2 Table "public.a2" Column │ Type │ Modifiers ┼─┼─── a │ integer │ Foreign-key constraints: "a2_a_fkey" FOREIGN KEY (a) REFERENCES a1(a) I agree so access to related table is not simple, but you know constraint name, and you can take referenced table from constraint definition. so any special column is not necessary. It can be done in future, but for this moment I would add only really necessary fields. > It doesn't include the name of the foreign-key table (except in the > human-readable error message). But in principle you need to know that > table name to reliably identify the constraint that was violated. > > I think what's going on is a mismatch between the way the constraint > namespace works in the SQL standard and in PostgreSQL: it looks like the > standard expects constraint names to be unique within a schema, while > PostgreSQL only requires them to be unique within a table. (A similar > issue makes information_schema less useful than the pg_ tables for > foreign key constraints.) > > So I think it would be helpful to go beyond the standard in this case > and include the foreign-key table name somewhere in the report. > > Possibly the enhanced-error reports could somehow add the table name to > the string in the CONSTRAINT NAME field, so that the interface > PostgreSQL provides looks like the one the standard envisages (which > ought to make it easier to write cross-database client code). same situation is with triggers I prefer add two new fields CONSTRAINT_TABLE and TRIGGER_TABLE so NAME, TABLE and SCHEMA is unique Regards and thank you for comments Pavel > > Or it might be simpler to just add a new enhanced-error field; I can > imagine cases where that table name would be the main thing I'd be > interested in. > > -M- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug #6593, extensions, and proposed new patch policy
Alvaro Herrera writes: > Applied to master, 9.2 and 9.1. Thank you! Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Oracle porting sample instr function
Robert Haas wrote: >> I can confirm that Oracle returns 0 if the third argument to >> INSTR is 0. > Can someone provide a suitable doc patch? Here you are. Yours, Laurenz Albe instr-doc.patch Description: instr-doc.patch -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgfoundry references in docs
On Wed, Jul 4, 2012 at 9:18 AM, Albe Laurenz wrote: > Magnus Hagander wrote: >> Attached are two patches, one of which I'd like to apply. Open for >> discussion on which one. >> >> The smaller one, pgfoundry_1.diff, removes the suggestion to apply for >> new projects on pgfoundry. The reason for this being that pgfoundry >> doesn't *accept* new projects anymore. >> >> The second one removes the reference to pgfoundry completely. As a >> step in the deprecation. >> >> I'd prefer to apply the second one, but will settle for the first one >> if people object ;) > > Well, I don't object to the documentation change, but I have a problem > with the fact. > > Are there any other places that could be recommended for hosting > my pgFoundry projects? > If yes, that should be mentioned in the documentation. Exiting pgfoundry projects are perfectly safe for now - but *new* projects are not accepted. There is a project underway (for a *long* time - it keeps getting stalled) working on migration paths. Until such paths are available and documented, existing projects will still be safe on pgfoundry. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgfoundry references in docs
Magnus Hagander wrote: > Attached are two patches, one of which I'd like to apply. Open for > discussion on which one. > > The smaller one, pgfoundry_1.diff, removes the suggestion to apply for > new projects on pgfoundry. The reason for this being that pgfoundry > doesn't *accept* new projects anymore. > > The second one removes the reference to pgfoundry completely. As a > step in the deprecation. > > I'd prefer to apply the second one, but will settle for the first one > if people object ;) Well, I don't object to the documentation change, but I have a problem with the fact. Are there any other places that could be recommended for hosting my pgFoundry projects? If yes, that should be mentioned in the documentation. Yours, Laurenz Albe -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Solaris docs
On Tue, Jul 3, 2012 at 9:58 PM, Bjorn Munch wrote: > On 03/07 21.24, Magnus Hagander wrote: >> Our documentation still refers to PostgreSQL bundled in solaris, and >> references downloads to the pgfoundry project. > > Oh! I wasn't aware of that. > >> Attached patch removes all those references completely, since Solaris >> packages are now downloaded the same way as other binaries. It leaves >> all the notes about how to compile on solaris and other platform >> details, of course. >> >> Any objections? > > No objections from here. ok, applied. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgfoundry references in docs
On Tue, Jul 3, 2012 at 10:01 PM, David E. Wheeler wrote: > On Jul 3, 2012, at 9:20 PM, Magnus Hagander wrote: > >> The smaller one, pgfoundry_1.diff, removes the suggestion to apply for >> new projects on pgfoundry. The reason for this being that pgfoundry >> doesn't *accept* new projects anymore. > > Should you not perhaps recommend that they go somewhere else? Not really. We have nowhere else to recommend, since we don't run a replacement for it. And we really don't want to get involved in listing all the different third party sites out there. (For example, we had a reference to sourceforge.net in the same paragraph. And while that was certainly "state of the art" when the docs were written, I don't think anybody sane would recommend that today. The reality keeps changing on those things, so it really doesn't belong in the docs). We could put a set of links on the wiki if we want something "more live". -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgfoundry references in docs
On Tue, Jul 3, 2012 at 10:32 PM, Dave Page wrote: > > > On Tuesday, July 3, 2012, Peter Geoghegan wrote: >> >> On 3 July 2012 20:20, Magnus Hagander wrote: >> > The second one removes the reference to pgfoundry completely. As a >> > step in the deprecation. >> > >> > I'd prefer to apply the second one, but will settle for the first one >> > if people object ;) >> >> I'd also prefer if you applied the second one. > > > +1 Since all those who commented preferred that option, I've applied that patch. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proof concept - access to session variables on client side
Le 3 juil. 2012 à 19:53, Peter Eisentraut a écrit : > I think it would be better if DO could be extended into some kind of > "lambda", taking parameters and returning a value. Then you can use > existing infrastructure for passing values and saving the return. It > would also extend better to other languages. +1 -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
