Re: [HACKERS] Index AM change proposals, redux
>> * Proposed changes to allow amgetnext to return the actual index keys, >> allowing certain types of "unindexable" quals to be checked without >> having to fetch the heap entry. For example a LIKE condition could be >> fully checked against the index entry. In the extreme we could build tuples and push them up several nodes -- even including joins -- before fetching the rest of the attributes from the heap. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting a pre-existing index as a primary key
On Tue, Apr 8, 2008 at 10:03 PM, Jonah H. Harris <[EMAIL PROTECTED]> wrote: > On Tue, Apr 8, 2008 at 9:04 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > > "Jonah H. Harris" <[EMAIL PROTECTED]> writes: > > > I've run into a couple cases now where it would be helpful to easily > > > assign an already-existing unique index as a primary key. > > > > You need to present a more convincing use-case than this unsupported > > assertion. There's hardly any effective difference between a unique > > index + NOT NULL constraints and a declared primary key ... so what > > did you really need it for? > > Agreed, functionally there's not much of a difference. It's more of a > matter of proper design identifying a primary key. > set right constraints it's good for documenting the system itself, i like the idea... -- regards, Jaime Casanova -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Apr 10, 2008 at 3:24 PM, Tom Lane wrote: > "Brendan Jurd" writes: > > Personally I don't feel that using a patch tracker or wiki is any more > > onerous than using an email list, and it's a whole lot more responsive > > There are a couple of differences in my mind. One is that the email > list provides an automatic historical archive, whereas trackers tend to > be all about current state. They do? We must be thinking about different species of trackers then, because not only do trackers like bugzilla and Trac have automatic historical archives, but they are much easier to query than an email archive. Even a wiki page has a built-in history (list of changes) although admittedly this is not friendly to queries like "have we already rejected similar patches" and such. > Another is that the email list provides a > "push" mechanism for putting the proposed patch under the noses of a > bunch of people, a few of whom will hopefully take a sniff ;-). > A tracker is very much more of a "pull" scenario where someone has to > actively go looking for pending/proposed changes. > The typical way to solve this is to have the tracker send an automatic notification email to a list saying "Hey, there's a new ticket at , come and check it out". This is trivial to configure in a "real" tracker. Less so for a wiki page, but it could still be accomplished with the careful application of script-fu. Cheers, BJ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: http://getfiregpg.org iD8DBQFH/asy5YBsbHkuyV0RAkJoAJ9yJCZ1da5FN015xxmKY1g5LbAzygCfUpXQ xJ0S/rcBzv2S3L8mQxz9aAs= =MzEJ -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Index AM change proposals, redux
Tom Lane wrote: * Proposed change to let both amgetnext and amgetmulti mark returned tuples as "candidate matches", that is in need of rechecking of quals against the real heap tuple. I had originally objected to this on the grounds that it would require setup work that doesn't happen now, but looking at the code I see that's wrong --- the setup work *does* happen anyway, see indexqualorig. nodeIndexscan.c comments "The indexqualorig expression is always initialized even though it will only be used in some uncommon cases --- would be nice to improve that". So this complaint is probably a red herring. I'm still a bit concerned by the fact that the patch only tracks this on a page basis in the amgetmulti case: if any of the tuples on a page are in-doubt then everything on the page will have to be rechecked. Is that good enough? I think it's good enough. If we wanted to track it on a per-tuple basis, we'd need to store two bits per tuple, AFAICS, doubling the size of the bitmaps. A bigger issue is whether this is worth applying when nobody seems to be working on either of the main uses for it (bitmap indexes and GIT indexes). There seemed to be some possible marginal use for it in GIST indexes, but I'm not convinced that's a sufficient reason to complicate the APIs. It has some merit on its own. Consider the case where you bitmap AND a lossy page and a non-lossy one. We currently make the result lossy, because we can't know which tuples on the page match, and then we need to recheck all tuples on that page. With the support for candidate matches, we could instead keep the non-lossy version of the bitmap and mark the matches as candidates. In the very best case, we might even apply a further AND to the page, which eliminates all the remaining candidate matches, and skip the heap access of that page altogether. * Proposed change to let amgetnext mark returned tuples as not necessarily in order, requiring somebody downstream to sort them again. I was pretty desperately unhappy with that because it required injection of sorting knowledge into code that really shouldn't have anything to do with it --- not least because not all indexscans even have a defined ordering. Given that it is only needed for one particular possible implementation of GIT, and Heikki was leaning away from that implementation anyway at last report, I vote against considering this any further. Agreed, there's no use for that without GIT. * Proposed changes to refactor the TIDBitmap support to include a concept of a "stream bitmap" being read from disk. (Not strictly an index AM change, but closely related.) While this is clean and nice on its own, it doesn't seem to have any use unless bitmap indexes happen. If we leave the code sit it'll probably acquire bit rot, but I'm disinclined to add a bunch of unused code, too. Thoughts? If the code is unused, it can easily bit rot even if it's in CVS. Let's not apply it. The patch is out there if/when someone picks up the bitmap index patch again. * Proposed changes to allow amgetnext to return the actual index keys, allowing certain types of "unindexable" quals to be checked without having to fetch the heap entry. For example a LIKE condition could be fully checked against the index entry. Since certain types of indexes (GIN now, possibly hash in future) are incapable of doing this, there'd need to be a way of marking index AMs (or perhaps individual indexes) as able or not able to return their keys, so that the planner could know whether quals could be pushed into the indexscan. We don't have any actual submitted patch for this, but AFAIR everyone agrees it's a good idea. Agreed :-). * GIT (Grouped Index Tuple) indexes, which achieve index space savings in btrees by having a single index tuple represent multiple heap tuples (on a single heap page) containing a range of key values. I am not sure what the development status is --- Heikki had submitted a completed patch but there seemed to be agreement on making changes, and that's not been done AFAIK. The really serious problem I've got with it is that it'd foreclose the possibility of returning actual index keys from btree indexes, thus basically killing the usefulness of that idea. I'm not convinced it would offer enough gain to be worth paying that price. Another issue is that we'd need to check how much of the use-case for GIT has been taken over by HOT. I don't think there's much overlap with HOT. On the contrary, HOT makes GIT much more useful, because GIT was very sensitive to the cluster order of the heap, and HOT helps with that. There is, however, a ton of overlap with index-only scans, and the possibility to return keys from indexes, as you pointed out. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
"Brendan Jurd" <[EMAIL PROTECTED]> writes: > Personally I don't feel that using a patch tracker or wiki is any more > onerous than using an email list, and it's a whole lot more responsive There are a couple of differences in my mind. One is that the email list provides an automatic historical archive, whereas trackers tend to be all about current state. Another is that the email list provides a "push" mechanism for putting the proposed patch under the noses of a bunch of people, a few of whom will hopefully take a sniff ;-). A tracker is very much more of a "pull" scenario where someone has to actively go looking for pending/proposed changes. Obviously there are virtues on both sides of this, which is why I think we need both mechanisms. The simplest way to integrate them AFAICS is to use the tracker as an index on the email traffic. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Free Space Map data structure
PFC wrote: About the FSM : Would it be possible to add a flag marking pages where all tuples are visible to all transactions ? (kinda like frozen I think) Ah, the visibility map. That's another line of discussion. The current plan is to not tie that to the FSM, but implement it separately. There's some infrastructure changes that are needed for both, like the "map forks" (see recent FSM discussions), which is why we need to have a design for FSM as well before we start implementing the visibility map. It's definitely something I want to do for 8.4. Here's my rough plan: 1. Common "map forks" support 2. Rewrite FSM 3. Implement visibility map, to allow partial vacuums 4. Implement index-only scans, using the visibility map. We'll see how far I get in the 8.4 cycle. Any help is welcome, of course :-). -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Wednesday, April 09, 2008 19:06:30 -0700 "Joshua D. Drake" <[EMAIL PROTECTED]> wrote: > On Wed, 09 Apr 2008 22:59:43 -0300 > "Marc G. Fournier" <[EMAIL PROTECTED]> wrote: > >> Damn, this is starting to get to be a trend ... but, I can't but >> agree 100% with this ... we *can* enforce, and I doubt it will have >> much (if any) affect on the # of patches that come in, since ppl want >> to see their work committed, and will follow any *reaonable* >> procedure we have for them to do so ... >> >> Do other large projects accept patches 'ad hoc' like we do? >> FreeBSD? Linux? KDE? >> > > KDE is fairly structured. Linux... I don't know. I do know that at one > time a very well known Alan slammed a very well known Linux for using > his inbox as a patch queue (funny isn't it, no offense Bruce). FreeBSD I > would think you would know better than I. However even Debian for all > of its warts is *very* structured. FreeBSD, *everything* goes through GnATs ... bug reports *and* patches ... - -- Marc G. FournierHub.Org Hosting Solutions S.A. (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.8 (FreeBSD) iEYEARECAAYFAkf9g/4ACgkQ4QvfyHIvDvNRhgCgtxXpF9+9ruMRfOmVBQsHctfe Z2AAoJbKGw05+BAxLaw38MxjiU41dMrS =O5g6 -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
On Wed, 9 Apr 2008, Marc G. Fournier wrote: Do other large projects accept patches 'ad hoc' like we do? FreeBSD? Linux? KDE? The Linux procedure is documented at http://www.mjmwired.net/kernel/Documentation/SubmittingPatches Linux was forced into some structure by the SCO lawsuit circa 2004, in that they track who patches came from more carefully now. But the process of submission to the Linux kernel developer's mailing list is even less organized than here; as stated in that document, they will drop patches without comment whenever they please. However, they do have a person designated "Trivial Patch Monkey" which is such a great title that you have to forgive the rest of the problems in the process. FreeBSD includes a program called send-pr just to submit "problem reports" into their system which can include feature changes. You can get an idea how sophisticated their tracking for bug patches is by looking at http://www.freebsd.org/cgi/query-pr-summary.cgi?query KDE's process works similarly to here, e-mail based with specific people assigned to track submissions to the various portions of the project: http://developer.kde.org/documentation/other/developer-faq.html#q2.21 GNOME makes all submitters create a report in bugzilla and tracks from there: http://live.gnome.org/GnomeLove/SubmittingPatches Apache also pushes everything through bugzilla: http://httpd.apache.org/dev/patches.html The interesting quote there is: "Traditionally, patches have been submitted on the developer's mailing list as well as through the bug database. Unfortunately, this has made it hard to easily track the patches. And without being able to easily track them, too many of them have been ignored. Patches must now be submitted through the bug database..." The thing that will obviously go away if this project were to switch to such a model is that right now, there are lots of ideas that go by that would never be submitted as patches like that. But Bruce snags them and turns them into todo items and such rather than letting the idea just get lost in the archives. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
On Thu, 10 Apr 2008 12:02:37 +1000 "Brendan Jurd" <[EMAIL PROTECTED]> wrote: > It's not like we'd be asking submitters to draw a chalk circle on > their floor and conduct an arcane ritual. =) It's just a website. No, but that is essentially what we do now :P. Joshua D. Drake -- The PostgreSQL Company since 1997: http://www.commandprompt.com/ PostgreSQL Community Conference: http://www.postgresqlconference.org/ United States PostgreSQL Association: http://www.postgresql.us/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
On Wed, 09 Apr 2008 23:01:30 -0300 "Marc G. Fournier" <[EMAIL PROTECTED]> wrote: > I could see it with older submitters, who are used to just sending an > email, but the new guys will go with whatever procedure is laid out > for them *as long as* it is enforced ... Just as a note... email can be used as a submission procedure to a patch tracker. Joshua D. Drake -- The PostgreSQL Company since 1997: http://www.commandprompt.com/ PostgreSQL Community Conference: http://www.postgresqlconference.org/ United States PostgreSQL Association: http://www.postgresql.us/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
On Wed, 09 Apr 2008 22:59:43 -0300 "Marc G. Fournier" <[EMAIL PROTECTED]> wrote: > Damn, this is starting to get to be a trend ... but, I can't but > agree 100% with this ... we *can* enforce, and I doubt it will have > much (if any) affect on the # of patches that come in, since ppl want > to see their work committed, and will follow any *reaonable* > procedure we have for them to do so ... > > Do other large projects accept patches 'ad hoc' like we do? > FreeBSD? Linux? KDE? > KDE is fairly structured. Linux... I don't know. I do know that at one time a very well known Alan slammed a very well known Linux for using his inbox as a patch queue (funny isn't it, no offense Bruce). FreeBSD I would think you would know better than I. However even Debian for all of its warts is *very* structured. Joshua D. Drake -- The PostgreSQL Company since 1997: http://www.commandprompt.com/ PostgreSQL Community Conference: http://www.postgresqlconference.org/ United States PostgreSQL Association: http://www.postgresql.us/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Marc G. Fournier wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > > > - --On Wednesday, April 09, 2008 21:38:29 -0400 Bruce Momjian <[EMAIL > PROTECTED]> > wrote: > > > > I think there is concern that trivial patches wouldn't be submitted to a > > patch tracker, especially by new submitters. > > Can I see a show of hands as to who (other then Bruce) is concerned that new > submitters are too lazy to submit their patches through a proper patch > tracker > vs simply sending via email? Uh, I am not concerned. I was just stating what others have said. -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Wednesday, April 09, 2008 21:38:29 -0400 Bruce Momjian <[EMAIL PROTECTED]> wrote: > I think there is concern that trivial patches wouldn't be submitted to a > patch tracker, especially by new submitters. Can I see a show of hands as to who (other then Bruce) is concerned that new submitters are too lazy to submit their patches through a proper patch tracker vs simply sending via email? I could see it with older submitters, who are used to just sending an email, but the new guys will go with whatever procedure is laid out for them *as long as* it is enforced ... - -- Marc G. FournierHub.Org Hosting Solutions S.A. (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.8 (FreeBSD) iEYEARECAAYFAkf9dPoACgkQ4QvfyHIvDvP3vwCg5+PG39xHUDhiDUKPcEzH8fZi Ei4An140iNV/q/Ne/B7NpsziNDXQaIoU =PlXR -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Apr 10, 2008 at 11:38 AM, Bruce Momjian wrote: > I think there is concern that trivial patches wouldn't be submitted to a > patch tracker, especially by new submitters. Again, I am willing to > track the ones that aren't in the patch tracker, but then we have two > places where patches exist (perhaps three with the wiki). For new submitters, there's always going to be some set of hoops to jump through. Whether it's emailing -patches, adding a ticket to a Trac or bugzilla, or adding a wiki entry, there will be some process that they need to conform to if they want their patch to get reviewed. Personally I don't feel that using a patch tracker or wiki is any more onerous than using an email list, and it's a whole lot more responsive (i.e., doesn't require Bruce to personally and manually shift the entry from -patches to the queue). It's not like we'd be asking submitters to draw a chalk circle on their floor and conduct an arcane ritual. =) It's just a website. Cheers, BJ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: http://getfiregpg.org iD8DBQFH/XU55YBsbHkuyV0RAg5DAJ9faAwOxlGPGmqkroHLjwkDpE0lWwCfcMrH dDHcBjpQy5uspFljnv463/k= =1Tuq -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Wednesday, April 09, 2008 18:33:30 -0700 "Joshua D. Drake" <[EMAIL PROTECTED]> wrote: > On Wed, 9 Apr 2008 20:50:28 -0400 (EDT) > Bruce Momjian <[EMAIL PROTECTED]> wrote: > >> Greg Smith wrote: >> > Making sure nothing falls through the cracks is exactly the point >> > of an enforced workflow. It might be a manual operation, it might >> > be some piece of software, but ultimately you need a well-defined >> > process where things move around but don't get dropped. Exactly >> > how said enforcement happens is certainly open to discussion though. >> >> As a volunteer organization we don't have much enforcement control. > > We don't? It's like this :) > > "You want to submit a patch, this is how it's done." > "Oh... You don't want to do it that way?" > "Tough" > > Why is it that because we are a volunteer organization we can't have > enforcement? You document the procedure, and every single time the > issue arises you paste a link with that procedure :) Damn, this is starting to get to be a trend ... but, I can't but agree 100% with this ... we *can* enforce, and I doubt it will have much (if any) affect on the # of patches that come in, since ppl want to see their work committed, and will follow any *reaonable* procedure we have for them to do so ... Do other large projects accept patches 'ad hoc' like we do? FreeBSD? Linux? KDE? - -- Marc G. FournierHub.Org Hosting Solutions S.A. (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.8 (FreeBSD) iEYEARECAAYFAkf9dI8ACgkQ4QvfyHIvDvOFgQCfZ74Yefkh3TGxlmoxf6ujI4La VxIAn3dJRWm4pLUn9Qr7Y2zobyCpXHeG =pazk -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
On Wed, 9 Apr 2008 21:40:25 -0400 (EDT) Bruce Momjian <[EMAIL PROTECTED]> wrote: > Joshua D. Drake wrote: > > > And when we did whittle it down to a short list there were still > > > very few people helping. > > > > > > > Yes and that is a much bigger problem. > > I was looking for some more text after that sentence. :-( > Sorry man :). I was afraid if I continued down that path it wouldn't be good. I actually have some ideas on this but I need time to align some ducks first. Sincerely, Joshua D. Drake -- The PostgreSQL Company since 1997: http://www.commandprompt.com/ PostgreSQL Community Conference: http://www.postgresqlconference.org/ United States PostgreSQL Association: http://www.postgresql.us/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
On Wed, 9 Apr 2008 21:38:29 -0400 (EDT) Bruce Momjian <[EMAIL PROTECTED]> wrote: > I think there is concern that trivial patches wouldn't be submitted > to a patch tracker, especially by new submitters. Again, I am > willing to track the ones that aren't in the patch tracker, but then > we have two places where patches exist (perhaps three with the wiki). > Which is horrible. We need to have one place to find this information. Joshua D. Drake -- The PostgreSQL Company since 1997: http://www.commandprompt.com/ PostgreSQL Community Conference: http://www.postgresqlconference.org/ United States PostgreSQL Association: http://www.postgresql.us/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Joshua D. Drake wrote: > > And when we did whittle it down to a short list there were still very > > few people helping. > > > > Yes and that is a much bigger problem. I was looking for some more text after that sentence. :-( -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Brendan Jurd wrote: > > Does that move us in the direction of the patch tracker? That does > > raise the bar for patch submitters, though I would catch any patches > > that weren't in the tracker. > > > > I suppose you could say that it would raise the bar, but for what it's > worth I would much *rather* be maintaining a wiki page / row in a wiki > table than sending emails with attachments to a list. Especially if > the wiki is equipped with clever templates for doing same*. > > When you consider the hours spent reading and understanding existing > code, making changes and compiling/recompiling/regression testing that > a patch author needs to do in order to even create a patch, the extra > five minutes it takes to add a line to a wiki table doesn't really > signify. And pretty much pays for itself in terms of the immediate > satisfaction of knowing that your patch is now safely in the correct > queue. I think there is concern that trivial patches wouldn't be submitted to a patch tracker, especially by new submitters. Again, I am willing to track the ones that aren't in the patch tracker, but then we have two places where patches exist (perhaps three with the wiki). -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
On Wed, 9 Apr 2008 20:50:28 -0400 (EDT) Bruce Momjian <[EMAIL PROTECTED]> wrote: > Greg Smith wrote: > > Making sure nothing falls through the cracks is exactly the point > > of an enforced workflow. It might be a manual operation, it might > > be some piece of software, but ultimately you need a well-defined > > process where things move around but don't get dropped. Exactly > > how said enforcement happens is certainly open to discussion though. > > As a volunteer organization we don't have much enforcement control. We don't? It's like this :) "You want to submit a patch, this is how it's done." "Oh... You don't want to do it that way?" "Tough" Why is it that because we are a volunteer organization we can't have enforcement? You document the procedure, and every single time the issue arises you paste a link with that procedure :) > We can control what we do, but not require others to do anything. We do it all the time now, or does Tom actually like going through all the patches and rewriting half of them? There is a very distinct way we allow things to happen in this community, let's not fool ourselves. It is just a lot of it isn't documented so we have this fuzzy feeling of being all malleable and not in control. > > And when we did whittle it down to a short list there were still very > few people helping. > Yes and that is a much bigger problem. Joshua D. Drake -- The PostgreSQL Company since 1997: http://www.commandprompt.com/ PostgreSQL Community Conference: http://www.postgresqlconference.org/ United States PostgreSQL Association: http://www.postgresql.us/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Apr 10, 2008 at 12:33 AM, Bruce Momjian wrote: > Heikki Linnakangas wrote: > > I still think it would be best if the patch authors did the work. They > > are the ones who care about the patch and want the review, and they're > > in the best position to know what the status of a patch is. Others can > > do it as well of course, in the spirit of a Wiki. > I very much agree. > Does that move us in the direction of the patch tracker? That does > raise the bar for patch submitters, though I would catch any patches > that weren't in the tracker. > I suppose you could say that it would raise the bar, but for what it's worth I would much *rather* be maintaining a wiki page / row in a wiki table than sending emails with attachments to a list. Especially if the wiki is equipped with clever templates for doing same*. When you consider the hours spent reading and understanding existing code, making changes and compiling/recompiling/regression testing that a patch author needs to do in order to even create a patch, the extra five minutes it takes to add a line to a wiki table doesn't really signify. And pretty much pays for itself in terms of the immediate satisfaction of knowing that your patch is now safely in the correct queue. Cheers, BJ * I'd be interested in helping to build such templates -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: http://getfiregpg.org iD8DBQFH/WxK5YBsbHkuyV0RAmQXAJ43vvIPse1HNa3Me92W706kcLeYGgCgiCM3 JjrsH7Ot5uVQrwC9JHT88hQ= =GlX7 -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Index AM change proposals, redux
I am glad you have summarized this. The details of exactly what was being proposed was too complex for me to understand before. --- Tom Lane wrote: > I just finished looking through the various threads about index AM > changes that Bruce has been holding onto in the commit-fest queue. > There seem to be the following issues: > > * Proposed change to have amgetmulti return its results by ORing bits > into a caller-supplied TIDBitmap, rather than the current interface > involving an intermediate TID array. The TID-array design was intended > to be agnostic about what would happen on either side of the API, but it > seems that it's really just equally inconvenient for everybody :-(. > > Although the original motivation for this was to open a door for bitmap > indexes, it seems to me it's worth doing whether or not bitmap indexes > ever happen. It's logically simpler (only one amgetmulti call per scan) > and should save at least a few cycles. I recommend we just go ahead > with this. > > * Proposed change to let both amgetnext and amgetmulti mark returned > tuples as "candidate matches", that is in need of rechecking of quals > against the real heap tuple. I had originally objected to this on > the grounds that it would require setup work that doesn't happen now, > but looking at the code I see that's wrong --- the setup work *does* > happen anyway, see indexqualorig. nodeIndexscan.c comments "The > indexqualorig expression is always initialized even though it will only > be used in some uncommon cases --- would be nice to improve that". > So this complaint is probably a red herring. I'm still a bit concerned > by the fact that the patch only tracks this on a page basis in the > amgetmulti case: if any of the tuples on a page are in-doubt then > everything on the page will have to be rechecked. Is that good enough? > > A bigger issue is whether this is worth applying when nobody seems to be > working on either of the main uses for it (bitmap indexes and GIT > indexes). There seemed to be some possible marginal use for it in GIST > indexes, but I'm not convinced that's a sufficient reason to complicate > the APIs. > > * Proposed change to let amgetnext mark returned tuples as not > necessarily in order, requiring somebody downstream to sort them again. > I was pretty desperately unhappy with that because it required injection > of sorting knowledge into code that really shouldn't have anything to do > with it --- not least because not all indexscans even have a defined > ordering. Given that it is only needed for one particular possible > implementation of GIT, and Heikki was leaning away from that > implementation anyway at last report, I vote against considering this > any further. > > * Proposed changes to refactor the TIDBitmap support to include a > concept of a "stream bitmap" being read from disk. (Not strictly an > index AM change, but closely related.) While this is clean and nice on > its own, it doesn't seem to have any use unless bitmap indexes happen. > If we leave the code sit it'll probably acquire bit rot, but I'm > disinclined to add a bunch of unused code, too. Thoughts? > > * Proposed changes to allow amgetnext to return the actual index keys, > allowing certain types of "unindexable" quals to be checked without > having to fetch the heap entry. For example a LIKE condition could be > fully checked against the index entry. Since certain types of indexes > (GIN now, possibly hash in future) are incapable of doing this, there'd > need to be a way of marking index AMs (or perhaps individual indexes) as > able or not able to return their keys, so that the planner could know > whether quals could be pushed into the indexscan. > > We don't have any actual submitted patch for this, but AFAIR everyone > agrees it's a good idea. > > * Bitmap indexes themselves. As far as I can tell, this development > effort has stalled because of intractable problems with VACUUM. > Can anyone provide a status update on that? > > * GIT (Grouped Index Tuple) indexes, which achieve index space savings > in btrees by having a single index tuple represent multiple heap tuples > (on a single heap page) containing a range of key values. I am not sure > what the development status is --- Heikki had submitted a completed > patch but there seemed to be agreement on making changes, and that's not > been done AFAIK. The really serious problem I've got with it is that > it'd foreclose the possibility of returning actual index keys from btree > indexes, thus basically killing the usefulness of that idea. I'm not > convinced it would offer enough gain to be worth paying that price. > Another issue is that we'd need to check how much of the use-case for > GIT has been taken over by HOT. > > * "Thick" indexes containing copies of tuple visibility info. As far > as I'm concerned, this patch isn't going anywhere :-(. It's got the
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow wrote:
Tom Lane wrote:
Perhaps we could do a partial exposure, where the exported struct
declaration contains "public" fields and there are some "private" ones
after that.
I have another idea. It would remove a boat load of members that would
need to be exposed (may remove them all).
Can we make a variant of PQmakeEmptyPGresult? Maybe something like this:
Here is a quick implementation demonstrating the idea. It is very similar to
the patches internal dupresult function (handlers/utils.c).
/* numParameters, paramDescs, errFields, curBlock, curOffset and spaceLeft
* are not assigned at all, initialized to zero. errMsg is handled by
* PQmakeEmptyPGresult.
*/
PGresult *PQdupPGresult(
PGconn *conn,
PGresult *source,
int numAttributes,
int ntups)
{
PGresult *r;
if(!source || numAttributes < 0 || ntups < 0)
return NULL;
r = PQmakeEmptyPGresult(conn, source->resultStatus);
if(!r)
return NULL;
r->binary = source->binary;
strcpy(r->cmdStatus, source->cmdStatus);
/* assigned by PQmakeEmptyPGresult when conn is not NULL */
if(!conn)
{
r->noticeHooks = source->noticeHooks;
r->client_encoding = source->client_encoding;
}
r->attDescs = (PGresAttDesc *)
pqResultAlloc(r, numAttributes * sizeof(PGresAttDesc), TRUE);
if(!r->attDescs)
{
PQclear(r);
return NULL;
}
r->numAttributes = numAttributes;
r->tuples = (PGresAttValue **)
malloc(ntups * sizeof(PGresAttValue *));
if(!r->tuples)
{
PQclear(r);
return NULL;
}
r->ntups = ntups;
r->tupArrSize = ntups;
return r;
}
--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Tom Lane wrote: Perhaps we could do a partial exposure, where the exported struct declaration contains "public" fields and there are some "private" ones after that. I have another idea. It would remove a boat load of members that would need to be exposed (may remove them all). Can we make a variant of PQmakeEmptyPGresult? Maybe something like this: PGresult *PQdupPGresult( // maybe not the best name? PGconn *conn, PGresult *source, int numAttributes, int ntups); This starts off by calling PQmakeEmptyPGresult and then copying the below members from the provided 'source' result argument. - ExecStatusType resultStatus; - char cmdStatus[CMDSTATUS_LEN]; - int binary; - PGNoticeHooks noticeHooks; - int client_encoding; It would then preallocate attDescs and tuples which would also set numAttributes, ntups & tupArrSize. attdescs and tuples are not duplicated, only allocated based on the 'numAttributes' and 'ntups' arguments. Now libpqtypes only needs direct access to attDescs and tuples, for when it loops array elements or composite fields and copies stuff from the source result. Any interest in this direction? I am still thinking about how to abstract attDesc and tuples, I think it would require more API calls as well. curBlock, curOffset and spaceLeft were never copied (intialized to zero). We don't touch these. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Heikki Linnakangas wrote: > Bruce Momjian wrote: > > Tom Lane wrote: > >> This doesn't seem particularly hard, just a matter of following the > >> relevant mailing lists (mostly -patches, but various offenders send > >> patches elsewhere) and adding links to the current wiki page. > >> > >>> That's where I'd love to have Bruce to help. > >> Bruce has made it perfectly clear that he doesn't want to take on any > >> added maintenance work. > > > > Yea, I would like to reduce the amount of maintenance work I do, not add > > to it. Now, you might say that if this works I will have less > > maintenance work to do because more people will be doing it, but I will > > believe it when I see it. > > Ok. In that case I'd suggest that we do roughly the same thing we did > this commit fest. You, Bruce, collect all the relevant threads into the > patch queue as before, and at the beginning of commit fest someone else > goes through that list and puts the threads that have a commit-fest > worthy patch or design proposal in them to the Wiki (thanks Alvaro for > doing that in this fest!). Then we can use the Wiki page as the official > list during the commit fest. Fine with me, but I was hoping someone would come up with an idea that would reduce what I need to do, like perhaps a new vacuum cleaner. ;-) -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Greg Smith wrote: > Making sure nothing falls through the cracks is exactly the point of an > enforced workflow. It might be a manual operation, it might be some piece > of software, but ultimately you need a well-defined process where things > move around but don't get dropped. Exactly how said enforcement happens > is certainly open to discussion though. As a volunteer organization we don't have much enforcement control. We can control what we do, but not require others to do anything. This makes trying to enforce uniform behavior difficult. Companies have a much easier time with enforcement because there is a paycheck attached to it. > As a sideline observer here it seems to me that Bruce has a good and hard > to replace process to kick this all off already going, so don't mess with > that. It would be nice to find vict...err, volunteers to pull him out of > the later steps though for a net reduction in his time. Simply getting > things organized better from the start should help with getting more > people helping out with review; the common complaint seemed to be "I can't > figure out what to help with in this big mess" which having a summary from > the start should improve. And when we did whittle it down to a short list there were still very few people helping. -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Index AM change proposals, redux
I just finished looking through the various threads about index AM changes that Bruce has been holding onto in the commit-fest queue. There seem to be the following issues: * Proposed change to have amgetmulti return its results by ORing bits into a caller-supplied TIDBitmap, rather than the current interface involving an intermediate TID array. The TID-array design was intended to be agnostic about what would happen on either side of the API, but it seems that it's really just equally inconvenient for everybody :-(. Although the original motivation for this was to open a door for bitmap indexes, it seems to me it's worth doing whether or not bitmap indexes ever happen. It's logically simpler (only one amgetmulti call per scan) and should save at least a few cycles. I recommend we just go ahead with this. * Proposed change to let both amgetnext and amgetmulti mark returned tuples as "candidate matches", that is in need of rechecking of quals against the real heap tuple. I had originally objected to this on the grounds that it would require setup work that doesn't happen now, but looking at the code I see that's wrong --- the setup work *does* happen anyway, see indexqualorig. nodeIndexscan.c comments "The indexqualorig expression is always initialized even though it will only be used in some uncommon cases --- would be nice to improve that". So this complaint is probably a red herring. I'm still a bit concerned by the fact that the patch only tracks this on a page basis in the amgetmulti case: if any of the tuples on a page are in-doubt then everything on the page will have to be rechecked. Is that good enough? A bigger issue is whether this is worth applying when nobody seems to be working on either of the main uses for it (bitmap indexes and GIT indexes). There seemed to be some possible marginal use for it in GIST indexes, but I'm not convinced that's a sufficient reason to complicate the APIs. * Proposed change to let amgetnext mark returned tuples as not necessarily in order, requiring somebody downstream to sort them again. I was pretty desperately unhappy with that because it required injection of sorting knowledge into code that really shouldn't have anything to do with it --- not least because not all indexscans even have a defined ordering. Given that it is only needed for one particular possible implementation of GIT, and Heikki was leaning away from that implementation anyway at last report, I vote against considering this any further. * Proposed changes to refactor the TIDBitmap support to include a concept of a "stream bitmap" being read from disk. (Not strictly an index AM change, but closely related.) While this is clean and nice on its own, it doesn't seem to have any use unless bitmap indexes happen. If we leave the code sit it'll probably acquire bit rot, but I'm disinclined to add a bunch of unused code, too. Thoughts? * Proposed changes to allow amgetnext to return the actual index keys, allowing certain types of "unindexable" quals to be checked without having to fetch the heap entry. For example a LIKE condition could be fully checked against the index entry. Since certain types of indexes (GIN now, possibly hash in future) are incapable of doing this, there'd need to be a way of marking index AMs (or perhaps individual indexes) as able or not able to return their keys, so that the planner could know whether quals could be pushed into the indexscan. We don't have any actual submitted patch for this, but AFAIR everyone agrees it's a good idea. * Bitmap indexes themselves. As far as I can tell, this development effort has stalled because of intractable problems with VACUUM. Can anyone provide a status update on that? * GIT (Grouped Index Tuple) indexes, which achieve index space savings in btrees by having a single index tuple represent multiple heap tuples (on a single heap page) containing a range of key values. I am not sure what the development status is --- Heikki had submitted a completed patch but there seemed to be agreement on making changes, and that's not been done AFAIK. The really serious problem I've got with it is that it'd foreclose the possibility of returning actual index keys from btree indexes, thus basically killing the usefulness of that idea. I'm not convinced it would offer enough gain to be worth paying that price. Another issue is that we'd need to check how much of the use-case for GIT has been taken over by HOT. * "Thick" indexes containing copies of tuple visibility info. As far as I'm concerned, this patch isn't going anywhere :-(. It's got the same showstopper problem as retail vacuum and some of the earlier forms of the HOT patch: it assumes that during tuple update or delete, it can re-find the tuple's index entries by re-computing the indexed expressions. That's just not reliable enough. Is that a reasonable summary of where we are? regards, tom lane -- Sent via pgsql-hackers mailing list (pgs
Re: [HACKERS] Commit fest queue
On Wed, 9 Apr 2008, Tom Lane wrote: Gregory Stark <[EMAIL PROTECTED]> writes: What would move us in the direction of this mythical "patch tracker" would be if we knew exactly what our workflow was. Once we know what our workflow is then we could pick a tool which enforces that workflow. Well, I don't think we want or need an "enforced" workflow. What we need is just a list of pending patches so that nothing falls through the cracks. Making sure nothing falls through the cracks is exactly the point of an enforced workflow. It might be a manual operation, it might be some piece of software, but ultimately you need a well-defined process where things move around but don't get dropped. Exactly how said enforcement happens is certainly open to discussion though. Last time I chimed in on this subject I tried unsuccessfully to move discussion into this area--trying to nail down the structure of a patch processing workflow--but all I managed to do was kick off was a discussion of the trivia involved with one step. A better attempt is below. As you say, most of the work is in recognizing which emails deserve to be entered into the list, and that's not subject to automation (not in this decade anyway). Sure, but that can still be an input to the workflow. Since I'm unphased by criticism and have been watching this whole 'Fest fairly closely, I'll even throw out a sample for a more formal workflow outline. Always easier to map this stuff out when you've got a dummy proposal to beat up. This is aimed to look somewhat like what happened this time around (except using the newer tools that are basically built now) rather than to be a more grand vision: Input: submissions to -patches and -hackers Processing: Saved via mail reader software Output: mbox file with relevant items Person: Bruce Input: mbox file Processing: Run script Output: Patch queue detail wiki page, with links to the archives Person: Greg Stark via his script Input: Patch queue detail Processing: Manually editing page, perhaps with some tool assistance Output: Patch queue summary wiki page Person: Alvaro Input: Patch queue summary Processing: Patch committed, removed from page Output: Updated patch queue summary, e-mail to author Person: Tom, Bruce, other committers Input: Patch queue summary Processing: Patch changed to be a TODO item Output: Expanded TODO list, updated patch queue summary, e-mail to author Person: Bruce Input: Patch queue summary Processing: Patch rejected or bounced back with comments Output: Reduced patch queue summary, e-mail to author Person: Bruce There's a clear hole for messages to fall into when they're being summarized into the patch summary step, I recall Tom saying something about items that didn't make it into the current summary. That needs to be improved a bit. I also note that I didn't diagram separate review steps because I didn't see them happen in a formal way this time around that I could use as a model. As a sideline observer here it seems to me that Bruce has a good and hard to replace process to kick this all off already going, so don't mess with that. It would be nice to find vict...err, volunteers to pull him out of the later steps though for a net reduction in his time. Simply getting things organized better from the start should help with getting more people helping out with review; the common complaint seemed to be "I can't figure out what to help with in this big mess" which having a summary from the start should improve. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Free Space Map data structure
About the FSM : Would it be possible to add a flag marking pages where all tuples are visible to all transactions ? (kinda like frozen I think) This could be useful to implement index-only scans, for count(), or to quickly skip rows when OFFSET is used, or to use only the index when the selected columns are all in the index. Of course if the page is flagged as "may contain updated tuples", then it would have to look in the heap. But, for tables that are not randomly updated (for instance tables that are mostly appended to, like forum posts, or logs, or the huge archive table, etc) it could save a lot of heap lookups and IO. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Free Space Map data structure
On Wed, 2008-04-09 at 21:09 +0300, Heikki Linnakangas wrote: > Hannu Krosing wrote: > > Saving 1 byte is an atomic op > > Unfortunately, it's not. Most if not all modern CPUs will perform one > byte modification as "load word" + "modify word" + "save word". Hmm, maybe we I should change my design to modify page free info and its parent together ? or what is word ? 2 bytes ? 4 bytes ? even 8 bytes ? Hannu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] jaguar is failing
On Mon, Mar 31, 2008 at 4:02 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] writes: > > Did you commit a patch already, do you want me to rerun the build? > > Yes; no --- looks like it'll run by itself in an hour anyay. I set up a new box with -DCLOBBER_CACHE_ALWAYS called pigeon so that we now have 2 boxes running this option. The build is scheduled once a day. For the record, it takes nearly 6 hours to run all the tests. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] Integer datetime by default
On Sun, Mar 30, 2008 at 6:29 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > Neil Conway <[EMAIL PROTECTED]> writes: > > Applied to HEAD. > > At this point it would probably be a good idea if a couple of buildfarm > machines were to start testing builds with --disable-integer-datetimes > ... any volunteers out there? I set up a new animal called marten with --disable-integer-datetimes. http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=marten&dt=2008-04-09%2020:45:22 -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Gregory Stark <[EMAIL PROTECTED]> writes: > I suggested eliminating pgsql-patches as a > separate mailing list for people to send mail to. > Instead you could subscribe to a version of pgsql-hackers which automatically > had large attachments removed and replaced with a link to the file on a web > page. Interesting thought. I dunno how implementable it is either, but it definitely would help to cut down fragmentation of discussions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Hmm. I guess it wouldn't be completely out of the question to expose >> the contents of PGresult as part of libpq's API. > How about a proxy header (if such an animal exists). A separate header might be a good idea to discourage unnecessary reliance on the struct, but it doesn't change the basic fact that we'd be adding the struct to our ABI and couldn't change it without a major library version number bump. Perhaps we could do a partial exposure, where the exported struct declaration contains "public" fields and there are some "private" ones after that. This would still work for external creation of PGresults, so long as all PGresults are initially manufactured by PQmakeEmptyPGresult. That would give us a little bit of wiggle room ... in particular I'd be very tempted not to expose the fields associated with space allocation (we could export pqResultAlloc instead), nor anything not foreseeably needed by libpgtypes. > Maybe it is > possible to take pg_result, and all structs it references, and put it in > result-int.h. I'd think something like libpq-result.h would be a better choice of name. The other seems likely to collide with who-knows-what from other packages, regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Florian Pflug <[EMAIL PROTECTED]> writes: > But isn't that an argument *for* having support for the binary format in > libpq in a form similar to what this patch offers? Then at least you'd > be safe as long as your libpq-version is >= your server version. > Currently, there seems to be no safe way to use the binary format, That's right, there isn't, and it's folly to imagine that anything like this will make it "safe". What we really put in the binary I/O support for was for COPY BINARY, with a rather limited use-case of fast dump and reload between similar server versions (you'll notice pg_dump does NOT use it). Yeah, we expose it for clients to use, but they had better understand that they are increasing their risk of cross-version portability problems. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault using heap_form_tuple
"Claudio Rossi" <[EMAIL PROTECTED]> writes: > You guessed it right, I was trying to "CStringGetDatum" into a text field, > now i solved it with > values[n] = DirectFunctionCall1(textin, CStringGetDatum(...string...)); If you're working in CVS HEAD there's an easier way --- see CStringGetTextDatum. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Concurrent psql API
Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: Shane Ambler wrote: So I am thinking something like C-z that will allow you to switch out of a task that is waiting for results without having to stop it with C-c. I agree -- we would need to have a mode on which it is "not on any connection", to which we could switch on C-z. If all connections are busy, there's no way to create a new one otherwise. That would work okay for interactive use and not at all for scripts, which makes it kind of a nonstarter. I'm far from convinced that the case must be handled anyway. If you fat-finger a SQL command the consequences are likely to be far worse than having to wait a bit, so why is it so critical to be able to recover from a typo in a \join argument? I can see that a non-connected prompt would interfere with a script but I would think that a prompt should always be linked to a connection. It may work to get an un-connected prompt made available from C-z which could be limited to only allow new connections or \join commands which would also be limited to interactive input. My first thoughts where that C-z would either drop back to the previous connection or create a new connection either based on the initial login or the connection you are C-z'ing out of. This would be the tricky decider though which may make a limited prompt viable. C-z input detection may also be limited to the wait for query response loop so that it is only available if the current connection is without a prompt. I do think it is useful for more than typo's in the \join command. What about a slip where you forget to \g& the command. Or you start a query that seems to be taking too long, background it and look into what is happening. This would be more helpful to those that ssh into a machine then run psql from there. (I'm also unconvinced that there won't be severe implementation difficulties in supporting a control-Z-like interrupt --- we don't have any terminal signals left to use AFAIK. And what about Windows?) That may be so and could be the decider over whether this can be added or not. Unless Windows steals the input before psql gets it I don't see there will be a problem there. Windows may be a factor in deciding which key to use for this command if it is to be uniform across platforms. -- Shane Ambler pgSQL (at) Sheeky (dot) Biz Get Sheeky @ http://Sheeky.Biz -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Allow COPY from STDIN to absorb all input before throwing an error
On Apr 8, 2008, at 5:10 PM, Stephen Denne wrote: I had an annoying experience with COPY within psql yesterday. I had a dump of just three tables, which I wanted to investigate. I tried loading them into an empty database, using psql's \i command. The table creation failed as dependent tables/sequences where absent. The copy command failed as the tables did not exist. The data intended as the input to the copy statement resulted in a large number of error messages. My idea to avoid this situation is to add an option to COPY that tells it not to throw an error until it runs out of input data. This will not solve the problem, since again it only works if the COPY command gets to execution I brought this up because of a very similar problem a coworker ran into. He did a pg_dumpall and tried to restore it into an existing cluster. One of the tables already existed and didn't have the same columns, so the copy command ran and then failed. And then all hell broke lose. :) This was on 8.1, which AFAIK is using the v3 protocol, so it's still an issue. I can see that there would be a problem if you wrapped the dump into a transaction and something up-stream of the copy failed... I'm not sure on a good way to handle that, perhaps other than switching to \COPY. -- Decibel!, aka Jim C. Nasby, Database Architect [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 smime.p7s Description: S/MIME cryptographic signature
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Tom Lane wrote: But I'll agree that cross-version hazards are a much more clear and present danger. We've already broken binary compatibility at least once since the current binary-I/O system was instituted (intervals now have three fields not two) and there are obvious candidates for future breakage, such as text locale/encoding support. But isn't that an argument *for* having support for the binary format in libpq in a form similar to what this patch offers? Then at least you'd be safe as long as your libpq-version is >= your server version. Currently, there seems to be no safe way to use the binary format, despite it's benefits for moving around large amounts of data. regards, Florian Pflug -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Concurrent psql API
On Apr 9, 2008, at 12:27 PM, Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: Shane Ambler wrote: So I am thinking something like C-z that will allow you to switch out of a task that is waiting for results without having to stop it with C-c. I agree -- we would need to have a mode on which it is "not on any connection", to which we could switch on C-z. If all connections are busy, there's no way to create a new one otherwise. That would work okay for interactive use and not at all for scripts, which makes it kind of a nonstarter. I can't see any need to do this in a script, and in fact I don't think shell scripting supports it. Totally different story for interactive use. Anyone using *nix is likely to be familiar with how job control works in shells and expecting psql to work the same way. We should try and follow the shell standard as much as possible just so that people don't have to re-train themselves. I'm far from convinced that the case must be handled anyway. If you fat-finger a SQL command the consequences are likely to be far worse than having to wait a bit, so why is it so critical to be able to recover from a typo in a \join argument? I find myself doing this frequently with any long-running command, but currently it's a PITA because I'd doing it at the shell level and firing up a new psql: more work than should be necessary, and psql sometimes gets confused when you resume it from the background in interactive mode (stops echoing characters, though maybe this has been fixed). (I'm also unconvinced that there won't be severe implementation difficulties in supporting a control-Z-like interrupt --- we don't have any terminal signals left to use AFAIK. And what about Windows?) That might be true. I don't know if we could use ^z anyway; the shell might have different ideas there. It makes sense if we continue with the shell analogy: the shell prompt is not any particular task. Either there is a task running in foreground (in which case we have no prompt, but we can press C-z to suspend the current task and get a prompt), or there isn't (in which case we have a prompt.) This is nonsense. When you have a shell prompt, you are connected to a shell that will take a command right now. You're always connected to the shell, but if you background something in the shell it becomes a stand-alone job that you're not connected to. You could even think of it as every command you run being a job, it's just a question of if you're actually connected to it or not. -- Decibel!, aka Jim C. Nasby, Database Architect [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 smime.p7s Description: S/MIME cryptographic signature
Re: [HACKERS] Free Space Map data structure
Tom Lane wrote: Heikki Linnakangas <[EMAIL PROTECTED]> writes: ... what I really wanted to discuss is the data structure needed for the Free Space Map. The FSM data structure needs to support two basic operations: 1. Fast lookup of page with >= X bytes of free space 2. Update of arbitrary, individual pages. Our current code doesn't support 2, as we always update the FSM in bulk after vacuum, but we will need that capability to be able to do partial vacuums in the future. Note that the lack of such infrastructure is mainly because we didn't need it, not because it couldn't be done. Yep. I actually remember seeing a function in freespace.c in CVS history to do partial updates, but it was removed at some point because it wasn't needed. One brilliant idea I had, is a binary heap/tree kind of structure, where each heap page is represented by one leaf node. I'm worried about a couple of things here: * Loss of concurrency in access to the FSM itself, due to updates having to change large amounts of interrelated entries, and due to all processes always wanting to touch the root first. When searching for free space, we start from the top, and go down from there. We only need to take a shared lock on the (page that contains) the upper nodes, so that shouldn't become a bottleneck. When we update a node, we can stop propagating the change upwards as soon as we hit a node where the other child has less space than the current node. For example: 5 5 3 45 3 4 3 2 5 2 3 To update the first leaf node from 4 to 2, we need to update it's parent to 3. But we don't need to update it's parent (5), and we can stop at that point without accessing the root. We probably want to mark new pages as full, as Hannu suggested, to avoid repeatedly updating the FSM during bulk loads. Given that the current FSM is protected by a single lock, shared by all relations, and always taken in exclusive mode, I'm not too worried. I'm not sure how long an FSM page needs to be locked in my scheme compared to the current FSM, but at least there will be a separate FSM for each relation. I do want to beat the current FSM in terms of scalability, though, I think I've seen FreeSpaceLock become contented on some tests. As a little optimization, we could optimistically start from somewhere else than the top node when searching for free space. Perhaps from the top of the FSM page that contained our previous rd_targetblock. I don't think we need to, though. * Loss of concurrency in subsequent heap updates. The current FSM gets one thing right: if different backends ask for space, they will (if at all possible) be given different heap pages to insert into, and therefore will not suffer page-level contention while they do their insertions. I don't see what you'll do to ensure a comparable behavior with this structure. Whenever we have a choice to traverse to either left or right child because there's heap pages with enough free space on both sides, we can randomize that decision to achieve the spreading. This actually brings up another nice property this data structure has: instead of randomizing, we can choose the child node that's "closer" to some specific heap page. That can be used to keep an updated tuple version close to the old version, or to maintain cluster order. That's a goal that's at least somewhat at odds with the goal of spreading the inserts, of course, but the data structure has the flexibility, which is nice. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault using heap_form_tuple
> You've omitted the details that probably matter. My guess is that > you're inappropriately converting one of these values to a Datum, > or converting it to a Datum that's not really of the type the > tuple descriptor specifies. > > regards, tom lane Mate, you just won a beer :) You guessed it right, I was trying to "CStringGetDatum" into a text field, now i solved it with values[n] = DirectFunctionCall1(textin, CStringGetDatum(...string...)); Thank you very much! Claudio Rossi -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault using heap_form_tuple
> > nulls = (bool *)palloc(natts*sizeof(bool *)); > > > > May not be related to segfault you are seeing, but this looks completely > wrong. > You want array of bool and not (bool *). Yeah, you are right but in the original code it's: values = (Datum *) palloc(natts * sizeof(Datum)); nulls = (bool *) palloc(natts * sizeof(bool)); I just typed it wrong, I didn't copy and paste. I also forgot to add this line (from valgrind log) at the beginning of error sequence: ==30549== Warning: set address range perms: large range 496753892 (undefined) I'm clueless, any hints? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Bruce Momjian wrote: Tom Lane wrote: This doesn't seem particularly hard, just a matter of following the relevant mailing lists (mostly -patches, but various offenders send patches elsewhere) and adding links to the current wiki page. That's where I'd love to have Bruce to help. Bruce has made it perfectly clear that he doesn't want to take on any added maintenance work. Yea, I would like to reduce the amount of maintenance work I do, not add to it. Now, you might say that if this works I will have less maintenance work to do because more people will be doing it, but I will believe it when I see it. Ok. In that case I'd suggest that we do roughly the same thing we did this commit fest. You, Bruce, collect all the relevant threads into the patch queue as before, and at the beginning of commit fest someone else goes through that list and puts the threads that have a commit-fest worthy patch or design proposal in them to the Wiki (thanks Alvaro for doing that in this fest!). Then we can use the Wiki page as the official list during the commit fest. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault using heap_form_tuple
"Claudio Rossi" <[EMAIL PROTECTED]> writes: > values[0] = ...GetDatum(...my datum...); > ... > values[natts-1] = ...GetDatum(...my datum...); You've omitted the details that probably matter. My guess is that you're inappropriately converting one of these values to a Datum, or converting it to a Datum that's not really of the type the tuple descriptor specifies. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Free Space Map data structure
Hannu Krosing wrote: Sabing 1 byte is an atomic op Unfortunately, it's not. Most if not all modern CPUs will perform one byte modification as "load word" + "modify word" + "save word". -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Gregory Stark <[EMAIL PROTECTED]> writes: > What would move us in the direction of this mythical "patch tracker" would be > if we knew exactly what our workflow was. Once we know what our workflow is > then we could pick a tool which enforces that workflow. Well, I don't think we want or need an "enforced" workflow. What we need is just a list of pending patches so that nothing falls through the cracks. As you say, most of the work is in recognizing which emails deserve to be entered into the list, and that's not subject to automation (not in this decade anyway). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
"Bruce Momjian" <[EMAIL PROTECTED]> writes: >> I still think it would be best if the patch authors did the work. They >> are the ones who care about the patch and want the review, and they're >> in the best position to know what the status of a patch is. Others can >> do it as well of course, in the spirit of a Wiki. > > Does that move us in the direction of the patch tracker? That does > raise the bar for patch submitters, though I would catch any patches > that weren't in the tracker. Not really, whatever tool it's in the hard part is reading the emails and deciding what to do with them. What would move us in the direction of this mythical "patch tracker" would be if we knew exactly what our workflow was. Once we know what our workflow is then we could pick a tool which enforces that workflow. That's the whole point of a special purpose tool for things like this, enforcing that all the ts are crossed, is dotted, and all procedures followed. Eg, requiring that the "Patch status" field be filled in or that two reviewers don't grab the same patch. What a request tracker does *not* do is magically make decisions for us. We still have to review the patches, we still have to make technical decisions. Patches which are too hard to wrap our head around would still sit there until someone stands up and applies or rejects it. (Anyone who's reported a Mozilla bug in the past 8 years might know what such a tracker looks like...) Frankly until we decide what our workflow is and what information we want to track I don't see why -- or even how -- we would pick a tool to track it. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
"Tom Lane" <[EMAIL PROTECTED]> writes: >> If not, would it be possible to some how force reply-to of pg-patches to >> -hackers? > > No, we aren't going to do that. It wouldn't work anyway; you can't > force people to send messages to one list rather than another, and > the mail list software is surely not bright enough to distinguish > "patch" from "not a patch" on its own. I did suggest something a while back that I think died for because too many other things were changing at the same time. Perhaps would be more practical with the current infrastructure. I suggested eliminating pgsql-patches as a separate mailing list for people to send mail to. Instead you could subscribe to a version of pgsql-hackers which automatically had large attachments removed and replaced with a link to the file on a web page. So all followups would be on -hackers because they would follow the headers on the original message. The only place the -noattachments list would show up would be buried in the Received headers. I could help make a perl script to do the message munging but this assumes there's a way to hook it into the mail server and get the files to the web server. I'm not familiar with the infrastructure so I don't know how accessible these things are to each other. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's 24x7 Postgres support! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Tom Lane wrote: Hmm. I guess it wouldn't be completely out of the question to expose the contents of PGresult as part of libpq's API. We haven't changed it often, and it's hard to imagine a change that wouldn't be associated with a major-version change anyhow. We could do some things to make it a bit more bulletproof too, like change cmdStatus from fixed-size array to pointer to allocated space so that CMDSTATUS_LEN doesn't become part of the API. Alternatively, we could keep it opaque and expose a bunch of manipulator routines, but that might be a lot more cumbersome than it's worth. regards, tom lane How about a proxy header (if such an animal exists). Maybe it is possible to take pg_result, and all structs it references, and put it in result-int.h. libpq-int.h would obviously include this. Also, any external library, like libpqtypes, that need direct access to a result can include result-int.h. This keeps pg_result and referenced structs out of libpq-fe.h. From a libpq user's viewpoint, nothing has changed. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Concurrent psql API
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Shane Ambler wrote: >> So I am thinking something like C-z that will allow you to switch out of >> a task that is waiting for results without having to stop it with C-c. > I agree -- we would need to have a mode on which it is "not on any > connection", to which we could switch on C-z. If all connections are > busy, there's no way to create a new one otherwise. That would work okay for interactive use and not at all for scripts, which makes it kind of a nonstarter. I'm far from convinced that the case must be handled anyway. If you fat-finger a SQL command the consequences are likely to be far worse than having to wait a bit, so why is it so critical to be able to recover from a typo in a \join argument? (I'm also unconvinced that there won't be severe implementation difficulties in supporting a control-Z-like interrupt --- we don't have any terminal signals left to use AFAIK. And what about Windows?) > It makes sense if we continue with the shell analogy: the shell prompt > is not any particular task. Either there is a task running in > foreground (in which case we have no prompt, but we can press C-z to > suspend the current task and get a prompt), or there isn't (in which > case we have a prompt.) This is nonsense. When you have a shell prompt, you are connected to a shell that will take a command right now. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault using heap_form_tuple
On Wed, Apr 9, 2008 at 10:48 PM, Claudio Rossi <[EMAIL PROTECTED]> wrote: > nulls = (bool *)palloc(natts*sizeof(bool *)); > May not be related to segfault you are seeing, but this looks completely wrong. You want array of bool and not (bool *). Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Segfault using heap_form_tuple
Greetings, I'm having a problem using heap_form_tuple. I'm trying to form a HeapTuple to use it later with simple_heap_insert. My code looks like this: -- Datum *values; bool *nulls; int natts; TupleDesc tupDesc; HeapTuple tuple; ... tupDesc = RelationGetDescr(...my previously opened relation...); natts = tupDesc->natts; values = (Datum *)palloc(natts*sizeof(Datum *)); nulls = (bool *)palloc(natts*sizeof(bool *)); memset(nulls, false, natts * sizeof(nulls)); values[0] = ...GetDatum(...my datum...); ... values[natts-1] = ...GetDatum(...my datum...); ... tuple = heap_form_tuple(tupDesc, values, nulls); -- When I come to last line, I get these errors (I'm using valgrind): ==25850== Source and destination overlap in memcpy(0x8BCB070, 0x4CF2480, 496753820) ==25850==at 0x4024586: memcpy (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==25850==by 0x808C811: heap_fill_tuple (in /usr/local/pgsql/bin/postgres) ==25850==by 0x808D72F: heap_form_tuple (in /usr/local/pgsql/bin/postgres) ... ==25850== Invalid read of size 1 ==25850==at 0x40245A1: memcpy (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==25850==by 0x808C811: heap_fill_tuple (in /usr/local/pgsql/bin/postgres) ==25850==by 0x808D72F: heap_form_tuple (in /usr/local/pgsql/bin/postgres) ... ==25850== Address 0x8BCB027 is 1 bytes before a block of size 496,753,892 alloc'd ==25850==at 0x4022825: malloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==25850==by 0x82BE278: AllocSetAlloc (in /usr/local/pgsql/bin/postgres) ==25850==by 0x82BEE91: MemoryContextAllocZero (in /usr/local/pgsql/bin/postgres) ==25850==by 0x808D694: heap_form_tuple (in /usr/local/pgsql/bin/postgres) ... ==25850== Address 0x8BCB027 is 2 bytes before a block of size 496,753,892 alloc'd ... ==25850== Address 0x8BCB027 is 3 bytes before a block of size 496,753,892 alloc'd ... ==25850== Address 0x8BCB027 is 4 bytes before a block of size 496,753,892 alloc'd ... ==25850== Process terminating with default action of signal 11 (SIGSEGV) ==25850== Access not within mapped region at address 0x8BCAFFF ==25850==at 0x40245A1: memcpy (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==25850==by 0x808C811: heap_fill_tuple (in /usr/local/pgsql/bin/postgres) ==25850==by 0x808D72F: heap_form_tuple (in /usr/local/pgsql/bin/postgres) If I work on tables with only 1 attribute, I use normal variables (Datum values, bool nulls) instead of arrays, and using heap_form_tuple(TupleDesc td, &values, &nulls) works flawlessly. What am i missing? Thanks in advance for help, Claudio Rossi -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow <[EMAIL PROTECTED]> writes: >>> What parts of PGconn/PGresult do you need to touch that aren't exposed >>> already? > Don't need direct access to PGconn at all. Oh, good, that makes things much easier. > Also, we basically need write access to every member inside a result > object ... since we create our own for arrays and composites by copying > the standard result members over. Hmm. I guess it wouldn't be completely out of the question to expose the contents of PGresult as part of libpq's API. We haven't changed it often, and it's hard to imagine a change that wouldn't be associated with a major-version change anyhow. We could do some things to make it a bit more bulletproof too, like change cmdStatus from fixed-size array to pointer to allocated space so that CMDSTATUS_LEN doesn't become part of the API. Alternatively, we could keep it opaque and expose a bunch of manipulator routines, but that might be a lot more cumbersome than it's worth. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
On Tue, 2008-04-08 at 21:49 -0400, Merlin Moncure wrote:
> > * an "escapeIdent" function.
>
> not sure what this is...
>
Similar to the quote_ident() function in postgresql:
=> select quote_ident('foo"');
quote_ident
-
"foo"""
(1 row)
It could be called something like PQquoteIdent or PQescapeIdent.
Regards,
Jeff Davis
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Tom Lane wrote: That's exactly what I'm strongly suggesting. If you need to include libpq-int.h at all, then your library will be forever fragile, and could very well end up classified as "don't ship this at all, it's too likely to break". regards, tom lane I see your point, and it has a lot of merit. We am completely open to hearing how this can be solved. How do we duplicate a result object and customize many member values after the dup? Do we create a PGresultInfo struct containing all members of a result object that gets passed to "PGresuolt *PQresultDup(PGresult *source, PGresultInfo *info);"? Maybe it has a flags member that indicates which PQresultInfo members contain values that should override the source result. Any suggestions? This is where we are stumped. Everything else has several solutions. We are not debating this anymore, we are trying to implement it. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL fast in PSQL, very slow using MS.NET driver
On Wed, Apr 9, 2008 at 10:56 AM, Hannu Krosing <[EMAIL PROTECTED]> wrote: > > On Wed, 2008-04-09 at 18:33 +0530, Ashish Sharma wrote: > > Hi, > > Hi, all!! > > The setup in question includes PostGRESQL v8.2.4, Java based web > > servers and MS.NET based web servers. Following is the fuzzy > > situation: > > > > 1. Our SQL queries run very fast using PSQL (both, from the > > server as well as the client). > > > > 2. The Java app also retrieves the results very fast (of course, > > we are using Postgres JDBC driver). > > > > 3. But, the same SQL queries perform pathetically slow when > > called from .NET application. The driver being used is NPGSQL. > > > > What queries are you running? What version of Npgsql? Are you using prepared statements? We have performance issues with prepared statements. If it is so, can you try without prepared statements? You can discuss this also in our forums: forums.npgsql.org Thanks in advance. -- Regards, Francisco Figueiredo Jr. fxjr.blogspot.com www.npgsql.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql \du and \dg commands.
David BOURIAUD wrote: Hello, I don't really know since when those commands are provided by psql, but I found them recently and was quite annoyed by the output given by both of them. Not certain since when but I would think from a very early version. Though I find that the \du command's output is quite accurate, I wonder why \dg gives the same informations ? I would have expected to see the group names in a left column, and the list of users that belongs to this group. I know I can get the information by fetching rows of pg_group system table, but I was just wondering about this issue and see what you here would think of it. Thanks for any suggestions about this behavior. Historically old versions had a clear definition between groups and users. 8.1 introduced the role as we use today replacing the users and groups. The views in pg_catalog that are used to replace the old users and groups defines a group as a role that cannot login. This is an approximation only as any role can have members assigned to them as if it was a group, inheriting privileges of that role. And nologin may be assigned to any role for more reasons than to define it as a group. The \dg and \du commands in psql need to remain as the new version can still connect to old servers that don't support the new roles. But with an 8.1 or newer server it can't reliably distinguish between a user and a group role so returns the same info. I guess there is always a chance of someone making a patch that would hide the two options (maybe replace them with \dr?) when connected to an 8.1 or higher server. But I wouldn't expect it any time soon. -- Shane Ambler pgSQL (at) Sheeky (dot) Biz Get Sheeky @ http://Sheeky.Biz -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Aidan Van Dyk <[EMAIL PROTECTED]> writes: > I've often been confused that discussion seem to seamlessly be on either > -patches, or -hackers. From the understanding I got on the mailing > list pages (http://archives.postgresql.org/), it seems like -patches is > supposed to be only for patches, and -hackers for the general > discussion, issues, features, etc on anything development related. That's the theory. > But from observation, it seems like -patches and -hackers are different > lists of the same thing, except that -patches has a much bigger message > size limit. Practice is often different from theory ;-). I don't mind discussion about a patch on -patches, as long as it's not getting into major design decisions --- if it does, then the thread should get moved to -hackers, though that doesn't always happen. > If not, would it be possible to some how force reply-to of pg-patches to > -hackers? No, we aren't going to do that. It wouldn't work anyway; you can't force people to send messages to one list rather than another, and the mail list software is surely not bright enough to distinguish "patch" from "not a patch" on its own. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow <[EMAIL PROTECTED]> writes: > If you are suggesting that libpqtypes should not access internal libpq, > than this idea won't work. We can pull all the code out and hook in, as > you suggested, but we had no plans of abstracting from internal libpq. That's exactly what I'm strongly suggesting. If you need to include libpq-int.h at all, then your library will be forever fragile, and could very well end up classified as "don't ship this at all, it's too likely to break". regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Concurrent psql API
Shane Ambler wrote: > Think jobs in a shell, you can suspend a long running process then send > it to the background to work and go on with something else. > > So I am thinking something like C-z that will allow you to switch out of > a task that is waiting for results without having to stop it with C-c. I agree -- we would need to have a mode on which it is "not on any connection", to which we could switch on C-z. If all connections are busy, there's no way to create a new one otherwise. It makes sense if we continue with the shell analogy: the shell prompt is not any particular task. Either there is a task running in foreground (in which case we have no prompt, but we can press C-z to suspend the current task and get a prompt), or there isn't (in which case we have a prompt.) -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Calling GSoc Mentors
On Wed, 9 Apr 2008 11:18:38 -0400 Alvaro Herrera <[EMAIL PROTECTED]> wrote: > Josh Berkus wrote: > > >> I have seen stupid decisions in the past, but requesting unused > >> addresses to be used for GSoC registration should certainly win a > >> prize somewhere. > > > > Well, why did you use that address for your Google account, then? > > If Google weren't so opaque about creating Google accounts that are > not gmail.com perhaps I would have thought it was at all possible. > (To be fair, I still don't know if it's possible.) It is. I have a [EMAIL PROTECTED] and a [EMAIL PROTECTED] J -- The PostgreSQL Company since 1997: http://www.commandprompt.com/ PostgreSQL Community Conference: http://www.postgresqlconference.org/ United States PostgreSQL Association: http://www.postgresql.us/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Tom Lane wrote: The key phrase in that being "some way or another". Red Hat works with a concept of core vs extras (or another way to look at it being what comes on the CDs vs what you have to download from someplace). I think it's highly likely that libpgtypes would end up in extras. If you do not make it possible to package it that way (ie, separately from libpq), it's more likely that it won't get packaged at all than that it will be put in core. regards, tom lane I'll buy this. Better to be safe then sorry. This patch becomes more flexible and distributable when separated. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
On Wed, Apr 9, 2008 at 11:17 AM, Andrew Chernow <[EMAIL PROTECTED]> wrote:
> We can read args->get.result properties using PQfuncs with no problem. But
> we have no way of assign these values to our result 'r'.
By the way, our decision to 'create the result' when exposing arrays
and composites saved us from creating lot of interface code to access
these structures from user code...the result already gave us what we
needed. Just in case anybody missed it, the way arrays of composites
would be handled in libpq would be like this:
PGarray array;
PQgetf(res, 0, "%foo[]", 0, &array); // foo being a composite(a int, b float)
PQclear(res);
for (i = 0; i < PQntuples(array.res); i++)
{
a int;
b float;
PQgetf(array.res, i, "%int %float", 0, &a, 1, &b);
[...]
}
PQclear(array.res);
In the getf call, the brackets tell libpq that this is an array and a
new result is created to present the array...one column for simple
array, multiple columns if the array is composite. This can be
recursed if the composite is nested. We support all the PGget*
functions for the newly created array, providing the OIDs of the
internal composite elements for example. This is, IMO, payoff of
doing all the work with the type handlers...we don't have to make a
special version of getf that operates over arrays for example.
The upshot of this is that, however separation occurs, the PGresult is
a first class citizen to the types library.
merlin
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Andrew Chernow wrote: >> The core of what I am trying to ask is, there doesn't appear to be an >> advantage to separating libpqtypes from libpq in terms of space. > My guess is that if we provide an useful library, Redhat will distribute > it some way or another. The key phrase in that being "some way or another". Red Hat works with a concept of core vs extras (or another way to look at it being what comes on the CDs vs what you have to download from someplace). I think it's highly likely that libpgtypes would end up in extras. If you do not make it possible to package it that way (ie, separately from libpq), it's more likely that it won't get packaged at all than that it will be put in core. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Calling GSoc Mentors
Josh Berkus wrote: >> I have seen stupid decisions in the past, but requesting unused >> addresses to be used for GSoC registration should certainly win a prize >> somewhere. > > Well, why did you use that address for your Google account, then? If Google weren't so opaque about creating Google accounts that are not gmail.com perhaps I would have thought it was at all possible. (To be fair, I still don't know if it's possible.) -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow wrote: What parts of PGconn/PGresult do you need to touch that aren't exposed already? Don't need direct access to PGconn at all. result: null_field tupArrSize client_encoding (need a PGconn for this which might be dead) pqSetResultError pqResultAlloc pqResultStrdup Also, we basically need write access to every member inside a result object ... since we create our own for arrays and composites by copying the standard result members over. /* taken from dupresult inside handlers/utils.c */ PGresult *r = (PGresult *)malloc(sizeof(PGresult)); memset(r, 0, sizeof(PGresult)); /* copy some data from source result */ r->binary = args->get.result->binary; r->resultStatus= args->get.result->resultStatus; r->noticeHooks = args->get.result->noticeHooks; r->client_encoding = args->get.result->client_encoding; strcpy(r->cmdStatus, args->get.result->cmdStatus); [snip...] We can read args->get.result properties using PQfuncs with no problem. But we have no way of assign these values to our result 'r'. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Aidan Van Dyk wrote: -- Start of PGP signed section. > * Tom Lane <[EMAIL PROTECTED]> [080409 10:40]: > > > This doesn't seem particularly hard, just a matter of following the > > relevant mailing lists (mostly -patches, but various offenders send > > patches elsewhere) and adding links to the current wiki page. > > I've often been confused that discussion seem to seamlessly be on either > -patches, or -hackers. From the understanding I got on the mailing > list pages (http://archives.postgresql.org/), it seems like -patches is > supposed to be only for patches, and -hackers for the general > discussion, issues, features, etc on anything development related. > > But from observation, it seems like -patches and -hackers are different > lists of the same thing, except that -patches has a much bigger message > size limit. > > Is that the intended operational status? > > If not, would it be possible to some how force reply-to of pg-patches to > -hackers? I know, that blows chunks with the usual mailling-list > reply-to issues, but it would make -hackers the single place to follow > discussions, and maybe make -patches a more pointed "list of things to > track". > > Tracking 2 lists really isn't a big deal - I'm guessing most others dump > both -patches and -hackers into the same mailbox anyways too, but it > does seem like the 2 lists are a needless split as they currently are > used. Yea, the split is kind of odd. I think the idea is that discussion about patch details happens on 'patches' and more general discussion on hackers, though it doesn't work that way all the time. -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
* Tom Lane <[EMAIL PROTECTED]> [080409 10:40]: > This doesn't seem particularly hard, just a matter of following the > relevant mailing lists (mostly -patches, but various offenders send > patches elsewhere) and adding links to the current wiki page. I've often been confused that discussion seem to seamlessly be on either -patches, or -hackers. From the understanding I got on the mailing list pages (http://archives.postgresql.org/), it seems like -patches is supposed to be only for patches, and -hackers for the general discussion, issues, features, etc on anything development related. But from observation, it seems like -patches and -hackers are different lists of the same thing, except that -patches has a much bigger message size limit. Is that the intended operational status? If not, would it be possible to some how force reply-to of pg-patches to -hackers? I know, that blows chunks with the usual mailling-list reply-to issues, but it would make -hackers the single place to follow discussions, and maybe make -patches a more pointed "list of things to track". Tracking 2 lists really isn't a big deal - I'm guessing most others dump both -patches and -hackers into the same mailbox anyways too, but it does seem like the 2 lists are a needless split as they currently are used. a. -- Aidan Van Dyk Create like a god, [EMAIL PROTECTED] command like a king, http://www.highrise.ca/ work like a slave. signature.asc Description: Digital signature
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Tom Lane wrote: Andrew Chernow <[EMAIL PROTECTED]> writes: There is no need to pass hookData to the hook function. libpqtypes already accesses PGconn and PGresult directly so it can just access the hookData member. That's a habit you'd really be best advised to stop, if you're going to be a separate library. Otherwise, any time we make an internal change in the PGconn struct, it'll break your library --- and we have and will feel free to do that without an external soname change. What parts of PGconn/PGresult do you need to touch that aren't exposed already? regards, tom lane Well, we manually create a result for arrays and composites. We also use pqResultAlloc in several places. I will have to look at the code again but I'm not sure we can be completely abstracted from the result struct. If you are suggesting that libpqtypes should not access internal libpq, than this idea won't work. We can pull all the code out and hook in, as you suggested, but we had no plans of abstracting from internal libpq. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Bruce Momjian wrote: I think Andrew Chernow is fundamentally confused about dynamic linking, which apache has to use because it doesn't know what type of file it has to handle, with linking, which is bound to the application code. pgtypes is bound to the application code so it is not like apache --- an application isn't going to be fed arbitrary pgtypes function calls it has to handle. This discussion is now completely pointless as we have conceeded to a separate library. The community has spoken. We are trying to move on and open a discussion on the hook design. there are numbered questions: http://archives.postgresql.org/pgsql-hackers/2008-04/msg00565.php -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Tom Lane wrote: > This doesn't seem particularly hard, just a matter of following the > relevant mailing lists (mostly -patches, but various offenders send > patches elsewhere) and adding links to the current wiki page. > > > That's where I'd love to have Bruce to help. > > Bruce has made it perfectly clear that he doesn't want to take on any > added maintenance work. Yea, I would like to reduce the amount of maintenance work I do, not add to it. Now, you might say that if this works I will have less maintenance work to do because more people will be doing it, but I will believe it when I see it. -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > I still think it would be best if the patch authors did the work. They > are the ones who care about the patch and want the review, and they're > in the best position to know what the status of a patch is. Unfortunately, a lot of submitters are way too optimistic about that ... > it's bad because good patches from one-off submitters might fall through > the cracks. Exactly. Somebody (or preferably somebodies) has to accept the responsibility of seeing to it that everything gets onto the commit-fest page; requiring the authors to do it simply won't work reliably. And it'd be more work for them anyway --- consider an author who hasn't got an account on our wiki and/or has never edited a wiki page before. Somebody who does it every day will certainly be a lot faster. This doesn't seem particularly hard, just a matter of following the relevant mailing lists (mostly -patches, but various offenders send patches elsewhere) and adding links to the current wiki page. > That's where I'd love to have Bruce to help. Bruce has made it perfectly clear that he doesn't want to take on any added maintenance work. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Dunstan wrote: > > All I am trying to say is, redhat's core packages are normally very > > inclusive. Like apache, which includes many/most modules in the core > > package. > > > > There are plenty of modules that they don't include, e.g. mod_fastcgi. > If you want that you download and build it against your installed > apache. Or you get mod_fcgid from extras. > > Your bolt-on client library would be in exactly the same boat as one of > those. I think Andrew Chernow is fundamentally confused about dynamic linking, which apache has to use because it doesn't know what type of file it has to handle, with linking, which is bound to the application code. pgtypes is bound to the application code so it is not like apache --- an application isn't going to be fed arbitrary pgtypes function calls it has to handle. -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] psql \du and \dg commands.
Hello, I don't really know since when those commands are provided by psql, but I found them recently and was quite annoyed by the output given by both of them. Though I find that the \du command's output is quite accurate, I wonder why \dg gives the same informations ? I would have expected to see the group names in a left column, and the list of users that belongs to this group. I know I can get the information by fetching rows of pg_group system table, but I was just wondering about this issue and see what you here would think of it. Thanks for any suggestions about this behavior. signature.asc Description: This is a digitally signed message part.
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow wrote: Andrew Dunstan wrote: I don't get what you're not seeing about this. All I am trying to say is, redhat's core packages are normally very inclusive. Like apache, which includes many/most modules in the core package. There are plenty of modules that they don't include, e.g. mod_fastcgi. If you want that you download and build it against your installed apache. Or you get mod_fcgid from extras. Your bolt-on client library would be in exactly the same boat as one of those. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Heikki Linnakangas wrote: > Tom Lane wrote: > > This isn't really about tools. It's about who wants to put in the > > day-after-day, year-after-year drudge work to maintain the queue. > > Whoever wants to do the work can pick their tools... > > I still think it would be best if the patch authors did the work. They > are the ones who care about the patch and want the review, and they're > in the best position to know what the status of a patch is. Others can > do it as well of course, in the spirit of a Wiki. Does that move us in the direction of the patch tracker? That does raise the bar for patch submitters, though I would catch any patches that weren't in the tracker. -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Concurrent psql API
Tom Lane wrote: Shane Ambler <[EMAIL PROTECTED]> writes: When switching to a conn we also need a non-destructive way out if it is busy. Uh, why? Why would you switch to a connection at all, if you didn't want its result? What if you switch to the wrong connection and it hasn't finished. Do you then have to wait until you get the results before you can issue another command? Or will we be able to type commands while we wait for results? I am thinking as currently happens - you can't type a command as you are waiting for a result. So if the connection you switch to is busy but you want to go to another connection then how do you? This may tie into an 'auto new connection'. You start psql enter a command that will take a while then think of something else you can do as you wait. Do you open another shell and start psql again, or send the working task to the background and enter another command in a new connection? Think jobs in a shell, you can suspend a long running process then send it to the background to work and go on with something else. So I am thinking something like C-z that will allow you to switch out of a task that is waiting for results without having to stop it with C-c. -- Shane Ambler pgSQL (at) Sheeky (dot) Biz Get Sheeky @ http://Sheeky.Biz -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow <[EMAIL PROTECTED]> writes: > There is no need to pass hookData to the hook function. libpqtypes already > accesses PGconn and PGresult directly so it can just access the hookData > member. That's a habit you'd really be best advised to stop, if you're going to be a separate library. Otherwise, any time we make an internal change in the PGconn struct, it'll break your library --- and we have and will feel free to do that without an external soname change. What parts of PGconn/PGresult do you need to touch that aren't exposed already? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Dunstan wrote: I don't get what you're not seeing about this. All I am trying to say is, redhat's core packages are normally very inclusive. Like apache, which includes many/most modules in the core package. I am still not convinced there is merit to a separate library. But honestly, I'm indifferent at this point. If the community wants it this way, whether or not I agree with the reasons, then consider it done. It would be helpful to get some feedback about what the requirements are for a hooking mechanism for libpqtypes: 1. Do we make the hooking system generic, for other library to use? 2. If #1 is YES, then does this hooking system need to allow registering multiple hooks per app instance. Meaning, should we implement a table of callbacks/hooks? Like a linked list of them. 3. What design is preferred? *) A single msg proc that uses a msg object which is either a big union or something that gets up casted. *) A separate function pointer per hook, like conn_create, conn_destroy *) A combo, where conn hooks are handled by one funcptr and result hooks by another. But each only has one function. Please think carefully about question #1, as this could lead to over-design or guess-design. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow wrote: > The core of what I am trying to ask is, there doesn't appear to be an > advantage to separating libpqtypes from libpq in terms of space. If > redhat follows their normal policy of include all (probably to make > their distro as feature rich out-of-the-box as possible), then they > would distribute libpqtypes.so which would use the same amount of space > as if it were part of libpq. My guess is that if we provide an useful library, Redhat will distribute it some way or another. In the worst case (i.e. Redhat does not distribute it at all), it will still be available on PGDG rpm repositories. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow wrote: Andrew Chernow wrote: Well, I can get it working with a very small patch. We actually don't need very much in libpq. Although, making it somehow generic enough to be useful to other extensions is a bit tricky. Please, suggestions would be helpful. Quick question on the hook concept before I try to supply a new patch. From my experience, redhat normally compiles everything into their packages; like apache modules. Why would libpq be any different in regards to libpqtypes? If they don't distribute libpqtypes, how does a libpq user link with libpqtypes? They don't have the library. Where would they get a libpqtypes.so that is compatible with redhat's supplied libpq.so? The core of what I am trying to ask is, there doesn't appear to be an advantage to separating libpqtypes from libpq in terms of space. If redhat follows their normal policy of include all (probably to make their distro as feature rich out-of-the-box as possible), then they would distribute libpqtypes.so which would use the same amount of space as if it were part of libpq. They would get it the same way they would get anything else that uses libpq that isn't packaged with it (e.g. DBD::Pg). They would either get the package that contains it, if it exists, or get the source and build it. The package with the dependent library might belong in the extras collection, while Tom's goal (and it's a good one) is to keep libpq in the core collection. I don't get what you're not seeing about this. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL fast in PSQL, very slow using MS.NET driver
On Wed, 2008-04-09 at 18:33 +0530, Ashish Sharma wrote: > Hi, > > The setup in question includes PostGRESQL v8.2.4, Java based web > servers and MS.NET based web servers. Following is the fuzzy > situation: > > 1. Our SQL queries run very fast using PSQL (both, from the > server as well as the client). > > 2. The Java app also retrieves the results very fast (of course, > we are using Postgres JDBC driver). > > 3. But, the same SQL queries perform pathetically slow when > called from .NET application. The driver being used is NPGSQL. > > > > I have tried making some alterations to TCP related system variables > like TCP_NoDelay and TCPAckFrequency on Windows, but to no profit. > Same setup is already running on Oracle backend, and, there is no > difference in SQL response timings from either application. > > > > Please advice, as this has become a major hurdle for us to push > PostGRESQL to be productionized, in place of currently running Oracle > DB. You have to give more info than that to get any meaningful advice . Unless you are just looking for some paid-support guy to contact you ;) And as this seems to be a .NET related problem, you may get a better answer from some dedicated list, maybe http://archives.postgresql.org/pgsql-interfaces/ - Hannu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow wrote: Andrew Chernow wrote: Well, I can get it working with a very small patch. We actually don't need very much in libpq. Although, making it somehow generic enough to be useful to other extensions is a bit tricky. Please, suggestions would be helpful. Quick question on the hook concept before I try to supply a new patch. From my experience, redhat normally compiles everything into their packages; like apache modules. Why would libpq be any different in regards to libpqtypes? If they don't distribute libpqtypes, how does a libpq user link with libpqtypes? They don't have the library. Where would they get a libpqtypes.so that is compatible with redhat's supplied libpq.so? The core of what I am trying to ask is, there doesn't appear to be an advantage to separating libpqtypes from libpq in terms of space. If redhat follows their normal policy of include all (probably to make their distro as feature rich out-of-the-box as possible), then they would distribute libpqtypes.so which would use the same amount of space as if it were part of libpq. By the way, I offered up the idea of compiling our patch in or out, ./configure --with-pqtypes. But Tom had said >>[shrug...] So the packagers will compile it out, and you're >>still hosed, or at least any users who'd like to use it are. If redhat would compile out our patch, no questions asked, why would they distribute libpqtypes.so. Meaning, separating it out doesn't seem to unhose us :) -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow wrote: Well, I can get it working with a very small patch. We actually don't need very much in libpq. Although, making it somehow generic enough to be useful to other extensions is a bit tricky. Please, suggestions would be helpful. Quick question on the hook concept before I try to supply a new patch. From my experience, redhat normally compiles everything into their packages; like apache modules. Why would libpq be any different in regards to libpqtypes? If they don't distribute libpqtypes, how does a libpq user link with libpqtypes? They don't have the library. Where would they get a libpqtypes.so that is compatible with redhat's supplied libpq.so? The core of what I am trying to ask is, there doesn't appear to be an advantage to separating libpqtypes from libpq in terms of space. If redhat follows their normal policy of include all (probably to make their distro as feature rich out-of-the-box as possible), then they would distribute libpqtypes.so which would use the same amount of space as if it were part of libpq. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow wrote: Yeah, currently composites and arrays only support binary results in libpqtypes. This forces any array elementType or any member of a composite to have a send/recv routine. Using the "fallback to text output" approach, this limitation on array elements and composite members would be removed. Actually, I am confusing the way the protocol handles arrays and composites (as a single column value, vs. the way libpqtypes handles these (as a separate result object). for instance, the members of a composite inherit the format of the column within the protocol. To allow one member of a composite to be text formatted and another be binary, would require a change to the protocol, an additional format value per member header. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
On Tue, Apr 08, 2008 at 07:18:31PM -0400, Tom Lane wrote: > sure that there's much potential commonality. The thing that's most > problematic about ecpg is that it wants to offer client-side equivalents > of some server datatype-manipulation functions; and I don't actually see > much of any of that in the proposed patch. All that's really here is > format conversion stuff, so there's no hope of unifying that code > unless this patch adopts ecpg's choices of client-side representation ecpg for instance does not use a struct for time, so there doesn't seem to be an advantance for ecpg in switching. On the other side there's a disadvantage in that you can binary transfer right now given that you're on the same architecture. But if the datatypes differ this would be lost. > (which I believe are mostly Informix legacy choices, so I'm not sure we > want that). Not really. ecpg's pgtypeslib uses the same datatypes as the backend uses (well, mostly because numeric was changed in the backend after the creation of pgtypeslib). Then there's a compatibility layer that maps Informix functions and datatypes to our functions and datatypes. Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] SQL fast in PSQL, very slow using MS.NET driver
Hi, The setup in question includes PostGRESQL v8.2.4, Java based web servers and MS.NET based web servers. Following is the fuzzy situation: 1. Our SQL queries run very fast using PSQL (both, from the server as well as the client). 2. The Java app also retrieves the results very fast (of course, we are using Postgres JDBC driver). 3. But, the same SQL queries perform pathetically slow when called from .NET application. The driver being used is NPGSQL. I have tried making some alterations to TCP related system variables like TCP_NoDelay and TCPAckFrequency on Windows, but to no profit. Same setup is already running on Oracle backend, and, there is no difference in SQL response timings from either application. Please advice, as this has become a major hurdle for us to push PostGRESQL to be productionized, in place of currently running Oracle DB. Regards, Ashish Sharma Core Database Systems HYPERLINK "http://www.coredbsystems.com"http://www.coredbsystems.com Internal Virus Database is out-of-date. Checked by AVG. Version: 7.5.519 / Virus Database: 269.22.5/1356 - Release Date: 4/2/2008 4:14 PM
Re: [HACKERS] Free Space Map data structure
On Wed, 2008-04-09 at 10:17 +0530, Pavan Deolasee wrote: > On Wed, Apr 9, 2008 at 12:22 AM, Heikki Linnakangas > <[EMAIL PROTECTED]> wrote: > > > > > Well, if you add the higher levels, we're no longer talking about O(1), but > > O(log n) (for a pretty steep logarithm), just like my proposal. > > > > For updates, I would still call it O(1) because the number of levels is > limited > and a very small number. > > > > > > It's actually not that orthogonal. You describe it as a hierarchical > > bitmap, but it's essentially the same idea as the binary heap/tree, just > > with way more than 2 children per parent. > > > > Yes, I agree. > > Btw, I agree with Tom's point about the lock contention on the higher levels > for FSM updates. What we can do is during normal operations, FSM pages > are updated with a SHARE lock - so the information can best be considered > only as hint. Hopefully, if we are only doing bit flipping, we won't go wrong > often. And periodically, VACUUM would correct any mistakes in FSM info. Sabing 1 byte is an atomic op so if we update going from branches to root and read from root to branches, then we can probably detect inconsistencies when reading and then re-read. What I'm more concerned about is L1 cache swapping between several processors/cores, which could lead to content switch storms we used to have in locking. anyway, one thing to do for sure is to avoid storing info for new/last page(s) appended by any process before that process is done filling it, as that would lead to excessive up-tree max_free updates. BTW, I'm pretty sure I have figured out the method for storing minimal required binary tree as an array, where adding each page adds exactly one upper node. The node added is often not the immediate parent, but is always the one needed for covering all nodes. I just have to write it up in an understandable way and then you all can look at it and tell if it is something well-known from Knuth or Date ;) -- Hannu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SET TRANSACTION not compliant with SQL:2003
Tom wrote: > So I'm of the opinion that there's no good reason to change either our > code or our docs. The standard-incompatibility is with BEGIN, not > SET TRANSACTION, and it's already documented. Yes. > PS: the proposed patch is buggy as can be anyway: it applies the change > even if !doit, and it causes START TRANSACTION ISOLATION LEVEL xxx > to affect not only the current but the next transaction, which surely > cannot be justified by any reading of the spec ;-) In IBM Informix the command SET TRANSACTION ISOLATION LEVEL xxx, returns an error when issued outside a BEGIN WORK -- COMMIT transaction block. set transaction isolation level read uncommitted; 255: Not in transaction. In their latest docs they state: "The SET TRANSACTION statement complies with ANSI SQL-92." So I agree that there is no need to change what we have. Andreas -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Merlin Moncure wrote: On Wed, Apr 9, 2008 at 8:04 AM, Andrew Dunstan <[EMAIL PROTECTED]> wrote: \> Merlin Moncure wrote: However, due to libpq limitations, if any datatype must return text the entire result must be text (resultFormat)...this is also interestingly true for functions that return 'void'. So, at present, to use PQgetf, you result set must be binary. I'm surprised you didn't try to address that limitation. whoops! we did...thinko on my part. Text results are fully supported save for composites and arrays. merlin Yeah, currently composites and arrays only support binary results in libpqtypes. This forces any array elementType or any member of a composite to have a send/recv routine. Using the "fallback to text output" approach, this limitation on array elements and composite members would be removed. >>That makes it sound more like a protocol limitation, rather than a >>libpq limitation. Or am I misunderstanding? It looks like libpq, message 'T' handling in getRowDescriptions, reads a separate format for each column off the wire. The protocol already has this ability. The backend needs a ?minor? adjustment to make use of the existing capability. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Chernow wrote: Andrew Dunstan wrote: Merlin Moncure wrote: However, due to libpq limitations, if any datatype must return text the entire result must be text (resultFormat)...this is I'm surprised you didn't try to address that limitation. That would change the existing behavior of resultFormat, although not terribly. Currently, the server will spit back an error if you use binary results but some type hasn't implemented a send/recv. Instead of an error, the server could "fallback" to the type's in/out routines and mark the column as text format. I think the "fallback" approach is more intelligent behavior but such a change could break libpq clients. They might be blindly ASSuming if the exec worked with resultFormat=1, that everything returned by PQgetvalue will be binary (I'm guilty of this one, prior to libpqtypes). Our patch would work with no changes because it supports text and binary results. So, each type handler already toggles itself based on PQfformat. That makes it sound more like a protocol limitation, rather than a libpq limitation. Or am I misunderstanding? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Andrew Dunstan wrote: Merlin Moncure wrote: However, due to libpq limitations, if any datatype must return text the entire result must be text (resultFormat)...this is I'm surprised you didn't try to address that limitation. That would change the existing behavior of resultFormat, although not terribly. Currently, the server will spit back an error if you use binary results but some type hasn't implemented a send/recv. Instead of an error, the server could "fallback" to the type's in/out routines and mark the column as text format. I think the "fallback" approach is more intelligent behavior but such a change could break libpq clients. They might be blindly ASSuming if the exec worked with resultFormat=1, that everything returned by PQgetvalue will be binary (I'm guilty of this one, prior to libpqtypes). Our patch would work with no changes because it supports text and binary results. So, each type handler already toggles itself based on PQfformat. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
On Wed, Apr 9, 2008 at 8:04 AM, Andrew Dunstan <[EMAIL PROTECTED]> wrote: \> Merlin Moncure wrote: > > > However, due to libpq limitations, if any datatype must > > return text the entire result must be text (resultFormat)...this is > > also interestingly true for functions that return 'void'. So, at > > present, to use PQgetf, you result set must be binary. > > I'm surprised you didn't try to address that limitation. whoops! we did...thinko on my part. Text results are fully supported save for composites and arrays. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
Merlin Moncure wrote: However, due to libpq limitations, if any datatype must return text the entire result must be text (resultFormat)...this is also interestingly true for functions that return 'void'. So, at present, to use PQgetf, you result set must be binary. I'm surprised you didn't try to address that limitation. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
On Wed, Apr 9, 2008 at 6:13 AM, Gregory Stark <[EMAIL PROTECTED]> wrote:
> > The exclusive use of binary formats is worrisome to me. This circumvents
> > one level of indirection that we have (i.e. that everything moves
> > through in/out functions), and will impose a backwards-compatibility
> > requirement on the internal representation of data types, including
> > user-defined data types. As far as I know, we currently have no such
> > requirement, even for built-in types.
>
> This is actually incorrect. Binary I/O still goes through a function call,
> the
> send/recv functions instead of the in/out functions. In theory these are also
> supposed to be cross-platform.
>
> In practice they are, uhm, less cross-platform. For example they send floats
> as raw (presumably) IEEE floats. There have also been fewer clients using
> binary mode, so you're more likely to run into bugs.
small correction: you _were_ more likely to run into bugs. in the
process of developing this patch during the 8.3 cycle, we caught,
fixed, and submitted a number of binary format related issues,
including some wacky things that are not possible through the text
parser (a polygon with zero points for example). we used regression
testing heavily in development.
> But the reason fewer clients use binary mode is because they would have to
> implement all this type-specific knowledge. It doesn't make sense to do it
> for
> every driver or application but if you're implementing it once in a library
> it
> does start to make more sense.
>
> Note however that not every data type will necessarily provide a binary
> send/recv function. The built-in data types do, but only as a matter of
> policy. It's not an error to create a type with no binary i/o functions.
> So I think you have to support using text mode as an option.
You have this option. there is nothing keeping you from using getvalue
vs. getf. However, due to libpq limitations, if any datatype must
return text the entire result must be text (resultFormat)...this is
also interestingly true for functions that return 'void'. So, at
present, to use PQgetf, you result set must be binary.
In some of the early versions of the patch we introduced parsing code
to compute text results in addition to binary results (so for
getf('%int), the library does the atoi for you, checking range and
such). We ended up dropping this because we were getting nervous
about the size of the patch at that point.
In any event, I think a better solution would be to change
resultformat to mean 'give me binary format if it's available' on a
per column basis...libpq already has a method to check field format of
the result and getf can just raise a run-time error if you try to pass
a native type in when it's not available. This would be a pretty
amazing sequence of events...only likely to occur when developing new
types for example. Is this (mixed text/binary results) possible with
the v3 protocol?
merlin
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Concurrent psql API
"Tom Lane" <[EMAIL PROTECTED]> writes: > Alvaro Herrera <[EMAIL PROTECTED]> writes: >> Tom Lane wrote: >>> It strikes me that with these semantics, \cwait is a lot like a thread >>> join operation, so we could call it \join or \j. > >> FWIW on POSIX shell there's something similar called "wait". >> http://www.opengroup.org/onlinepubs/009695399/utilities/wait.html >> Perhaps we should define the operator after these semantics -- these >> guys have probably hashed up a good interface. Basically it means we >> would have a "\cwait [n ...]" command meaning "wait for the connection >> 'n' to return". > > I was thinking about this some more while out running an errand, and > came to the same conclusion that "\cwait connID" would be a good thing > to have. I threw out cwait because it seemed to me that to write any kind of reliable regression test you would end up having to put a cwait with a timeout on every connection switch. Consider a simple regression test to test that update locks out concurrent updaters: 1 begin; 1 update t where i=1 UPDATE 1 2 begin; 2 update t where i=1 2 commit; COMMIT UPDATE 1 So here what you really want to test is that the second update blocks. If we don't wait at all we might very well miss the UPDATE message because we just flew past it too fast. In fact IIRC that's exactly what I saw. > While there's still a possible use for \D (disconnect) in this > scheme, I'm not sure how interesting it is. In any case disconnecting > the active session is a bogus behavior; you should only be able > to disconnect a non-active, idle one. Unless you're specifically trying to test that things get cleaned up properly when the session rolls back... But yeah, I only put it in for the sake of completeness at the time. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] libpq type system 0.9a
"Jeff Davis" <[EMAIL PROTECTED]> writes: > * Binary formatting > > The exclusive use of binary formats is worrisome to me. This circumvents > one level of indirection that we have (i.e. that everything moves > through in/out functions), and will impose a backwards-compatibility > requirement on the internal representation of data types, including > user-defined data types. As far as I know, we currently have no such > requirement, even for built-in types. This is actually incorrect. Binary I/O still goes through a function call, the send/recv functions instead of the in/out functions. In theory these are also supposed to be cross-platform. In practice they are, uhm, less cross-platform. For example they send floats as raw (presumably) IEEE floats. There have also been fewer clients using binary mode, so you're more likely to run into bugs. But the reason fewer clients use binary mode is because they would have to implement all this type-specific knowledge. It doesn't make sense to do it for every driver or application but if you're implementing it once in a library it does start to make more sense. Note however that not every data type will necessarily provide a binary send/recv function. The built-in data types do, but only as a matter of policy. It's not an error to create a type with no binary i/o functions. So I think you have to support using text mode as an option. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SET TRANSACTION not compliant with SQL:2003
"Tom Lane" <[EMAIL PROTECTED]> writes: > I believe the reason the spec is written in the particular way that > it is is that they wanted to allow, e.g., > > set transaction isolation level serializable; > set transaction read only; > sql-command; > sql-command; > ... > commit; So that works currently. I think you're right that the spec has to be read assuming autocommit off. postgres=# \set AUTOCOMMIT off postgres=# set transaction isolation level serializable; SET postgres=# set transaction read only; SET postgres=# create table foo (i integer); ERROR: transaction is read-only postgres=# rollback; ROLLBACK postgres=# set transaction read only; SET postgres=# set transaction isolation level serializable; SET postgres=# create table i (integer i); ERROR: transaction is read-only -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's 24x7 Postgres support! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
Tom Lane wrote: This isn't really about tools. It's about who wants to put in the day-after-day, year-after-year drudge work to maintain the queue. Whoever wants to do the work can pick their tools... I still think it would be best if the patch authors did the work. They are the ones who care about the patch and want the review, and they're in the best position to know what the status of a patch is. Others can do it as well of course, in the spirit of a Wiki. That leaves out most of the discussion threads, potential TODO items etc. that Bruce collects in the patches queue. Depending on your viewpoint that's either a good or a bad thing. It's good because it keeps the patch queue short and relevant; we'll only have patches or design proposals in the list that are genuinely waiting for review. But it's bad because good patches from one-off submitters might fall through the cracks. That's where I'd love to have Bruce to help. He's good at following up items and making sure nothing falls through the cracks. I don't mind what tool he uses for doing that, the mailbox probably is the easiest for that task. And that's the kind of work that's hard to do as a team. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
