Re: [HACKERS] linked list rewrite
On Tue, Mar 23, 2004 at 01:16:51PM -0500, Bruce Momjian wrote: I agree a renaming of list functions is good. If we had kept the original Berkeley code as-is, we would have a lot fewer developers today. :-) Making drastic cleanups is often worthwile. I write backend code and still can't remember which list function does what, so clearer naming would help me a lot, and I am sure others too. Maybe I already ask, is there any coding style recommendation for _new_ written things? I agree with Bruce. The list functions and a lot of other functions can be candidates for rename. I read Mono docs last night and it's perfect if they can write to docs about functions names: mono_type_action. I think sometime are people too much focus on the shortest way to target and forgot that with code work others people and not CPU only. (sorry of this pontification:-) Karel -- Karel Zak [EMAIL PROTECTED] http://home.zf.jcu.cz/~zakkr/ ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] linked list rewrite
On Mar 23, 2004, at 11:45, Tom Lane wrote: AFAICS, though, CVS is not broken for our needs. I don't see an adequate reason to change. Yes, of course. I guess my point is that if you're going to move away from CVS, please don't just jump to ``slightly better CVS.'' I'm looking forward to a world with more open source projects in distributed revision control. The ones I'm tracking and doing personal branches against (vim, tla, a couple of other small projects) are liberating. Branching a tree I didn't expect to even have to edit while on an airplane recently was a very pleasant experience. Actually, being able to branch without thinking about the horrible consequences of CVS has been wonderful in general. As long as CVS works for you, though, you should continue to use it. -- SPY My girlfriend asked me which one I like better. pub 1024/3CAE01D5 1994/11/03 Dustin Sallings [EMAIL PROTECTED] |Key fingerprint = 87 02 57 08 02 D0 DA D6 C8 0F 3E 65 51 98 D8 BE L___ I hope the answer won't upset her. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] [COMMITTERS] pgsql-server/ oc/src/sgml/ref/alter_table.sgml ...
On 23-Mar-04, at 3:03 PM, Bruce Momjian wrote: Does removing SET WITHOUT OIDS cause compatibility problems? ALTER TABLE ... SET WITHOUT OIDS has not been removed. -Neil ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Chapter on PostreSQL in a book
On Wednesday 24 March 2004 01:38, Christopher Kings-Lynne wrote: I'm at a loss as to how much we should focus on these sections. Do we use what's in GBorg ? Do the hackers have any suggestions ? There are 33 DBA tools and 19 Design tools in GBorg .. are there any specific tools that are recommended ? Well, I think there are really just two major active free database administration tools: pgAdmin3 and phpPgAdmin There's RedHat's java-based stuff too. That's freely downloadable and GPLed too IIRC. -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] where can I get the HTML docs
On Wednesday 24 March 2004 00:30, Paul Tillotson wrote: Can anyone show me where to download a zipped tarball of .html files of what exists at the following link? http://www.postgresql.org/docs/7.4/static/index.html Not really a hackers question, you might have been better with the -general list. Go to the website, click download, select a mirror, look in v7.4.2 and there are docs packages there. -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] pg_advisor schema proof of concept
On Monday 22 March 2004 09:38, Fabien COELHO wrote: Hello hackers, please find attached a quick proof of concept for a 'pg_advisor' schema. I'm still pushing my agenda, despite lack of reaction on the list;-) I had time this week-end to improve my current 'pg_advisor' prototype schema. Had a look, and it seems good to me - pretty much what I was thinking of. This new version is now less a proof of concept and more a preliminary implementation for discussion. Some thoughts and questions about a pg_advisor schema design: (1) should it use pg_catalog.* or information_schema.*? - is portability desirable? - my initial version is based on pg_catalog. - information_schema could make it more portable? Not sure portability is important, but using information_schema will presumably make it less likely that things will change between versions. well, I'm not sure it would do the job. I need to know what are the system schemas, and it is likely that this would differ? what about support functions? - should it be compatible with old versions of postgreSQL? if yes, what about support functions? Not sure it's worth the trouble to support 7.3, and anything below that is going to be a lot of work. (2) advices should be associated: - a kind (design/model, performance... what else?) - a severity (info, notice, warning, error... others? different?) - a title - an abstract - a description with examples - what about a subject, such as referencial integrity or index... if so, what could be the sujects? or maybe it is not needed? Might be useful to be able to run all relevant tests against a single table, especially if we end up with lots of tests. - should we use the COMMENT infrastructure for that? I don't think so, but it could be done. No - I think the separate table (advice_classification) is right. (3) needed support function - should be added to pg_catalog? implemented in C? - can we use plpgsql? SQL? others? I would try to avoid anything other that pg_catalog and sql functions, but I needed to add several functions that were missing. If plpgsql works OK, I say stick with it. (4) advices implementations. - I implemented 11 basic design advices at the time. I tested them with existing databases, and I'm pretty happy with the result: I had very few comments on good design/model, and a lot of warnings or notice on badly designed tables. Actually picked up a genuine mistake on one of my databases (mismatched pkey=fkey sizes). It's been worth the money already :-) - what other design advices would be useful? how to grade them (from info to error)? Probably a matter of opinion. It'll give people something to argue about, anyway. . cross schema contraints/tables? - what about performance advices? Well, I can see how you could examine the stats tables, but you'd probably need to be able to see the queries too. what support functions are useful for those? - others? (5) documentation - should include design notes for new advices? I think so. - how to make things more modular? We probably need a good list of tests before deciding what to make into libraries - let us use comments about every view and columns... - how to 'localise' pg_advisor? a more general issue is how to 'localise' COMMENTS. Not sure we want any of the text in the comments. Put all the messages/titles in a description table like you already have and people can translate the text in that file. (6) possible inclusion in postgresql? - among other contributions? what about contrib/advisor? - added to template1 on default installation? maybe not for a first release? or yes? it is easier to communicate about I think we're going to want a gborg project for developing/coordinating tests anyway. Having the schema included in contrib/ might help adoption, but so would pgadmin/phpgadmin. Any client-builders reading this? What do you think? -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Log rotation
This thread seems to have died without a conclusion. AFAICS, we have 5 options: . the apache program - see below pro: robust, portable, extremely well tested, no effort to import con: possible license issues, limited features . Peter Eisentraut's program pro: portable, better featured, no license issues con: code state uncertain, less well tested . the Afilias script pro: well featured, no license issues con: not portable (relies on perl), testing status uncertain . something new pro: no license issues, can make as featured as desired, portable con: lots of effort, untested . nothing pro: no effort, no license issues :-) con: feature is desired The issues seem to have been thrashed out ad nauseam. Surely we can put *something* in contrib for this? After all, nobody has to use it if they don't want to. cheers andrew I wrote: Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Did anything ever come from this thread? http://archives.postgresql.org/pgsql-hackers/2003-05/msg00603.php (Heading: Plan B for log rotation support: borrow Apache code) Only an entry on my depressingly long personal to-do list :-( I did take a look at the Apache rotator program, and found that it was probably more trouble to adopt than it's worth. It seemed to depend on a lot of configuration and library-routine infrastructure that we don't share. (No big surprise; I suppose someone trying to pull out a random bit of our backend code would be at least as unhappy.) I suspect it would be less trouble, as well as legalistically cleaner, to write our own from scratch. Andrew Sullivan offered Afilias' rotator script awhile back also. I think that works fine if you like a Perl script. FWIW, in less than 30 minutes I took the log rotator from apache 1.3.29 (i.e. the latest non-APR version) and imported it into a fresh postgreql tree. With very little massaging it built happily (see below). If it will advance matters, I can submit this as a patch filling the currently empty contrib/apache_logging directory. You could be right about the legal stuff - worth talking to the apache folks? ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] pg_advisor schema proof of concept
Dear Richard, (1) should it use pg_catalog.* or information_schema.*? Not sure portability is important, but using information_schema will presumably make it less likely that things will change between versions. Another issue I found is that, although all the contents of information_schema can be found in pg_catalog (as it derives from it!) not all of pg_catalog may be found in information_schema... In particular, for performance advices about indexes, operators and casts, I'm not sure the all information is available in information_schema, from the quick look I had about it. Might be useful to be able to run all relevant tests against a single table, especially if we end up with lots of tests. That could be done quite easily, I've added a feature in my working version about which schemas should be tested. It is easy to have both a schema/table names and to be able to filter those of interest to the user. I'll resend later an updated version for discussion. If plpgsql works OK, I say stick with it. Hmmm. I'm not very happy with plpgsql, as I had an infinite recursion which is partly due to plpgsql, partly to a very bad plan by the optimiser, and partly to the fact that I want to do strange things with tables querying about tables, so it goes bad quickly if the table starts querying about itself to count it's own lines:-) (4) advices implementations. - I implemented 11 basic design advices at the time. [...] Actually picked up a genuine mistake on one of my databases (mismatched pkey=fkey sizes). It's been worth the money already :-) That is a point. The other question is how many false positive. That's why I put a grade, for things that are matters of opinions, as you say... So that controversial advices can be downgraded to notice or info. - what about performance advices? Well, I can see how you could examine the stats tables, but you'd probably need to be able to see the queries too. I was thinking along the kind of missing index Tom was arguing about for RI checks, that may be helped if an appropriate index is available. I'm not sure what could be done, even with the query, in the general case. How to guess what index would help make a better plan? It depends on the optimiser itself, on what kind of indexes could be built, and so on. That's more human expect work than tool work. - let us use comments about every view and columns... - how to 'localise' pg_advisor? a more general issue is how to 'localise' COMMENTS. Not sure we want any of the text in the comments. Put all the messages/titles in a description table like you already have and people can translate the text in that file. Ok. but the system should be able to store several locales. I guess it is possible to know about the current locale within SQL, e.g. by querying lc_message in pg_settings for instance. (6) possible inclusion in postgresql? I think we're going to want a gborg project for developing/coordinating tests anyway. Why not. How much work in the infrastructure ? What would be the added value ? better communication ? Having the schema included in contrib/ might help adoption, but so would pgadmin/phpgadmin. Sure. Thanks a lot for your comments. I'll submitted an updated version later. -- Fabien Coelho - [EMAIL PROTECTED] ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: subversion vs cvs (Was: Re: [HACKERS] linked list rewrite)
On Tue, 23 Mar 2004 23:03:03 -0400 (AST) Marc G. Fournier [EMAIL PROTECTED] wrote: On Tue, 23 Mar 2004, Sailesh Krishnamurthy wrote: Which brings me to another question .. has anybody considered using subversion instead of CVS ? Why? not that I'm for a chance from something that isn't broken, but what advantages does subversion give us over what we already have? Subversion has lots of little benefits, but nothing that would be a major incentive to switch. The biggest benefits I can think of of the top of my head are: * Commits are actually atomic * protocol sends diffs in both directions which speeds up everything * branching and tagging are cheap constant time operations * the time it takes to make changes is based on the size of the change, not the size of the project * whole directories are versioned not just files. So for example if you for some reason wanted to rename src/backend/bootstrap.c to src/backend/bootup.c you wouldn't lose your revision history information. Same thing goes for complete reorganizations of the file layouts. * You can checkout parts of a project so if you need to fix a bug in 7.3.6's src/backend/ you only have to transfer that portion to you. * cvs diff ( well svn diff ) can be done offline. Same with 'status' which shows you your local modifications and 'revert' which reverts your changes back to your last checkout/update/etc. * Revisions numbers are repository wide instead of by file. You can refer to revision #14328 on hackers and everyone knows exactly what you are talking about and can switch their working copies to it easily ( svn switch -r 14328 ). It's sort of like having a tag for every commit made to the repository. It does have some downsides that I have found, most notibly that the size of your sources you have in your working copy are essentially doubled. There is a copy in your .svn directory that allows the offline status, diff, and revert commands to work. - Frank Wiles [EMAIL PROTECTED] http://frank.wiles.org - ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_advisor schema proof of concept
Fabien COELHO [EMAIL PROTECTED] writes: (1) should it use pg_catalog.* or information_schema.*? Not sure portability is important, but using information_schema will presumably make it less likely that things will change between versions. Another issue I found is that, although all the contents of information_schema can be found in pg_catalog (as it derives from it!) not all of pg_catalog may be found in information_schema... This is necessarily so, as the information_schema by definition covers only concepts standardized by the SQL spec. Since the SQL spec considers things like indexes to be implementation details, it is simply not possible for information_schema to tell you everything you want to know to give performance advice. If plpgsql works OK, I say stick with it. Hmmm. I'm not very happy with plpgsql, I don't know where you are planning on going with this. If it's only to be a contrib tool, it's okay to depend on plpgsql. But we couldn't incorporate it into the base system because plpgsql isn't part of the base system. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Log rotation
On Wed, Mar 24, 2004 at 09:55:34AM -0500, Andrew Dunstan wrote: This thread seems to have died without a conclusion. AFAICS, we have 5 options: . the apache program - see below pro: robust, portable, extremely well tested, no effort to import con: possible license issues, limited features One of the folks who works with me at Afilias, Andrew Hammond, is trying to clean up a patch he made to include the apache log rotator natively. We've been looking at it because I'm worried about the I/O our current approach imposes, and we need to have something to compare it with. I think he told me that he'll have a patch ready to offer in a day or two. Of course, it would depend on actually having the rotator program sitting around somewhere. A -- Andrew Sullivan | [EMAIL PROTECTED] The fact that technology doesn't work is no bar to success in the marketplace. --Philip Greenspun ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Log rotation
On Wed, 24 Mar 2004, Andrew Dunstan wrote: . Peter Eisentraut's program pro: portable, better featured, no license issues con: code state uncertain, less well tested Where is Peter's code available, I'd like to try it out. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Chapter on PostreSQL in a book
Sailesh, First off, I'd suggest reopening this topic on Advocacy or Docs. Hackers is really not the list for this, and I think you'd get more feedback on those other lists. - Database Design and Querying Tools Well, I think there is no question that phpPgAdmin and pgAdminIII are our two leading Admin/Database Browsing tools. If you drop to the runners-up, though, you have a choice of 8 or 9 different tools which are still current. Perhaps this page will give some perspective? http://techdocs.postgresql.org/guides/GUITools I might suggest, for a single chapter, going over the top two but mentioning that there are many others. - Replication, Distribution and External Data For replication we have and will continute to have several solutions, both OSS and commercial. Replication is not a problem that suits itself to a single, monolithic solution, and our diversity is superiority. You should mention eRServer, SlonyI, DBmirror, and Mammoth Replication. What are distribution external data? - Database Administration Tools How is this different from the first item? -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] pg_advisor schema proof of concept
Dear Tom, This is necessarily so, as the information_schema by definition covers only concepts standardized by the SQL spec. Since the SQL spec considers things like indexes to be implementation details, it is simply not possible for information_schema to tell you everything you want to know to give performance advice. Well, it makes sense. As pg_catalog will be necessary for some advices, let us avoid information_schema for a greater homogeneity. If plpgsql works OK, I say stick with it. Hmmm. I'm not very happy with plpgsql, I don't know where you are planning on going with this. If it's only to be a contrib tool, it's okay to depend on plpgsql. But we couldn't incorporate it into the base system because plpgsql isn't part of the base system. Well, the ultimate status of the tool basically depends on the patchers (we above) decision;-) If you veto the inclusion of advisor stuff into the base system because you do not want it there anyway, which may be perfectly legitimate, then I would not bother to port the plpgsql stuff just for the fun of it. On the otherhand, if you would be ready to consider it for inclusion in the base system some day, provided that the quality is fine and that there is no plpgsql in it, then it would make sense to discuss needed functions to be added to the base system. The current preliminary implementation requires plpgsql for : - array_index (find index of item in array, to deal with pg_constraint attribute lists) - array_ceq (whether two arrays contains the same values, possibly in a different order, idem) - count_tuples (count the number of tuples in a relation) I think these functions could be included in the base system, anyway. As for performance advices, such as missing indexes for ri check that you suggested, I don't know. Some functions that already exists in the backend would be welcome to be called from sql, such as selecting an = operator variant given the oid of the expected types... but maybe they can be developped within SQL (i.e. without plpgsql). I haven't looked at it yet. As for what is not foreseen yet, who knows? ;-) Have a nice day, -- Fabien Coelho - [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Log rotation
Andrew Dunstan wrote: This thread seems to have died without a conclusion. AFAICS, we have 5 options: . the apache program - see below pro: robust, portable, extremely well tested, no effort to import con: possible license issues, limited features . Peter Eisentraut's program pro: portable, better featured, no license issues con: code state uncertain, less well tested . the Afilias script pro: well featured, no license issues con: not portable (relies on perl), testing status uncertain . something new pro: no license issues, can make as featured as desired, portable con: lots of effort, untested . nothing pro: no effort, no license issues :-) con: feature is desired The issues seem to have been thrashed out ad nauseam. Surely we can put *something* in contrib for this? After all, nobody has to use it if they don't want to. Shouldn't all of these just be on gborg? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_advisor schema proof of concept
On Wednesday 24 March 2004 15:52, Tom Lane wrote: If plpgsql works OK, I say stick with it. Hmmm. I'm not very happy with plpgsql, I don't know where you are planning on going with this. If it's only to be a contrib tool, it's okay to depend on plpgsql. But we couldn't incorporate it into the base system because plpgsql isn't part of the base system. So Tom, are you suggesting: 1. A core in the base distribution (C / SQL) 2. command-line tool in the base distro (pg_advisor) 3. more open project (gborg?) to let people design/add tests, some of which will eventually end up in the standard set in the base distro. -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] My chat addresses
Here are my instant messaging addresses: AIM bmomjian ICQ 151255111 Yahoo bmomjian MSN [EMAIL PROTECTED] IRC bmomjian on #postgresql via FreeNode or EFNet I find it useful to help developers work on PostgreSQL and keep in touch with folks. I usually see PostgreSQL 20 people online, and have a total of 40 in my PostgreSQL list. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] pg_advisor schema proof of concept
Richard Huxton [EMAIL PROTECTED] writes: On Wednesday 24 March 2004 15:52, Tom Lane wrote: I don't know where you are planning on going with this. If it's only to be a contrib tool, it's okay to depend on plpgsql. But we couldn't incorporate it into the base system because plpgsql isn't part of the base system. So Tom, are you suggesting: 1. A core in the base distribution (C / SQL) 2. command-line tool in the base distro (pg_advisor) 3. more open project (gborg?) to let people design/add tests, some of which will eventually end up in the standard set in the base distro. I'm not suggesting anything ;-). Just pointing out a constraint that might affect you guys' choices of where to go with this. There has been some talk of installing plpgsql by default, in which case the constraint would vanish anyway. So I wouldn't put a huge amount of emphasis on it at this stage. I just wanted to point it out so you wouldn't paint yourselves into a corner without realizing it. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: subversion vs cvs (Was: Re: [HACKERS] linked list rewrite)
On Mar 24, 2004, at 7:29, Frank Wiles wrote: [cool feature list] Arch has all of that except for the checking out part of a directory thing (would you really just check out the backend, submit a change, and not build and test it?). Additionally: * Repositories can be easily replicated so checkouts don't have to cross slow networks. I replicate every repository I work with to every machine I use. This is not only my backup strategy, but it makes checkouts faster. * You can work completely offline. In addition to being able to diff, undo changes, redo changes, etc... while offline, you can completely branch a project do multiple commits, and merge them back into the main archive when your plane lands. * Branches are not only cheap, but can easily cross repository boundaries. Any given user can create a branch from the head-of-line tree and maintain changes, and the head-of-line maintainers can pull those changes back in. * Its storage is immutable. It never modifies a file so it does not provide any possibility for corruption. This is also what makes replication so trivial. * Changesets have cryptographic checksums and may be cryptographically signed. Checkouts are authenticated against both of these. * Repositories can be accessed via a wide variety of means. Most of mine are local file, WebDAV, or SFTP. I allow read-only access via plain HTTP for anyone who wants to check out one of my projects. * Files can be tracked in a nearly (or completely) automatic fashion. Depending on project configuration, you can avoid having to interact with the revision control system other than writing changelogs and submitting patches. You an also use CVS-style tracking (manual adds and deletes (plus moves)) if you're more comfortable that way. * The design is way, way more simple and transparent, there are far fewer requirements. At least for me, this translates to a higher confidence that my stuff will always be available. The advantage I see to Subversion is that it's designed to be a better CVS. Since many people are comfortable with CVS and that style of centralized development, it may feel a little more natural for new converts. I feel that that's because it doesn't seem to take you very far. -- Dustin Sallings ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
LOOK - KITTENS! (was Re: [HACKERS] pg_advisor schema proof of concept)
On Wednesday 24 March 2004 18:02, Tom Lane wrote: Richard Huxton [EMAIL PROTECTED] writes: So Tom, are you suggesting: 1. A core in the base distribution (C / SQL) 2. command-line tool in the base distro (pg_advisor) 3. more open project (gborg?) to let people design/add tests, some of which will eventually end up in the standard set in the base distro. I'm not suggesting anything ;-). Just pointing out a constraint that might affect you guys' choices of where to go with this. The direction (and it's all Fabien's work so far) will depend on where it will live. That depends on whether anyone else thinks it's worthwhile. So howabout some feedback on Fabian's work people? -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] subversion vs cvs
Sailesh Krishnamurthy wrote: I've had plenty of pain with cvs in terms of directories not being first-class etc .. but I don't really contribute to pgsql so you guys probably don't have the same experience. I was just curious as it looks like eventually subversion (or arch :-) will be an alternative to cvs. Eventually it (either subversion, or arch, or something else) will. You just have to be patient :-) The movement will be very slow, we'll probably see Apache 1.3.x disappear first before we see CVS disappear. It _is_ frustrating to have to use something new, especially something so frequently used like source control tool. -- dave ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] rotatelogs integration in pg_ctl
I've attached a patch for pg_ctl which integrates the Apache project's rotatelogs for logging. Is there any interested in the community for such a thing? I have not yet added the appropriate stuff to autoconf to completely integrate this. I would appreciate any suggestions for improvement. Drew --- pg_ctl 2004-03-19 14:01:40.0 -0500 +++ pg_ctl_new 2004-03-24 15:30:29.0 -0500 @@ -39,6 +39,9 @@ Options for start or restart: -l FILENAME write (or append) server log to FILENAME. The use of this option is highly recommended. + -r ROTATIONTIME use apache's rotatelogs instead of writing + dirrectly tto FILENAME. Rotate the logs every + ROTATIONTIME seconds (see man for rotatelogs) -o OPTIONS command line options to pass to the postmaster (PostgreSQL server executable) -p PATH-TO-POSTMASTER normally not necessary @@ -62,6 +65,10 @@ VERSION='7.4.2' DEF_PGPORT='5432' +# have autoconf detect this? +apache_bindir='/opt/OXRS/apache/bin' +rotatelogs_path=$apache_bindir/rotatelogs + # protect the log file umask 077 @@ -114,6 +121,7 @@ wait= wait_seconds=60 logfile= +rotation_time= silence_echo= shutdown_mode=smart PGDATAOPTS= @@ -143,6 +151,12 @@ -l*) logfile=`echo $1 | sed 's/^-l//'` ;; + -r) + rotation_time=$2 + shift;; + -r*) + rotation_time=`echo $1 | sed 's/^-m//'` + ;; -m) shutdown_mode=$2 shift;; @@ -207,6 +221,18 @@ exit 1 fi +if [ ! x$rotation_time = x ]; then +if [ ! -x $rotatelogs_path ]; then +echo $CMDNAME: log rotation specified but can't find $rotatelogs_path 12 +exit 1 +fi + +if [ x$logfile = x ]; then +echo $CMDNAME: log rotation specified but no logfile given. Try the -l option. 12 +exit 1 +fi +fi + if [ -z $wait ]; then case $op in start) wait=no;; @@ -338,7 +364,11 @@ fi if [ -n $logfile ]; then +if [ -n $rotation_time ]; then # use rotatelogs for logging + $po_path ${1+$@} ${PGDATAOPTS+$PGDATAOPTS} /dev/null 21 | $rotatelogs_path $logfile $rotation_time + else $po_path ${1+$@} ${PGDATAOPTS+$PGDATAOPTS} /dev/null $logfile 21 + fi else # when starting without log file, redirect stderr to stdout, so # pg_ctl can be invoked with $logfile and still have pg_ctl's ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: subversion vs cvs (Was: Re: [HACKERS] linked list rewrite)
Dustin Sallings wrote: On Mar 24, 2004, at 11:45, David Garamond wrote: So one might ask, what *will* motivate a die-hard CVS user? A real-close Bitkeeper clone? :-) Since it's illegal for anyone who uses Bitkeeper's free license to contribute to another project, does anyone know if there are any features in Bitkeeper missing from arch (specifically tla) that matter to developers? Or is there anything that may be a better match than arch? From what I read here and there, BitKeeper excels primarily in merging (good merging is apparently a very complex and hard problem) and GUI stuffs. Unfortunately, I have never and will never use Bitkeeper unless someone buys me a license for some reason. The distributed model seems like the only way to go for the open source development of the future. Not necessarily. For small to medium projects, a centralized model might work better. -- dave ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] rotatelogs integration in pg_ctl
Andrew Hammond [EMAIL PROTECTED] writes: I've attached a patch for pg_ctl which integrates the Apache project's rotatelogs for logging. Why bother? You just pipe pg_ctl's output to rotatelogs and you're done. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: subversion vs cvs (Was: Re: [HACKERS] linked list rewrite)
On Mar 24, 2004, at 13:22, David Garamond wrote: From what I read here and there, BitKeeper excels primarily in merging (good merging is apparently a very complex and hard problem) and GUI stuffs. There's not a lot of GUI in arch, but star-merge is fairly incredible. This is how tla (the main arch implementation) itself is developed. Lots of branches in lots of archives by lots of people. Unfortunately, I have never and will never use Bitkeeper unless someone buys me a license for some reason. The distributed model seems like the only way to go for the open source development of the future. Not necessarily. For small to medium projects, a centralized model might work better. I make use of the distributed nature of arch in my personal projects with no other developers. Offline work is just a branch in another archive that gets merged in later. Arch supports a centralized model as well as anything else, and I've got a big centralized set of archives, but I don't always have good connectivity to the master. This is where the distributed model wins. A server/network/whatever outage does not have the opportunity to slow me down. In the worst case, a long outage causes my branch to drift a little further from head of line than it normally would. -- Dustin Sallings ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] PostgreSQL ES3.0 problems?
Hello, About a week ago we had a report of replicator not working correctly on ES 3.0 when running with more than one CPU. Replicator worked perfectly when utilizing only one CPU. The machine was a Quad Xeon with HyperThreading, SCSI RAID array and 4 Gig of RAM. During our testing we found issues with - nocachegetattr. Our tests showed the following output using gdb: == GDB LOG == [EMAIL PROTECTED] postgresql-7.3.6]$ gdb /usr/local/pgsql/bin/postmaster /home/postgres/data_master/base/16976/core.858 GNU gdb Red Hat Linux (5.3.90-0.20030710.40rh) Copyright 2003 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as i386-redhat-linux-gnu...Using host libthread_db library /lib/tls/libthread_db.so.1. Core was generated by `postgres: postgres test [local] SELECT '. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /usr/lib/libreadline.so.4...done. Loaded symbols for /usr/lib/libreadline.so.4 Reading symbols from /lib/libtermcap.so.2...done. Loaded symbols for /lib/libtermcap.so.2 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/tls/libm.so.6...done. Loaded symbols for /lib/tls/libm.so.6 Reading symbols from /lib/tls/libc.so.6...done. Loaded symbols for /lib/tls/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 #0 0x0806b729 in nocachegetattr (tuple=0x82992d8, attnum=17, tupleDesc=0x0, isnull=0xbfffae2f ) at heaptuple.c:406 406 usecache = false; (gdb) l 401 } 402 403 off = att_addlength(off, att[i]-attlen, tp + off); 404 405 if (usecache att[i]-attlen = 0) 406 usecache = false; 407 } 408 409 off = att_align(off, att[attnum]-attalign); 410 (gdb) Our testing was performed on: Dual Athlon MP 2800 (Barton) 2 Gig of Registered ECC ram 3Ware 0+1 IDE Array LVM The log output was thus: [EMAIL PROTECTED] postgresql-7.3.6]$ /usr/local/pgsql/bin/postmaster -D ~/data_master DEBUG: /usr/local/pgsql/bin/postmaster: PostmasterMain: initial environ dump: DEBUG: - DEBUG: HOSTNAME=es30 DEBUG: TERM=xterm DEBUG: SHELL=/bin/bash DEBUG: HISTSIZE=1000 DEBUG: SSH_CLIENT=192.168.1.65 1745 22 DEBUG: SSH_TTY=/dev/pts/2 DEBUG: USER=postgres DEBUG: LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.zip=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.bz=00;31:*.tz=00;31:*.rpm=00;31:*.cpio=00;31:*.jpg=00;35:*.gif=00;35:*.bmp=00;35:*.xbm=00;35:*.xpm=00;35:*.png=00;35:*.tif=00;35: DEBUG: MAIL=/var/spool/mail/postgres DEBUG: PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/postgres/bin DEBUG: INPUTRC=/etc/inputrc DEBUG: PWD=/home/postgres/postgresql-7.3.6 DEBUG: LANG=en_US.UTF-8 DEBUG: SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass DEBUG: SHLVL=1 DEBUG: HOME=/home/postgres DEBUG: LOGNAME=postgres DEBUG: SSH_CONNECTION=192.168.1.65 1745 192.168.1.199 22 DEBUG: LESSOPEN=|/usr/bin/lesspipe.sh %s DEBUG: G_BROKEN_FILENAMES=1 DEBUG: _=/usr/local/pgsql/bin/postmaster DEBUG: OLDPWD=/home/postgres/src/SOURCES/postgresql-7.3.6 DEBUG: - DEBUG: FindExec: found /usr/local/pgsql/bin/postgres using argv[0] DEBUG: invoking IpcMemoryCreate(size=36421632) DEBUG: FindExec: found /usr/local/pgsql/bin/postmaster using argv[0] LOG: database system was shut down at 2004-03-19 06:48:17 PST LOG: checkpoint record is at 0/58E2B4F8 LOG: redo record is at 0/58E2B4F8; undo record is at 0/0; shutdown TRUE LOG: next transaction id: 630; next oid: 217646 LOG: database system is ready DEBUG: proc_exit(0) DEBUG: shmem_exit(0) DEBUG: exit(0) DEBUG: reaping dead processes DEBUG: /usr/local/pgsql/bin/postmaster child[858]: starting with ( DEBUG: postgres DEBUG: -v131072 DEBUG: -p DEBUG: test DEBUG: ) DEBUG: InitPostgres DEBUG: BackendStartup: forked pid=858 socket=8 DEBUG: StartTransactionCommand DEBUG: ProcessUtility DEBUG: CommitTransactionCommand DEBUG: StartTransactionCommand DEBUG: ProcessQuery DEBUG: ProcessUtility DEBUG: CommitTransactionCommand DEBUG: StartTransactionCommand DEBUG: ProcessUtility DEBUG:
Re: [HACKERS] pg_advisor schema proof of concept
(6) possible inclusion in postgresql? - among other contributions? what about contrib/advisor? - added to template1 on default installation? maybe not for a first release? or yes? it is easier to communicate about I think we're going to want a gborg project for developing/coordinating tests anyway. Having the schema included in contrib/ might help adoption, but so would pgadmin/phpgadmin. Any client-builders reading this? What do you think? Both phpPgAdmin (me) and the pgAdmin team have added or have thought about adding some 'schema analysis' features to our products. If pg_advisor is available, I certainly won't bother and I will just recommend to people that they install it. I think it probably should live in userland... Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_advisor schema proof of concept
I was thinking along the kind of missing index Tom was arguing about for RI checks, that may be helped if an appropriate index is available. I'm not sure what could be done, even with the query, in the general case. How to guess what index would help make a better plan? It depends on the optimiser itself, on what kind of indexes could be built, and so on. That's more human expect work than tool work. Also, if they have a partial index on the FK, it's not good enough! In CVS, IS NOT NULL partial indexes should be used, but in general all others still won't... Chris ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [GENERAL] unicode error and problem
, 24.03.2004, 11:33, Paolo Supino : Hi I received a unicode CSV file from someone (the file was created on a windows system) and I'm trying to import it into postgresql. When it gets to a line that isn't ascii it prints the following error and aborts: ERROR: copy: line 33, Invalid UNICODE character sequence found (0xd956). Try to convert the file from UTF-16 (which might be the encoding of the file) to UTF-8 with iconv: iconv --from UTF-16 --to UTF-8 file file.UTF-8 Maybe the file is not in UTF-16 but in some other encoding - convert accordingly then. By the way, Unicode is just a number - glyph mapping, it doesn't say anything about the representation of that number in the byte stream. UTF-8 and UTF-16 are such representation specifications. The encoding name in PostgreSQL should be changed from UNICODE to UTF-8 because UNICODE really just isn't an encoding. -- Markus Bertheau [EMAIL PROTECTED] ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: subversion vs cvs (Was: Re: [HACKERS] linked list rewrite)
It does have some downsides that I have found, most notibly that the size of your sources you have in your working copy are essentially doubled. There is a copy in your .svn directory that allows the offline status, diff, and revert commands to work. What's needed is a good window client like WinCVS, however... Chris ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: subversion vs cvs (Was: Re: [HACKERS] linked list rewrite)
Christopher Kings-Lynne wrote: What's needed is a good window client like WinCVS, however... Chris There is a number of those, our shop uses (and makes programs for) both windows and unix (and might soon use mac's aswell), so it's very important that there exists a good client for each. Especially if you version html pages and such that is edited by people that isn't so techy. We're using TortoiseSvn right now, it's implemented as an explorer extension, so you just rightclick on a file or directory to update/commit/whatever. What i like with svn is that it's a nobrainer for old cvs guys like me to use it. It solves all the problems with CVS right now, and promises more features later on (like much better than CVS merging). The new buzz is distributed versioning systems these days, but i question if that is called for in the vast majority of projects out there. If the only reason is for offline work that can be achieved with subversion too, with svk for example (haven't tried it, but been told that it works fine). Svk handles or will(?) handle distributed repos in the bk sense aswell, i believe. But ofcourse arch has alot of features that are extremly cool, the reason why i didn't evaluate it further was that it didn't work on windows well, the fixed weird branching/version naming and the complexity of learning for our developers since they already use cvs. Surely the two systems should be evaluated against their competiors within the same distribution models, not cross the boundries, since the design is very different. Subversions strength is it's percieved simplicity, and archs strength is it's complexity. Regards, Magnus ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: subversion vs cvs (Was: Re: [HACKERS] linked list rewrite)
On Mar 24, 2004, at 18:22, Magnus Naeslund(t) wrote: The new buzz is distributed versioning systems these days, but i question if that is called for in the vast majority of projects out there. You can use distributed revision control systems as centralized systems, but not vice-versa. But ofcourse arch has alot of features that are extremly cool, the reason why i didn't evaluate it further was that it didn't work on windows well, the fixed weird branching/version naming and the complexity of learning for our developers since they already use cvs. Depends on how you set it up. In the recommended ``tagline'' method, you don't have to tell the revision control system when you add, remove, or move files (for the most part, binary files are not suitable for tagline). All you have to do is commit and type a message. Subversions strength is it's percieved simplicity, and archs strength is it's complexity. It is subversion's complexity that drove me away from it, in fact. Arch is incredibly simple all the way through. It sounds like it must be complex to be able to do all of the things people do with it, but it's most assuredly not. The requirements are few and basic (diff, patch, and tar). The most common stuff is at least as simple as CVS (commit, update and in explicit mode, add, rm, and mv), and the advanced stuff is just as easy as the simple stuff (star-merge, tag (branching), etc...). As far as understanding the simplicity of arch (if you wanted to understand the problems it solves and implement it yourself), a really good presentation was posted today to the arch list that sums it up quickly and concisely. It's hard to go through that and not think, ``I could write this.'' http://web.verbum.org/tla/grokking-arch/grokking-arch.html -- SPY My girlfriend asked me which one I like better. pub 1024/3CAE01D5 1994/11/03 Dustin Sallings [EMAIL PROTECTED] |Key fingerprint = 87 02 57 08 02 D0 DA D6 C8 0F 3E 65 51 98 D8 BE L___ I hope the answer won't upset her. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] Email addresses on developer bios site
On this page: http://developer.postgresql.org/bios.php Is there any chance we could get our email addresses obfuscated to prevent spam? Chris ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: subversion vs cvs (Was: Re: [HACKERS] linked list rewrite)
On Wednesday 24 March 2004 06:03 pm, Dustin Sallings wrote: There's not a lot of GUI in arch, but star-merge is fairly incredible. This is how tla (the main arch implementation) itself is developed. Lots of branches in lots of archives by lots of people. I would guess that better merging might be a real motivation for people. If a patch that takes a month to develop can still apply cleanly despite significant code drift in the interrem, I could see that as a real motivating factor. Matthew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: subversion vs cvs (Was: Re: [HACKERS] linked list rewrite)
Matthew T. O'Connor [EMAIL PROTECTED] writes: I would guess that better merging might be a real motivation for people. If a patch that takes a month to develop can still apply cleanly despite significant code drift in the interrem, I could see that as a real motivating factor. Not here. You want me to trust some bit of code (with absolutely zero understanding of the source text it's hacking on) to figure out how to resolve conflicting patches? That sounds like a recipe for big-time unhappiness. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: subversion vs cvs (Was: Re: [HACKERS] linked list rewrite)
On Mar 24, 2004, at 20:29, Tom Lane wrote: Not here. You want me to trust some bit of code (with absolutely zero understanding of the source text it's hacking on) to figure out how to resolve conflicting patches? That sounds like a recipe for big-time unhappiness. The idea is that it's the responsibility of the branch owner to keep it up-to-date. For example, I've got a branch of tla (an arch implementation) I made soon after I started using it in order to add a command I wanted and refactor a bit of the insides. Over time, a lot of stuff has changed, but I still want my stuff to work, so as I update my branch against head of line, I make minor changes to it as things go. The difference is that instead of having a patch sitting in a queue somewhere suffering from bit-rot, you've got a pointer to a branch that you can merge when you get around to it. You can still view it as a diff if you want, but the diff you get six months after the original submission may be quite a bit different from what you would've got at the beginning if a lot of the code around it has changed. It's definitely not a magic tool that makes bad code good and conflicting patches happy. It solves other problems, though. Many of the problems you don't realize you have until you go back to something else and try to do something simple like undo all of the changes you've made since your last checkin. -- SPY My girlfriend asked me which one I like better. pub 1024/3CAE01D5 1994/11/03 Dustin Sallings [EMAIL PROTECTED] |Key fingerprint = 87 02 57 08 02 D0 DA D6 C8 0F 3E 65 51 98 D8 BE L___ I hope the answer won't upset her. ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] HEAD compile troubles
Kind people, I just tried to compile HEAD on fedora, and it broke as per http://rafb.net/paste/results/W1942548.html More info... uname -a Linux fetter.org 2.4.22-1.2166.nptl #1 Fri Jan 30 13:48:31 EST 2004 i686 i686 i386 GNU/Linux gcc -v Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.3.2/specs Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --host=i386-redhat-linux Thread model: posix gcc version 3.3.2 20031022 (Red Hat Linux 3.3.2-1) cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 2 model name : Intel(R) Pentium(R) 4 CPU 3.00GHz stepping: 9 cpu MHz : 3006.892 cache size : 512 KB fdiv_bug: no hlt_bug : no f00f_bug: no coma_bug: no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid bogomips: 6003.09 Any ideas what i buggered up? Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 510 893 6100 mobile: +1 415 235 3778 Remember to vote! ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] HEAD compile troubles
David Fetter [EMAIL PROTECTED] writes: Kind people, I just tried to compile HEAD on fedora, and it broke as per http://rafb.net/paste/results/W1942548.html Looks to me like someone got confused about ifdef ENABLE_THREAD_SAFETY versus ifdef USE_SSL ... too tired to figure it out exactly though ... regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] [HACKERS] fsync method checking
Tom Lane wrote: [EMAIL PROTECTED] writes: I could certainly do some testing if you want to see how DBT-2 does. Just tell me what to do. ;) Just do some runs that are identical except for the wal_sync_method setting. Note that this should not have any impact on SELECT performance, only insert/update/delete performance. I've made a test run that compares fsync and fdatasync: The performance was identical: - with fdatasync: http://khack.osdl.org/stp/290607/ - with fsync: http://khack.osdl.org/stp/290483/ I don't understand why. Mark - is there a battery backed write cache in the raid controller, or something similar that might skew the results? The test generates quite a lot of wal traffic - around 1.5 MB/sec. Perhaps the writes are so large that the added overhead of syncing the inode is not noticable? Is the pg_xlog directory on a seperate drive? Btw, it's possible to request such tests through the web-interface, see http://www.osdl.org/lab_activities/kernel_testing/stp/script_param.html -- Manfred ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [GENERAL] unicode error and problem
By the way, Unicode is just a number - glyph mapping, it doesn't say anything about the representation of that number in the byte stream. UTF-8 and UTF-16 are such representation specifications. The encoding name in PostgreSQL should be changed from UNICODE to UTF-8 because UNICODE really just isn't an encoding. Actually you can use UTF-8 instead of UNICODE when using PostgreSQL. However the primary name is still UNICODE, and I agree it's better to change to UTF-8 for the primary name. Maybe for 7.5? -- Tatsuo Ishii ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match