Re: [HACKERS] Any reason why the default_with_oids GUC is still there?
On tis, 2010-09-21 at 18:31 -0400, Bruce Momjian wrote: Also, doesn't some SQL standard require oids, so we should have a way to enable them by default for all tables? From some DB2 example: CREATE TYPE BusinessUnit_t AS (Name VARCHAR(20), Headcount INT); CREATE TABLE BusinessUnit OF BusinessUnit_t (REF IS oid USER GENERATED); The DB2 documentation consistently refers to this column as oid, but there is no requirement to name it that way. The SQL standard also contains this sentence: Let OID be the name of the self-referencing column of S. which refers to the thing defined in the example above, but OID is just a placeholder here. I think there was a mention of OIDs in the SQL3 draft that eventually became SQL99, but that's long past now. Current standards don't have it, except in the, perhaps more generalized, form above. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] knngist patch preliminary review (2010-09 commitfest)
I haven't quite finished reviewing this, but here is the position so far. I'm going to continue with some performance and other tests, but I'm posting this for discussion in the mean time. 1) General patch issues - applies cleanly and passes regression - one small warning with ecpg parser build, which I assume is just due to the patch having touched the main parser in a way the ecpg build doesn't expect (presumably this will be cleaned up at some later stage) - *no* documentation (see below) 2) Design questions Reading through the previous discussion on -hackers, I didn't get the impression that there was agreement on the question of how to represent the ordering operators in the catalog. This patch takes the approach of adding a boolean column pg_amop.amoporder and doing changes that require touching unrelated code in a fair number of places. In addition there are the points Robert Haas expressed in his earlier response. This approach doesn't feel right to me, but I don't know enough about it (especially any possible other features that might want to do something similar) to express a strong opinion on it. So that point is open for discussion. 3) Documentation There are problems with the GiST docs that go much deeper than this patch alone; the manual sections on writing GiST support functions are wholly inadequate, and many features that have been around for a long time, such as secondary split in GiST picksplit functions, are essentially undocumented other than as (uncommented!) example code in contrib modules. This patch would, as it stands, make that issue worse. My opinion is that the gist-implementation section of the docs needs to be substantially expanded so that it explains, for each support function, exactly what the function is expected to do. If it would help, I'm prepared to put some time towards actually writing something up for this. -- Andrew (irc:RhodiumToad) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Do we need a ShmList implementation?
Hi, On 09/21/2010 05:48 PM, Kevin Grittner wrote: OK, I'd say it's a little rough yet, but it works. Is this reasonable?: http://git.postgresql.org/gitweb?p=users/kgrittn/postgres.git;a=commitdiff;h=b8eca245ab63725d0fbfc3b5969f4a17fc765f2c I only get a: 404 - Unknown commit object on that link. Did you push your work? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On 22/09/10 03:25, Joshua D. Drake wrote: Why is this in the config file at all. It should be: synchronous_replication = TRUE/FALSE Umm, what does this do? then ALTER CLUSTER ENABLE REPLICATION FOR FOO; ALTER CLUSTER SET keep_connect ON FOO TO TRUE; Or some such thing. I like a configuration file more because you can easily add comments, comment out lines, etc. It also makes it easier to have a different configuration in master and standby. We don't support cascading slaves, yet, but you might still want a different configuration in master and slave, waiting for the moment that the slave is promoted to a new master. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Needs Suggestion
How can I increase the stack depth limit ? Is it only by modifying the postgres.conf file, but there I cannot increase the stack depth beyond 4 MB. Actually, my problem is that, when I set the stack base address of the child thread using the POSIX library function pthread_setstackaddr(), I am unable to access the memory contents of its parent. The data-structures in the parent are either getting destroyed or unaccessible when moving to the context of the child thread. But when I create the child thread without setting its stack base address then it is able to access the parent's memory contents. But if the child thread's code size is large then it is getting stack fault and displaying the message MAX stack depth limit reached Can anyone give any reason why is that so ? OR If I cannot set the child thread's stack base address. Then How can I increase the stack depth limit to a large value ? -- Thank You, Subham Roy. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQLSTATE of notice PGresult
Hey all, Okay, as Robert points, 0 code in successful messages seems as waste of bytes. But according to the documentation, All messages emitted by the PostgreSQL server are assigned five-character error codes that follow the SQL standard's conventions for SQLSTATE codes. - the first sentence of http://www.postgresql.org/docs/9.0/static/errcodes-appendix.html -- Regards, Dmitriy
Re: [HACKERS] Configuring synchronous replication
On 21/09/10 18:12, Tom Lane wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes: On 21/09/10 11:52, Thom Brown wrote: My fear would be standby.conf would be edited by users who don't really know XML and then we'd have 3 different styles of config to tell the user to edit. I'm not a big fan of XML either. ... Then again, maybe we should go with something like json or yaml The fundamental problem with all those machine editable formats is that they aren't people editable. If you have to have a tool (other than a text editor) to change a config file, you're going to be very unhappy when things are broken at 3AM and you're trying to fix it while ssh'd in from your phone. I'm not very familiar with any of those formats, but I agree it needs to be easy to edit by hand first and foremost. I think the ini file format suggestion is probably a good one; it seems to fit this problem, and it's something that people are used to. We could probably shoehorn the info into a pg_hba-like format, but I'm concerned about whether we'd be pushing that format beyond what it can reasonably handle. The ini file format seems to be enough for the features proposed this far, but I'm a bit concerned that even that might not be flexible enough for future features. I guess we'll cross the bridge when we get there and go with an ini file for now. It should be possible to extend it in various ways, and in the worst case that we have to change to a completely different format, we can provide a how to guide on converting existing config files to the new format. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
Hi, On 09/21/2010 08:05 PM, Simon Riggs wrote: Hmm, no reason? The reason is that the alternative is that the session would hang until a standby arrived that offered that level of service. Why would you want that behaviour? Would you really request that option? I think I now agree with Simon on that point. It's only an issue in multi-master replication, where continued operation would lead to a split-brain situation. With master-slave, you only need to make sure your master stays the master even if the standby crash(es) are followed by a master crash. If your cluster-ware is too clever and tries a fail-over on a slave that's quicker to come up, you get the same split-brain situation. Put another way: if you let your master continue, don't ever try a fail-over after a full-cluster crash. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Needs Suggestion
On 22/09/10 11:14, sub...@cse.iitb.ac.in wrote: How can I increase the stack depth limit ? Is it only by modifying the postgres.conf file, but there I cannot increase the stack depth beyond 4 MB. Actually, my problem is that, when I set the stack base address of the child thread using the POSIX library function pthread_setstackaddr(), I am unable to access the memory contents of its parent. The data-structures in the parent are either getting destroyed or unaccessible when moving to the context of the child thread. It is not a good idea to use threads in server code. PostgreSQL server code is not thread-safe, things will break. Assuming that you're not actually doing that but using threads in the client instead, max_stack_depth should have no effect on you as it only affects the server. But you really should find another way to communicate between threads. Stacks should be treated as thread-private. Use malloc() or something and global variables. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BUG #5661: The character encoding in logfile is confusing.
[moving to pgsql-hackers; this isn't the simple bug I initially suspected it might be] On 20/09/10 03:10, Tom Lane wrote: Craig Ringer cr...@postnewspapers.com.au writes: One of the correctly encoded messages is Unexpected EOF received on client connection One of the incorrectly encoded (shift-JIS) messages is: Fast Shutdown request received. Another is Aborting any active transactions. ... question now is where the messages are converted from UTF-8 to shift-JIS and why that conversion is being applied inconsistently. Given those three examples, I wonder whether all the mis-encoded messages are emitted by the postmaster, rather than backends. Anyway it seems that you ought to look for some pattern in which messages are correctly vs incorrectly encoded. I think you're right. Looking into it more, though, I'm not even sure what the correct behaviour even is. I don't think this is a simple bug where Pg fails to convert between encodings in a few places; rather, it's a design oversight where the effect of having a system encoding different from the encoding of the database(s) isn't considered. A single log file should obviously be in a single encoding, it's the only sane way to do things. But which encoding is it in? And which *should* it be in? - The system text encoding? This is what the postmaster will have from its environment, and is what the user will expect the logs to be in. Postmaster will emit messages in this encoding at least during startup, as it doesn't know what encoding the cluster uses yet. (In fact it seems to stick to the system encoding throughout its life). - The default database encoding supplied to initdb during cluster creation? - The encoding of the database emitting a message? This makes sense when considering RAISE messages, for example. Backends will currently use this encoding when emitting log messages, whether user-supplied or translated from po files. This confusion leads to the mixed encoding issues reported by the OP. It's not a simple bug, it's a design issue. Unfortunately, it's not as simple as picking one of the above encodings for all logging. The system encoding isn't a good choice, because it might not be capable of representing all characters emitted by user RAISE statements in databases with a different encoding, nor all double quoted identifiers, parameter values, etc etc etc. For example, if the system encoding is SHIFT-JIS, but user databases emit messages with Chinese, Cyrillic, extended latin, or pretty much any non-Japanese characters, there's no sane way to convert messages containing any user text to shift-JIS for logging. The same applies with a latin-1 (iso-8859-1) system encoding and a utf-8 or shift-jis database emitting Japanese messages. Scratch using the system encoding for logging. What about the encoding used by initdb to create the cluster? It seems sensible, but: - The postmaster doesn't know what it is when it's doing it's initial startup. How can the postmaster complain that it can't find / open the cluster datadir when it doesn't know what encoding to use for the complaint? - If the cluster isn't created as utf-8, the same issue as with the system encoding applies. Using the encoding of the emitting database will permit all messages to be represented, but will give rise to mixed encodings in the log file, and still won't help the postmaster know what to do before it's found and read the cluster. I'm now inclined to propose that all logging be done unconditionally in utf-8, with a BOM written to the start of every log file. Backends with non-utf-8 databases should convert messages to utf-8 for logging. Because PostgreSQL supports the use of different encodings in different databases this is the only way to ensure sane logging with consistent encoding in a single log file. The only alternative I see is to break logging out into separate files: - postmaster.log for postmaster etc - [databasename].log for each database, in that database's encoding ... but I'm not confident that'd be worth the confusion. Neither scheme solves the question of what to do when logging to syslog, though. Syslog expects messages in the system encoding, and Pg would be wrong to log in any other encoding. Yet as databases may have characters that cannot be represented in the system encoding, the system encoding isn't good enough. Should syslog messages be converted to the system encoding with non-representable characters replaced by ? or some other placeholder? Blech. Ideas? Suggestions? -- Craig Ringer Tech-related writing: http://soapyfrogs.blogspot.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] snapshot generation broken
Hi all! It seems that the git move has broken the generation of the automatic snapshot tarballs - has anybody yet looked into what it would take to move those to fetching from git? Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Standby registration
(starting yet another thread to stay focused) Having mulled through all the recent discussions on synchronous replication, ISTM there is pretty wide consensus that having a registry of all standbys in the master is a good idea. Even those who don't think it's *necessary* for synchronous replication seem to agree that it's nevertheless a pretty intuitive way to configure it. And it has some benefits even if we never get synchronous replication. So let's put synchronous replication aside for now, and focus on standby registration first. Once we have that, the synchronous replication patch will be much smaller and easier to review. The consensus seems to be use a configuration file called standby.conf. Let's use the ini file format for now [1]. Aside from synchronous replication, there are three nice things we can do with a standby registry: A) Make monitoring easier. Let's have a system view to show the status of all standbys [2]. B) Improve authorization. At the moment, we require superuser rights to connect for connecting in replication mode. That's pretty ugly, because superuser rights imply that you can do anything; you'd typically want to restrict access from the standby to do replication only and nothing else. You can lock it down in pg_hba.conf to allow the superuser to only connect in replication mode, but it still gives me the creeps. When each standby has a name, we can associate standbys with roles, so that you have to be user X to replicate as standby Y. C) Smarter replacement for wal_keep_segments. Instead of always keeping wal_keep_segments WAL files around, once we know how far each standby has replicated, we can keep just the right amount. I think we'll still want a global upper limit to avoid running out of disk space in the master in case of emergency though. Any volunteers on implementing that? Fujii-san? [1] http://archives.postgresql.org/pgsql-hackers/2010-09/msg01195.php [2] http://archives.postgresql.org/pgsql-hackers/2010-09/msg00932.php -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On 09/22/2010 04:18 AM, Heikki Linnakangas wrote: On 21/09/10 18:12, Tom Lane wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes: On 21/09/10 11:52, Thom Brown wrote: My fear would be standby.conf would be edited by users who don't really know XML and then we'd have 3 different styles of config to tell the user to edit. I'm not a big fan of XML either. ... Then again, maybe we should go with something like json or yaml The fundamental problem with all those machine editable formats is that they aren't people editable. If you have to have a tool (other than a text editor) to change a config file, you're going to be very unhappy when things are broken at 3AM and you're trying to fix it while ssh'd in from your phone. I'm not very familiar with any of those formats, but I agree it needs to be easy to edit by hand first and foremost. I think the ini file format suggestion is probably a good one; it seems to fit this problem, and it's something that people are used to. We could probably shoehorn the info into a pg_hba-like format, but I'm concerned about whether we'd be pushing that format beyond what it can reasonably handle. The ini file format seems to be enough for the features proposed this far, but I'm a bit concerned that even that might not be flexible enough for future features. I guess we'll cross the bridge when we get there and go with an ini file for now. It should be possible to extend it in various ways, and in the worst case that we have to change to a completely different format, we can provide a how to guide on converting existing config files to the new format. The ini file format is not flexible enough, IMNSHO. If we're going to adopt a new config file format it should have these characteristics, among others: * well known (let's not invent a new one) * supports hierarchical structure * reasonably readable I realize that the last is very subjective. Personally, I'm very comfortable with XML, but then I do a *lot* of work with it, and have for many years. I know I'm in a minority on that, and some people just go bananas when they see it. Since we're just about to add a JSON parser to the backend, by the look of it, that looks like a reasonable bet. Maybe it uses a few too many quotes, but that's not really so hard to get your head around, even if it offends you a bit aesthetically. And it is certainly fairly widely known. cheers andrew
Re: [HACKERS] snapshot generation broken
On 22/09/10 11:33, Stefan Kaltenbrunner wrote: Hi all! It seems that the git move has broken the generation of the automatic snapshot tarballs - has anybody yet looked into what it would take to move those to fetching from git? BTW, there is a nice command called in git to create tarballs: git archive master | gzip postgresql.tar.gz I doubt we can use it because we add some generated files to our tarballs that are not in the repository, but I thought I'd mention it. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On Wed, Sep 22, 2010 at 10:47, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 22/09/10 11:33, Stefan Kaltenbrunner wrote: Hi all! It seems that the git move has broken the generation of the automatic snapshot tarballs - has anybody yet looked into what it would take to move those to fetching from git? BTW, there is a nice command called in git to create tarballs: git archive master | gzip postgresql.tar.gz I doubt we can use it because we add some generated files to our tarballs that are not in the repository, but I thought I'd mention it. It is useful, yes, but not for this. I'll take a peek at the tarball generation today, unless beaten to it (meaning if someone wants to star tlooking into it, send me an email or ping me on irc to make sure we don't duplicate the work) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstan and...@dunslane.net wrote: The ini file format is not flexible enough, IMNSHO. If we're going to adopt a new config file format it should have these characteristics, among others: well known (let's not invent a new one) supports hierarchical structure reasonably readable The ini format meets all of those requirements - and it's certainly far more readable/editable than XML and friends. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Needs Suggestion
On 22/09/10 12:03, sub...@cse.iitb.ac.in wrote: Actually, I used palloc() to set the stack base address. And I am trying to create only a single thread, then also it is causing problem. Actually, I created all the data-structures using palloc(), then I am passing these to the child thread. Even if I make these variables global then also it is not working. But if I don't set its stack address then it is working. But I need to do that because when my thread body is big then it is causing stack fault. So if I cannot set its stack address then Can I increase the stack depth limit to a large value ? It's not clear what you're trying to do, but it's just not going to work. The backend code is not thread-safe, so you can't safely create any threads in server code. Not even a single one. And even if you could, you should not mess with stack base addresses. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: It seems that the git move has broken the generation of the automatic snapshot tarballs - has anybody yet looked into what it would take to move those to fetching from git? Depends on what's broken about it, but I notice that the developer docs and the NLS builds are also not updating. Perhaps something wrong with the anoncvs service. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BUG #5661: The character encoding in logfile is confusing.
On ons, 2010-09-22 at 16:25 +0800, Craig Ringer wrote: A single log file should obviously be in a single encoding, it's the only sane way to do things. But which encoding is it in? And which *should* it be in? We need to produce the log output in the server encoding, because that's how we need to send it to the client. If you have different databases with different server encodings, you will get inconsistently encoded output in the log file. Conceivably, we could create a configuration option that specifies the encoding for the log file, and strings a recoded from whatever gettext() produces to the specified encoding. initdb could initialize that option suitably, so in most cases users won't have to do anything. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: It seems that the git move has broken the generation of the automatic snapshot tarballs - has anybody yet looked into what it would take to move those to fetching from git? Depends on what's broken about it, but I notice that the developer docs and the NLS builds are also not updating. Perhaps something wrong with the anoncvs service. anoncvs still serves the old cvs repository. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On Wed, Sep 22, 2010 at 11:11 AM, Magnus Hagander mag...@hagander.net wrote: On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: It seems that the git move has broken the generation of the automatic snapshot tarballs - has anybody yet looked into what it would take to move those to fetching from git? Depends on what's broken about it, but I notice that the developer docs and the NLS builds are also not updating. Perhaps something wrong with the anoncvs service. anoncvs still serves the old cvs repository. Whens that due to be fixed? I imagine most of the buildfarm is testing that still... -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
Peter Eisentraut wrote: On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: It seems that the git move has broken the generation of the automatic snapshot tarballs - has anybody yet looked into what it would take to move those to fetching from git? Depends on what's broken about it, but I notice that the developer docs and the NLS builds are also not updating. Perhaps something wrong with the anoncvs service. heh - well nagios complained that the current tarballs are outdated, I have not actually investigated further :) But iirc snapshot generation is running against the main cvs repo and not anoncvs. Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On Wed, Sep 22, 2010 at 12:19, Dave Page dp...@pgadmin.org wrote: On Wed, Sep 22, 2010 at 11:11 AM, Magnus Hagander mag...@hagander.net wrote: On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: It seems that the git move has broken the generation of the automatic snapshot tarballs - has anybody yet looked into what it would take to move those to fetching from git? Depends on what's broken about it, but I notice that the developer docs and the NLS builds are also not updating. Perhaps something wrong with the anoncvs service. anoncvs still serves the old cvs repository. Whens that due to be fixed? I imagine most of the buildfarm is testing that still... Don't have an actual time for that. Figured we'd leave the old one running until things have settled down. Mind you, I think bf members will need tweaking anyway, since they will assume the old style cvs layout. AFAICT, git-cvsserver will export each branch as a module, rather than one module as pgsql. We can obviously get that working for the HEAD branch (by just mapping it to a branch called pgsql), but I don't know how doable that is for backbranches. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Make tuples_per_page pr. table configureable.
On Tue, Sep 21, 2010 at 10:33 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: I didn't read previous discussions, but did we have consensus that number of tuples is better than other units for the parameter? For example, threshold bytes or percentage in a page also seems to be reasonable for me. I think either of those is more intuitive than tuples per page, and especially the first. But I'm not sure that's really the knob we want either. I feel like what people are really trying to tune here is more like - if a single datum is larger than X bytes, push it out. But I might be all wet. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On Wed, Sep 22, 2010 at 10:33, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: Hi all! It seems that the git move has broken the generation of the automatic snapshot tarballs - has anybody yet looked into what it would take to move those to fetching from git? I think this should be working now. I made this change to mk-snapshot, and a similar one to mk-stable_snapshot. I have run the snapshot generation for master, and the one for 8.4 should finish in a minute or so. I have disabled the cronjobs until someone else has verified the tarballs. When I do, I'll also enable generation of snapshots of REL9_0_STABLE. We currently do back to 8.2 - should we keep that, or drop 8.2? //Magnus *** old/mk-snapshot Wed Sep 22 06:02:02 2010 --- mk-snapshot Wed Sep 22 07:09:13 2010 *** *** 8,14 export MAKE=gmake export MD5SUM=md5 ! export CVSROOT=:ext:scra...@cvs.postgresql.org:/cvsroot export MAKEFLAGS=VERSION=snapshot export TMPDIR=/usr/local/pgsql/snapshot --- 8,14 export MAKE=gmake export MD5SUM=md5 ! export GIT_DIR=/usr/local/pgsql/git/postgresql/.git export MAKEFLAGS=VERSION=snapshot export TMPDIR=/usr/local/pgsql/snapshot *** *** 19,25 then rm -fr pgsql fi ! /usr/bin/cvs -q export -rHEAD pgsql cd pgsql ./configure export OUTPUTFILE=$($MAKE -s $MAKEFLAGS distdir-location) --- 19,28 then rm -fr pgsql fi ! mkdir pgsql ! git fetch ! git archive origin/master | tar xf - -C pgsql ! cd pgsql ./configure export OUTPUTFILE=$($MAKE -s $MAKEFLAGS distdir-location) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQLSTATE of notice PGresult
On Wed, Sep 22, 2010 at 4:18 AM, Dmitriy Igrishin dmit...@gmail.com wrote: Okay, as Robert points, 0 code in successful messages seems as waste of bytes. But according to the documentation, All messages emitted by the PostgreSQL server are assigned five-character error codes that follow the SQL standard's conventions for SQLSTATE codes. - the first sentence of http://www.postgresql.org/docs/9.0/static/errcodes-appendix.html Sounds like that wording needs some adjustment. I'm not even sure that it would be correct to say All error messages..., unless elog(ERROR, can't happen) throws something into that field. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
Magnus Hagander wrote: On Wed, Sep 22, 2010 at 10:33, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: Hi all! It seems that the git move has broken the generation of the automatic snapshot tarballs - has anybody yet looked into what it would take to move those to fetching from git? I think this should be working now. I made this change to mk-snapshot, and a similar one to mk-stable_snapshot. I have run the snapshot generation for master, and the one for 8.4 should finish in a minute or so. I have disabled the cronjobs until someone else has verified the tarballs. When I do, I'll also enable generation of snapshots of REL9_0_STABLE. We currently do back to 8.2 - should we keep that, or drop 8.2? hmm I would say we should do all supported branches because those are the ones that people might be interested in when looking for a fix on a branch... Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Get the offset of a tuple inside a table
On Tue, Sep 21, 2010 at 10:58 PM, Pei He hepeim...@gmail.com wrote: Hi Tom, The bitmapset works for me. I want to implement the operator for the following query: Select * from a left join b on a.id = b.id order by b.id; In a left outer join, I want the tuples that have matches in the inner table appear first. So, the order by clause is need. Why can't you just write SELECT * FROM a LEFT JOIN b ON a.id = b.id ORDER BY b.id NULLS FIRST? I want my query results in a different order is almost never something that requires modifying the source code. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Documentation, window functions
In http://www.postgresql.org/docs/9.0/static/tutorial-window.html it say Although avg will produce the same result no matter what order it processes the partition's rows in, this is not true of all window functions. When needed, you can control that order using ORDER BY within OVER. While it's true that avg() produce the same result no matter what order. A ORDER BY clause will affect what rows are included in the computation and thus change the result (the default window frame is RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW). So one can not in general add an ORDER BY to the example in the tutorial and get the same result as without an ORDER BY. Maybe we can find some better wording of the above? /Dennis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compute the number of tuples in a block
On Tue, Sep 21, 2010 at 11:42 PM, Pei He hepeim...@gmail.com wrote: Hi, In ctid, there are the block num, and the tuple offset inside the block. How can I know the maximum number of tuples in a block? The block size would be BLCKSZ. See MaxHeapTuplesPerPage and MaxIndexTuplesPerPage. I am not quite sure where is the best place to find the size of the tuple in a table. You might want to look at contrib/pageinspect. See also PageGetItemId and ItemIdGetLength. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multi-branch committing in git, revisited
At 2010-09-21 21:20:06 -0400, t...@sss.pgh.pa.us wrote: So it seems like maybe we still need some more thought about how to recognize related commits in different branches. I'd suggest using git cherry-pick -x (or something similar) to mark backported patches: -x When recording the commit, append to the original commit message a note that indicates which commit this change was cherry-picked from. Append the note only for cherry picks without conflicts. Do not use this option if you are cherry-picking from your private branch because the information is useless to the recipient. If on the other hand you are cherry-picking between two publicly visible branches (e.g. backporting a fix to a maintenance branch for an older release from a development branch), adding this information can be useful. I don't think it makes any sense to contort your workflow to commit to different branches at the same instant just to be able to group commits by timestamp. Using the trail left by cherry-pick -x is much better. You can just commit your changes to master and cherry-pick them wherever you want to. This is independent of doing the work in a topic branch. (Of course, with git it's less troublesome to merge forward rather than pick backwards, but that's a workflow change that's a lot harder to adjust to.) -- ams -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Per-column collation, work in progress
On Thu, Sep 16, 2010 at 5:46 AM, Peter Eisentraut pete...@gmx.net wrote:
Following up on my previous patch [0], here is a fairly complete
implementation of this feature. The general description and
implementation outline of the previous message still apply. This patch
contains documentation and regression tests, which can serve as further
explanations.
I tested the patch on database with encoding=UTF8 and locale-C.
I have a couple of questions and comments.
* CREATE TABLE (LIKE table_with_collation) doesn't inherit collations.
We need to copy collations by default, or add INCLUDING COLLATE option.
* upper() doesn't work if a column has a collation.
It still works if a column doesn't have a collation.
postgres=# \d tbl
Table public.tbl
Column | Type | Modifiers
+--+
c | text | collate C
ja | text | collate ja_JP.utf8
postgres=# SELECT name, setting FROM pg_settings WHERE name IN
('lc_ctype', 'lc_collate');
name| setting
+-
lc_collate | C
lc_ctype | C
(2 rows)
postgres=# SELECT upper(c) FROM tbl;
ERROR: invalid multibyte character for locale
HINT: The server's LC_CTYPE locale is probably incompatible with the
database encoding.
postgres=# SELECT upper(ja) FROM tbl;
ERROR: invalid multibyte character for locale
HINT: The server's LC_CTYPE locale is probably incompatible with the
database encoding
* Comparison of strings with different collations is forbidden,
but assignment is allowed, right?
postgres=# SELECT * FROM tbl WHERE c = ja;
ERROR: collation mismatch between implicit collations C and ja_JP.utf8
LINE 1: SELECT * FROM tbl WHERE c = ja;
^
HINT: You can override the collation by applying the COLLATE clause
to one or both expressions.
postgres=# INSERT INTO tbl(c, ja) SELECT ja, c FROM tbl;
INSERT 0 6
* psql \d needs a separator between collate and not null modifiers.
postgres=# ALTER TABLE tbl ALTER COLUMN c SET NOT NULL;
ALTER TABLE
postgres=# \d tbl
Table public.tbl
Column | Type | Modifiers
+--+
c | text | collate Cnot null= HERE
ja | text | collate ja_JP.utf8
the feature overall only works on Linux/glibc.
We could support it also on MSVC.
http://msdn.microsoft.com/en-us/library/a7cwbx4t(v=VS.90).aspx -- _strcoll_l
http://msdn.microsoft.com/en-us/library/45119yx3(v=VS.90).aspx -- _towupper_l
--
Itagaki Takahiro
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On 09/22/2010 04:54 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstanand...@dunslane.net wrote: The ini file format is not flexible enough, IMNSHO. If we're going to adopt a new config file format it should have these characteristics, among others: well known (let's not invent a new one) supports hierarchical structure reasonably readable The ini format meets all of those requirements - and it's certainly far more readable/editable than XML and friends. No, it's really not hierarchical. It only has goes one level deep. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multi-branch committing in git, revisited
On Wed, Sep 22, 2010 at 05:47, Tom Lane t...@sss.pgh.pa.us wrote:
Bruce Momjian br...@momjian.us writes:
However, keep in mind that creating a branch in every existing backpatch
branch is going to create even more backpatching monkey-work.
Monkey-work is scriptable though. It'll all be worth it if git
cherry-pick is even marginally smarter about back-merging the actual
patches. In principle it could be less easily confused than plain
old patch, but I was a bit discouraged by the upthread comment that
it's just a shorthand for git diff | patch :-(
FWIW, here's the workflow I just tried for the gitignore patch (blame
me and not the workflow if I broke the patch, btw :P)
* Have a master committers repo, with all active branches checked out
(and a simple script that updates and can reset them all automatically)
* Have a working repo, where I do my changes. Each branch gets a checkout
when necessary here, and this is where I apply it. I've just used
inline checkouts,
but I don't see why it shouldn't work with workdirs etc.
* In the working repo, apply patch to master branch.
* Then use git cherry-pick to get it into the back branches (still in
the working repo)
At this point, also do the testing of the backpatch.
At this point we have commits with potentially lots of time in between them.
So now we squash these onto the committers repository, using a small script that
does this:
#!/bin/sh
set -e
CMSG=/tmp/commitmsg.$$
editor $CMSG
if [ ! -f $CMSG ]; then
echo No commit message, aborting.
exit 0
fi
export BRANCHES=master REL9_0_STABLE REL8_4_STABLE REL8_3_STABLE
REL8_2_STABLE REL8_1_STABLE REL8_0_STABLE REL7_4_STABLE
echo Fetching local changes so they are available to merge
git fetch local
for B in ${BRANCHES} ; do
echo Switching and merging $B...
git checkout $B
git merge --squash local/$B --no-commit
git commit -F $CMSG
done
rm -f $CMSG
BTW, before pushing, I like to do something like this:
git push --dry-run 21 |egrep -v ^To | awk '{print $1}'|xargs git
log --format=fuller
just to get a list of exactly what I'm about to push :-) That doesn't
mean there won't
be mistake, but maybe fewer of them...
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On Wed, Sep 22, 2010 at 12:07 PM, Andrew Dunstan and...@dunslane.net wrote: On 09/22/2010 04:54 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstanand...@dunslane.net wrote: The ini file format is not flexible enough, IMNSHO. If we're going to adopt a new config file format it should have these characteristics, among others: well known (let's not invent a new one) supports hierarchical structure reasonably readable The ini format meets all of those requirements - and it's certainly far more readable/editable than XML and friends. No, it's really not hierarchical. It only has goes one level deep. I guess pgAdmin/wxWidgets are broken then :-) [Servers] Count=5 [Servers/1] Server=localhost Description=PostgreSQL 8.3 ServiceID= DiscoveryID=/PostgreSQL/8.3 Port=5432 StorePwd=true Restore=false Database=postgres Username=postgres LastDatabase=postgres LastSchema=public DbRestriction= Colour=#FF SSL=0 Group=PPAS Rolename= [Servers/1/Databases] [Servers/1/Databases/postgres] SchemaRestriction= [Servers/1/Databases/pphq] SchemaRestriction= [Servers/1/Databases/template_postgis] SchemaRestriction= [Servers/2] ... ... -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote: Mind you, I think bf members will need tweaking anyway, since they will assume the old style cvs layout. AFAICT, git-cvsserver will export each branch as a module, rather than one module as pgsql. We can obviously get that working for the HEAD branch (by just mapping it to a branch called pgsql), but I don't know how doable that is for backbranches. We need to have a fix for this before we can do any more backbranch releases. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote: Mind you, I think bf members will need tweaking anyway, since they will assume the old style cvs layout. AFAICT, git-cvsserver will export each branch as a module, rather than one module as pgsql. We can obviously get that working for the HEAD branch (by just mapping it to a branch called pgsql), but I don't know how doable that is for backbranches. We need to have a fix for this before we can do any more backbranch releases. Fix for what, exactly? (assuming that people update their bf animals of course) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BUG #5661: The character encoding in logfile is confusing.
On 22/09/2010 5:45 PM, Peter Eisentraut wrote: On ons, 2010-09-22 at 16:25 +0800, Craig Ringer wrote: A single log file should obviously be in a single encoding, it's the only sane way to do things. But which encoding is it in? And which *should* it be in? We need to produce the log output in the server encoding, because that's how we need to send it to the client. That doesn't mean it can't be recoded for writing to the log file, though. Perhaps it needs to be. It should be reasonably practical to detect when the database and log encoding are the same and avoid the transcoding performance penalty, not that it's big anyway. If you have different databases with different server encodings, you will get inconsistently encoded output in the log file. I don't think that's an OK answer, myself. Mixed encodings with no delineation in one file = bug as far as I'm concerned. You can't even rely on being able to search the log anymore. You'll only get away with it when using languages that mostly stick to the 7-bit ASCII subset, so most text is still readable; with most other languages you'll get logs full of what looks to the user like garbage. Conceivably, we could create a configuration option that specifies the encoding for the log file, and strings a recoded from whatever gettext() produces to the specified encoding. initdb could initialize that option suitably, so in most cases users won't have to do anything. Yep, I tend to think that'd be the right way to go. It'd still be a bit of a pain, though, as messages written to stdout/stderr by the postmaster should be in the system encoding, but messages written to the log files should be in the encoding specified for logs, unless logging is being done to syslog, in which case it has to be in the system encoding after all... And, of course, the postmaster still doesn't know how to log anything it might emit before reading postgresql.conf, because it doesn't know what encoding to use. I still wonder if, rather than making this configurable, the right choice is to force logging to UTF-8 (with BOM) across the board, right from postmaster startup. It's consistent, all messages in all other encodings can be converted to UTF-8 for logging, it's platform independent, and text editors etc tend to understand and recognise UTF-8 especially with the BOM. Unfortunately, because many unix utilities (grep etc) aren't encoding aware, that'll cause problems when people go to search log files. For (eg) 広告掲載 The log files will contain the utf-8 bytes: \xe5\xba\x83\xe5\x91\x8a\xe6\x8e\xb2\xe8\xbc\x89 but grep on a shift-jis system will be looking for: \x8d\x4c\x8d\x90\x8cf\x8d\xda so it won't match. Ugh. If only we could say PostgreSQL requires a system locale with a UTF-8 encoding. Alas, I don't think that'd go down very well with packagers or installers. [Insert rant about how stupid it is that *nix systems still aren't all UTF-8 here]. -- Craig Ringer Tech-related writing at http://soapyfrogs.blogspot.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
Peter Eisentraut wrote: On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote: Mind you, I think bf members will need tweaking anyway, since they will assume the old style cvs layout. AFAICT, git-cvsserver will export each branch as a module, rather than one module as pgsql. We can obviously get that working for the HEAD branch (by just mapping it to a branch called pgsql), but I don't know how doable that is for backbranches. We need to have a fix for this before we can do any more backbranch releases. yeah having to touch ever BF member despite the fact that we have a git-cvs gateway seems heavily annoying and not really expected. However if this must be done oen should send a proper headsup and howto to the buildfarm member list (people need to add the 9.0 branch as well so we better tell them this now instead of having to do work twice). Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
Magnus Hagander wrote: On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote: Mind you, I think bf members will need tweaking anyway, since they will assume the old style cvs layout. AFAICT, git-cvsserver will export each branch as a module, rather than one module as pgsql. We can obviously get that working for the HEAD branch (by just mapping it to a branch called pgsql), but I don't know how doable that is for backbranches. We need to have a fix for this before we can do any more backbranch releases. Fix for what, exactly? (assuming that people update their bf animals of course) how exactly should people fix their BF members - is there any howto or such on how to deal with the new git2cvs gateway? Switching to git entirely is rather hard for some of my members for example - at least not without investing hours of work... Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BUG #5661: The character encoding in logfile is confusing.
On Wed, Sep 22, 2010 at 12:25 PM, Craig Ringer cr...@postnewspapers.com.au wrote: I don't think that's an OK answer, myself. Mixed encodings with no delineation in one file = bug as far as I'm concerned. You can't even rely on being able to search the log anymore. You'll only get away with it when using languages that mostly stick to the 7-bit ASCII subset, so most text is still readable; with most other languages you'll get logs full of what looks to the user like garbage. This issue crops up periodically in the pgAdmin lists as well, as the mixed encoding sometimes break the log viewer. I still wonder if, rather than making this configurable, the right choice is to force logging to UTF-8 (with BOM) across the board, right from postmaster startup. It's consistent, all messages in all other encodings can be converted to UTF-8 for logging, it's platform independent, and text editors etc tend to understand and recognise UTF-8 especially with the BOM. That would be ideal for us. Unfortunately, because many unix utilities (grep etc) aren't encoding aware, that'll cause problems when people go to search log files. For (eg) 広告掲載 But not for others! -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote: On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut pete...@gmx.net wrote: We need to have a fix for this before we can do any more backbranch releases. Fix for what, exactly? (assuming that people update their bf animals of course) For the NLS workflow, we need to have some kind of CVS server that serves up all supported branches. Changing the URL or the module name or whatever is fine. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On Wed, Sep 22, 2010 at 13:28, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: Magnus Hagander wrote: On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote: Mind you, I think bf members will need tweaking anyway, since they will assume the old style cvs layout. AFAICT, git-cvsserver will export each branch as a module, rather than one module as pgsql. We can obviously get that working for the HEAD branch (by just mapping it to a branch called pgsql), but I don't know how doable that is for backbranches. We need to have a fix for this before we can do any more backbranch releases. Fix for what, exactly? (assuming that people update their bf animals of course) how exactly should people fix their BF members - is there any howto or such on how to deal with the new git2cvs gateway? Andrew has posted info on how to convert them to git, I believe. I don't think there are any for the git2cvs gateway, since it's not up yet. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote: On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut pete...@gmx.net wrote: We need to have a fix for this before we can do any more backbranch releases. Fix for what, exactly? (assuming that people update their bf animals of course) For the NLS workflow, we need to have some kind of CVS server that serves up all supported branches. Changing the URL or the module name or whatever is fine. I thought you had/were going to convert those to work natively with git? I guess I misunderstood you. Anyway, it's on my list to get done pretty quickly. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multi-branch committing in git, revisited
On 09/22/2010 07:07 AM, Magnus Hagander wrote:
On Wed, Sep 22, 2010 at 05:47, Tom Lanet...@sss.pgh.pa.us wrote:
Bruce Momjianbr...@momjian.us writes:
However, keep in mind that creating a branch in every existing backpatch
branch is going to create even more backpatching monkey-work.
Monkey-work is scriptable though. It'll all be worth it if git
cherry-pick is even marginally smarter about back-merging the actual
patches. In principle it could be less easily confused than plain
old patch, but I was a bit discouraged by the upthread comment that
it's just a shorthand for git diff | patch :-(
FWIW, here's the workflow I just tried for the gitignore patch (blame
me and not the workflow if I broke the patch, btw :P)
* Have a master committers repo, with all active branches checked out
(and a simple script that updates and can reset them all automatically)
* Have a working repo, where I do my changes. Each branch gets a checkout
when necessary here, and this is where I apply it. I've just used
inline checkouts,
but I don't see why it shouldn't work with workdirs etc.
* In the working repo, apply patch to master branch.
* Then use git cherry-pick to get it into the back branches (still in
the working repo)
At this point, also do the testing of the backpatch.
At this point we have commits with potentially lots of time in between them.
So now we squash these onto the committers repository, using a small script that
does this:
#!/bin/sh
set -e
CMSG=/tmp/commitmsg.$$
editor $CMSG
if [ ! -f $CMSG ]; then
echo No commit message, aborting.
exit 0
fi
export BRANCHES=master REL9_0_STABLE REL8_4_STABLE REL8_3_STABLE
REL8_2_STABLE REL8_1_STABLE REL8_0_STABLE REL7_4_STABLE
echo Fetching local changes so they are available to merge
git fetch local
for B in ${BRANCHES} ; do
echo Switching and merging $B...
git checkout $B
git merge --squash local/$B --no-commit
git commit -F $CMSG
done
rm -f $CMSG
BTW, before pushing, I like to do something like this:
git push --dry-run 21 |egrep -v ^To | awk '{print $1}'|xargs git
log --format=fuller
just to get a list of exactly what I'm about to push :-) That doesn't
mean there won't
be mistake, but maybe fewer of them...
What a rigmarole! This seems to be getting positively gothic.
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multi-branch committing in git, revisited
On Wed, Sep 22, 2010 at 13:47, Andrew Dunstan and...@dunslane.net wrote:
On 09/22/2010 07:07 AM, Magnus Hagander wrote:
On Wed, Sep 22, 2010 at 05:47, Tom Lanet...@sss.pgh.pa.us wrote:
Bruce Momjianbr...@momjian.us writes:
However, keep in mind that creating a branch in every existing backpatch
branch is going to create even more backpatching monkey-work.
Monkey-work is scriptable though. It'll all be worth it if git
cherry-pick is even marginally smarter about back-merging the actual
patches. In principle it could be less easily confused than plain
old patch, but I was a bit discouraged by the upthread comment that
it's just a shorthand for git diff | patch :-(
FWIW, here's the workflow I just tried for the gitignore patch (blame
me and not the workflow if I broke the patch, btw :P)
* Have a master committers repo, with all active branches checked out
(and a simple script that updates and can reset them all automatically)
* Have a working repo, where I do my changes. Each branch gets a checkout
when necessary here, and this is where I apply it. I've just used
inline checkouts,
but I don't see why it shouldn't work with workdirs etc.
* In the working repo, apply patch to master branch.
* Then use git cherry-pick to get it into the back branches (still in
the working repo)
At this point, also do the testing of the backpatch.
At this point we have commits with potentially lots of time in between
them.
So now we squash these onto the committers repository, using a small
script that
does this:
#!/bin/sh
set -e
CMSG=/tmp/commitmsg.$$
editor $CMSG
if [ ! -f $CMSG ]; then
echo No commit message, aborting.
exit 0
fi
export BRANCHES=master REL9_0_STABLE REL8_4_STABLE REL8_3_STABLE
REL8_2_STABLE REL8_1_STABLE REL8_0_STABLE REL7_4_STABLE
echo Fetching local changes so they are available to merge
git fetch local
for B in ${BRANCHES} ; do
echo Switching and merging $B...
git checkout $B
git merge --squash local/$B --no-commit
git commit -F $CMSG
done
rm -f $CMSG
BTW, before pushing, I like to do something like this:
git push --dry-run 21 |egrep -v ^To | awk '{print $1}'|xargs git
log --format=fuller
just to get a list of exactly what I'm about to push :-) That doesn't
mean there won't
be mistake, but maybe fewer of them...
What a rigmarole! This seems to be getting positively gothic.
FWIW, it's shorter and simpler than what I use for cvs ;) But maybe
that's because I'm being overly careful...
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On ons, 2010-09-22 at 12:20 +0100, Dave Page wrote: No, it's really not hierarchical. It only has goes one level deep. I guess pgAdmin/wxWidgets are broken then :-) [Servers] Count=5 [Servers/1] Server=localhost Well, by that logic, even what we have now for postgresql.conf is hierarchical. I think the criterion was rather meant to be - can represent hierarchies without repeating intermediate node names (Note: no opinion on which format is better for the task at hand) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On ons, 2010-09-22 at 13:45 +0200, Magnus Hagander wrote: On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut pete...@gmx.net wrote: For the NLS workflow, we need to have some kind of CVS server that serves up all supported branches. Changing the URL or the module name or whatever is fine. I thought you had/were going to convert those to work natively with git? I guess I misunderstood you. I think CVS should be eliminated from that process during the 9.1 release cycle, but we are looking for backbranch releases within the next few weeks, and there is no time to get that done by then. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On 09/22/2010 07:20 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 12:07 PM, Andrew Dunstanand...@dunslane.net wrote: On 09/22/2010 04:54 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstanand...@dunslane.net wrote: The ini file format is not flexible enough, IMNSHO. If we're going to adopt a new config file format it should have these characteristics, among others: well known (let's not invent a new one) supports hierarchical structure reasonably readable The ini format meets all of those requirements - and it's certainly far more readable/editable than XML and friends. No, it's really not hierarchical. It only has goes one level deep. I guess pgAdmin/wxWidgets are broken then :-) [Servers] Count=5 [Servers/1] Server=localhost Description=PostgreSQL 8.3 ServiceID= DiscoveryID=/PostgreSQL/8.3 Port=5432 StorePwd=true Restore=false Database=postgres Username=postgres LastDatabase=postgres LastSchema=public DbRestriction= Colour=#FF SSL=0 Group=PPAS Rolename= [Servers/1/Databases] [Servers/1/Databases/postgres] SchemaRestriction= [Servers/1/Databases/pphq] SchemaRestriction= [Servers/1/Databases/template_postgis] SchemaRestriction= [Servers/2] ... ... Well, that's not what I'd call a hierarchy, in any sane sense. I've often had to dig all over the place in ini files to find related bits of information in disparate parts of the file. Compared to a meaningful tree structure this is utterly woeful. In a sensible hierarchical format, all the information relating to, say, Servers/1 above, wopuld be under a stanza with that heading, instead of having separate and unnested stanzas like Servers/1/Databases/template_postgis. If you could nest stanzas in ini file format it would probably do, but you can't, leading to the above major ugliness. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On Wed, Sep 22, 2010 at 12:50 PM, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-09-22 at 12:20 +0100, Dave Page wrote: No, it's really not hierarchical. It only has goes one level deep. I guess pgAdmin/wxWidgets are broken then :-) [Servers] Count=5 [Servers/1] Server=localhost Well, by that logic, even what we have now for postgresql.conf is hierarchical. Well, yes - if you consider add-in GUCs which use prefixing like foo.setting=... I think the criterion was rather meant to be - can represent hierarchies without repeating intermediate node names If this were data, I could understand that as it could lead to tremendous bloat, but as a config file, I'd rather have the readability of the ini format, despite the repeated node names, than have to hack XML files by hand. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On 09/22/2010 06:22 AM, Magnus Hagander wrote: Mind you, I think bf members will need tweaking anyway, since they will assume the old style cvs layout. AFAICT, git-cvsserver will export each branch as a module, rather than one module as pgsql. We can obviously get that working for the HEAD branch (by just mapping it to a branch called pgsql), but I don't know how doable that is for backbranches. Buildfarm owners should be moving aggressively towards using git. I'm not aware of any platform it can't be used on. I was planning on doing an audit around the end of the week and following up with people who haven't migrated. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] .gitignore files, take two
On Tue, Sep 21, 2010 at 22:23, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Tue, Sep 21, 2010 at 22:15, Tom Lane t...@sss.pgh.pa.us wrote: Global patterns look ok to me. Thought you were going to stick leading slashes on all the others? Oh, misunderstood. I thought the idea was just slashes in the top-level ones, not the leaf ones. But I'll add it to those as well then :-) I think it'd be wise to have a convention of leading slash anywhere the pattern is not meant to be global. It won't matter to git in leaf dirs, but it might prevent somebody from making a copy-and-paste error later; or perhaps more likely, might prevent a problem if what had been a leaf directory acquires children. Done and applied. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Standby registration
On Wed, Sep 22, 2010 at 5:43 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: So let's put synchronous replication aside for now, and focus on standby registration first. Once we have that, the synchronous replication patch will be much smaller and easier to review. Though I agree with standby registration, I'm still unclear what's standby registration ;) What if the number of standby entries in standby.conf is more than max_wal_senders? This situation is allowed if we treat standby.conf as just access control list like pg_hba.conf. But if we have to ensure that all the registered standbys can connect to the master, we should emit the error in that case. Should we allow standby.conf to be changed and reloaded while the server is running? This seems to be required if we use standby.conf for replacement of wal_keep_segments. Because we need to register the backup starting location as the last receive location of upcoming standby when taking a base backup for that standby. But what if the reloaded standby.conf has no entry for already connected standby? If we treat standby.conf as just access control list, we can easily allow it to be reloaded as well as pg_hba.conf is. Otherwise, we would need a careful design. Should we allow multiple standbys with the same name to connect to the master? That is, entry in standby.conf and real standby should be one-to-one relationship? Or we should add new parameter specifying the number of standbys with the name? Any volunteers on implementing that? Fujii-san? I'm willing to implement that. But I'll be busy for a few days because of presentation at LinuxCon and so on. So please feel free to try that if time allows. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On 09/22/2010 07:57 AM, Dave Page wrote:
On Wed, Sep 22, 2010 at 12:50 PM, Peter Eisentrautpete...@gmx.net wrote:
On ons, 2010-09-22 at 12:20 +0100, Dave Page wrote:
No, it's really not hierarchical. It only has goes one level deep.
I guess pgAdmin/wxWidgets are broken then :-)
[Servers]
Count=5
[Servers/1]
Server=localhost
Well, by that logic, even what we have now for postgresql.conf is
hierarchical.
Well, yes - if you consider add-in GUCs which use prefixing like foo.setting=...
I think the criterion was rather meant to be
- can represent hierarchies without repeating intermediate node names
If this were data, I could understand that as it could lead to
tremendous bloat, but as a config file, I'd rather have the
readability of the ini format, despite the repeated node names, than
have to hack XML files by hand.
XML is not the only alternative - please don't use it as a straw man.
For example, here is a fragment from the Bacula docs using their
hierarchical format:
FileSet {
Name = Test
Include {
File = /home/xxx/test
Options {
regex = .*\.c$
}
}
}
Or here is a piece from the buildfarm client config (which is in fact
perl, but could also be JSON or similar fairly easily):
mail_events =
{
all = [],
fail = [],
change = ['f...@bar.com', 'b...@blurfl.org' ],
green = [],
},
build_env =
{
CCACHE_DIR = /home/andrew/pgfarmbuild/ccache/$branch,
},
cheers
andrew
Re: [HACKERS] Configuring synchronous replication
On Wed, Sep 22, 2010 at 1:25 PM, Andrew Dunstan and...@dunslane.net wrote:
XML is not the only alternative - please don't use it as a straw man. For
example, here is a fragment from the Bacula docs using their hierarchical
format:
FileSet {
Name = Test
Include {
File = /home/xxx/test
Options {
regex = .*\.c$
}
}
}
Or here is a piece from the buildfarm client config (which is in fact perl,
but could also be JSON or similar fairly easily):
mail_events =
{
all = [],
fail = [],
change = ['f...@bar.com', 'b...@blurfl.org' ],
green = [],
},
build_env =
{
CCACHE_DIR = /home/andrew/pgfarmbuild/ccache/$branch,
},
Both of which I've also used in the past, and also find uncomfortable
and awkward for configuration files.
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BUG #5661: The character encoding in logfile is confusing.
On ons, 2010-09-22 at 19:25 +0800, Craig Ringer wrote: Yep, I tend to think that'd be the right way to go. It'd still be a bit of a pain, though, as messages written to stdout/stderr by the postmaster should be in the system encoding, but messages written to the log files should be in the encoding specified for logs, unless logging is being done to syslog, in which case it has to be in the system encoding after all... I think that should not be a problem to implement. Those two go through different routines anyway. And, of course, the postmaster still doesn't know how to log anything it might emit before reading postgresql.conf, because it doesn't know what encoding to use. That should also not be a big issue. The postmaster needs the configuration file to know where to write the log file anyway. I still wonder if, rather than making this configurable, the right choice is to force logging to UTF-8 (with BOM) across the board, right from postmaster startup. It's consistent, all messages in all other encodings can be converted to UTF-8 for logging, it's platform independent, and text editors etc tend to understand and recognise UTF-8 especially with the BOM. I don't think this would make things better or easier. At some point you're going to have to insert a recode call, and it doesn't matter much whether the destination argument is a constant or a variable. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On 09/22/2010 08:32 AM, Dave Page wrote:
On Wed, Sep 22, 2010 at 1:25 PM, Andrew Dunstanand...@dunslane.net wrote:
XML is not the only alternative - please don't use it as a straw man. For
example, here is a fragment from the Bacula docs using their hierarchical
format:
FileSet {
Name = Test
Include {
File = /home/xxx/test
Options {
regex = .*\.c$
}
}
}
Or here is a piece from the buildfarm client config (which is in fact perl,
but could also be JSON or similar fairly easily):
mail_events =
{
all = [],
fail = [],
change = ['f...@bar.com', 'b...@blurfl.org' ],
green = [],
},
build_env =
{
CCACHE_DIR = /home/andrew/pgfarmbuild/ccache/$branch,
},
Both of which I've also used in the past, and also find uncomfortable
and awkward for configuration files.
I can't imagine trying to configure Bacula using ini file format - the
mind just boggles. Frankly, I'd rather stick with our current config
format than change to something as inadequate as ini file format.
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQLSTATE of notice PGresult
Robert Haas robertmh...@gmail.com wrote: I'm not even sure that it would be correct to say All error messages..., unless elog(ERROR, can't happen) throws something into that field. If it doesn't, it should. Probably 'XX000' (internal_error). -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Do we need a ShmList implementation?
Markus Wanner mar...@bluegap.ch wrote: I only get a: 404 - Unknown commit object on that link. Did you push your work? Yeah, but it has since been blown away (at my request) as part of my attempt to get it based on the new git conversion. Sorry about that. Attached is an mbox representation of what was there. Hopefully I can get it out there again today some time. -Kevin xactlist.mbox Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On Wed, Sep 22, 2010 at 13:45, Magnus Hagander mag...@hagander.net wrote: On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote: On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut pete...@gmx.net wrote: We need to have a fix for this before we can do any more backbranch releases. Fix for what, exactly? (assuming that people update their bf animals of course) For the NLS workflow, we need to have some kind of CVS server that serves up all supported branches. Changing the URL or the module name or whatever is fine. I thought you had/were going to convert those to work natively with git? I guess I misunderstood you. Anyway, it's on my list to get done pretty quickly. Started again on this. Whenever I try to do something useful, it crashes with sqlite errors that appear to be FreeBSD specific. I'm going to try to upgrade all the ports on the box and give it another try. Unfortunately, that tends to take a couple of hours - but it'll get done eventually :-) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multi-branch committing in git, revisited
On Tue, Sep 21, 2010 at 9:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: So it seems like maybe we still need some more thought about how to recognize related commits in different branches. Or at the very least, we need a best-practice document explaining how to manage this --- we shouldn't expect every committer to reinvent this wheel for himself. Comments? I don't think there's one right way to do this. In fact, there are probably at least 50 reasonable ways to do it, depending on your own workflow preferences. And you needn't do it the same way every time, either. I don't. git is designed to treat commits the way that databases treat rows. They are objects. You can create them, throw them out, move them around, replace them with updated versions, etc. And there are multiple ways of doing each of these things (just as there's no single right way to design a database schema). Of course, in the One True Source Tree, we only every create them at the heads of existing branches. But in your own workspace, you can do all of those things - and you should, because they make you able to get the same things done faster. What I plan to do, I think, is use one clone nearly all the time. If I need to back-patch, I'll switch branches and either apply from the stash or cherry-pick off the master. But if I need to to go back more than one or two branches and there are merge conflicts, I'll create a bunch of temporary clones off my main clone and push/pull from there. Then I'll remove them when I'm done. But from what I gather, there are probably going to be at least as many workflows as there are committers, and maybe more, since I just said I'm going to use two different approaches depending on the situation. One option is just to update the date stamp on each commit before you push. You could check out each branch you've updated and do something like: GIT_EDITOR=/usr/bin/true git commit --amend --date=now Of course that'll only update the top commit, and you want to be sure not to do it on branches where the top commit is something that's already been pushed. But to reiterate my main point, I think we should only dictate the contents of the commits that must be pushed (e.g. time stamps close together, so git_topo_order can match them up) and not the process by which someone creates those commits (because the chances of getting more than 1 person to agree on the way to generate such commits seems near zero, and the fact that I plan to do it different ways depending on the situation means that even getting 1 person to agree with themselves on how to do it may be out of reach). People have repeatedly suggested that timestamp/author/log message matching is a lousy way of matching up commits. I agree, but we have 14.25 years of history for which that's the only practical method. We could decide on a different method going forward, such as embedding a token (the format of which we can argue about until we're blue in the face - could be whatever git cherry-pick does or could be a reference to our issue tracking system if we had one, could be something else) in each commit in a very specific format which a script can then recognize. The advantage of that is that it might feel a bit less ad hoc than what we're doing now; the disadvantage is that then our scripts would have to know about both methods, unless of course we go back and annotate all of the old commits (using git-notes) with reconstructed information on which commits go together, which we already have from git_topo_order. No matter what we pick, it's going to require non-zero effort to not screw it up; and with the exception of doing forward merges which I think is a terrible idea, none of the methods so far proposed seem likely to take significantly more time than any of the others. So if we're going to change anything at all, we ought to focus on how it's going to improve the overall way that we manage the project, not on the exact sequence of commands that will be required to create it, which I'm fairly confident will settle down to a quite small number as you and everyone else get used to the new tool (but it won't be the same sequence for everyone). -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multi-branch committing in git, revisited
Tom Lane wrote: Okay, so now that I've actually done a couple of multi-branch commits... I'm using the multiple-work-directory arrangement suggested on our wiki page. The work flow seems to boil down to: * Prepare patch in master * Stage patch with git add * git diff --staged /tmp/patch-head * cd into REL9_0_STABLE workdir * patch -p0 /tmp/patch-head * Adjust patch if needed * Stage patch with git add * git diff --staged /tmp/patch-90 * cd into REL8_4_STABLE workdir * patch -p0 /tmp/patch-90 * ... lather, rinse, repeat ... * cd back to master * git commit -F /tmp/commitmsg * cd into REL9_0_STABLE workdir * git commit -F /tmp/commitmsg * cd into REL8_4_STABLE workdir * git commit -F /tmp/commitmsg * ... lather, rinse, repeat ... * git push Uh, just to be clear, the above is more complex than necessary because git diff will show all uncommitted modifications. You could just do: * Prepare patch in master * git diff /tmp/patch-head * cd into REL9_0_STABLE workdir * patch -p0 /tmp/patch-head * Adjust patch if needed ... There is no need for 'git add' because once you are done you can use git commmit -a in each branch to add all modifications and commit them. I think this exactly matches how we did thing with CVS. A final 'git push' sends them to the remote repository. This causes commits to all happen around the same time. I am not saying that is the way we should to it, but it is clearly possible. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Per-column collation, work in progress
On ons, 2010-09-22 at 19:44 +0900, Itagaki Takahiro wrote: * CREATE TABLE (LIKE table_with_collation) doesn't inherit collations. We need to copy collations by default, or add INCLUDING COLLATE option. OK, should be easy to fix. * upper() doesn't work if a column has a collation. It still works if a column doesn't have a collation. I think what you are observing is the result of mixing C and non-C locales. Of course that should also be fixed, but it doesn't have much to do with what upper() does. Note that there is a regression test case for lower(), which works mostly the same way. * Comparison of strings with different collations is forbidden, but assignment is allowed, right? Correct. * psql \d needs a separator between collate and not null modifiers. OK. We could support it also on MSVC. http://msdn.microsoft.com/en-us/library/a7cwbx4t(v=VS.90).aspx -- _strcoll_l http://msdn.microsoft.com/en-us/library/45119yx3(v=VS.90).aspx -- _towupper_l Great. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BUG #5661: The character encoding in logfile is confusing.
Craig Ringer cr...@postnewspapers.com.au writes: On 22/09/2010 5:45 PM, Peter Eisentraut wrote: We need to produce the log output in the server encoding, because that's how we need to send it to the client. That doesn't mean it can't be recoded for writing to the log file, though. Perhaps it needs to be. It should be reasonably practical to detect when the database and log encoding are the same and avoid the transcoding performance penalty, not that it's big anyway. We have seen ... and rejected ... such proposals before. The problem is that transcode to some other encoding is not a simple and guaranteed error-free operation. As an example, if you choose to name some table using a character that doesn't exist in the log encoding, you have just ensured that no message about that table will ever get to the log. Nice way to hide your activities from the DBA ;-) Transcoding also eats memory, which might be in exceedingly short supply while trying to report an out of memory error; and IIRC there are some other failure scenarios to be concerned about. We could maybe accept a design for this that included a sufficiently well-thought-out set of fallback behaviors. But we haven't seen one yet. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On Wed, Sep 22, 2010 at 9:01 AM, Andrew Dunstan and...@dunslane.net wrote: I can't imagine trying to configure Bacula using ini file format - the mind just boggles. Frankly, I'd rather stick with our current config format than change to something as inadequate as ini file format. Perhaps we need to define a little better what information we think we might eventually need to represent in the config file. With one exception, nobody has suggested anything that would actually require hierarchical structure. The exception is defining the policy for deciding when a commit has been sufficiently acknowledged by an adequate quorum of standbys, and it seems to me that doing that in its full generality is going to require not so much a hierarchical structure as a small programming language. The efforts so far have centered around reducing the use cases that $AUTHOR cares about to a set of GUCs which would satisfy that person's needs, but not necessarily everyone else's needs. I think efforts to encode arbitrary algorithms using configuration settings are doomed to failure, so I'm unimpressed by the argument that we should design the config file to support our attempts to do so. For everything else, no one has suggested that we need anything more complex than, essentially, a group of GUCs per server. So we could do: [server] guc=value or server.guc=value ...or something else. Designing this to support: server.hypothesis.experimental.unproven.imaginary.what-in-the-world-could-this-possibly-be = 42 ...seems pretty speculative at this point, unless someone can imagine what we'd want it for. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Standby registration
On Wed, Sep 22, 2010 at 8:21 AM, Fujii Masao masao.fu...@gmail.com wrote: What if the number of standby entries in standby.conf is more than max_wal_senders? This situation is allowed if we treat standby.conf as just access control list like pg_hba.conf. But if we have to ensure that all the registered standbys can connect to the master, we should emit the error in that case. I don't think a cross-check between these settings makes much sense. We should either get rid of max_wal_senders and make it always equal to the number of defined standbys, or we should treat them as independent settings. Should we allow standby.conf to be changed and reloaded while the server is running? Yes. But what if the reloaded standby.conf has no entry for already connected standby? We kick him out? Should we allow multiple standbys with the same name to connect to the master? No. The point of naming them is to uniquely identify them. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BUG #5661: The character encoding in logfile is confusing.
Peter Eisentraut pete...@gmx.net writes: On ons, 2010-09-22 at 19:25 +0800, Craig Ringer wrote: I still wonder if, rather than making this configurable, the right choice is to force logging to UTF-8 (with BOM) across the board, I don't think this would make things better or easier. At some point you're going to have to insert a recode call, and it doesn't matter much whether the destination argument is a constant or a variable. It'd avoid the problem of having possibly-unconvertable messages ... at the cost of pissing off users who have a uniform server encoding selection already and don't see why they should be forced to deal with UTF8 in the log. It's pretty much just one step from here to deciding that the server should work exclusively in UTF8 and never mind all those other legacy encodings. We've resisted that attitude for quite some years now, and are probably not really ready to adopt it for the log either. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multi-branch committing in git, revisited
Bruce Momjian br...@momjian.us writes: There is no need for 'git add' because once you are done you can use git commmit -a in each branch to add all modifications and commit them. git commit -a is not a universal solution. In particular, the patch I was dealing with yesterday involved additions and removals of files, neither of which will be implemented by git commit -a. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Git cvsserver serious issue
So, I found (with some helpful hints from Robert who caught the final nail in the coffin) a good reason why we really can't run a git-cvsserver globally. Any user can point their cvs client at the repository. And check out an arbitrary branch, tag *or individual commit*. Doing so will create a 50Mb sqlite database on the server with cache information about that head. That basically means that git-cvsserver is completely useless in a public scenario as it stands. An easier way to DOS our server is hard to find, really. Now, if we can limit this by IP address, that would be ok. I assume we can do this for the NLS stuff - peter? As for buildfarm members needing CVS - is it workable to require that the maintainers of these set up their own git clone with git cvsserver (over ssh or pserver) and restrict it locally to the IP(s) of their machines? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Standby registration
On 22/09/10 16:54, Robert Haas wrote: On Wed, Sep 22, 2010 at 8:21 AM, Fujii Masaomasao.fu...@gmail.com wrote: What if the number of standby entries in standby.conf is more than max_wal_senders? This situation is allowed if we treat standby.conf as just access control list like pg_hba.conf. But if we have to ensure that all the registered standbys can connect to the master, we should emit the error in that case. I don't think a cross-check between these settings makes much sense. We should either get rid of max_wal_senders and make it always equal to the number of defined standbys, or we should treat them as independent settings. Even with registration, we will continue to support anonymous asynchronous standbys that just connect and start streaming. We need some headroom for those. But what if the reloaded standby.conf has no entry for already connected standby? We kick him out? Sounds reasonable. Should we allow multiple standbys with the same name to connect to the master? No. The point of naming them is to uniquely identify them. Hmm, that situation can arise if there's a network glitch which leads the standby to disconnect, but the master still considers the connection as alive. When the standby reconnects, the master will see two simultaneous connections from the same standby. In that scenario, you clearly want to disconnect the old connetion in favor of the new one. Is there a scenario where you'd want to keep the old connection instead and refuse the new one? Perhaps that should be made configurable, so that you wouldn't need to list all standbys in the config file if you don't want to. Then you don't get any of the benefits of standby registration, though. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] repository size differences
On Tue, Sep 21, 2010 at 10:32 PM, Abhijit Menon-Sen a...@toroid.org wrote: That's not it. I ran the same git gc command on my old repository, and it didn't make any difference to the size. (I didn't try with a larger window size, though.) Probably lots of it has to do with the delta chains themselves. The old repository was an incremental conversion, so each new delta (as it's added) has only (and all) repository wide objects to look at for choosing a base. git has some limits and hueristics on deciding how far and wide to look for the best delta base. The cvs2* scripts are more direct, they first reference the files, then commit graph, etc, so all revisions of a particular file are added before moving on to the next. This means that all previous versions of a file are likely hot in the path git will look for the best fit delta. By changing the order of how the objects are added to the git repository, it makes it easier for git to find the best/better delta bases. You can adjust the delta window git-repack uses, see the man page for git-repack, and git-gc. If you're willing to do a monster repack on the old repository (using a *huge* window) you can likely get it close in size. a. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git cvsserver serious issue
Magnus Hagander mag...@hagander.net writes: Any user can point their cvs client at the repository. And check out an arbitrary branch, tag *or individual commit*. Doing so will create a 50Mb sqlite database on the server with cache information about that head. That basically means that git-cvsserver is completely useless in a public scenario as it stands. An easier way to DOS our server is hard to find, really. Ugh. Now, if we can limit this by IP address, that would be ok. I assume we can do this for the NLS stuff - peter? As for buildfarm members needing CVS - is it workable to require that the maintainers of these set up their own git clone with git cvsserver (over ssh or pserver) and restrict it locally to the IP(s) of their machines? If we're going to let people in by IP address, maybe we could let legacy buildfarm members in by IP address. It doesn't seem particularly helpful to expect each buildfarm owner to solve this problem for themselves. I'd also note that if they could run git locally, they wouldn't be needing cvsserver in the first place. Also, couldn't we just set up the cvsserver on its own VM with a limited amount of disk space, and not worry too much about any DOS threat? If somebody does do this, block them and reinitialize that server. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git cvsserver serious issue
On Wed, Sep 22, 2010 at 16:23, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Any user can point their cvs client at the repository. And check out an arbitrary branch, tag *or individual commit*. Doing so will create a 50Mb sqlite database on the server with cache information about that head. That basically means that git-cvsserver is completely useless in a public scenario as it stands. An easier way to DOS our server is hard to find, really. Ugh. Indeed. Now, if we can limit this by IP address, that would be ok. I assume we can do this for the NLS stuff - peter? As for buildfarm members needing CVS - is it workable to require that the maintainers of these set up their own git clone with git cvsserver (over ssh or pserver) and restrict it locally to the IP(s) of their machines? If we're going to let people in by IP address, maybe we could let legacy buildfarm members in by IP address. It doesn't seem particularly helpful to expect each buildfarm owner to solve this problem for themselves. I'd also note that if they could run git locally, they wouldn't be needing cvsserver in the first place. We could. It's currently on a freebsd vm though and I don't think we can set per-server IP filters on those. (I was thinking iptables). We could move it though - it doesn't *have* to be on the anonymous git VM. It's just some extra resources. Well, the use-case I was thinking of was Stefan. While he can't run git on each and every animal, he certainly has *some* machine(s) on the correct side of whatever firewall there may be that can run git. Also, couldn't we just set up the cvsserver on its own VM with a limited amount of disk space, and not worry too much about any DOS threat? If somebody does do this, block them and reinitialize that server. We could do that, but that could end up fighting a losing battle in case some bot hits it. I don't like deploying something with a known issue on it, sandboxed or not. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] knngist patch preliminary review (2010-09 commitfest)
Andrew Gierth and...@tao11.riddles.org.uk writes: My opinion is that the gist-implementation section of the docs needs to be substantially expanded so that it explains, for each support function, exactly what the function is expected to do. Yes, the GIST (and GIN) parts of the docs are really pretty bad. If it would help, I'm prepared to put some time towards actually writing something up for this. That would be outstanding. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git cvsserver serious issue
On 09/22/2010 10:23 AM, Tom Lane wrote: If we're going to let people in by IP address, maybe we could let legacy buildfarm members in by IP address. It doesn't seem particularly helpful to expect each buildfarm owner to solve this problem for themselves. I'd also note that if they could run git locally, they wouldn't be needing cvsserver in the first place. Also, couldn't we just set up the cvsserver on its own VM with a limited amount of disk space, and not worry too much about any DOS threat? If somebody does do this, block them and reinitialize that server. I'm not convinced we need any such thing yet. 13 of the 38 animals that have reported in the last 5 days are using git already (OK, factoring out my animals that's 8 out of 33). I'm going to send out email in a few days prodding people to migrate. Let's see how far we get. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Needs Suggestion
sub...@cse.iitb.ac.in writes: How can I increase the stack depth limit ? Is it only by modifying the postgres.conf file, but there I cannot increase the stack depth beyond 4 MB. Actually, my problem is that, when I set the stack base address of the child thread using the POSIX library function pthread_setstackaddr(), I am unable to access the memory contents of its parent. You've got worse problems than the stack depth limit. Creating multiple threads inside a backend process is entirely unsupported and just about guaranteed to result in massive breakage. Don't do it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQLSTATE of notice PGresult
Robert Haas robertmh...@gmail.com writes: On Wed, Sep 22, 2010 at 4:18 AM, Dmitriy Igrishin dmit...@gmail.com wrote: Okay, as Robert points, 0 code in successful messages seems as waste of bytes. But according to the documentation, All messages emitted by the PostgreSQL server are assigned five-character error codes that follow the SQL standard's conventions for SQLSTATE codes. - the first sentence of http://www.postgresql.org/docs/9.0/static/errcodes-appendix.html Sounds like that wording needs some adjustment. That wording is correct as it stands. If I recall the previous discussion here, the problem is that the OP is reading that and thinking that it applies also to errors generated internally by libpq. We should, but don't, have any support for assigning SQLSTATEs to those. But the server always emits a SQLSTATE when sending a notice or error message --- read send_message_to_frontend() if you doubt it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
Stefan Kaltenbrunner ste...@kaltenbrunner.cc writes: Magnus Hagander wrote: I think this should be working now. I made this change to mk-snapshot, and a similar one to mk-stable_snapshot. I have run the snapshot generation for master, and the one for 8.4 should finish in a minute or so. I have disabled the cronjobs until someone else has verified the tarballs. Do you still need someone to do that, and what do you want done exactly? When I do, I'll also enable generation of snapshots of REL9_0_STABLE. We currently do back to 8.2 - should we keep that, or drop 8.2? hmm I would say we should do all supported branches because those are the ones that people might be interested in when looking for a fix on a branch... FWIW, I think back to 8.2 is sufficient. The branches earlier than that are going to be unsupported in a matter of weeks anyway. And if no one has complained about the lack of nightly tarballs for them to date, it's unlikely that they'll start complaining now. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Opening a plpgsql cursor parameter by name
Hello list,
We intend to implement $subject, so instead of
mycursor CURSOR (myparm text) IS SELECT myparm;
OPEN mycursor('A');
it would be possible to do
OPEN mycursor(myparm := 'A');
The idea is to
* in pl_exec.exec_stmt_forc, detect if a positional parameter or named
parameter is used.
* if named, use plpgsql_ns_lookup to find the cursur arguments position
in the plpgsql name space.
* normal processing for the found position from here.
Mixing named and positional could probably be made to work as well.
Does this approach sound reasonable?
regards,
Yeb Havinga Willem Dijkstra
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On Wed, Sep 22, 2010 at 16:50, Tom Lane t...@sss.pgh.pa.us wrote: Stefan Kaltenbrunner ste...@kaltenbrunner.cc writes: Magnus Hagander wrote: I think this should be working now. I made this change to mk-snapshot, and a similar one to mk-stable_snapshot. I have run the snapshot generation for master, and the one for 8.4 should finish in a minute or so. I have disabled the cronjobs until someone else has verified the tarballs. Do you still need someone to do that, and what do you want done exactly? Just a second set of eyes that the output looks reasonable for being a snapshot generated now. When I do, I'll also enable generation of snapshots of REL9_0_STABLE. We currently do back to 8.2 - should we keep that, or drop 8.2? hmm I would say we should do all supported branches because those are the ones that people might be interested in when looking for a fix on a branch... FWIW, I think back to 8.2 is sufficient. The branches earlier than that are going to be unsupported in a matter of weeks anyway. And if no one has complained about the lack of nightly tarballs for them to date, it's unlikely that they'll start complaining now. Ok. 8.2 it is then - and I'll add 9.0 -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On Tue, 2010-09-21 at 17:04 -0700, Josh Berkus wrote: That said, the timeout option also feels a bit wishy-washy to me. With a timeout, acknowledgment of a commit means your transaction is safely committed in the master and slave. Or not, if there was some glitch with the slave. That doesn't seem like a very useful guarantee; if you're happy with that why not just use async replication? Ah, I wasn't clear. My thought was that a standby which exceeds the timeout would be marked as nonresponsive and no longer included in the list of standbys which needed to be synchronized. That is, the timeout would be a timeout which says this standby is down. So the only case where standby registration is required is where you deliberately choose to *not* have N+1 redundancy and then yet still require all N standbys to acknowledge. That is a suicidal config and nobody would sanely choose that. It's not a large or useful use case for standby reg. (But it does raise the question again of whether we need quorum commit). This is becoming very confusing. Some people advocating standby registration have claimed it allows capabilities which aren't possible any other way; all but one of those claims has so far been wrong - the remaining case is described above. If I'm the one that is wrong, please tell me where I erred. Thinking of this as a sysadmin, what I want is to have *one place* I can go an troubleshoot my standby setup. If I have 12 synch standbys and they're creating too much load on the master, and I want to change half of them to async, I don't want to have to ssh into 6 different machines to do so. If one standby needs to be taken out of the network because it's too slow, I want to be able to log in to the master and instantly identify which standby is lagging and remove it there. The above case is one where I can see your point and it does sound easier in that case. But I then think: What happens after failover?. We would then need to have 12 different standby.conf files, one on each standby that describes what the setup would look like if that standby became the master. And guess what, every time we made a change on the master, you'd need to re-edit all 12 standby.conf files to reflect the new configuration. So we're still back to having to edit in multiple places, ISTM. Please, please, somebody write down what the design proposal is *before* we make a decision on whether it is a sensible way to proceed. It would be good to see a few options written down and some objective analysis of which way is best to let people decide. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
Magnus Hagander mag...@hagander.net writes: On Wed, Sep 22, 2010 at 16:50, Tom Lane t...@sss.pgh.pa.us wrote: Do you still need someone to do that, and what do you want done exactly? Just a second set of eyes that the output looks reasonable for being a snapshot generated now. Sure, I can look. Where are these tarballs anyway? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] knngist patch preliminary review (2010-09 commitfest)
On Wed, Sep 22, 2010 at 10:30 AM, Tom Lane t...@sss.pgh.pa.us wrote: If it would help, I'm prepared to put some time towards actually writing something up for this. That would be outstanding. +1. Or really, plus several. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git cvsserver serious issue
Andrew Dunstan and...@dunslane.net writes: On 09/22/2010 10:23 AM, Tom Lane wrote: If we're going to let people in by IP address, maybe we could let legacy buildfarm members in by IP address. It doesn't seem particularly helpful to expect each buildfarm owner to solve this problem for themselves. I'd also note that if they could run git locally, they wouldn't be needing cvsserver in the first place. I'm not convinced we need any such thing yet. 13 of the 38 animals that have reported in the last 5 days are using git already (OK, factoring out my animals that's 8 out of 33). I'm going to send out email in a few days prodding people to migrate. Let's see how far we get. Even if we get 100% compliance on the buildfarm side, Peter's already stated that moving the NLS support over to git is going to take more time than we have available right now. We need a cvsserver for awhile yet. We can't just suddenly announce CVS service is terminated as of yesterday and expect that that's not going to have any serious consequences. Is there anything we could do to patch the problem out of git-cvsserver? Maybe hack it to only accept requests for the active branch tips? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] moving development branch activity to new git repo
Elvis Pranskevichus e...@prans.net wrote: Instead of filtering non-merge commits I would suggest doing git rebase on the latest revision of the old git repo. That way you will get a set of commits that should apply cleanly. The reason it is failing now is that it is impossible for git am to do a 3-way merge without the original git objects, which are obviously not available in the new repo. Well, that didn't work much better. It applied, but it started in June, and the big file, which has been in constant flux since January suddenly springs into existence in July. :-( The last few commits look right, but it gets pretty trashy pretty quickly before that. What *did* work is to take a fresh of the new repo, branch it as of the point in time that I created my original branch, and take the first mbox entry of the 167, which starts like this: From 07ea78aaafe22cbbb0ffdedbfcf78404abbdbc70 Mon Sep 17 00:00:00 2001 From: Kevin Grittner kevin.gritt...@wicourts.gov Date: Fri, 8 Jan 2010 15:39:12 -0600 and change the first line to point to the point at which this was applied: From 369494e41fe408f103884032f477555ba134a0a8 Fri Jan 8 09:06:05 2010 That applies fine. It's an encouraging start, but I'm not clear on exactly what I have to do to get the rest of these commits merged in with commits from the master branch cleanly. Some fix-up work is OK with me. Do I need to look at the old log and manually interleave merges to the branch and manual commits in the original pattern to make such a scheme work? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Opening a plpgsql cursor parameter by name
Yeb Havinga yebhavi...@gmail.com writes:
We intend to implement $subject, so instead of
mycursor CURSOR (myparm text) IS SELECT myparm;
OPEN mycursor('A');
it would be possible to do
OPEN mycursor(myparm := 'A');
Is this really worth the trouble? Is it supported by any other DBMS?
Are cursors used so much, or with so many parameters, that there's a
real benefit to be gained? (I tend to think that FOR loops are better
than cursors 99% of the time.)
I wouldn't be so obstructionist if this syntax weren't in flux.
But seeing that we have hopes of migrating from := to = before
very long, adding another dependency on that choice where it's
not buying a lot of functionality doesn't seem like a good idea.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Standby registration
On Wed, Sep 22, 2010 at 10:19 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: No. The point of naming them is to uniquely identify them. Hmm, that situation can arise if there's a network glitch which leads the standby to disconnect, but the master still considers the connection as alive. When the standby reconnects, the master will see two simultaneous connections from the same standby. In that scenario, you clearly want to disconnect the old connetion in favor of the new one. +1 for making that the behavior. Is there a scenario where you'd want to keep the old connection instead and refuse the new one? I doubt it. Perhaps that should be made configurable, so that you wouldn't need to list all standbys in the config file if you don't want to. Then you don't get any of the benefits of standby registration, though. I think it's fine to have async slaves that don't want any special features (like sync rep, or tracking how far behind they are in the xlog stream) not mentioned in the config file. But allowing multiple slaves with the same name seems like complexity without any attendant benefit. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Documentation, window functions
On Wed, Sep 22, 2010 at 6:03 AM, Dennis Björklund d...@zigo.dhs.org wrote: In http://www.postgresql.org/docs/9.0/static/tutorial-window.html it say Although avg will produce the same result no matter what order it processes the partition's rows in, this is not true of all window functions. When needed, you can control that order using ORDER BY within OVER. While it's true that avg() produce the same result no matter what order. A ORDER BY clause will affect what rows are included in the computation and thus change the result (the default window frame is RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW). So one can not in general add an ORDER BY to the example in the tutorial and get the same result as without an ORDER BY. Maybe we can find some better wording of the above? Yeah, that doesn't seem right. rhaas=# create table foo (a integer); CREATE TABLE rhaas=# insert into foo values (1); INSERT 0 1 rhaas=# insert into foo values (2); INSERT 0 1 rhaas=# insert into foo values (3); INSERT 0 1 rhaas=# select a, avg(a) over () from foo; a |avg ---+ 1 | 2. 2 | 2. 3 | 2. (3 rows) rhaas=# select a, avg(a) over (order by a) from foo; a | avg ---+ 1 | 1. 2 | 1.5000 3 | 2. (3 rows) But I confess that I'm sort of murky on how ORDER affects the window frame, or how to rephrase this more sensibly. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Any reason why the default_with_oids GUC is still there?
Peter Eisentraut wrote: On tis, 2010-09-21 at 18:31 -0400, Bruce Momjian wrote: Also, doesn't some SQL standard require oids, so we should have a way to enable them by default for all tables? From some DB2 example: CREATE TYPE BusinessUnit_t AS (Name VARCHAR(20), Headcount INT); CREATE TABLE BusinessUnit OF BusinessUnit_t (REF IS oid USER GENERATED); The DB2 documentation consistently refers to this column as oid, but there is no requirement to name it that way. The SQL standard also contains this sentence: Let OID be the name of the self-referencing column of S. which refers to the thing defined in the example above, but OID is just a placeholder here. I think there was a mention of OIDs in the SQL3 draft that eventually became SQL99, but that's long past now. Current standards don't have it, except in the, perhaps more generalized, form above. Thanks for those details. I did remember it appearing at one point, which I guess was SQL3. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git cvsserver serious issue
On ons, 2010-09-22 at 16:03 +0200, Magnus Hagander wrote: That basically means that git-cvsserver is completely useless in a public scenario as it stands. An easier way to DOS our server is hard to find, really. Now, if we can limit this by IP address, that would be ok. I assume we can do this for the NLS stuff - peter? Well, let's see. If someone can figure out the git equivalent of if cvs -q update | egrep -q '^(U|P) '; then # ... something changed, so run the update ... fi (assuming, for simplicity, that the current directory has the appropriate branch checked out already) then I might be able to get this fixed. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
Robert Haas wrote: [server] guc=value or server.guc=value Yes, this was my idea too. It uses our existing config file format. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multi-branch committing in git, revisited
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: There is no need for 'git add' because once you are done you can use git commmit -a in each branch to add all modifications and commit them. git commit -a is not a universal solution. In particular, the patch I was dealing with yesterday involved additions and removals of files, neither of which will be implemented by git commit -a. Uh, why is that? I see -a saying: -a, --all Tell the command to automatically stage files that have been modified and deleted, but new files you have not told git about are not affected. What is it about add/deletes that it doesn't do? Is the problem 'git add' creates a stage already? How is that a problem? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git cvsserver serious issue
On Wed, Sep 22, 2010 at 12:21 PM, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-09-22 at 16:03 +0200, Magnus Hagander wrote: That basically means that git-cvsserver is completely useless in a public scenario as it stands. An easier way to DOS our server is hard to find, really. Now, if we can limit this by IP address, that would be ok. I assume we can do this for the NLS stuff - peter? Well, let's see. If someone can figure out the git equivalent of if cvs -q update | egrep -q '^(U|P) '; then # ... something changed, so run the update ... fi (assuming, for simplicity, that the current directory has the appropriate branch checked out already) then I might be able to get this fixed. Can you just check whether the commit SHA of HEAD has changed? e.g. git show-ref --heads -s master git log --format=format:%H -n 1 master ...and compare with previous results of same? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On 22 September 2010 17:23, Bruce Momjian br...@momjian.us wrote:
Robert Haas wrote:
[server]
guc=value
or
server.guc=value
Yes, this was my idea too. It uses our existing config file format.
So...
sync_rep_services = {critical: recv=2, fsync=2, replay=1;
important: fsync=3;
reporting: recv=2, apply=1}
becomes ...
sync_rep_services.critical.recv = 2
sync_rep_services.critical.fsync = 2
sync_rep_services.critical.replay = 2
sync_rep_services.important.fsync = 3
sync_rep_services.reporting.recv = 2
sync_rep_services.reporting.apply = 1
I actually started to give this example to demonstrate how cumbersome
it would look... but now that I've just typed it out, I've changed my
mind. I actually like it!
--
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git cvsserver serious issue
Excerpts from Peter Eisentraut's message of mié sep 22 12:21:45 -0400 2010: Well, let's see. If someone can figure out the git equivalent of if cvs -q update | egrep -q '^(U|P) '; then # ... something changed, so run the update ... fi (assuming, for simplicity, that the current directory has the appropriate branch checked out already) then I might be able to get this fixed. Would it work to save the previous commit hash in a file and compare to the current one? -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git cvsserver serious issue
At 2010-09-22 19:21:45 +0300, pete...@gmx.net wrote: Well, let's see. If someone can figure out the git equivalent of if cvs -q update | egrep -q '^(U|P) '; then # ... something changed, so run the update ... fi I think you want: git pull if [ $(git rev-parse HEAD) != $(git rev-parse ORIG_HEAD) ]; then # ... the pull changed something ... fi -- ams -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
Thom Brown wrote:
On 22 September 2010 17:23, Bruce Momjian br...@momjian.us wrote:
Robert Haas wrote:
[server]
guc=value
or
server.guc=value
?
Yes, this was my idea too. ?It uses our existing config file format.
So...
sync_rep_services = {critical: recv=2, fsync=2, replay=1;
important: fsync=3;
reporting: recv=2, apply=1}
becomes ...
sync_rep_services.critical.recv = 2
sync_rep_services.critical.fsync = 2
sync_rep_services.critical.replay = 2
sync_rep_services.important.fsync = 3
sync_rep_services.reporting.recv = 2
sync_rep_services.reporting.apply = 1
I actually started to give this example to demonstrate how cumbersome
it would look... but now that I've just typed it out, I've changed my
mind. I actually like it!
It can be prone to mistyping, but it seems simple enough. We already
through a nice error for mistypes in the sever logs. :-)
I don't think we support 3rd level specifications, but we could. Looks
very Java-ish.
--
Bruce Momjian br...@momjian.ushttp://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multi-branch committing in git, revisited
On Wed, Sep 22, 2010 at 12:40 PM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Bruce Momjian br...@momjian.us writes: There is no need for 'git add' because once you are done you can use git commmit -a in each branch to add all modifications and commit them. git commit -a is not a universal solution. In particular, the patch I was dealing with yesterday involved additions and removals of files, neither of which will be implemented by git commit -a. Uh, why is that? I see -a saying: -a, --all Tell the command to automatically stage files that have been modified and deleted, but new files you have not told git about are not affected. What is it about add/deletes that it doesn't do? Is the problem 'git add' creates a stage already? How is that a problem? Tom is slightly incorrect. Deletions work fine with git commit -a. git already knows about the files, so everything just works. However, it won't pick up on added files, because it can't distinguish between a file that you want added to the repository and a stray file you left lying around and assumes the latter. But I don't see that this takes anything away from your point. You can certainly just work on the patch in each repository separately and then commit everything all at once at the end, if you're so inclined. Of course, as Tom points out, it's a lot nicer to apply patches in a way that allows git to try to auto-merge for you. Sometimes it works, and when it doesn't work having the merge conflict stuff in the file is still better than having a .rej hunk leftover that you have to figure out what to do with. So personally I don't intend to do it that way, but as Larry Wall said about Perl, There's More Than One Way To Do It. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] snapshot generation broken
On Wed, Sep 22, 2010 at 17:08, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Wed, Sep 22, 2010 at 16:50, Tom Lane t...@sss.pgh.pa.us wrote: Do you still need someone to do that, and what do you want done exactly? Just a second set of eyes that the output looks reasonable for being a snapshot generated now. Sure, I can look. Where are these tarballs anyway? They're up on the ftp site - ftp://ftp.postgresql.org/pub/snapshot/. dev and stable/8.4 are updated. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuring synchronous replication
On Wed, 2010-09-22 at 17:43 +0100, Thom Brown wrote:
So...
sync_rep_services = {critical: recv=2, fsync=2, replay=1;
important: fsync=3;
reporting: recv=2, apply=1}
becomes ...
sync_rep_services.critical.recv = 2
sync_rep_services.critical.fsync = 2
sync_rep_services.critical.replay = 2
sync_rep_services.important.fsync = 3
sync_rep_services.reporting.recv = 2
sync_rep_services.reporting.apply = 1
I actually started to give this example to demonstrate how cumbersome
it would look... but now that I've just typed it out, I've changed my
mind. I actually like it!
With respect, this is ugly. Very ugly. Why do we insist on cryptic
parameters within a config file which should be set within the database
by a super user.
I mean really?
ALTER CLUSTER ENABLE [SYNC] REPLICATION ON db.foobar.com PORT 5432 ALIAS
CRITICAL;
ALTER CLUSTER SET REPLICATION CRITICAL RECEIVE FOR 2;
ALTER CLUSTER SET REPLICATION CRITICAL FSYNC FOR 2;
ALTER CLUSTER SET REPLICATION CRITICAL REPLAY FOR 2;
Or some such thing. I saw Heiiki's reply but really the idea that we are
shoving this all into the postgresql.conf is cumbersome.
Sincerely,
Joshua D. Drake
--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579
Consulting, Training, Support, Custom Development, Engineering
http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
