Re: [HACKERS] Assisting developers
On Jul 13, 2004, at 17:02, Bruce Momjian wrote: One failing that has appeared during the 7.5 development cycle is that we as a community haven't been able to provide timely feedback to developers working on large feature additions. I am particularly thinking of Alvaro (nested transactions) and Simon (PITR), where we haven't been able to give them sufficient feedback to make them fully productive. I'm just a bystander here, but it seems to me that in-depth discussion of a feature only starts when someone realizes that he must speak now or the darn thing might get committed. In other words, the emphasis is placed in preventing something half-baked getting included. And that's perfectly natural because it is much easier and quicker than commenting thoughtfully on every idea that someone might come up with. But it of course means that the price of admission is a patch that poses a real risk of getting committed. From a pure resource utilization perspective, I don't see a way around this. There's not enough expertise of pgsql internals to go around. As long as that's the case, there will always be a barrier to entry. But a high-risk patch isn't the only thing that can get you over such a barrier; the only thing to control the distribution of this scarce resource. Cash comes to mind as an alternative. mk ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Converting postgresql.conf parameters to kilobytes
Frank Wiles wrote: shared_buffers = 1 ( shared_buffers in pages ) shared_buffers = 100M ( 100 MBs of shared_buffers ) shared_buffers = 2048K ( 2MBs of shared_buffers ) I don't know if this is pedantic or just obsessive-compulsive, but I think it should be MB and KB (or more properly, kB) instead of just M and K, to distinguish from kilopages or megapages. mk ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [COMMITTERS] attempt at a multi file commit, to seee how it
Marc G. Fournier wrote: Not sure if I like the URLs, myself ... opinions? You could try cvsspam, which is what I use in-house. Neil posted the URL. It does HTML mail, so it'd probably belong on a dedicated list. It's a huge help with the links, though, see this sample: http://www.badgers-in-foil.co.uk/projects/cvsspam/example.html The good thing is that it has a line limit (default 1000 lines), and if a patch is under that, the complete diff is shown. Despite PostgreSQL being an unixy open source project, I'm sure that there are a ton of people on the lists with HTML emailers, and for us this would be a very welcome alternative :-) mk ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Call for 7.5 feature completion
Peter Eisentraut wrote: How about this one: Everything we have moved from the core to gborg so far has been a miserable failure. The code is no longer maintained, or maintained by three different competing groups, the documentation has disappeared, the portability is no longer taken care of, and only the bravest souls even dare look at the stuff. I think before you move anything more, you need to have a strong, convincing community on the receiving side rather than just kicking things out and hoping someone will pick it up. Just because it can be built separately doesn't mean everything needs to be. I guess the key thing is that pgFoundry shouldn't be a community distinct from the core. The same community standards need to apply on both sides of the fence. What has happened here echoes the experiences of the PHP project very closely. PHP, too, thought that the core was getting too big for efficient release coordination, and started moving extensions to PECL, an extension library bolted on the side of PEAR. Trouble is, nobody wants to be forced into the ghetto. The only way to make pgFoundry work is to minimize the downside of living there. Consider these: - Don't move just inessential projects. Move absolutely essential projects to push end-user and developer adoption. - Have a tier of official projects that are actively endorsed by and within the core distribution. Don't be afraid to pick a favorite solution within a class (for example in replication). - Discourage other developers from ignoring pgFoundry projects. For the official tier, this could mean sending commit messages to pgsql-committers, patches to pgsql-patches, and having the discussions here. Resist the urge to start project-specific mailing lists unless absolutely essential. Have everyone know that the pgFoundry/core distinction only exists for release engineering purposes, and that it can't be allowed to split the community. - Invest in integrating pgFoundry tools into the core distribution. For example, official projects should have makefiles included in the core that'd download and compile the latest compatible version. Components as important as JDBC and some of the procedural languages could be distributed this way. mk ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Rough draft for Unicode-aware UPPER()/LOWER()/INITCAP()
Tatsuo Ishii wrote: initdb could even emit a warning if the --encoding option was used without also specifying --no-locale. Please don't do that. Most Asian chasets does not work with locale enabled PostgreSQL installation. i.e. it returns WRONG SELECT results. I've been telling this to Japanese users for hundreds of times when they ask me why SELECT returns results. If that kind of wanings are installed, I think more Japanese users will be confused. You've advocated a default of --no-locale yourself for this reason. If using a Japanese --encoding setting without --no-locale emitted a warning suggesting the use of --no-locale, I'd imagine you wouldn't have had to give human support to most of those hundreds of people? Wouldn't that be a halfway point to your goal? mk ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] enabling tcpip_socket by default
On 17. touko 2004, at 10:40, Tatsuo Ishii wrote: Consider a program using JDBC on localhost. It can only reach to PostgreSQL via TCP/IP. Ah! Of course. That makes sense, and listening on 127.0.0.1 never hurt anyone (except, of course, the tinfoil hat crowd nmapping localhost in a frenzy...) mk ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] enabling tcpip_socket by default
Tatsuo Ishii wrote: Is there any security risk if we enable tcpip_socket by default? We restrict connection from localhost only by default so I think enabling tcpip_socket adds no security risk. Please correct me if I am wrong. Bruce Momjian wrote: Right, and 7.5 will ship with tcp and localhost enabled. If the default will be to listen on all interfaces, not just 127.0.0.1, then this IS a security risk. And if that's not the plan, what good does this change do? Any real use of tcp would still require a configuration change anyway. Listening on public network interfaces by default would multiply by orders of magnitude the number of machines vulnerable to potential future remote exploits. I gather that the pre-authentication code paths are pretty well known, and that the chances of such an attack are slim. Nevertheless I cannot help but note that it is exactly this default setting that caused Microsoft SQL Server to lose a big, big chunk of its reputation, and gain notoriety as a launchpad for Windows worms. mk ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Rough draft for Unicode-aware UPPER()/LOWER()/INITCAP()
Tom Lane wrote: This code will only work if the database is running under an LC_CTYPE setting that implies the same encoding specified by server_encoding. However, I don't see that as a fatal objection, because in point of fact the existing upper/lower code assumes the same thing. I think this interaction between the locale and server_encoding is confusing. Is there any use case for running an incompatible mix? If not, would it not make sense to fetch initdb's default database encoding with nl_langinfo(CODESET) instead of using SQL_ASCII? initdb could even emit a warning if the --encoding option was used without also specifying --no-locale. Using nl_langinfo(CODESET) was discussed and quietly dismissed a year ago (although the topic was the client encoding back then). But I think that the idea is worth revisiting because it would allow UPPER() and LOWER() to work correctly with international alphabets -- out of the box and without configuration -- on a wide variety of modern systems. mk ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Rough draft for Unicode-aware UPPER()/LOWER()/INITCAP()
Marko Karppinen wrote: I think this interaction between the locale and server_encoding is confusing. Is there any use case for running an incompatible mix? If not, would it not make sense to fetch initdb's default database encoding with nl_langinfo(CODESET) instead of using SQL_ASCII? Peter Eisentraut wrote: This would be fine and dandy if we had any sort of idea about what sort of strings nl_langinfo(CODESET) returns and how to map them to our encoding names. Karel Zak posted an answer to this last year, here on pgsql-hackers: http://archives.postgresql.org/pgsql-hackers/2003-05/msg00744.php It's not complete, but it's sort of an idea. The code is under LGPL, but copyright doesn't reach down to the actual information about the encoding strings used by various operating systems, so it's possible to reappropriate. I'd imagine that it covers many, if not most, of the likely cases. The current situation of upper/lower/collating/etc just being broken by default on many non-C locales is bad enough to warrant bailing out during initdb when this situation is detected (with a reasonably cautious heuristic). It used to be that you got what you deserved if you were stupid enough to define a non-C, non-ASCII-based locale. You had only yourself to blame for everything breaking. These days, however, millions of systems get shipped and installed with UTF-8 locales on by default, so it's not possible to portray this as an user error. Requiring every one of these people to configure initdb's encoding manually would be harsh, however, so I think that an heuristic that'd work with most modern systems would strike an appropriate balance of correctness and path-of-least-surprise. mk ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Tablespaces
On 3 March 2004, at 19:52, Bruce Momjian wrote: The advantage of symlinks is that an administrator could see how things are laid out from the command line. One thing to keep in mind is that system administrators don't see symlinks as being informational -- they see them as the actual UI for the redirection in question. So their expectation is that they'll be able to move the actual directory around at will (as long as they update the symlink to match). If symlinks are used, the rule of least surprise would mean that no information whatsoever about the physical location of a tablespace should be stored in the system catalogs. Otherwise their relationship with the information stored in the symlink is ambiguous. mk ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] About the partial tarballs
On 11 Nov 2003, at 20:44, Peter Eisentraut wrote: Do we have any data on how many people download the partial tarballs (-base, -opt, etc.)? I have a feeling that more people are confused by them than use them. Even if they weren't useful for anything else, I think there's value in the developers having to consider what is optional and what is not. This need for constant review probably reduces the chance of bloat, over time even in the full tarball. I agree that the partial tarballs can confuse an ftp user, though. I think a good solution to this would be to put them one level deeper, into a subfolder. The full tarball would then be the only thing a casual user would encounter, but the source-based port systems could still benefit from the partial tarballs. mk ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Making PostgreSQL 7.4 (CVS) work properly on OS X 10.3 (7B85)
On 8 Nov 2003, at 22:19, Tom Lane wrote: After fixing the ps_status problems, I cannot observe any problem, with or without system.c. However, I agree that it's a bad idea to propagate that hack forward when it's no longer needed. I've #ifdef'd out system.c for OS X 10.3 and later. (I was tempted to disable it on 10.2.* as well, but I can only prove system() works in 10.2.6, not having older OS X handy to test.) As a general rule of thumb, I don't think it's necessary to support anything other than the latest revision in 10.2.x and 10.3.x at any time. The revisions are free, they mainly fix bugs, they don't affect hardware requirements and other software vendors see requiring them as a non-issue. This, at the very least, applies to major pgsql versions such as 7.4 and 7.5 -- you might disagree about a OS version requirement changing in a point release. But in my opinion, if system() is fixed in 10.2.6, you can require Jaguar users to upgrade to that version for 7.4. mk ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Call for port reports
On Oct 24, 2003, at 18:37, Bruce Momjian wrote: It is time for people to report their port testing. Please test against current CVS or beta5 and report your 'uname -a'. This is with beta 5. Darwin marko.karppinen.fi 7.0.0 Darwin Kernel Version 7.0.0: Wed Sep 24 15:48:39 PDT 2003; root:xnu/xnu-517.obj~1/RELEASE_PPC Power Macintosh powerpc ProductName:Mac OS X ProductVersion: 10.3 BuildVersion: 7B85 6 out of 93 tests failed: date ... FAILED timetz ... FAILED inet ... FAILED comments ... FAILED test horology ... FAILED test misc ... FAILED The regression.diffs file is at: http://www.karppinen.fi/pgsql74b4-darwin7-regression.diffs mk ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Call for port reports
Looking a bit further into this, it looks like random tests are failing. Seems like an issue with the test harness on this platform. Does someone want a shell account to debug? mk On Oct 24, 2003, at 21:39, Marko Karppinen wrote: 6 out of 93 tests failed: date ... FAILED timetz ... FAILED inet ... FAILED comments ... FAILED test horology ... FAILED test misc ... FAILED ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Call for port reports
I'm just being an idiot, it's obviously a limits problem on the platform. It has a default max user processes limit of 100, which I was hitting. I shut down a bunch of desktop apps, and it's now passing: 92 of 93 tests passed, 1 failed test(s) ignored. (random was the one failing). So I guess you can add it to the list. mk ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PERFORM] Sun performance - Major discovery!
On 14.10.2003, at 19:52, Tom Lane wrote: This means that relaxing the check would require (a) finding out which of the sub-flags break our code and which don't; (b) finding out how the answer to (a) has varied with gcc release; and (c) finding out how we can test whether a given sub-flag is set --- are there #defines for each of them in gcc 3? Okay, I can see how that makes this unpractical to implement. Thanks. The current error message is do not put -ffast-math in CFLAGS; does someone have an idea for a better text that doesn't imply that you actually /have/ --ffast-math in CFLAGS? It'd be good to acknowledge that it can be set implicitly, too. And on the same subject: On 14.10.2003, at 18:13, Peter Eisentraut wrote: That sounds perfectly reasonable to me. Why should we develop elaborate workarounds for compiler flags that are known to create broken code? I also want to point out that I'm getting kind of tired of developing more and more workarounds for sloppy Apple engineering. Peter, you are free to consider your current environment to be the peak of perfection, but that doesn't mean that the only reason for differences between your system and others' is the sloppiness of their engineering. I'm not aware of any Darwin-specific workarounds in the tree right now; the only thing close to that is the support for Apple's two-level namespaces feature. And while you can argue the relative merits of Apple's approach, the reason for its existence isn't sloppiness and the support for it that was implemented by Tom most certainly isn't a workaround. The fact of the matter is that Mac OS X has about ten million active users, and when one of these people is looking for an RDBMS, he's gonna go for one that compiles and works great on his system, rather worrying if his platform is optimal for running PostgreSQL. Supporting this platform well is absolutely crucial to the overall adoption of pg, and even if you consider yourself to be above such pedestrian concerns, many people who have to make the business case for putting money into PostgreSQL development most definitely think otherwise. mk ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Hacking PostgreSQL to work in Mac OS X 10.3 (Panther 7B85)
On 12.10.2003, at 03:52, James Wilson wrote: I've done a checkout from CVS and performed a build under OS X Panther 7B85 with the non-hacked header files. The problem with the param.h header file seems to be fixed, however the build is failing with undefined symbols in libs: I believe you have some old files lingering on your tree: [marko:~/src] mka% rm -rf pgsql [marko:~/src] mka% cvs -z3 -d :pserver:[EMAIL PROTECTED]:/projects/cvsroot co -P pgsql [marko:~/src] mka% cd pgsql [marko:~/src/pgsql] mka% autoconf [marko:~/src/pgsql] mka% ./configure . . . [marko:~/src/pgsql] mka% make . . . All of PostgreSQL successfully made. Ready to install. [marko:~/src/pgsql] mka% Either that, or your usage of mismatching OS X and dev tools versions is causing you problems. mk ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Hacking PostgreSQL to work in Mac OS X 10.3 (Panther 7B85)
James, we've spent some time lately with Mac OS X related changes to the code base. Please try this again with a fresh CVS copy of PostgreSQL and let us know if there are any further changes you think we'd need to work properly with Panther. Thanks, mk On 11.10.2003, at 04:32, James Wilson wrote: After toying with a few builds of the up coming OS X 10.3 (Panther) release, I've written a doc on how to get postgresql to build using Panther build 7B85 and Xcode build 7B85 (which are rumored to be the gold master and the builds that will be shipping later this month). ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] [GENERAL] Can't Build 7.3.4 on OS X
On 23.9.2003, at 19:01, Tom Lane wrote: 1. Doesn't this break backward compatibility with Darwin 1.0-1.2? (If so, does anyone care anymore?) No. The two-level namespace feature was added in 10.1, and the -flat_namespace hack was included in pgsql to unbreak the compile on 10.1 and later. This change hence doesn't have any effect on 10.0 / Darwin 1.x -- except for gcc warnings about unknown flags. The other change we discussed, ie. using -no-cpp-precomp instead of -traditional-cpp, will break Darwin 1.x support. 2. Why is the postgres executable added to LINK.shared? Doesn't this break building shlibs that are not intended to be linked into the backend (eg libpq)? When linking a shared object, the Darwin linker wants to know that all the symbols are accounted for -- it doesn't allow undefined symbols. The bundle_loader flag gives the linker a hint: these symbols will be provided by the application that loads us, so you don't need raise an error about them being undefined. In other words, the -bundle_loader /path/to/postgresql doesn't affect the build results at all. In the case where there are no undefined symbols (like in libpq), the flag is ignored. 3. What's with the WITHOUT_DARWIN_BUNDLE_LOADER hack for ecpg? The linker doesn't like symbols that are defined both in the shared object that's being linked and the bundle_loader binary. ecpg contains symbols that are in the postgresql binary, too, leading to the link failing with multiply-defined errors. The easiest way for me to fix this -- I don't really know my way around the source base -- was to specify by hand that the -bundle-loader flag isn't used on these occasions. Perhaps a more prudent way would be to examine which shared objects actually have undefined symbols and use the bundle-loader flag only when linking them, but I think it'd lead to much more maintenance and bloated Makefiles. cheers mk Marko Karppinen [EMAIL PROTECTED] writes: While we're on a Mac-related note, I managed to compile PostgreSQL on Mac OS X 10.2 Jaguar with two-level namespace support. http://developer.apple.com/documentation/ReleaseNotes/DeveloperTools/ TwoLevelNamespaces.html This is quite useful, so I hope you can look into incorporating the changes. http://www.markokarppinen.com/pg-two-level-namespace.diff ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] [GENERAL] Can't Build 7.3.4 on OS X
Tom: BTW, is anyone interested in looking into whether we can be made to build without using either flag? I tried it and saw a number of failures that looked like they traced to incompatible macro expansion. This wouldn't surprise me if PG were some halfbaked package that only got tested with stock GCC, but considering that we actually build fine on quite a range of compilers, it seems like Apple's cpp ought to work too. It would be interesting to understand what the problem is. Does this happen with gcc 3.3? My understanding is that pre-3.3, the default C preprocessor was cpp-precomp, a NeXT-era concoction that supports precompiled header files for Objective-C. This preprocessor is about as bad as it seems to be, so even Apple recommends turning it off for other languages. In gcc 3.3, Apple has generalized the precompiled headers feature to a mainline gcc feature called PFE, so cpp-precomp should be history. If I've gotten this right, you shouldn't get (the same) compiler errors with gcc 3.3 if you leave out the -no-cpp-precomp switch. . . . While we're on a Mac-related note, I managed to compile PostgreSQL on Mac OS X 10.2 Jaguar with two-level namespace support. http://developer.apple.com/documentation/ReleaseNotes/DeveloperTools/ TwoLevelNamespaces.html This is quite useful, so I hope you can look into incorporating the changes. http://www.markokarppinen.com/pg-two-level-namespace.diff This involves getting rid of undefined symbols in all shared libraries (or bundles) that get built. I tested this with ./configure and ./configure --with-tcl but didn't try the jdbc or python extensions. Hopefully someone else can test those, and if all seems fine, commit the changes. Thanks, Marko ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] [GENERAL] Can't Build 7.3.4 on OS X
On lauantai, syys 20, 2003, at 23:37 Europe/Helsinki, Tom Lane wrote: Is anyone on the list still running OS X 10.1, or anyway still using a version of the OS X developer tools older than the Dec 2002 release? It would be good to check if -no-cpp-precomp creates any problems on any release that anyone still cares about. For the moment, I've made src/template/darwin unconditionally use -no-cpp-precomp, but we could probably hack it to use -traditional-cpp if there's still any interest in Darwin compiler versions that don't recognize -no-cpp-precomp. -no-cpp-precomp replaced -traditional-cpp as the preferred syntax for turning off the precompiled header support in Mac OS X 10.1 (Darwin 5.0). Supporting Mac OS X 10.0 (Darwin 1.x) would require -traditional-cpp, but I don't think it's necessary. 10.1 was a free upgrade and it didn't remove any hardware support, so I really can't see a reason why somebody would continue to use it. mk ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
