[HACKERS] Re: beta testing version
Hi, how long is PG7.1 already in beta testing? can it be released before Christmas day? can PG7.1 will recover database from system crash? Thanks, XuYifeng
RE: [HACKERS] SQL 'in' vs join.
Now, given the two components, each with very low costs, it chooses to do a sequential scan on the table. I don't get it. Read the FAQ? http://www.postgresql.org/docs/faq-english.html#4.23 "4.23) Why are my subqueries using IN so slow?") - Andrew
Re: [HACKERS] beta testing version
Judging by the information below, taken *directly* from PostgreSQL, Inc. website, it appears that they will be releasing all code into the main source code branch -- with the exception of "Advanced Replication and Distributed Information capabilities" (to which capabilities they are referring is not made clear) which may remain proprietary for up to 24 months "in order to assist us in recovering development costs and continue to provide funding for our other Open Source contributions." I have interpreted this to mean that basic replication (server - server, server - client, possibly more) will be available shortly for Postgres (with the release of 7.1?) and that those more advanced features will follow behind. This is one of the last features that was missing from Postgres (along with recordset returning functions and clusters, among others) that was holding it back from the enterprise market -- and I do not blame PostgreSQL, Inc. one bit for withholding some of the more advanced features to recoup their development costs -- it was *their time* and *their money* they spent developing the *product* and it must be recoup'ed for projects like this to make sense in the future (who knows, maybe next they will implement RS returning SP's or clusters, projects that are funded with their profit off the advanced replication and distributed information capabilities that they *may* withhold -- would people still be whining then?) Michael Fork - CCNA - MCP - A+ Network Support - Toledo Internet Access - Toledo Ohio (http://www.pgsql.com/press/PR_5.html) "At the moment we are limiting our test groups to our existing Platinum Partners and those clients whose requirements include these features." advises Jeff MacDonald, VP of Support Services. "We expect to have the source code tested and ready to contribute to the open source community before the middle of October. Until that time we are considering requests from a number of development companies and venture capital groups to join us in this process." Davidson explains, "These initial Replication functions are important to almost every commercial user of PostgreSQL. While we've fully funded all of this development ourselves, we will be immediately donating these capabilities to the open source PostgreSQL Global Development Project as part of our ongoing commitment to the PostgreSQL community." http://www.erserver.com/ eRServer development is currently concentrating on core, universal functions that will enable individuals and IT professionals to implement PostgreSQL ORDBMS solutions for mission critical datawarehousing, datamining, and eCommerce requirements. These initial developments will be published under the PostgreSQL Open Source license, and made available through our sites, Certified Platinum Partners, and others in PostgreSQL community. Advanced Replication and Distributed Information capabilities are also under development to meet specific business and competitive requirements for both PostgreSQL, Inc. and clients. Several of these enhanced PostgreSQL, Inc. developments may remain proprietary for up to 24 months, with availability limited to clients and partners, in order to assist us in recovering development costs and continue to provide funding for our other Open Source contributions. On Sun, 3 Dec 2000, Hannu Krosing wrote: The Hermit Hacker wrote: IIRC, this thread woke up on someone complaining about PostgreSQl inc promising to release some code for replication in mid-october and asking for confirmation that this is just a schedule slip and that the project is still going on and going to be released as open source. What seems to be the answer is: "NO, we will keep the replication code proprietary". I have not seen this answer myself, but i've got this impression from the contents of the whole discussion. Do you know if this is the case ? --- Hannu
Re: [HACKERS] beta testing version
In fact, it might seem to be common courtesy... An odd choice of words coming from you Don. We are offering our services and expertise to a community outside -hackers, as a business formed in a way that this new community expects to see. Nothing special or sinister here. Other than it seems to have raised the point that you expected each of us to be working for you, gratis, on projects you find compelling, using all of our available time, far into the future just as each of us has over the last five years. After your recent spewing, it irks me a little to admit that this will not change, and that we are likely to continue to each work on OS PostgreSQL projects using all of our available time, just as we have in the past. A recent example of non-sinister change in another area is the work done to release 7.0.3. This is a release which would not have happened in previous cycles, since we are so close to beta on 7.1. But GB paid Tom Lane to work on it as part of *their* business plan, and he sheparded it through the cycle. There was no outcry from you at this presumption, and on this diversion of community resources for this effort. Not sure why, other than you chose to pick some other fight. And no matter which fight you chose, you're wasting the time of others as you fight your demons. - Thomas
Re: AW: [HACKERS] broken locale in 7.0.2 without multibyte support (F reeBSD 4.1-RELEASE) ?
Zeugswetter Andreas SB [EMAIL PROTECTED] writes: Today I inserted (unsigned char) casts into all the ctype.h function calls I could find. This issue should be fixed as of current cvs. Please try it again when you have time. I am a sceptic to the many casts. Would'nt the clean solution be, to use unsigned char througout the code ? No; see the prior discussion. The casts only help to avoid compiler warnings or errors. They do not solve the underlying problem. You are mistaken. regards, tom lane
Re: [HACKERS] Using Threads?
I maybe wrong but I think that PGSQL is not threaded mostly due to historical reasons. It looks to me like the source has developed over time where much of the source is not reentrant with many global variables throughout. In addition, the parser is generated by flex which can be made to generate reentrant code but is still not thread safe b/c global variables are used. That being said, I experimented with the 7.0.2 source and came up with a multithreaded backend for PGSQL which uses Solaris Threads. It seems to work, but I drifted very far from the original source. I had to hack flex to generate threadsafe code as well. I use it as a linked library with my own fe-be protocol. This ended up being much much more than I bargained for and looking back would probably not have tried had I known any better. Myron Scott On Mon, 27 Nov 2000, Junfeng Zhang wrote: Hello all, I am new to postgreSQL. When I read the documents, I find out the Postmaster daemon actual spawns a new backend server process to serve a new client request. Why not use threads instead? Is that just for a historical reason, or some performance/implementation concern? Thank you very much. Junfeng
Re: [HACKERS] beta testing version
On Sun, 3 Dec 2000, Don Baccus wrote: At 11:59 PM 12/3/00 -0400, The Hermit Hacker wrote: the sanctity of the *core* server is *always* foremost in our minds, no matter what other projects we are working on ... What happens if financially things aren't entirely rosy with your company? The problem in taking itty-bitty steps in this direction is that you're involving outside money interests that don't necessarily adhere to this view. Having taken the first steps to a proprietary, closed source future, would you pledge to bankrupt your company rather than accept a large captital investment with an ROI based on proprietary extensions to the core that might not be likely to come out of the non-tainted side of the development house? You mean sort of like Great Bridge investing in core developers? Quite frankly, I have yet to see anything but good come out of Tom as a result of that, as now he has more time on his hands ... then again, maybe Outer Joins was a bad idea? *raised eyebrow* PgSQL is *open source* ... that means that if you don't like it, take the code, fork off your own version if you don't like what's happening to the current tree and build your own community *shrug*
[HACKERS] Bitmap index
Hi, on other RDBMS (Oracle,etc...),there is an index called bitmap index that greatly improve performance compared to btree index for boolean value (such as for a sex value,it's either M or F),i would like to know if such index will be implemented inside PostgreSQL.
Re: [HACKERS] Using Threads?
On Mon, 27 Nov 2000, Junfeng Zhang wrote: Hello all, I am new to postgreSQL. When I read the documents, I find out the Postmaster daemon actual spawns a new backend server process to serve a new client request. Why not use threads instead? Is that just for a historical reason, or some performance/implementation concern? Several reasons, 'historical' probably being the strongest right now ... since PostgreSQL was never designed for threading, its about as 'un-thread-safe' as they come, and cleaning that up will/would be a complete nightmare (should eventually be done, mind you) ... The other is stability ... right now, if one backend drops away, for whatever reason, it doesn't take down the whole system ... if you ran things as one process, and that one process died, you just lost your whole system ...
Re: [HACKERS] beta testing version
On Mon, 4 Dec 2000, Don Baccus wrote: A recent example of non-sinister change in another area is the work done to release 7.0.3. This is a release which would not have happened in previous cycles, since we are so close to beta on 7.1. But GB paid Tom Lane to work on it as part of *their* business plan, and he sheparded it through the cycle. There was no outcry from you at this presumption, and on this diversion of community resources for this effort. Not sure why, other than you chose to pick some other fight. There's a vast difference between releasing 7.0.3 in open source form TODAY and eRServer, which may not be released in open source form for up to two years after it enters the market on a closed source, proprietary footing. To suggest there is no difference, as you seem to be doing, is a hopelessly unconvincing argument. Except, eRServer, the basic model, will be released Open Source, and, if all goes as planned, in time for inclusion in contrib of v7.1 ...
[HACKERS] EAN13 for postgresql
i have wrote an application dealing with ean13 and ean8 type,how can i submit it ??
re : Re: [HACKERS] Add-on
Ean13 and ean8 are bar codes for european. You can convert an ISBN or iSSN to Ean13. My addon add a new type and can convert isbn to an EAN and calculate th key of ean. More over in few day add on can store the png or jpg images of bar codes in blob type or TODO: add upc-A upc-E ean128 and other type of bar code Best regards PEJAC Pascal
[HACKERS] Bitmap index
Hi, on other RDBMS (Oracle,etc...),there is an index called bitmap index that greatly improve performance compared to btree index for boolean value (such as for a sex value,it's either M or F),i would like to know if such index will be implemented inside PostgreSQL. Best regards, PEJAC Pascal
Re: [HACKERS] beta testing version
On Thu, 30 Nov 2000, Nathan Myers wrote: Second, the transaction log is not, as has been noted far too frequently for Vince's comfort, really written atomically. The OS has promised to write it atomically, and given the opportunity, it will. If you pull the plug, all promises are broken. Say what? Vince. -- == Vince Vielhaber -- KA8CSHemail: [EMAIL PROTECTED]http://www.pop4.net 128K ISDN from $22.00/mo - 56K Dialup from $16.00/mo at Pop4 Networking Online Campground Directoryhttp://www.camping-usa.com Online Giftshop Superstorehttp://www.cloudninegifts.com ==
Re: [HACKERS] compiling pg 7.0.3 on sco 5.0.5
* Tom Lane [EMAIL PROTECTED] [001204 09:27]: "Billy G. Allie" [EMAIL PROTECTED] writes: ... The DISABLE_COMPLEX_MACRO definition was originally put in to work around a macro size limitation of the UnixWare 2.1 C compiler (and later the SCO UDK (Universal Development Kit)). If the gnu C compiler is being used it should not be defined. Hm. Is anyone likely to still be using a version of that compiler that still has such limitations? I ask because we recently pulled "#define DISABLE_COMPLEX_MACRO" from port/sco.h, on the grounds that various people were seeing more harm than good from it. But I'm suddenly wondering whether those people might've been using gcc. I wonder if #ifndef __GNUC__ #define DISABLE_COMPLEX_MACRO #endif in port/sco.h would be the smart way to go. Based on my running both CURRENT UDK and GCC on my UnixWare 7 boxes with CURRENT sources, I think we may need to see if anyone complains. LER regards, tom lane -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Re: [HACKERS] redundancy and disk i/o
On Tue, 28 Nov 2000, Sandeep Joshi wrote: 1. Is it possible to set up a set of redundant disks for a database? one of them being remote from the database? Call IBM Global Services, and tell them you are interested in purchasing an RS/6000 with a 7133 SSA drives, one tray off-site using the fiber extenders. With those, you can have your drives up to 2.4 km from the server they're connected to, while they still are local to the machine. (And you still get 4 simultaneous reads/writes in each direction of the loop, for a total of 160 Mbyte/sec transfer.) -- Dominic J. Eidson "Baruk Khazad! Khazad ai-menu!" - Gimli --- http://www.the-infinite.org/ http://www.the-infinite.org/~dominic/
Re: [HACKERS] compiling pg 7.0.3 on sco 5.0.5
Tom Lane wrote: "Billy G. Allie" [EMAIL PROTECTED] writes: ... The DISABLE_COMPLEX_MACRO definition was originally put in to work around a macro size limitation of the UnixWare 2.1 C compiler (and later the SCO UDK (Universal Development Kit)). If the gnu C compiler is being used it should not be defined. Hm. Is anyone likely to still be using a version of that compiler that still has such limitations? I ask because we recently pulled "#define DISABLE_COMPLEX_MACRO" from port/sco.h, on the grounds that various people were seeing more harm than good from it. But I'm suddenly wondering whether those people might've been using gcc. I wonder if #ifndef __GNUC__ #define DISABLE_COMPLEX_MACRO #endif in port/sco.h would be the smart way to go. regards, tom lane Well I recompilied with the stock cc shipped in the SCO development package for OpenServer 5. It was released in 97'.
Re: [HACKERS] redundancy and disk i/o
If you're talking about replication, PostgreSQL, Inc. will be offering a solution to its $19,000/yr Platinum Partners shortly. It will be released in open source form no more than two years after its release in proprietary form. Check out http://www.erserver.com for more details, and http://www.pgsql.com for more details on the PostgreSQL, Inc. partnership program. Thanks Don for the reference. As you know, there will also be a "roll your own" replication toolset contributed by PostgreSQL Inc. under the BSD license in the PostgreSQL contrib/ directory for the 7.1 release, assuming that this inclusion is acceptable to the community. Given the general interest, I hope that this won't be an issue, and that the recent flames will have died down enough to not be a continued distraction. - Thomas
[HACKERS] update on compiling postgres on sco
posting for others who may need, hopfuly the searchable mail list works in the future commenting out the define complex macro allowed me to compile on sco 5.0.5 using udk compiler on sco, without the c++ stuff. but scos udk solution breaks almost every thing else i compile on sco 5.0.5 when u install there compatability stuff they introduce 2 sets of libraries one for sco one for unixware compatability, the compatability librarsy the give you DO NOT replace all the shared libraies on the system, notably /lib/prot.so or anything under /lib has no compatable libs installed under /udk/usr/lib, the udk compilers both c and c++ use the new libraries by default soon as you need a libray that is not available under /udk/usr/lib your screwed, there is a skunkware version of gcc but it passes a non existant option -b to the sco assember, and even the sco assebler uses the new udk libs, removing c++ on sco does not fix the problem as sco says it should, since there is no binutils ported to sco skunkware with gas and other tools this realy sucks. not sure if i can install my old sdk on sco 5.0.5 which was licsensed on sco 5.0.2, not sure if sdk and udk can co exist, and how do u manage the libs dirs that are searched autmaticly, or add the -Xo option that sco says will allow stuff to compile with less strict error checking - winblows model stuff compiles and links may - maynot work or unknown problems to test scos udk on sco openserver 5.0.5 breaks all open source code, or at least most of what i want to use. advice to future people that want to use sco open server, screw the udk, udk compatability for those wishing to use the backwards compatability from unixware to sco openserver expect conflicts, and unsuportted libraries, links static and pray. hope like hell caldara slaps some sense into sco fast, or at least port binutils, gcc then maybe u can just license the header files for 100 bucks instead of buing crapy compiler technoweldgy for 500 bucks when all you want is the header files, and -- My opinions are my own and not that of my employer even if I am self employed Tech Net, Inc.--FREE THE MACHINES-- 651.224.2223 627 Palace Ave. #2 [EMAIL PROTECTED] [EMAIL PROTECTED] St. Paul, MN 55102-3517www.tnss.com wanted: adsl/cable modem with static ip at reasonable price accept-txt: us-ascii,html,pdf accept-dat: ascii-delimited,sql insert statments
Re: [HACKERS] compiling pg 7.0.3 on sco 5.0.5
Tom Lane writes: I ask because we recently pulled "#define DISABLE_COMPLEX_MACRO" from port/sco.h, on the grounds that various people were seeing more harm than good from it. But I'm suddenly wondering whether those people might've been using gcc. We can be fairly certain that they weren't, unless GCC started accepting SCO's compiler flags (or someone altered the compiler flags and filed a *very* incomplete bug report). -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: [HACKERS] Postgresql on dynix/ptx system
Radek Fleks writes: I'm compiling (not, I'm trying to compile) last version of Postgresql on Sequent Dynix/ptx ver 4.4.7 system. Under compilation process with gcc (ver 2.7.2 ported on dynix/pt) is reporting several errors. It's not so interesting at this point to port PostgreSQL 7.0.*, given that PostgreSQL 7.1 should go beta sometime, er, this year. If you want to port 7.1 then you should be looking into the following files and/or directories for platform specific stuff: configure.in src/template src/makefiles src/include/port src/Makefile.shlib src/backend/port/dynloader src/include/storage/s_lock.h src/backend/storage/buffer/s_lock.c Once you have gotten past the fact that configure will complain about your system not being supported (which you should fix in configure.in and re-run autoconf), showing actual compiler output will help. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: [HACKERS] Using Threads?
if we were to do this in steps, I beliee that one of the major problems
irght now is that we have global variables up the wazoo ... my
'thread-awareness' is limited, as I've yet to use them, so excuse my
ignorance ... if we got patches that cleaned up the code in stages, moving
towards a cleaner code base, then we could get it into the main source
tree ... ?
On Mon, 4 Dec 2000, Ross J. Reedstrom wrote:
Myron -
Putting aside the fork/threads discussion for a moment (the reasons,
both historical and other, such as inter-backend protection, are well
covered in the archives), the work you did sounds like an interesting
experiment in code redesign. Would you be willing to release the hacked
code somewhere for others to learn from? Hacking flex to generate
thread-safe code is of itself interesting, and the question about PG and
threads comes up so often, that an example of why it's not a simple task
would be useful.
Ross
On Mon, Dec 04, 2000 at 12:20:20AM -0800, Myron Scott wrote:
I maybe wrong but I think that PGSQL is not threaded mostly due to
historical reasons. It looks to me like the source has developed over
time where much of the source is not reentrant with many global variables
throughout. In addition, the parser is generated by flex which
can be made to generate reentrant code but is still not thread safe b/c
global variables are used.
That being said, I experimented with the 7.0.2 source and came up with a
multithreaded backend for PGSQL which uses Solaris Threads. It seems to
work, but I drifted very far from the original source. I
had to hack flex to generate threadsafe code as well. I use it as a
linked library with my own fe-be protocol. This ended up being much much
more than I bargained for and looking back would probably not have tried
had I known any better.
Myron Scott
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: [EMAIL PROTECTED] secondary: scrappy@{freebsd|postgresql}.org
[HACKERS] [Fwd: Re: [CORE] Going Beta on Monday ...]
I browsed through the CVS logs and made this list of the important stuff. There's a ton of less important stuff... Shall we consider this the start of the list then? I think there may be a couple of things already mentioned in the release note stubs for 7.1 too. - Thomas Additional items: AT TIME ZONE clause for date/time types OVERLAPS operator support rewritten WAL --- fsync reliability without the performance hit TOAST --- 8K row limit is no longer significant outer joins (per SQL92 syntax, not Oracle's) subselects in FROM clause views and subselects now allow union/intersect/except, order by, limit views containing grouping, aggregates, DISTINCT work now bit-string types work now function manager overhaul: fixes portability problems, NULL-argument handling memory management overhaul: prevent memory leak accumulation during queries drop table and rename table are now rollback-able (transaction-safe) extensive overhaul of configure/build mechanism overhaul of parameter-setting mechanisms (postmaster flags, postmaster.opts, etc) more efficient large-object implementation pg_dump can dump large objects now pg_dump does the right thing with user-added objects in template1 support for binding postmaster's IP socket to a virtual host name support for placing postmaster's Unix socket file elsewhere than /tmp keep reference counts on syscache entries to avoid dropping still-used entries Protect against changes in LOCALE environment causing corrupted indexes better handling of unknown-type literals (default to string type more readily) inet/cidr datatypes cleaned up LIKE/ESCAPE implemented, also ILIKE (case-insensitive LIKE) aggregate-function support redesigned: only one transition function now, cleaner handling of NULLs STDDEV() and VARIANCE() aggregates added SUM() and AVG() on integer datatypes use NUMERIC accumulators Child tables are now scanned by default -- ie, if foo has children then SELECT FROM foo means SELECT FROM foo*. Ditto for UPDATE and DELETE. Use SELECT FROM ONLY foo if you don't want this behavior. vacuum analyze does the analyze part without holding exclusive lock
Re: [HACKERS] Wrong FOR UPDATE lock type
Mikheev, Vadim wrote: I'm about 99.67% sure that the lock type choosen in the FOR UPDATE case (line 511 of parse_relation.c) should be RowExclusiveLock instead of RowShareLock. Actually I get "Deadlock risk" debug messages when selecting FOR UPDATE and then really UPDATE. http://www.postgresql.org/users-lounge/docs/6.5/user/x3116.htm RowShareLock Acquired by SELECT FOR UPDATE and LOCK TABLE for IN ROW SHARE MODE statements. Conflicts with ExclusiveLock and AccessExclusiveLock modes. Tom, IIRC the "Deadlock risk" debug message is from you. I think it must get a little smarter. IMHO an application that want's to UPDATE something in a transaction but must SELECT the row(s) first to do it's own calculation on them, should use SELECT FOR UPDATE. Is that debug output really appropriate in this case (it raises from RowShareLock to RowExclusiveLock because of the UPDATE of the previous FOR UPDATE selected row)? Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] #
Re: [HACKERS] [Fwd: Re: [CORE] Going Beta on Monday ...]
Macaddr manufacturer table update now in SQL table syslog configurability improvements. * Thomas Lockhart [EMAIL PROTECTED] [001204 13:42]: I browsed through the CVS logs and made this list of the important stuff. There's a ton of less important stuff... Shall we consider this the start of the list then? I think there may be a couple of things already mentioned in the release note stubs for 7.1 too. - Thomas Additional items: AT TIME ZONE clause for date/time types OVERLAPS operator support rewritten WAL --- fsync reliability without the performance hit TOAST --- 8K row limit is no longer significant outer joins (per SQL92 syntax, not Oracle's) subselects in FROM clause views and subselects now allow union/intersect/except, order by, limit views containing grouping, aggregates, DISTINCT work now bit-string types work now function manager overhaul: fixes portability problems, NULL-argument handling memory management overhaul: prevent memory leak accumulation during queries drop table and rename table are now rollback-able (transaction-safe) extensive overhaul of configure/build mechanism overhaul of parameter-setting mechanisms (postmaster flags, postmaster.opts, etc) more efficient large-object implementation pg_dump can dump large objects now pg_dump does the right thing with user-added objects in template1 support for binding postmaster's IP socket to a virtual host name support for placing postmaster's Unix socket file elsewhere than /tmp keep reference counts on syscache entries to avoid dropping still-used entries Protect against changes in LOCALE environment causing corrupted indexes better handling of unknown-type literals (default to string type more readily) inet/cidr datatypes cleaned up LIKE/ESCAPE implemented, also ILIKE (case-insensitive LIKE) aggregate-function support redesigned: only one transition function now, cleaner handling of NULLs STDDEV() and VARIANCE() aggregates added SUM() and AVG() on integer datatypes use NUMERIC accumulators Child tables are now scanned by default -- ie, if foo has children then SELECT FROM foo means SELECT FROM foo*. Ditto for UPDATE and DELETE. Use SELECT FROM ONLY foo if you don't want this behavior. vacuum analyze does the analyze part without holding exclusive lock -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Re: [HACKERS] Wrong FOR UPDATE lock type
Jan Wieck [EMAIL PROTECTED] writes: I'm about 99.67% sure that the lock type choosen in the FOR UPDATE case (line 511 of parse_relation.c) should be RowExclusiveLock instead of RowShareLock. Actually I get "Deadlock risk" debug messages when selecting FOR UPDATE and then really UPDATE. Should I change it? Not sure, but if you do change it, that's *not* the only place. I coded that as RowShareLock because that was what was getting grabbed by the executor for SELECT FOR UPDATE. I believe the rewriter may need changed as well, since it can also be the first grabber of a lock for a rel. Note also that the docs say SELECT FOR UPDATE gets RowShareLock! The "deadlock risk" message is not very bright, and I wouldn't suggest changing the code just because of that. I'm not even sure I want to leave that check in the release version ... regards, tom lane
Re: [HACKERS] Using Threads?
The Hermit Hacker [EMAIL PROTECTED] writes: Why not use threads instead? Is that just for a historical reason, or some performance/implementation concern? Several reasons, 'historical' probably being the strongest right now ... since PostgreSQL was never designed for threading, its about as 'un-thread-safe' as they come, and cleaning that up will/would be a complete nightmare (should eventually be done, mind you) ... The other is stability ... right now, if one backend drops away, for whatever reason, it doesn't take down the whole system ... if you ran things as one process, and that one process died, you just lost your whole system ... Portability is another big reason --- using threads would create lots of portability headaches for platforms that had no threads or an incompatible threads library. (Not to mention buggy threads libraries, not-quite-thread-safe libc routines, yadda yadda.) The amount of work required looks far out of proportion to the payoff... regards, tom lane
Re: [HACKERS] broken locale in 7.0.2 without multibyte support (FreeBSD 4.1-RELEASE) ?
On Sun, 3 Dec 2000, Tom Lane wrote: Date: Sun, 03 Dec 2000 18:13:47 -0500 From: Tom Lane [EMAIL PROTECTED] To: Oleg Bartunov [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: [HACKERS] broken locale in 7.0.2 without multibyte support (FreeBSD 4.1-RELEASE) ? Oleg Bartunov [EMAIL PROTECTED] wrote a couple months ago: It's clear that we must use 'unsigned char' instead of 'char' and corrected version runs ok on both systems. That's why I suspect that gcc 2.95.2 has different default under FreeBSD which could cause problem with LC_CTYPE in 7.0.2 ok. will check this. I've recompile 7.0.2 on freebsd with -funsigned-char and the problem has gone away. This prove my suggestion. I also checked 6.5 and it has the same probelm on FreeBSD. Also, this makes clear many complains about broken locale under FreeBSD I got from people. Hmm, current cvs has the same problem :-( Today I inserted (unsigned char) casts into all the ctype.h function calls I could find. This issue should be fixed as of current cvs. Please try it again when you have time. Just tried on FreeBSD 3.4-STABLE, current cvs, gcc version 2.95.2 19991024 (release), ru-RU.KOI8-R locale, postgresql configured with --enable-locale, no gcc option like --unisgned-chars Looks like your changes did right job ! regards, Oleg regards, tom lane _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
Re: AW: AW: [HACKERS] broken locale in 7.0.2 without multibyte suppor t (F reeBSD 4.1-RELEASE) ?
Zeugswetter Andreas SB [EMAIL PROTECTED] writes: But don't they in general obfuscate cases where the callee does want unsigned/signed chars ? Well, it's ugly, but I don't think we have much choice. Seems to me that changing to "unsigned char" throughout the backend would obfuscate things *more* than coding ctype.h calls as char*p; ... x = tolower((unsigned char) *p); which is what I actually did. There are lots of places where "char" variables are used that will never see a ctype.h call. Do we institute a coding rule that plain "char" is verboten in *all* cases, whether or not they're relevant to ctype calls? If not, how do we check that "char" is being used safely? Aren't we likely to get compiler warnings from passing "unsigned char *" to libc functions that are declared to take plain "char *"? I don't think that path is an improvement over a coding rule that ctype functions must be applied to unsigned chars. IMHO the latter is less intrusive overall, and no harder to check for violations. My concern stems from a very bad experience with wrong signedness of chars on AIX. I agree that this is something we'll have to watch. I don't see any cleaner answer, though. regards, tom lane
Re: [HACKERS] Using Threads?
All the major operating systems should have POSIX threads implemented. Actually this can be configurable--multithreads or one thread. Thread-only server is unsafe, I agree. Maybe the following model can be a little better. Several servers, each is multi-threaded. Every server can support a maximum number of requests simultaneously. If anything bad happends, it is limited to that server. The cons side of processes model is not the startup time. It is about kernel resource and context-switch cost. Processes consume much more kernel resource than threads, and have a much higher cost for context switch. The scalability of threads model is much better than that of processes model. -Junfeng On Mon, 4 Dec 2000, Thomas Lockhart wrote: I am new to postgreSQL. When I read the documents, I find out the Postmaster daemon actual spawns a new backend server process to serve a new client request. Why not use threads instead? Is that just for a historical reason, or some performance/implementation concern? Both. Not all systems supported by PostgreSQL have a standards-compliant threading implementation (even more true for the systems PostgreSQL has supported over the years). But there are performance and reliability considerations too. A thread-only server is likely more brittle than a process-per-client implementation, since all threads share the same address space. Corruption in one server might more easily propagate to other servers. The time to start a backend is quite often small compared to the time required for a complete session, so imho the differences in absolute speed are not generally significant. - Thomas
[HACKERS] Debian build failing...
Hi. Could any kind soul tell me what's amiss here. I'm trying
to build pg7.0.3 on a friend's box - over the net.
Kind of like driving from the backseat. ;-)
My src builds but the linker barfs with:
make[2]: Leaving directory `/usr/local/postgresql-7.0.3/src/backend/utils'
gcc -I../include -I../backend -O2 -Wall -Wmissing-prototypes -Wmissing-declarations
-o postgres access/SUBSYS.o bootstrap/SUBSYS.o catalog/SUBSYS.o commands/SUBSYS.o
executor/SUBSYS.o lib/SUBSYS.o libpq/SUBSYS.o main/SUBSYS.o parser/SUBSYS.o
nodes/SUBSYS.o optimizer/SUBSYS.o port/SUBSYS.o postmaster/SUBSYS.o regex/SUBSYS.o
rewrite/SUBSYS.o storage/SUBSYS.o tcop/SUBSYS.o utils/SUBSYS.o ../utils/version.o
-lcrypt -lnsl -ldl -lm -lutil -lncurses -export-dynamic
/usr/lib/libdl.a(dlsym.o): In function `doit.2':
dlsym.o(.text+0x22): undefined reference to `_dl_default_scope'
dlsym.o(.text+0x4c): undefined reference to `_dl_default_scope'
make[1]: *** [postgres] Error 1
make[1]: Leaving directory `/usr/local/postgresql-7.0.3/src/backend'
make: *** [all] Error 2
TIA,
Tom
SVCMC - Center for Behavioral Health
Thomas Good tomg@ { admin | q8 } .nrnet.org
IS Coordinator / DBA Phone: 718-354-5528
Fax: 718-354-5056
Powered by: PostgreSQL s l a c k w a r e FreeBSD:
RDBMS |-- linux The Power To Serve
Re: [HACKERS] Wrong FOR UPDATE lock type
Jan Wieck [EMAIL PROTECTED] writes: Tom, IIRC the "Deadlock risk" debug message is from you. I think it must get a little smarter. IMHO an application that want's to UPDATE something in a transaction but must SELECT the row(s) first to do it's own calculation on them, should use SELECT FOR UPDATE. Is that debug output really appropriate in this case (it raises from RowShareLock to RowExclusiveLock because of the UPDATE of the previous FOR UPDATE selected row)? Well, there is a theoretical chance of deadlock --- not against other transactions doing the same thing, since RowShareLock and RowExclusiveLock don't conflict, but you could construct deadlock scenarios involving other transactions that grab ShareLock or ShareRowExclusiveLock. So I don't think it's appropriate for the "deadlock risk" check to ignore RowShareLock-RowExclusiveLock upgrades. But I'm not sure the check should be enabled in production releases anyway. I just put it in as a quick and dirty debug check. Perhaps it should be under an #ifdef that's not enabled by default. regards, tom lane
Re: [HACKERS] Using Threads?
On Mon, Dec 04, 2000 at 02:28:10PM -0600, Bruce Guenter wrote: On Mon, Nov 27, 2000 at 11:42:24PM -0600, Junfeng Zhang wrote: I am new to postgreSQL. When I read the documents, I find out the Postmaster daemon actual spawns a new backend server process to serve a new client request. Why not use threads instead? Is that just for a historical reason, or some performance/implementation concern? Once all the questions regarding "why not" have been answered, it would be good to also ask "why use threads?" Do they simplify the code? Do they offer significant performance or efficiency gains? What do they give, other than being buzzword compliant? Typically (on a well-written OS, at least), the spawning of a thread is much cheaper then the creation of a new process (via fork()). Also, since everything in a group of threads (I'll call 'em a team) shares the same address space, there can be some memory overhead savings. -- Adam Haberlach |"California's the big burrito, Texas is the big [EMAIL PROTECTED] | taco ... and following that theme, Florida is http://www.newsnipple.com| the big tamale ... and the only tamale that '88 EX500| counts any more." -- Dan Rather
[HACKERS] INSERT INTO ... SELECT problem
Hi everyone,
I've recently encountered a bizzare problem that manifests itself reliably
on my running copy of postgres. I have a system set up to track IPs. The
arrangement uses two mutually-exclusive buckets, one for free IPs and
the other for used ones. There are rules set up on the used pool to
remove IPs from the free on insert, and re-add them on delete.
The structure of the tables is:
CREATE TABLE "ips_free" (
"block_id" int4 NOT NULL,
"ip" inet NOT NULL,
"contact_id" int4,
"alloc_type" int4,
PRIMARY KEY ("block_id", "ip")
);
CREATE TABLE "ips_used" (
"block_id" int4 NOT NULL,
"ip" inet NOT NULL,
"contact_id" int4,
"alloc_type" int4,
PRIMARY KEY ("block_id", "ip")
);
The applicable rule that acts on inset to ips_used is:
CREATE RULE ip_allocated_rule AS
ON INSERT
TO ips_used
DO DELETE FROM ips_free
WHERE ips_free.block_id = NEW.block_id
AND ips_free.ip = NEW.ip;
When I tried to minimize the total number of queries in a data load, I
tried to get the block ID (see above for the schema definition) using
INSERT INTO ... SELECT. A query like
INSERT INTO ips_used
(
block_id,
ip,
contact_id
)
SELECT block_id
, ip
, '1000'
FROM ips_free
WHERE ip = '10.10.10.10'
simply reutrns with "INSERT 0 0" and in fact removes the IP from the
free bucket without adding it to the USED bucket. I really can't
explain this behavior and I'm hoping someone can shed a little bit of
light on it.
I am running PostgreSQL 7.0.0 on sparc-sun-solaris2.7, compiled by gcc 2.95.2
Thanks
Alex
--
Alex G. Perel -=- AP5081
[EMAIL PROTECTED] -=- [EMAIL PROTECTED]
play -=- work
Disturbed Networks - Powered exclusively by FreeBSD
== The Power to Serve -=- http://www.freebsd.org/
Re: [HACKERS] Using Threads?
Adam Haberlach writes: Typically (on a well-written OS, at least), the spawning of a thread is much cheaper then the creation of a new process (via fork()). This would be well worth testing on some representative sample systems. Within the past year and a half at one of my gigs some coworkers did tests on various platforms (Irix, Solaris, a few variations of Linux and *BSDs) and concluded that in fact the threads implementations were often *slower* than using processes for moving and distributing the sorts of data that they were playing with. With copy-on-write and interprocess pipes that are roughly equivalent to memcpy() speeds it was determined for that application that the best way to split up tasks was fork()ing and dup(). As always, your mileage will vary, but the one thing that consistently amazes me on the Un*x like operating systems is that usually the programmatically simplest way to implement something has been optimized all to heck. A lesson that comes hard to those of us who grew up on MS systems. Dan
Re: [HACKERS] 8192 BLCKSZ ?]
Don Baccus wrote: ... I expect TOAST to work even better). Users will still be able change to larger blocksizes (perhaps a wise thing to do if a large percentage of their data won't fit into a single PG block). Users using the default will be able to store rows of *awesome* length, efficiently. Depends... Actually the toaster already jumps in if your tuples exceed BLKSZ/4, so with the default of 8K blocks it tries to keep all tuples smaller than 2K. The reasons behind that are: 1. An average tuple size of 8K means an average of 4K unused space at the end of each block. Wasting space means to waste IO bandwidth. 2. Since big items are unlikely to be search criteria, needing to read them into memory for every chech for a match on other columns is a waste again. So the more big items are off from the main tuple, the smaller the main table becomes, the more likely it is that the main tuples (holding the keys) are cached and the cheaper a sequential scan becomes. Of course, especially for 2. there is a break even point. That is when the extra fetches to send toast values to the client cost more than there was saved from not doing it during the main scan already. A full table SELECT * definitely costs more if TOAST is involved. But who does unqualified SELECT * from a multi-gig table without problems anyway? Usually you pick a single or a few based on some other key attributes - don't you? Let's make an example. You have a forum server that displays one article plus the date and sender of all follow-ups. The article bodies are usually big (1-10K). So you do a SELECT * to fetch the actually displayed article, and another SELECT sender, date_sent just to get the info for the follow-ups. If we assume a uniform distribution of body size and an average of 10 follow-ups, that'd mean that we save 52K of IO and cache usage for each article displayed. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] #
Re: [HACKERS] Using Threads?
On Mon, Dec 04, 2000 at 03:17:00PM -0800, Adam Haberlach wrote: Typically (on a well-written OS, at least), the spawning of a thread is much cheaper then the creation of a new process (via fork()). Unless I'm mistaken, the back-end is only forked when starting a new connection, in which case the latency of doing the initial TCP tri-state and start-up queries is much larger than any process creation cost. On Linux 2.2.16 on a 500MHz PIII, I can do the fork/exit/wait sequence in about 164us. On the same server, I can make/break a PostgreSQL connection in about 19,000us (with 0% CPU idle, about 30% CPU system). Even if we can manage to get a thread for free, and assume that the fork from postmaster takes more than 164us, it won't make a big difference once the other latencies are worked out. Also, since everything in a group of threads (I'll call 'em a team) Actually, you call them a process. That is the textbook definition. shares the same address space, there can be some memory overhead savings. Only slightly. All of the executable and libraries should already be shared, as will all non-modified data. If the data is modified by the threads, you'll need seperate copies for each thread anyways, so the net difference is small. I'm not denying there would be a difference. Compared to seperate processes, threads are more efficient. Doing a context switch between threads means there is no PTE invalidations, which makes them quicker than between processes. Creation would be a bit faster due to just linking in the VM to a new thread rather than marking it all as COW. The memory savings would come from reduced fragmentation of the modified data (if you have 1 byte modified on each of 100 pages, the thread would grow by a few K, compared to 400K for processes). I'm simply arguing that the differences don't appear to be significant compared to the other costs involved. -- Bruce Guenter [EMAIL PROTECTED] http://em.ca/~bruceg/ PGP signature
RE: [HACKERS] Using Threads?
*snip* Once all the questions regarding "why not" have been answered, it would be good to also ask "why use threads?" Do they simplify the code? Do they offer significant performance or efficiency gains? What do they give, other than being buzzword compliant? The primary advantage that I see is that a single postgres process can benefit from multiple processors. I see little advantage to using thread for client connections.
Re: [HACKERS] Using Threads?
On Mon, Dec 04, 2000 at 02:30:31PM -0800, Dan Lyke wrote:
Adam Haberlach writes:
Typically (on a well-written OS, at least), the spawning of a thread
is much cheaper then the creation of a new process (via fork()).
This would be well worth testing on some representative sample
systems.
Using the following program for timing process creation and cleanup:
main() {
int i;
int pid;
for (i=0; i10; ++i) {
pid=fork();
if(pid==-1) exit(1);
if(!pid) _exit(0);
waitpid(pid,0,0);
}
exit(0);
}
And using the following program for timing thread creation and cleanup:
#include pthread.h
threadfn() { pthread_exit(0); }
main() {
int i;
pthread_t thread;
for (i=0; i10; ++i) {
if (pthread_create(thread, 0, threadfn, 0)) exit(1);
if (pthread_join(thread, 0)) exit(1);
}
exit(0);
}
On a relatively unloaded 500MHz PIII running Linux 2.2, the fork test
program took a minimum of 16.71 seconds to run (167us per
fork/exit/wait), and the thread test program took a minimum of 12.10
seconds to run (121us per pthread_create/exit/join). I use the minimums
because those would be the runs where the tasks were least interfered
with by other tasks. This amounts to a roughly 25% speed improvement
for threads over processes, for the null-process case.
If I add the following lines before the for loop:
char* m;
m=malloc(1024*1024);
memset(m,0,1024,1024);
The cost for doing the fork balloons to 240us, whereas the cost for
doing the thread is constant. So, the cost of marking the pages as COW
is quite significant (using those numbers, 73us/MB).
So, forking a process with lots of data is expensive. However, most of
the PostgreSQL data is in a SysV IPC shared memory segment, which
shouldn't affect the fork numbers.
--
Bruce Guenter [EMAIL PROTECTED] http://em.ca/~bruceg/
PGP signature
Re: [HACKERS] Using Threads?
Matthew wrote: The primary advantage that I see is that a single postgres process can benefit from multiple processors. I see little advantage to using thread for client connections. Multiprocessors best benefit multiple backends. And the current forked model lends itself admirably to SMP. And I say that even after using a multithreaded webserver (AOLserver) for three and a half years. Of course, AOLserver also sanely uses the multi process PostgreSQL backends in a pooled fashion, but that's beside the point. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Re: [HACKERS] Bitmap index
On Mon, Dec 04, 2000 at 04:28:47PM +0100, [EMAIL PROTECTED] wrote: on other RDBMS (Oracle,etc...),there is an index called bitmap index that greatly improve performance compared to btree index for boolean value (such as for a sex value,it's either M or F),i would like to know if such index will be implemented inside PostgreSQL. Yes, please do send in your implementation for review. Nathan Myers [EMAIL PROTECTED]
Re: [HACKERS] Using Threads?
I would love to distribute this code to anybody who wants it. Any suggestions for a good place? However, calling the work a code redesign is a bit generous. This was more like a brute force hack. I just moved all the connection related global variables to a thread local "environment variable" and bypassed much of the postmaster code. I did this so I could port my app which was originally designed for Oracle OCI and Java. My app uses very few SQL statements but uses them over and over. I wanted true prepared statements linked to Java with JNI. I got both as well as batched transaction writes ( which was more relevant before WAL). In my situation, threads seemed much more flexible to implement, and I probably could not have done the port without it. Myron On Mon, 4 Dec 2000, Ross J. Reedstrom wrote: Myron - Putting aside the fork/threads discussion for a moment (the reasons, both historical and other, such as inter-backend protection, are well covered in the archives), the work you did sounds like an interesting experiment in code redesign. Would you be willing to release the hacked code somewhere for others to learn from? Hacking flex to generate thread-safe code is of itself interesting, and the question about PG and threads comes up so often, that an example of why it's not a simple task would be useful. Ross
RE: [HACKERS] SQL to retrieve FK's, Update/Delete action, etc. (fwd)
There ya go, I figured it out :) Given the name a table, this query will return all foreign keys in that table, the table the primary key is in, the name of the primary key, if the are deferrable, if the are initially deffered, and the action to be performed (RESTRICT, SET NULL, etc.). To get the foreign keys and primary keys and tables, you must parse the null-terminated pg.tgargs. When I get the equivalent query working for primary keys I will send it your way -- or if you beat me to it, send it my way (I am working on some missing functionality from the ODBC driver) Michael Fork - CCNA - MCP - A+ Network Support - Toledo Internet Access - Toledo Ohio SELECT pt.tgargs, pt.tgnargs, pt.tgdeferrable, pt.tginitdeferred, pg_proc.proname, pg_proc_1.proname FROM pg_class pc, pg_proc pg_proc, pg_proc pg_proc_1, pg_trigger pg_trigger, pg_trigger pg_trigger_1, pg_proc pp, pg_trigger pt WHERE pt.tgrelid = pc.oid AND pp.oid = pt.tgfoid AND pg_trigger.tgconstrrelid = pc.oid AND pg_proc.oid = pg_trigger.tgfoid AND pg_trigger_1.tgfoid = pg_proc_1.oid AND pg_trigger_1.tgconstrrelid = pc.oid AND ((pc.relname='FOREIGN TABLE') AND (pp.proname LIKE '%%ins') AND (pg_proc.proname LIKE '%%upd') AND (pg_proc_1.proname LIKE '%%del') AND (pg_trigger.tgrelid=pt.tgconstrrelid) AND (pg_trigger_1.tgrelid = pt.tgconstrrelid)) On Tue, 5 Dec 2000, Christopher Kings-Lynne wrote: Hi Michael, I am on the phpPgAdmin development team, and I have been wanting to add this functionality to phpPgAdmin. I will start working with your query as soon as possible, and I will use phpPgAdmin as a testbed for the functionality. I really appreciate having your query as a working basis, because it's really hard trying to figure out the system tables! Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Michael Fork Sent: Sunday, December 03, 2000 12:23 PM To: [EMAIL PROTECTED] Subject: [HACKERS] SQL to retrieve FK's, Update/Delete action, etc. Given the name of a table, I need to find all foreign keys in that table and the table/column that they refer to, along with the action to be performed on update/delete. The following query works, but only when there is 1 foreign key in the table, when there is more than 2 it grows exponentially -- which means I am missing a join. However, given my limitied knowledge about the layouts of the postgres system tables, and the pg_trigger not being documented on the web site, I have been unable to get the correct query. Is this possible, and if so, what join(s) am I missing? SELECT pt.tgargs, pt.tgnargs, pt.tgdeferrable, pt.tginitdeferred, pg_proc.proname, pg_proc_1.proname FROM pg_class pc, pg_proc pg_proc, pg_proc pg_proc_1, pg_trigger pg_trigger, pg_trigger pg_trigger_1, pg_proc pp, pg_trigger pt WHERE pt.tgrelid = pc.oid AND pp.oid = pt.tgfoid AND pg_trigger.tgconstrrelid = pc.oid AND pg_proc.oid = pg_trigger.tgfoid AND pg_trigger_1.tgfoid = pg_proc_1.oid AND pg_trigger_1.tgconstrrelid = pc.oid AND ((pc.relname='tblmidterm') AND (pp.proname LIKE '%ins') AND (pg_proc.proname LIKE '%upd') AND (pg_proc_1.proname LIKE '%del')) Michael Fork - CCNA - MCP - A+ Network Support - Toledo Internet Access - Toledo Ohio
Re: [HACKERS] Using Threads?
Bruce Guenter [EMAIL PROTECTED] writes: [ some very interesting datapoints ] So, forking a process with lots of data is expensive. However, most of the PostgreSQL data is in a SysV IPC shared memory segment, which shouldn't affect the fork numbers. I believe (but don't have numbers to prove it) that most of the present backend startup time has *nothing* to do with thread vs process overhead. Rather, the primary startup cost has to do with initializing datastructures, particularly the system-catalog caches. A backend isn't going to get much real work done until it's slurped in a useful amount of catalog cache --- for example, until it's got the cache entries for pg_class and the indexes thereon, it's not going to accomplish anything at all. Switching to a thread model wouldn't help this cost a bit, unless we also switch to a shared cache model. That's not necessarily a win when you consider the increased costs associated with cross-backend or cross-thread synchronization needed to access or update the cache. And if it *is* a win, we could get most of the same benefit in the multiple-process model by keeping the cache in shared memory. The reason that a new backend has to do all this setup work for itself, rather than inheriting preloaded cache entries via fork/copy-on-write from the postmaster, is that the postmaster isn't part of the ring of processes that can access the database files directly. That was done originally for robustness reasons: since the PM doesn't have to deal with database access, cache invalidation messages, etc etc yadda yadda, it is far simpler and less likely to crash than a real backend. If we conclude that shared syscache is not a reasonable idea, it might be interesting to look into making the PM into a full-fledged backend that maintains a basic set of cache entries, so that these entries are immediately available to new backends. But we'd have to take a real hard look at the implications for system robustness/crash recovery. In any case I think we're a long way away from the point where switching to threads would make a big difference in connection startup time. regards, tom lane
Re: [HACKERS] Using Threads?
On Mon, 4 Dec 2000, Junfeng Zhang wrote: All the major operating systems should have POSIX threads implemented. Actually this can be configurable--multithreads or one thread. I don't understand this. The OS can be configured for one thread? How would that be any of use? Thread-only server is unsafe, I agree. Maybe the following model can be a little better. Several servers, each is multi-threaded. Every server can support a maximum number of requests simultaneously. If anything bad happends, it is limited to that server. There is no difference. If anything bad happens with the current multi-process server, all the postgres backends shutdown because the shared memory may be corrupted. The cons side of processes model is not the startup time. It is about kernel resource and context-switch cost. Processes consume much more kernel resource than threads, and have a much higher cost for context switch. The scalability of threads model is much better than that of processes model. What kernel resources do a process use? There is some VM mapping overhead, a process table entry, and a file descriptor table. It is possible to support thousands of processes today. For instance, ftp.freesoftware.com supports up to 5000 FTP connections using a slightly modified ftpd (doesn't use inetd anymore). That means with 5000 users connected, that works out to 5000 processes active. Amazing but true. Some OSes (Linux is the main one) implement threads as pseudo processes. Linux threads are processes with a shared address space and file descriptor table. Context switch cost for threads can be lower if you are switching to a thread in the same process. That of course assumes that all context switches will occur within the same process, or the Linux everything-is-a-process model isn't used. -Junfeng Tom
