Re: [HACKERS] Inefficient bytea escaping?
Marko Kreen wrote: On 5/28/06, Martijn van Oosterhout kleptog@svana.org wrote: With -lpthread lock.enabled 323s lock.disabled 50s lock.unlocked 36s I forgot to test with -lpthread, my bad. Indeed by default something less expensive that full locking is going on. The crux of the matter is though, if you're calling something a million times, you're better off trying to find an alternative anyway. There is a certain amount of overhead to calling shared libraries and no amount of optimisation of the library is going save you that. The crux of the matter was if its possible to use fwrite as easy string combining mechanism and the answer is no, because it's not lightweight enough. IIRC the windows port make use of multi-threading to simulate signals and it's likely that some add-on modules will bring in libs like pthread. It would be less ideal if PostgreSQL was designed to take a significant performance hit when that happens. Especially if a viable alternative exists. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] LIKE, leading percent, bind parameters and indexes
On Sat, May 27, 2006 at 11:52:40AM -0400, Tom Lane wrote: Martijn van Oosterhout kleptog@svana.org writes: How about the suggestion of using a sequential index scan like the recent changes to VACUUM in the case that there are no regular index quals? Nonstarter (hint: the solution we found for VACUUM assumes there can be only one). Bummer, I was envisioning allowing index AMs to have another access method, the unordered sequential scan. Just like we consider a random access of a real table to be more expensive than a seq scan, and index scan that seeks a lot would be more expensive that a sequential scan. So if you know you're going to scan most of an index, scanning sequentially would be cheaper... Ah well, Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] pg_proc probin misuse
On Sun, May 28, 2006 at 09:12:34PM -0400, Tom Lane wrote: But we're still avoiding the central issue: does it make sense to dump a probin clause at all for plpython functions? If it's a compiled form of prosrc then it probably doesn't belong in the dump. That's why I initially thought pg_dump or I was the dirty one. Even if CREATE FUNCTION would take it, the probin value would be ignored(well, overwritten). On reflection I'm kind of inclined to think that plpython is abusing the column. If it were really expensive to derive bytecode from source text then maybe it'd make sense to do what you're doing, but surely that's not all that expensive. Everyone else manages to parse prosrc on the fly and cache the result in memory; why isn't plpython doing that? Yeah, I don't think it's expensive. It wasn't a feature that I implemented out of any kind of demand or testing. Rather, I knew I could marshal code objects, and I figured it would likely yield some improvement on initial loads, so I implemented it. If we think that plpython is leading the wave of the future, I'd be kind of inclined to invent a new pg_proc column in which derived text can be stored, rather than trying to use probin for the purpose. Although arguably probin itself was once meant to do that, there's too much baggage now. I think having something like that in pg_proc could be useful. Certainly my case may not really be demanding, but I guess there may be some languages that could enjoy a good benefit from avoiding recompilation. Tho, such a column seems like it would be more of a mere convenience for PL authors. If initial load were truly that expensive, I would think that it would justify creating a table containing compiled code and taking the extra lookup hit on initial load. ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Updatable views/with check option parsing
While I don't think that making WITH a fully reserved word would cause any great damage, I'm unwilling to do it just to save a couple of lines of code. I think we should go on and do promote WITH to a reserved keyword now Oracle, MS-SQL, DB2, MySQL and Informix also have WITH reserved, so it would imho be ok to do it if it simplifies code. Andreas ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] some question about deadlock
Hi, Below is the notesfrompostgresql-8.1.3/src/backend/storage/lmgr/README: Lock acquisition (routines LockAcquire and ProcSleep) follows these rules: 1. A lock request is granted immediately if it does not conflict withany existing or waiting lock request, or if the process already holds aninstance of the same lock type (eg, there's no penalty to acquire a readlock twice). Note that a process never conflicts with itself, eg onecan obtain read lock when one already holds exclusive lock. 2. Otherwise the process joins the lock's wait queue. Normally it willbe added to the end of the queue, but there is an exception: if theprocess already holds locks on this same lockable object that conflictwith the request of any pending waiter, then the process will beinserted in the wait queue just ahead of the first such waiter. (If wedid not make this check, the deadlock detection code would adjust thequeue order to resolve the conflict, but it's relatively cheap to makethe check in ProcSleep and avoid a deadlock timeout delay in this case.)Note special case when inserting before the end of the queue: if theprocess's request does not conflict with any existing lock nor anywaiting request before its insertion point, then go ahead and grant thelock without waiting. I am confused with that exception(in bold), could some one give me an example? Best regards.
Re: [HACKERS] COPY FROM view
Hannu Krosing wrote: ?hel kenal p?eval, P, 2006-05-28 kell 13:53, kirjutas Alvaro Herrera: Hi, I've been having the COPY FROM patch that was posted on pgsql-patches some time ago (I think from Hannu Krossing), Not by/from me :) Patch was from Karel Zak: http://candle.pha.pa.us/mhonarc/patches_hold/msg00118.html -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] some question about deadlock
ipig wrote: Hi, Below is the notes from postgresql-8.1.3/src/backend/storage/lmgr/README: Lock acquisition (routines LockAcquire and ProcSleep) follows these rules: 1. A lock request is granted immediately if it does not conflict with any existing or waiting lock request, or if the process already holds an instance of the same lock type (eg, there's no penalty to acquire a read lock twice). Note that a process never conflicts with itself, eg one can obtain read lock when one already holds exclusive lock. 2. Otherwise the process joins the lock's wait queue. Normally it will be added to the end of the queue, but there is an exception: if the process already holds locks on this same lockable object that conflict with the request of any pending waiter, then the process will be inserted in the wait queue just ahead of the first such waiter. (If we did not make this check, the deadlock detection code would adjust the queue order to resolve the conflict, but it's relatively cheap to make the check in ProcSleep and avoid a deadlock timeout delay in this case.) Note special case when inserting before the end of the queue: if the process's request does not conflict with any existing lock nor any waiting request before its insertion point, then go ahead and grant the lock without waiting. I am confused with that exception(in bold), could some one give me an example? Well, first you are requiring people to be reading HTML email to see the bold text you added, which is not good: an exception: if the process already holds locks on this same lockable object that conflict with the request of any pending waiter, then the process will be inserted in the wait queue just ahead of the first such waiter. An example would be you have a read lock and want an exclusive lock, and someone else in the queue is waiting for a read lock too. In this case, the exclusive lock goes before the queued read lock. -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] pg_proc probin misuse
PFC [EMAIL PROTECTED] writes: If it were really expensive to derive bytecode from source text then maybe it'd make sense to do what you're doing, but surely that's not all that expensive. Everyone else manages to parse prosrc on the fly and cache the result in memory; why isn't plpython doing that? It depends on the number of imported modules in the function. If it imports a lot of modules, it can take some time to compile a python function (especially if the modules have some initialisation code which must be run on import). Surely the initialization code would have to be run anyway ... and if the function does import a pile of modules, do you really want to cache all that in its pg_proc entry? What happens if some of the modules get updated later? regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] non-transactional pg_class
Alvaro Herrera [EMAIL PROTECTED] writes: I've been taking a look at what's needed for the non-transactional part of pg_class. If I've understood this correctly, we need a separate catalog, which I've dubbed pg_ntclass (better ideas welcome), and a new pointer in RelationData to hold a pointer to this new catalog for each relation. Also a new syscache needs to be created (say, NTRELOID). Do you really need both a relcache slot and a syscache? Seems redundant. For that matter, do you need either? Both the relcache and syscache operate on the assumption of transactional updates, so I think that you're going to have semantic problems using the caches to hold these tuples. For instance we don't broadcast any sinval update messages from a rolled-back transaction. On the other hand, must this new catalog be boostrapped? If relation creation or row insertion is going to try to write into it, then yes. You could get away with not writing a row initially as long as the rows only hold reltuples/relpages, but I think that would stop working as soon as you put the unfreeze code in. Obviously the idea is that we would never heap_update tuples there; only heap_inplace_update (and heap_insert when a new relation is created.) Initial insertion (table CREATE) and deletion (table DROP) would both have to be transactional operations. This may be safe because we'd hold exclusive lock on the table and so no one else would be touching the table's row, but it bears thinking about, because after all the whole point of the exercise is to keep transactional and nontransactional updates separate. What happens if someone tries to do a manual UPDATE in this catalog? Maybe this can be in the category of superusers should know enough not to do that, but I'd like to be clear on exactly what the consequences might be. Perhaps nontransactional catalogs should be a new relkind that we disallow normal updates on. If we do disallow normal updates (and VACUUM FULL too, probably) then it'd be possible to say that a given entry has a fixed TID for its entire lifespan. Then we could store the TID in the table's regular pg_class entry and dispense with any indexes. This would be advantageous if we end up concluding that we can't use the syscache mechanism (as I suspect that we can't), because we're going to be making quite a lot of fetches from this catalog. A direct fetch by TID would be a lot cheaper than an index search. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] some question about deadlock
Hi, Thanks for your reply. I changed the format to plain text. For the question, suppose that process p0 held the lock of object A, and the wait queue for A is p1,p2,p3,, that process p1 is the first waiter in the queue. Since p1 is in the wait queue, the lock p1 requests must be conflict with the lock p0 held. That is to say, if p0 wants to lock A again, then p0 will be put before p1, and p0 will be at the head of the queue. Why do we need to find the first waiter which conflicts p0? I think that p0 must be added at the head of the wait queue. For your example, p0 has a read lock and wants an exclusive lock. Since p0 has a read lock, then in the queue, p1 must wait an exclusive lock. Then p0 will be put before p1, and p0 will be at the head of the queue. Is there anything I misunderstood? Best wishes. - Original Message - From: Bruce Momjian pgman@candle.pha.pa.us To: ipig [EMAIL PROTECTED] Cc: pgsql-hackers@postgresql.org Sent: Monday, May 29, 2006 9:49 PM Subject: Re: [HACKERS] some question about deadlock ipig wrote: Hi, Below is the notes from postgresql-8.1.3/src/backend/storage/lmgr/README: Lock acquisition (routines LockAcquire and ProcSleep) follows these rules: 1. A lock request is granted immediately if it does not conflict with any existing or waiting lock request, or if the process already holds an instance of the same lock type (eg, there's no penalty to acquire a read lock twice). Note that a process never conflicts with itself, eg one can obtain read lock when one already holds exclusive lock. 2. Otherwise the process joins the lock's wait queue. Normally it will be added to the end of the queue, but there is an exception: if the process already holds locks on this same lockable object that conflict with the request of any pending waiter, then the process will be inserted in the wait queue just ahead of the first such waiter. (If we did not make this check, the deadlock detection code would adjust the queue order to resolve the conflict, but it's relatively cheap to make the check in ProcSleep and avoid a deadlock timeout delay in this case.) Note special case when inserting before the end of the queue: if the process's request does not conflict with any existing lock nor any waiting request before its insertion point, then go ahead and grant the lock without waiting. I am confused with that exception(in bold), could some one give me an example? Well, first you are requiring people to be reading HTML email to see the bold text you added, which is not good: an exception: if the process already holds locks on this same lockable object that conflict with the request of any pending waiter, then the process will be inserted in the wait queue just ahead of the first such waiter. An example would be you have a read lock and want an exclusive lock, and someone else in the queue is waiting for a read lock too. In this case, the exclusive lock goes before the queued read lock. -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] question about security hole CVE-2006-2313 and UTF-8
I have been experimenting with the exploit described in http://www.postgresql.org/docs/techdocs.50 to see if our databases are affected. Server is 8.1.3, database encoding UTF8. Client is a C program compiled and linked against libpq version 8.1.3 that uses UTF8 encoding. I sent the following query: od -c bad.sql 000 S E L E C T ' h a r m l e s s 020 303 ' ' , c u r r e n t _ d a t 040 e a s m a l i c i o u s , 060 0 a s d e c o y 303 ' 074 but the server treats the sequence of 0xC3 (octal 303) and 0x27 (apostrophe) as two different characters. If I change the 0x27 after the 0xC3 to 0xA4 in both cases, the resulting sequence is correctly treated as a single character (German umlaut a). The question: Since neither the apostrophe nor the backslash can be a valid second byte of an UTF-8 sequence, how is it possible to inject code by exploiting an application that escapes quotes in strings and then uses them in queries sent to the server? It seems to me that UTF-8 databases are safe. Yours, Laurenz Albe ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] some question about deadlock
ipig wrote: Hi, Thanks for your reply. I changed the format to plain text. For the question, suppose that process p0 held the lock of object A, and the wait queue for A is p1,p2,p3,, that process p1 is the first waiter in the queue. Since p1 is in the wait queue, the lock p1 requests must be conflict with the lock p0 held. That is to say, if p0 wants to lock A again, then p0 will be put before p1, and p0 will be at the head of the queue. Why do we need to find the first waiter which conflicts p0? I think that p0 must be added at the head of the wait queue. For your example, p0 has a read lock and wants an exclusive lock. Since p0 has a read lock, then in the queue, p1 must wait an exclusive lock. Then p0 will be put before p1, and p0 will be at the head of the queue. Is there anything I misunderstood? I am guessing that p0 is put at the head _only_ if there are conflicting locks so that p0 does not starve other waiting processes. -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] some question about deadlock
Alvaro Herrera wrote: ipig wrote: Hi, Thanks for your reply. I changed the format to plain text. For the question, suppose that process p0 held the lock of object A, and the wait queue for A is p1,p2,p3,, that process p1 is the first waiter in the queue. Since p1 is in the wait queue, the lock p1 requests must be conflict with the lock p0 held. That is to say, if p0 wants to lock A again, then p0 will be put before p1, and p0 will be at the head of the queue. Why do we need to find the first waiter which conflicts p0? I think that p0 must be added at the head of the wait queue. For your example, p0 has a read lock and wants an exclusive lock. Since p0 has a read lock, then in the queue, p1 must wait an exclusive lock. Then p0 will be put before p1, and p0 will be at the head of the queue. Is there anything I misunderstood? You missed this: Note that a process never conflicts with itself, eg one can obtain read lock when one already holds exclusive lock. If p0 is holding a read lock and wants an exclusive lock, it will be granted right away. It will not be put in the waiting queue. Uh, unless other processes also hold a read lock on the object. In that case, p0 has to wait, and I think the description is saying p0 will be put ahead of other readers waiting for the object. -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] some question about deadlock
Hi, an exception: if the process already holds locks on this same lockable object that conflict with the request of any pending waiter, then the process will be inserted in the wait queue just ahead of the first such waiter. From the exception, I got that the process must be added at the head, since the first waiter in the queue must conflict with the lock-held process. Best wishes. - Original Message - From: Bruce Momjian pgman@candle.pha.pa.us To: ipig [EMAIL PROTECTED] Cc: pgsql-hackers@postgresql.org Sent: Monday, May 29, 2006 11:26 PM Subject: Re: [HACKERS] some question about deadlock ipig wrote: Hi, Thanks for your reply. I changed the format to plain text. For the question, suppose that process p0 held the lock of object A, and the wait queue for A is p1,p2,p3,, that process p1 is the first waiter in the queue. Since p1 is in the wait queue, the lock p1 requests must be conflict with the lock p0 held. That is to say, if p0 wants to lock A again, then p0 will be put before p1, and p0 will be at the head of the queue. Why do we need to find the first waiter which conflicts p0? I think that p0 must be added at the head of the wait queue. For your example, p0 has a read lock and wants an exclusive lock. Since p0 has a read lock, then in the queue, p1 must wait an exclusive lock. Then p0 will be put before p1, and p0 will be at the head of the queue. Is there anything I misunderstood? I am guessing that p0 is put at the head _only_ if there are conflicting locks so that p0 does not starve other waiting processes. -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] some question about deadlock
ipig [EMAIL PROTECTED] writes: That is to say, if p0 wants to lock A again, then p0 will be put before p1, and p0 will be at the head of the queue. Why do we need to find the first waiter which conflicts p0? I think that p0 must be added at the head of the wait queue. Your analysis is assuming that there are only two kinds of lock, which is not so. Process A might hold a weak lock and process B a slightly stronger lock that doesn't conflict with A's. In the wait queue there might be process C wanting a lock that conflicts with B's but not A's, followed by process D wanting a strong lock that conflicts with all three. Now suppose A wants to get a lock of the same type D wants. Since this conflicts with B's existing lock, A must wait. A must go into the queue before D (else deadlock) but if possible it should go after C, on fairness grounds. A concrete example here is A has AccessShareLock (reader's lock) B has RowExclusiveLock (writer's lock) C wants ShareLock (hence blocked by B but not A) D wants AccessExclusiveLock (must wait for all three) If A wants to upgrade to AccessExclusiveLock, it *must* queue in front of D, and we'd prefer that it queue behind C not in front of C. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] question about security hole CVE-2006-2313 and UTF-8
Albe Laurenz [EMAIL PROTECTED] writes: It seems to me that UTF-8 databases are safe. IIRC we determined that using UTF8 *on both the client and server sides* is safe. You can get burnt with combinations such as server_encoding = UTF8 and client_encoding = SJIS (exposing PQescapeString's naivete), or with client_encoding = UTF8 and server_encoding = anything else (exposing the server's weak validity checking during conversion). regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] some question about deadlock
Hi, In your example, it seems that process B is the first such waiter( the request of B conflicts AccessShareLock). Best regards. - Original Message - From: Tom Lane [EMAIL PROTECTED] To: ipig [EMAIL PROTECTED] Cc: Bruce Momjian pgman@candle.pha.pa.us; pgsql-hackers@postgresql.org Sent: Monday, May 29, 2006 11:51 PM Subject: Re: [HACKERS] some question about deadlock ipig [EMAIL PROTECTED] writes: That is to say, if p0 wants to lock A again, then p0 will be put before p1, and p0 will be at the head of the queue. Why do we need to find the first waiter which conflicts p0? I think that p0 must be added at the head of the wait queue. Your analysis is assuming that there are only two kinds of lock, which is not so. Process A might hold a weak lock and process B a slightly stronger lock that doesn't conflict with A's. In the wait queue there might be process C wanting a lock that conflicts with B's but not A's, followed by process D wanting a strong lock that conflicts with all three. Now suppose A wants to get a lock of the same type D wants. Since this conflicts with B's existing lock, A must wait. A must go into the queue before D (else deadlock) but if possible it should go after C, on fairness grounds. A concrete example here is A has AccessShareLock (reader's lock) B has RowExclusiveLock (writer's lock) C wants ShareLock (hence blocked by B but not A) D wants AccessExclusiveLock (must wait for all three) If A wants to upgrade to AccessExclusiveLock, it *must* queue in front of D, and we'd prefer that it queue behind C not in front of C. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] some question about deadlock
ipig [EMAIL PROTECTED] writes: In your example, it seems that process B is the first such waiter( the request of B conflicts AccessShareLock). No. Better go study http://developer.postgresql.org/docs/postgres/explicit-locking.html#LOCKING-TABLES After looking at the example again, consider the further assumption that C already has AccessShareLock (which is certainly a valid configuration). Then A *must* queue between C and D; there is no other valid order to grant the requests in. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] Proposal for debugging of server-side stored procedures
Hi everyone,
Having browsed the TODO list, one of the items that I would be interested
on working on is a debugger for stored procedures. Having searched on this
topic in the archives, I'm still short of some answers that would allow me
to come up with a complete proposal that I can use to start coding.
The most important question to answer in my mind is how should the
debugger communicate with the server? I can see 3 ways in which this could
happen:
1) Use the existing FE/BE protocol to allow the user to control
the debugging session using stored procedures/pseudo-tables, e.g.
SELECT pg_debug_enable('myfunction');
INSERT INTO pg_debug_breakpoints (function, line) VALUES
('myfunction', 2);
SELECT pg_debug_step('myfunction');
2) Spawn a separate debugger process listening on another port to
allow OOB communication with the server. This would involve the
design
and implementation of a debug FE/BE protocol, unless there is a
standard that already exists.
3) Extend the existing FE/BE protocol to allow transmission of an
OOB
debug channel.
My current thoughts are leaning towards 2 or 3, with the advantage of
approach 3 being that once you can connect to the database using a client,
the debugging functionality automatically becomes available without having
to worry about additional security checks on the debugger port (like
another version of pg_hba.conf just for the debugger) and firewalls
blocking debugger connections.
Thoughts?
Mark.
WebBased Ltd
17 Research Way
Plymouth
PL6 8BT
T: +44 (0)1752 797131
F: +44 (0)1752 791023
http://www.webbased.co.uk
http://www.infomapper.com
http://www.swtc.co.uk
This email and any attachments are confidential to the intended recipient
and may also be privileged. If you are not the intended recipient please
delete it from your system and notify the sender. You should not copy it
or use it for any purpose nor disclose or distribute its contents to any
other person.
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [HACKERS] Proposal for debugging of server-side stored procedures
Mark Cave-Ayland [EMAIL PROTECTED] writes:
The most important question to answer in my mind is how should the
debugger communicate with the server?
1) Use the existing FE/BE protocol to allow the user to control
the debugging session using stored procedures/pseudo-tables, e.g.
SELECT pg_debug_enable('myfunction');
INSERT INTO pg_debug_breakpoints (function, line) VALUES
('myfunction', 2);
SELECT pg_debug_step('myfunction');
I think this approach is utterly unworkable: it assumes that the
client application is cooperative, and it ignores the fact that the
operations you need to step through are inside a SQL command, which
is not a place where you can easily issue more SQL commands ---
certainly not without horrid damage to the FE/BE protocol.
You mustn't assume a cooperative application, either. Suppose that
you need to debug what your PL functions are doing when driven by a
complicated web application. Injecting debug commands through the
application would require the cooperation of quite a few layers of
code (eg, JDBC, a couple of levels of middleware, etc) and is just
not going to happen.
2) Spawn a separate debugger process listening on another port to
allow OOB communication with the server. This would involve the
design
and implementation of a debug FE/BE protocol, unless there is a
standard that already exists.
The way I usually debug server problems is to attach with GDB to an
already-started server process. This is fairly convenient and requires
no special client support, but it's less than desirable from a security
point of view: you have to be on the server's machine and be the same
user (or root) to be allowed to attach. I would imagine any
direct-attach protocol we might invent would have to have comparable
restrictions, because it's just too hard to tell whether someone
knocking on the door should actually be allowed to debug that specific
session over there.
3) Extend the existing FE/BE protocol to allow transmission of an
OOB
debug channel.
We could get around both the security and the application-cooperation
problems if we design the debugger as an intermediate process that sits
between the application and the server. To debug an app, you tell it to
connect to a host/port where the debugger is listening, and then the
debugger turns around and forwards the connection to the real database.
Now the debugger can watch the FE/BE messages going by, and can inject
debug commands into the data stream and read debug responses without
the application being any the wiser. So basically yeah, what we need
is a debug subchannel in the FE/BE protocol. I'd suggest inventing
a single Debug message type (sendable in both directions) with the
contents being specified by a separate protocol definition. Or perhaps
invert that and imagine the FE/BE protocol as embedded in a debug
protocol.
We might want to play the same game on the server side; that is, the
communication structure is
client - debugger user interface process --
debugger control process - backend
The advantage of this is the backend doesn't need to be explicitly
aware of debug operations, which would be a Good Thing because we
don't want it constantly checking the client port during queries
to see if messages have arrived. The control process would then
have both access to the message stream, and the ability to invoke
trace/step facilities on the backend process.
As far as the debug protocol details go, it'd be worth looking at the
GDB host protocol to see if it's usable; even if not directly usable
(it's probably too low-level) it would give ideas about the functions
you need to provide. It'd also be smart to understand how debugging is
commonly done in Perl, Python, etc, with an eye to being able to
piggyback on existing tools instead of inventing our own.
regards, tom lane
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [HACKERS] anoncvs still slow
On Sun, 28 May 2006, Magnus Hagander wrote: AFAICS, this is caused by the machine attempting to relay thousands and thousands of spam emails (some quick checked showed a rate of about 1 spam / 5 seconds enytering the queue - and I know I deleted almost 20,000 from the queue) And how exactly would you like me to fix *that*? The reason those were in the queue is because svr4 is a legit MX record for the mailing lists ... the messages are being delivered into svr4's mail queue, and mail.postgresql.org subsequently refusing htem because they are for invalid addresses ... If I remove svr4 as an MX record, its just going to move to a different machine ... So, how exactly would you like me to fix that problem? For bittorrent, I propose we take it out. We've suggested it before, I don't recall receiving any real requests to keep it, and IMHO it's way much more pain than it's worth. Therefor, unless someone objects, I'll pull the bittorrent links from the website in a couple of days, and then we can just remove it from the server. That works for me ... let me know once its is down, and then I can easily do the upgrade ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] anoncvs still slow
AFAICS, this is caused by the machine attempting to relay thousands and thousands of spam emails (some quick checked showed a rate of about 1 spam / 5 seconds enytering the queue - and I know I deleted almost 20,000 from the queue) And how exactly would you like me to fix *that*? The reason those were in the queue is because svr4 is a legit MX record for the mailing lists ... the messages are being delivered into svr4's mail queue, and mail.postgresql.org subsequently refusing htem because they are for invalid addresses ... If I remove svr4 as an MX record, its just going to move to a different machine ... So, how exactly would you like me to fix that problem? The complete fix is of course to apply the same ingress filtering on all machines. If that's not possible, do it as much as possible. As the email addresses existing on svr1 is fairly static, it shouldn't be too hard to teach svr4 (and other MXen if there are any) about them. To make graylisting properly effective that also needs to be applied on all entrypoints, otherwise svr4 will just solve the problems for the spammers who have software that won't retry. The quick fix is, as I wrote in one of my earlier mails, to configure svr1 not to tell svr4 to *retry delivery*, but to just junk the mail right away. It'll still cause joe-job style problems, but it won't load up the queue for days. For bittorrent, I propose we take it out. We've suggested it before, I don't recall receiving any real requests to keep it, and IMHO it's way much more pain than it's worth. Therefor, unless someone objects, I'll pull the bittorrent links from the website in a couple of days, and then we can just remove it from the server. That works for me ... let me know once its is down, and then I can easily do the upgrade ... I'll give it a day or two more for people to complain, and then junk it. //Magnus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] anoncvs still slow
On Mon, 29 May 2006, Magnus Hagander wrote: The quick fix is, as I wrote in one of my earlier mails, to configure svr1 not to tell svr4 to *retry delivery*, but to just junk the mail right away. It'll still cause joe-job style problems, but it won't load up the queue for days. But, from my look at the queue on svr4, this is already being done ... the queue contains a bunch of MAILER-DAEMON bounces back for 'recipient unknown', which is what is supposed to happen ... but, your point about the greylisting makes sense ... will work on implementing that one tonight ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] anoncvs still slow
The quick fix is, as I wrote in one of my earlier mails, to configure svr1 not to tell svr4 to *retry delivery*, but to just junk the mail right away. It'll still cause joe-job style problems, but it won't load up the queue for days. But, from my look at the queue on svr4, this is already being done ... the queue contains a bunch of MAILER-DAEMON bounces back for 'recipient unknown', which is what is supposed to happen ... That's because I've deleted thousands of emails already, and run the delete script once every hour or so in order to keep it living. (I bet your mailq command didn't take almost an hour - that's what it did when I ran it this morning) Run something like: mailq | grep Recipient address rejected This will currently show 283 emails, all backed to svr1. To clean up the queue (of this type of emails only), run mailq |./t.pl |postsuper -d - from roots homedir. The mails you are seeing are the ones generated after the other ones have been sitting in the queue for a couple of days. They were also in the thousands before, but since I try to cut down the queue at every chance I get now, it usually doesn't get that far, so they don't increase that much. but, your point about the greylisting makes sense ... will work on implementing that one tonight ... Great. //Magnus ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] anoncvs still slow
On Mon, 29 May 2006, Magnus Hagander wrote: The quick fix is, as I wrote in one of my earlier mails, to configure svr1 not to tell svr4 to *retry delivery*, but to just junk the mail right away. It'll still cause joe-job style problems, but it won't load up the queue for days. But, from my look at the queue on svr4, this is already being done ... the queue contains a bunch of MAILER-DAEMON bounces back for 'recipient unknown', which is what is supposed to happen ... That's because I've deleted thousands of emails already, and run the delete script once every hour or so in order to keep it living. (I bet your mailq command didn't take almost an hour - that's what it did when I ran it this morning) Run something like: mailq | grep Recipient address rejected I thought that the above was supposed to be a perm error, not temp? Does anyone know what I need to set in postfix on svr1 to change it to a perm? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Proposal for debugging of server-side stored procedures
Tom Lane wrote: Mark Cave-Ayland [EMAIL PROTECTED] writes: ... So basically yeah, what we need is a debug subchannel in the FE/BE protocol. I'd suggest inventing a single Debug message type (sendable in both directions) with the contents being specified by a separate protocol definition. Or perhaps invert that and imagine the FE/BE protocol as embedded in a debug protocol. I think this is a bad idea. PL/Java will use either shared memory or a socket to attach and as you already mentioned, when using C, a gdb will attach directly using the pid. I wouldn't be too surprised if Perl, Python, and PHP all have a similar solution and thus have no benefit from additions to the FE/BE protocol. IMO, debugging should be language specific and take place in a separate channel. There's no gain whatsoever mixing it with the FE/BE protocol. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] anoncvs still slow
The quick fix is, as I wrote in one of my earlier mails, to configure svr1 not to tell svr4 to *retry delivery*, but to just junk the mail right away. It'll still cause joe-job style problems, but it won't load up the queue for days. But, from my look at the queue on svr4, this is already being done ... the queue contains a bunch of MAILER-DAEMON bounces back for 'recipient unknown', which is what is supposed to happen ... That's because I've deleted thousands of emails already, and run the delete script once every hour or so in order to keep it living. (I bet your mailq command didn't take almost an hour - that's what it did when I ran it this morning) Run something like: mailq | grep Recipient address rejected I thought that the above was supposed to be a perm error, not temp? Does anyone know what I need to set in postfix on svr1 to change it to a perm? Yes, htat's what I sent before :-) c) Change svr1 parameters to: unknown_relay_recipient_reject_code = 550 and unknown_local_recipient_reject_code = 550 //Magnus ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] anoncvs still slow
On Mon, May 29, 2006 at 03:00:44PM -0300, Marc G. Fournier wrote: Run something like: mailq | grep Recipient address rejected I thought that the above was supposed to be a perm error, not temp? Does anyone know what I need to set in postfix on svr1 to change it to a perm? Do you have soft bounce turned on? A mailbox unavailable message should be a 550 error. Or is the problem that the relay gets back the rejection, and then queues a message to the originating mailer (which was, of course, a drone, and therefore won't be available)? The latter case will _also_ eat up a ton of resources. Unfortunately, there's no standards-compliant way to drop such connections on the floor instead of erroring, once you've accepted the mail. Better to reject at the time of connection, which is what the local_recipient_maps setting is for. A -- Andrew Sullivan | [EMAIL PROTECTED] In the future this spectacle of the middle classes shocking the avant- garde will probably become the textbook definition of Postmodernism. --Brad Holland ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] anoncvs still slow
Run something like: mailq | grep Recipient address rejected I thought that the above was supposed to be a perm error, not temp? Does anyone know what I need to set in postfix on svr1 to change it to a perm? Yes, htat's what I sent before :-) c) Change svr1 parameters to: unknown_relay_recipient_reject_code = 550 and unknown_local_recipient_reject_code = 550 By the way, the proper way to fix it it o use a relay_recipient_map. To this map, add all the users that are valid in postgresql.org, and install it on svr4. Then svr4 will reject the connections *before* it queues up the mail, and it'll also get rid of the incorrect bounces. //Magnus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Proposal for debugging of server-side stored procedures
Thomas Hallgren [EMAIL PROTECTED] writes: I think this is a bad idea. PL/Java will use either shared memory or a socket to attach and as you already mentioned, when using C, a gdb will attach directly using the pid. I wouldn't be too surprised if Perl, Python, and PHP all have a similar solution and thus have no benefit from additions to the FE/BE protocol. IMO, debugging should be language specific and take place in a separate channel. There's no gain whatsoever mixing it with the FE/BE protocol. It may well be that for plperl and friends we can kick the problem off to language-specific debuggers --- indeed, one of the first things we need to do is look at those to see what we can avoid reinventing. But what of plpgsql? Also, any solution of this type probably requires that the person doing debugging have database superuser access (in fact, be logged directly into the server machine as the postgres user). It'd be nice to have an approach that could be used by non-superusers to debug their trusted-PL functions. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Proposal for debugging of server-side stored procedures
Tom Lane wrote: Thomas Hallgren [EMAIL PROTECTED] writes: I think this is a bad idea. PL/Java will use either shared memory or a socket to attach and as you already mentioned, when using C, a gdb will attach directly using the pid. I wouldn't be too surprised if Perl, Python, and PHP all have a similar solution and thus have no benefit from additions to the FE/BE protocol. IMO, debugging should be language specific and take place in a separate channel. There's no gain whatsoever mixing it with the FE/BE protocol. It may well be that for plperl and friends we can kick the problem off to language-specific debuggers --- indeed, one of the first things we need to do is look at those to see what we can avoid reinventing. But what of plpgsql? Ideally, all pl's should use the same protocol. It should be language agnostic and allow different regions of the code to origin from different languages. That way, it would be possible to single step a plpgsql function that in turn calls a function in pljava. Incidentally, the JDWP (Java Debug Wire Protocol) was designed to do just that. But I think it would be very complicated to cross language boundaries even if we did use that. The JDWP and the architecture that surrounds it might be a good source for inspiration though. See: http://java.sun.com/j2se/1.5.0/docs/guide/jpda/architecture.html. Also, any solution of this type probably requires that the person doing debugging have database superuser access (in fact, be logged directly into the server machine as the postgres user). It'd be nice to have an approach that could be used by non-superusers to debug their trusted-PL functions. Indeed. In my case, it's a matter of who starts the VM and what options that are passed to it (given certain options, the JVM will listen to a port or a semaphore that controls a region of shared memory). That in turn is controlled using GUC settings so for PL/Java I think it would be possible to set it up that way. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] Compile libpq with vc8
Hi,
I am trying to compile libpq with vc8. I got errors in wchar.c
..\..\backend\utils\mb\wchar.c(99) : error C2054: expected '(' to follow
'inline'
..\..\backend\utils\mb\wchar.c(100) : error C2085: 'pg_euc_mblen' : not
in formal parameter list
..\..\backend\utils\mb\wchar.c(100) : error C2143: syntax error :
missing ';' before '{'
If I remove the inline, I got an error in:
[...]postgresql-8.1.4\src\include\utils/elog.h(105) : error C2365:
'errcode' : redefinition; previous definition was 'typedef'
Does anyone already compiled libpq with vc8?
Yannick.
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Compile libpq with vc8
Yes, I have a patch in the queue to fix it. Should be applied soon.
---
Yannick wrote:
Hi,
I am trying to compile libpq with vc8. I got errors in wchar.c
..\..\backend\utils\mb\wchar.c(99) : error C2054: expected '(' to follow
'inline'
..\..\backend\utils\mb\wchar.c(100) : error C2085: 'pg_euc_mblen' : not
in formal parameter list
..\..\backend\utils\mb\wchar.c(100) : error C2143: syntax error :
missing ';' before '{'
If I remove the inline, I got an error in:
[...]postgresql-8.1.4\src\include\utils/elog.h(105) : error C2365:
'errcode' : redefinition; previous definition was 'typedef'
Does anyone already compiled libpq with vc8?
Yannick.
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
--
Bruce Momjian http://candle.pha.pa.us
EnterpriseDBhttp://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
[HACKERS] type recv/send functions
Is it just me or are the send/recv strangely asymmetric? It seems like the recv function is designed to avoid copying so the type can pick the data straight out of the data stream without passing through intermediate representations. But the send function forces the type to copy the data into a temporary bytea, which is presumably then copied into the actual data stream. Wouldn't the natural thing to do be to provide the StringInfo buffer with a cursor for the type's send function to stuff the bytea into? -- greg ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Proposal for debugging of server-side stored procedures
Thomas Hallgren wrote: Ideally, all pl's should use the same protocol. It should be language agnostic and allow different regions of the code to origin from different languages. That way, it would be possible to single step a plpgsql function that in turn calls a function in pljava. Incidentally, the JDWP (Java Debug Wire Protocol) was designed to do just that. But I think it would be very complicated to cross language boundaries even if we did use that. The JDWP and the architecture that surrounds it might be a good source for inspiration though. See: http://java.sun.com/j2se/1.5.0/docs/guide/jpda/architecture.html. I have no idea if this would fit .. but just to throw something into the mix that was designed to be somewhat language agnostic (with a slant towards web scripting languages): http://xdebug.org/docs-dbgp.php If it seems like it could fit I can create contact with the two authors Derick and Shane to discuss this further. regards, Lukas ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] osprey buildfarm member has been failing for a long while
Le 28 mai 06 à 17:42, Tom Lane a écrit : =?ISO-8859-1?Q?R=E9mi_Zara?= [EMAIL PROTECTED] writes: Tom Lane wrote: Perhaps the swap space or ulimit setting on the box needs to be raised? What kind of ulimit did you think of ? I'll try upping the data segment size. Yeah, data segment size would be the most likely culprit if this is a ulimit thing. Changing this limit and removing ccache made the trick. Next run will try and re-enable ccache (this build lasted nearly 11.5 hours :-) Regards, Rémi Zara smime.p7s Description: S/MIME cryptographic signature
Re: [HACKERS] pg_proc probin misuse
Is there a TODO here? --- Tom Lane wrote: PFC [EMAIL PROTECTED] writes: If it were really expensive to derive bytecode from source text then maybe it'd make sense to do what you're doing, but surely that's not all that expensive. Everyone else manages to parse prosrc on the fly and cache the result in memory; why isn't plpython doing that? It depends on the number of imported modules in the function. If it imports a lot of modules, it can take some time to compile a python function (especially if the modules have some initialisation code which must be run on import). Surely the initialization code would have to be run anyway ... and if the function does import a pile of modules, do you really want to cache all that in its pg_proc entry? What happens if some of the modules get updated later? regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] pg_proc probin misuse
Hm, thinking again, I guess Tom Lane is right Surely the initialization code would have to be run anyway ... and if the function does import a pile of modules, do you really want to cache all that in its pg_proc entry? What happens if some of the modules get updated later? Besides, what happens if you store compiled bytecode in a table, then upgrade the python interpreter to a new version... would it be compatible ? I suppose so, but I don't really know... Persistent connections should be used anyway, this makes the RAM caching good... ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Inefficient bytea escaping?
Marko Kreen wrote: On 5/28/06, Martijn van Oosterhout kleptog@svana.org wrote: With -lpthread lock.enabled 323s lock.disabled 50s lock.unlocked 36s I forgot to test with -lpthread, my bad. Indeed by default something less expensive that full locking is going on. The crux of the matter is though, if you're calling something a million times, you're better off trying to find an alternative anyway. There is a certain amount of overhead to calling shared libraries and no amount of optimisation of the library is going save you that. The crux of the matter was if its possible to use fwrite as easy string combining mechanism and the answer is no, because it's not lightweight enough. So your patch to src/backend/storage/file/fd.c should be discarded? OK. -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Further reduction of bufmgr lock contention
Is this a TODO? --- Tom Lane wrote: Josh Berkus josh@agliodbs.com writes: BTW, we're going to be testing this patch on Sun Niagara servers. What's the outstanding bug with it? I don't quite follow. It's not acceptable as-is because of the risk of running out of shared memory for hashtable entries. In the existing code, there's a clear upper bound on the number of entries in the block-number-to-buffer hash table, ie, shared_buffers + 1 (the +1 because we acquire the new entry before releasing the old when reassigning a buffer). With multiple hashtables serving subsets of the buffers, the different tables might at different times need different numbers of entries, and that makes it a lot harder to be sure you won't run out of memory. I don't say it's insoluble, but the current patch wasn't even claimed to be safe by its author... regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] pg_resetxlog -f flag
A month ago, pg_resetxlog got an -f flag: -fforce reset xlog to be done, if the control file is corrupted, then try to restore it. However, the patch had identifiable problems that are still not fixed: http://archives.postgresql.org/pgsql-hackers/2006-05/msg00013.php Unless these problems are fixed in the next few days, the patch will be reverted. -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] type recv/send functions
Greg Stark [EMAIL PROTECTED] writes: Is it just me or are the send/recv strangely asymmetric? Not all that much: they both return a meaningful result. We cheated a little bit by allowing the recv functions to modify the state of their input argument, but they still deliver a valid result object. Wouldn't the natural thing to do be to provide the StringInfo buffer with a cursor for the type's send function to stuff the bytea into? Then the send function would return void, which is surely not more natural. Also, there would be a much larger disparity between the behaviors of the text and binary output paths. Anyway it's about three years too late to be debating this ;-) regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Altering view ownership doesn't work ...
Martijn van Oosterhout wrote: -- Start of PGP signed section. On Sun, Apr 30, 2006 at 12:34:42PM -0400, Tom Lane wrote: 2. Run setRuleCheckAsUser during rule load rather than rule store. #2 is a lot simpler, and would fix the problem for existing broken rules whereas #1 would not, so I'm kind of inclined to go with that. I doubt there'd be any meaningful performance hit --- parsing the stored form of a rule is relatively expensive anyway, so we cache the results. FWIW, I think #2 is better also. It's the easiest way to ensure the correct result and the performence isn't enough of a problem to worry about doing it a different way. Has this been completed? -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Altering view ownership doesn't work ...
Bruce Momjian pgman@candle.pha.pa.us writes: Has this been completed? No, still on the to-do list. IIRC that was about the time we got diverted by fixing the encoding security issues... regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] type recv/send functions
Tom Lane [EMAIL PROTECTED] writes: Stark [EMAIL PROTECTED] writes: Is it just me or are the send/recv strangely asymmetric? Not all that much: they both return a meaningful result. We cheated a little bit by allowing the recv functions to modify the state of their input argument, but they still deliver a valid result object. both return something seems like an odd axis to measure. In one case it's given pointer to the entire message, picks out the piece it's interested in and advances the cursor. I would expect the complementary function to get a pointer to the entire message, take the part it's responsible for and stuff it into that buffer advancing the cursor. It stands out to me because the recv api seems like it's intentionally designed around avoiding the extra copy. Then to see the send function not make any effort in the same direction seems odd. What I'm pondering here is that the extra copy to construct the bytea for every single data type being output seems like it would be a pretty big contribution to the complaint that postgres takes too much cpu in cases that should be entirely i/o bound. Anyway it's about three years too late to be debating this ;-) I suppose. Though it doesn't seem like it would be impossible to allow both the easy form that returns a bytea and the high performance zero-copy form. If there was a similar easy form for recv then it would be more feasible to implement data types without C programming too. I guess that's nowhere until I figure out a way to profile all these send functions separately though. -- greg ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] type recv/send functions
Greg Stark [EMAIL PROTECTED] writes: both return something seems like an odd axis to measure. In one case it's given pointer to the entire message, picks out the piece it's interested in and advances the cursor. This is just a trivial optimization compared to being handed a bytea input, which would be the clean version. (I had originally thought we could fake a bytea input without any copying, much as is done in the text input path, but that fails on machines that are persnickety about alignment: the bytea length word might not be word-aligned depending on message contents.) What I'm pondering here is that the extra copy to construct the bytea for every single data type being output seems like it would be a pretty big contribution to the complaint that postgres takes too much cpu in cases that should be entirely i/o bound. Since approximately zero percent of the people making that complaint are using binary output, I don't think it matters. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
