Re: [PERFORM] [HACKERS] high shared buffer and swap

2009-05-04 Thread Martijn van Oosterhout
et the kernel share the remaining memory between postgresql disk cache and other processes I might be running. On a NUMA machine you want to be keeping your memory on the local node and letting the kernel copy that data from elsewhere to your local memory when you need it. Have a nice day, -- Martij

Re: [PERFORM] [GENERAL] Shared memory usage

2007-08-25 Thread Martijn van Oosterhout
the system manage the memory itself, if it needs it, it'll use it. Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to > litigate. signature.asc Description: Digital signature

Re: [PERFORM] [HACKERS] EXISTS optimization

2007-03-24 Thread Martijn van Oosterhout
rest > of the class here.) You're right, I'm getting confused with the interaction of NULL and NOT IN. The multiple evaluation thing still applies, but that's minor. Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > From each according to his

Re: [PERFORM] [HACKERS] EXISTS optimization

2007-03-23 Thread Martijn van Oosterhout
irst row? That would mean the subquery to be materialised would not be equivalent if it called any non-immutable functions. It's also much less clear to be a win in the EXISTs case. But then, that's a costs issue the planner can deal with... Have a nice day, -- Martijn van Oosterhout

Re: [PERFORM] [HACKERS] Questions about planner methods

2006-12-24 Thread Martijn van Oosterhout
EADME's that explain these methods? There's plenty of the comments in the files that implement them (the executor directory. Have you checked them? Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > From each according to his ability. To each accord

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Martijn van Oosterhout
On Fri, Dec 15, 2006 at 09:56:57AM -0500, Tom Lane wrote: > Martijn van Oosterhout writes: > > On Fri, Dec 15, 2006 at 12:20:46PM +, Simon Riggs wrote: > >> Maybe sampling every 10 rows will bring things down to an acceptable > >> level (after the first N). You tr

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Martijn van Oosterhout
hat was to do with alarms and signals. On my system the manpage say setitimer() conforms to: SVr4, 4.4BSD. I'm unsure how many of the supported platforms fall under that catagorisation. Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to > litigate. signature.asc Description: Digital signature

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Martijn van Oosterhout
ach. setitimer is good, except it doesn't tell you if signals have been lost. Given they are most likely to be lost during high disk I/O, they're actually significant. I'm trying to think of a way around that. Then you don't need a cheap gettimeofday at all... Have a nice day, --

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Martijn van Oosterhout
counter but it has the problem that if you have multiple CPUs you need to calibrate the result. If the CPU goes to sleep, there's is no way for the userspace process to know. Only the kernel has all the relevent information about what "time" is to get a reasonable result. Have a nic

Re: [GENERAL] [PERFORM] Locking in PostgreSQL?

2006-12-06 Thread Martijn van Oosterhout
, writers don't block readers, so even if someone executes an update on a row, readers (SELECT statements) will not be blocked. So it's not a lock as such, more a "I've updated this row, go find the new version if that's appropriate for your snapshot". Have a nice

Re: [PERFORM] [HACKERS] Big IN() clauses etc : feature proposal

2006-05-11 Thread Martijn van Oosterhout
may not be what the standard means, but it still seems like a useful idea, to cut down on schema bloat. Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to > litigate. signature.asc Description: Digital signature

Re: [HACKERS] [PERFORM] Big IN() clauses etc : feature proposal

2006-05-11 Thread Martijn van Oosterhout
u start a savepoint you start a new transaction ID whose status is tied to its top-level transaction ID but can also be individually rolledback. Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to > litigate. signature.asc Description: Digital signature

Re: [PERFORM] [HACKERS] Big IN() clauses etc : feature proposal

2006-05-11 Thread Martijn van Oosterhout
t for create table, is that what we expect ? Hmm, I'm hoping ms means milliseconds... -- Martijn van Oosterhout http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to > litigate. signature.asc Description: Digital signature

Re: [HACKERS] [PERFORM] Big IN() clauses etc : feature proposal

2006-05-10 Thread Martijn van Oosterhout
rn between one of 1-10, 10-100, 100-1000, etc. You don't need exact number, you just need to get within an order of magnitude and a constant will work fine for that. How many functions sometimes return one and sometimes a million rows? Have a nice day, -- Martijn van Oosterhout http:/

Re: [HACKERS] [PERFORM] Big IN() clauses etc : feature proposal

2006-05-09 Thread Martijn van Oosterhout
end exit. They're about as close to in-memory tables as you're going to get... Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to > litigate. signature.asc Description: Digital signature

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-17 Thread Martijn van Oosterhout
ch a function and always return zero, the results will still be right. Not a new idea, but it would be very nice to implement. If would produce nice speedups for type where comparisons are expensive. And more importantly, the bulk of the comparisons can be moved inline and make the whole cac

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create

2006-02-17 Thread Martijn van Oosterhout
g operation once, we should not ever need to do it > again on the original data. Else most sorting algorithms wouldn't work ;-) We already do this with btree indexes. I'm not sure what you are proposing that improves on that. Have a nice day, -- Martijn van Oosterhout http://svana.

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Martijn van Oosterhout
n many ways and the sortKey must reflect that. So in postgres terms it's a property of the btree operator class. It's something I'd like to do if I get A Round Tuit. :) Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create

2006-02-16 Thread Martijn van Oosterhout
very rarely needs the tuples actualised in memory in the required order, just the pointers are enough. Have a ncie day, -- Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sit

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Martijn van Oosterhout
se algorithms much more applicable in my mind. Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do t

Re: [PERFORM] [HACKERS] qsort again

2006-02-16 Thread Martijn van Oosterhout
comparisons (which could easily swamp the time taken to do the actual sort in some cases)? Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around w

Re: [PERFORM] [GENERAL] Creation of tsearch2 index is very slow

2006-01-21 Thread Martijn van Oosterhout
al-world data, would you be willing to try some alternate picksplit algorithms to see if your gevel module shows any difference? Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5%

Re: [PERFORM] [GENERAL] Creation of tsearch2 index is very slow

2006-01-21 Thread Martijn van Oosterhout
nate algorithm I posted should produce equivalent results. No idea how to test it though... Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting a

Re: [PERFORM] [GENERAL] Creation of tsearch2 index is very slow

2006-01-21 Thread Martijn van Oosterhout
= $d; $m1 = $i; $m2 = $m; } } return($m1,$m2,$dist); } Full program available at: http://svana.org/kleptog/temp/picksplit.pl Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is

Re: [PERFORM] [GENERAL] Creation of tsearch2 index is very slow

2006-01-20 Thread Martijn van Oosterhout
On Fri, Jan 20, 2006 at 05:46:34PM -0500, Ron wrote: > At 04:37 PM 1/20/2006, Martijn van Oosterhout wrote: > >Given that all it's doing is counting bits, a simple fix would be to > >loop over bytes, use XOR and count ones. For extreme speedup create a > >lookup table wi

Re: [PERFORM] [GENERAL] Creation of tsearch2 index is very slow

2006-01-20 Thread Martijn van Oosterhout
s.postgresql.org/pgsql-hackers/2005-11/msg01069.php Hope this helps, -- Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone &g

Re: [PERFORM] [GENERAL] Creation of tsearch2 index is very slow

2006-01-20 Thread Martijn van Oosterhout
te a lookup table with 256 entries to give you the answer straight away... Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting

Re: [PERFORM] [GENERAL] Creation of tsearch2 index is very slow

2006-01-20 Thread Martijn van Oosterhout
mansec?1+gprof It's wierd how some man pages for gprof describe this feature, but the one on my local system doesn't mention it. > I'll see if I can link tsearch2 statically to resolve this question. That'll work too... -- Martijn van Oosterhout http://svana

Re: [PERFORM] [GENERAL] Creation of tsearch2 index is very slow

2006-01-20 Thread Martijn van Oosterhout
arch functions. I'm not 100% familiar with gprof, but is it possible those costs have been added somewhere else because it's in a shared library? Perhaps the costs went into FunctionCall1/3? Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius i

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-10-04 Thread Martijn van Oosterhout
process is probably CPU bound more than anything else. So, I don't think physical I/O is the problem. It's something further up the call tree. I wouldn't be surprised at all it it had to do with the creation and destruction of tuples. The cost of comparing tuples should not be under