Re: [PERFORM] Weird index or sort behaviour

2009-11-18 Thread Matthew Wakeling
On Sat, 14 Nov 2009, Tom Lane wrote: Matthew Wakeling matt...@flymine.org writes: [ discussion about applying materialize to a mergejoin's inner indexscan ] I have finally gotten round to doing something about this, and applied the attached patch to CVS HEAD. Could you test it on your

Re: [PERFORM] Too much blocks read

2009-11-18 Thread Matthew Wakeling
to read in all the rows where field1 = 'my_key', so that they can be sorted, but the sort will be completely unpredictable because all the values will be the same. If you wanted to grab any row, then remove the ORDER BY, and it will just return the first one it finds. Matthew -- The best way

Re: [PERFORM] database size growing continously

2009-10-29 Thread Matthew Wakeling
fairly soon, it is better to just VACUUM, or autovacuum. Just make sure the free space map can cope with it. Matthew -- import oz.wizards.Magic; if (Magic.guessRight())... -- Computer Science Lecturer -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org

Re: [PERFORM] Postgresql optimisation

2009-10-28 Thread Matthew Wakeling
it, causing a surprising delay at unpredictable times (although heavier near the start of the day). There *is* a benefit of running VACUUM ANALYSE rather than just ANALYSE. Matthew -- I suppose some of you have done a Continuous Maths course. Yes? Continuous Maths? menacing stares from audience Whoah

Re: [PERFORM] Issues with \copy from file

2009-10-19 Thread Matthew Wakeling
in the WAL, effectively writing the data twice. As Euler said, the trick is to tell Postgres that noone else will need to see the data, so it can skip the WAL step: BEGIN; TRUNCATE TABLE foo; COPY foo FROM ...; COMMIT; I see upward of 100MB/s over here when I do this. Matthew -- Patron: I

Re: [PERFORM] Query performance

2009-10-12 Thread Matthew Wakeling
if there are multiple rows with the same value. Does that answer your question? Matthew -- The only secure computer is one that's unplugged, locked in a safe, and buried 20 feet under the ground in a secret location...and i'm not even too sure about that one. --Dennis

Re: [PERFORM] Query performance

2009-10-12 Thread Matthew Wakeling
was used, and whether it was a disc sort or an in-memory sort. As it is only an EXPLAIN, the query hasn't actually been run, and we have no information about whether the sort would be performed on disc or not. Matthew -- Hi! You have reached 555-0129. None of us are here to answer the phone

Re: [PERFORM] Best suiting OS

2009-10-08 Thread Matthew Wakeling
stable release is scheduled for the new year. If you want the latest and greatest, then you can use Debian testing. Matthew -- The surest protection against temptation is cowardice. -- Mark Twain -- Sent via pgsql-performance mailing list (pgsql

Re: [PERFORM] position in DDL of columns used in indexes

2009-10-08 Thread Matthew Wakeling
the references in the actual row. Therefore, the order of columns should not matter. Moreover, whether a row is used in an index should not make any difference. The index stores the values too, right? Postgres will look up in the index, and then fetch the rows, in two separate operations. Matthew

[PERFORM] Query plan for NOT IN

2009-10-05 Thread Matthew Wakeling
rows=26341274 width=4) (4 rows) This query plan seems to me to be a little slow. Surely it could iterate through the ten project rows and perform ten index lookups in the big table? Matthew -- Riker: Our memory pathways have become accustomed to your sensory input. Data: I understand - I'm

Re: [PERFORM] Best suiting OS

2009-10-02 Thread Matthew Wakeling
On Thu, 1 Oct 2009, Greg Smith wrote: On Thu, 1 Oct 2009, Matthew Wakeling wrote: For comparison, with Red Hat, you will need to upgrade to a whole new distribution whenever you want updated software, which is a much bigger undertaking. This is somewhat true for larger packages, but it's

Re: [PERFORM] CPU cost of operators

2009-10-01 Thread Matthew Wakeling
On Wed, 30 Sep 2009, Robert Haas wrote: Er, wait... if you set the 'COST' parameter for the backing function, does that work? Ah, right. I was looking at CREATE OPERATOR, not CREATE FUNCTION. Thanks, Matthew -- Bashir: The point is, if you lie all the time, nobody will believe you, even

Re: [PERFORM] Best suiting OS

2009-10-01 Thread Matthew Wakeling
new distribution whenever you want updated software, which is a much bigger undertaking. As far as performance goes (and someone will probably contradict me here) all the different Linux distributions should be roughly equivalent, as long as they are up to date and well tuned. Matthew

Re: [PERFORM] long running insert statement

2009-10-01 Thread Matthew Wakeling
is quite reasonable for what you are asking it to do. To fix this, I suggest creating an index on NotReceivedTransport(SId, CId, ShipperTransportNumber). Then, the index will be able to immediately see that there are no rows to delete. Matthew -- We have always been quite clear that Win95

[PERFORM] CPU cost of operators

2009-09-30 Thread Matthew Wakeling
that the EXPLAINs are correct. Is there a way to tell Postgres that an operator has a large CPU cost? I can tell it what the join selectivity is, but I can't find anything about CPU cost. Matthew -- Unfortunately, university regulations probably prohibit me from eating small children in front of the lecture

Re: [PERFORM] query memory consumption

2009-09-22 Thread Matthew Wakeling
a query will not fit neatly into work_mem. At this point, Postgres will write the data to temporary files on disc. It is harder to predict what size those will be. However, EXPLAIN ANALYSE will sometimes give you a figure of how big a sort was for example. Matthew -- Reality is that which

Re: [PERFORM] Fwd: How to create a multi-column index with 2 dates using 'gist'?

2009-08-26 Thread Matthew Wakeling
floating point values, and bioseg (available from http://www.bioinformatics.org/bioseg/wiki/ which I am maintaining at the moment) will index integers. Matthew -- The early bird gets the worm. If you want something else for breakfast, get up later. -- Sent via pgsql-performance mailing list (pgsql

Re: [PERFORM] Indexing on a circle datatype

2009-08-24 Thread Matthew Wakeling
beneficial. Alternatively, if you really want to force its hand (just for testing purposes), then try running: SET enable_seqscan TO off; and see what happens. Matthew -- When I first started working with sendmail, I was convinced that the cf file had been created by someone bashing their head

Re: [PERFORM] Weird index or sort behaviour

2009-08-19 Thread Matthew Wakeling
table were cached already. Matthew, how big is this table compared to your RAM? Were you testing a case in which it'd be in cache? Oh, definitely. I have run this test so many times, it's all going to be in the cache. Luckily, that's what we are looking at as a normal situation in production

Re: [PERFORM] PG 8.3 and server load

2009-08-19 Thread Matthew Wakeling
? Matthew -- Surely the value of C++ is zero, but C's value is now 1? -- map36, commenting on the No, C++ isn't equal to D. 'C' is undeclared [...] C++ should really be called 1 response to C++ -- shouldn't it be called D? -- Sent via pgsql-performance mailing list (pgsql-performance

[PERFORM] Weird index or sort behaviour

2009-08-18 Thread Matthew Wakeling
that it uses this plan. I'm using Postgres 8.4.0 Matthew -- Interwoven alignment preambles are not allowed. If you have been so devious as to get this message, you will understand it, and you deserve no sympathy. -- Knuth, in the TeXbook -- Sent via pgsql-performance mailing list (pgsql

Re: [PERFORM] Weird index or sort behaviour

2009-08-18 Thread Matthew Wakeling
On Tue, 18 Aug 2009, Tom Lane wrote: Matthew Wakeling matt...@flymine.org writes: I'm seeing some interesting behaviour. I'm executing a query where I perform a merge join between two copies of the same table, completely symmetrically, and the two sides of the merge are sourced differently

Re: [PERFORM] Weird index or sort behaviour

2009-08-18 Thread Matthew Wakeling
BY objectid, bin) as a; ?column? - 57.5270393085641029 So on average, we will be rewinding by 57 rows each time. A materialise step really does sound like a win in this situation. Matthew -- Patron: I am looking for a globe of the earth. Librarian: We have a table-top

Re: [PERFORM] Memory reporting on CentOS Linux

2009-08-17 Thread Matthew Wakeling
than process memory in the totals. +1 on the idea that Linux memory reporting is incomprehensible nowadays. Matthew -- There once was a limerick .sig that really was not very big It was going quite fine Till it reached the fourth line -- Sent via pgsql-performance mailing list (pgsql-performance

[PERFORM] How to run this in reasonable time:

2009-08-13 Thread Matthew Wakeling
. Matthew -- If you're thinking Oh no, this lecturer thinks Turing Machines are a feasible method of computation, where's the door?, then you are in luck. There are some there, there, and by the side there. Oxygen masks will not drop from the ceiling... -- Computer

Re: [PERFORM] How to run this in reasonable time:

2009-08-13 Thread Matthew Wakeling
On Thu, 13 Aug 2009, Greg Stark wrote: On Thu, Aug 13, 2009 at 3:16 PM, Matthew Wakelingmatt...@flymine.org wrote: Now, I'd like to get this done this side of Christmas, so I was wondering if there's a neat trick I can use to get it to only consider the rows from s to e, instead of having

Re: [PERFORM] GiST, caching, and consistency

2009-08-05 Thread Matthew Wakeling
The location_object_bioseg index is 182 MB Matthew -- What goes up must come down. Ask any system administrator. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] GiST, caching, and consistency

2009-08-05 Thread Matthew Wakeling
On Wed, 5 Aug 2009, Robert Haas wrote: On Wed, Aug 5, 2009 at 6:20 AM, Matthew Wakelingmatt...@flymine.org wrote: It is certainly doing a sequential scan. So are you saying that it will start a sequential scan from a different part of the table each time, even in the absence of other

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-07-30 Thread Matthew Wakeling
. However, certainly for index creation, tests on servers over here have indicated that running four CREATE INDEX statements at the time runs four times as fast, assuming the table fits in maintenance_work_mem. Matthew -- I have an inferiority complex. But it's not a very good one. -- Sent via

Re: [PERFORM] hyperthreaded cpu still an issue in 8.4?

2009-07-29 Thread Matthew Wakeling
. Having said that, memory access latency is not scaling as quickly as CPU speed, so over time SMT is going to get more important. Matthew -- Take care that thou useth the proper method when thou taketh the measure of high-voltage circuits so that thou doth not incinerate both thee and the meter

Re: [PERFORM] hyperthreaded cpu still an issue in 8.4?

2009-07-28 Thread Matthew Wakeling
an explicit communication channel at all (I'm simplifying massively here). A real interconnect is only needed between CPUs and between different cores on a CPU, and of course to the outside world. Scott's explanation of why SMT works better now is much more likely to be the real reason. Matthew

Re: [PERFORM] select query performance question

2009-07-27 Thread Matthew Wakeling
the whole query to complete within a few milliseconds. Matthew -- And why do I do it that way? Because I wish to remain sane. Um, actually, maybe I should just say I don't want to be any worse than I already am. - Computer Science Lecturer -- Sent via pgsql-performance mailing list (pgsql

Re: [PERFORM] Calling conventions

2009-07-21 Thread Matthew Wakeling
image format, even if it is cached. The java program uses as near an on-disc format as Postgres does - just held in memory instead of in OS cache. Matthew -- Okay, I'm weird! But I'm saving up to be eccentric. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org

Re: [PERFORM] Full text search with ORDER BY performance issue

2009-07-21 Thread Matthew Wakeling
structure is just too general, because it needs to cope with the situation where a particular object type does not have a well defined order, or where the order is unuseful for indexing. Matthew -- A good programmer is one who looks both ways before crossing a one-way street. Considering

Re: [PERFORM] Calling conventions

2009-07-20 Thread Matthew Wakeling
. Matthew -- What goes up must come down. Ask any system administrator. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

[PERFORM] Calling conventions

2009-07-17 Thread Matthew Wakeling
I'm considering rewriting a postgres extension (GiST index bioseg) to make it use version 1 calling conventions rather than version 0. Does anyone have any ideas/opinions/statistics on what the performance difference is between the two calling conventions? Matthew -- Patron: I am looking

Re: [PERFORM] Calling conventions

2009-07-17 Thread Matthew Wakeling
On Fri, 17 Jul 2009, Peter Eisentraut wrote: On Friday 17 July 2009 16:40:40 Matthew Wakeling wrote: I'm considering rewriting a postgres extension (GiST index bioseg) to make it use version 1 calling conventions rather than version 0. Does anyone have any ideas/opinions/statistics on what

Re: Res: [PERFORM] Cost performace question

2009-07-13 Thread Matthew Wakeling
of the EXPLAIN result. Please read the manual at http://www.postgresql.org/docs/8.4/interactive/using-explain.html The width is the: * Estimated average width (in bytes) of rows output by this plan node Matthew -- There once was a limerick .sig that really was not very big It was going quite fine

Re: [PERFORM] Huge difference in query performance between 8.3 and 8.4 (possibly)

2009-07-09 Thread Hartman, Matthew
what it returns. There are so many cross joins, outer joins, and inner joins mixed up together, ugh. Rather than trying to puzzle out why it is slow, rewrite it. It will be faster than before on any version. Matthew Hartman Programmer/Analyst Information Management, ICP Kingston General Hospital

Re: [PERFORM] Sorting by an arbitrary criterion

2009-07-09 Thread Hartman, Matthew
by case(c.category) when 1 then 'z' when 2 then 'a' then 3 then 'b' when 4 then 'w' when 5 then 'h' end; Matthew Hartman Programmer/Analyst Information Management, ICP Kingston General Hospital -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your

Re: [PERFORM] Speeding up a query.

2009-07-07 Thread Hartman, Matthew
on 10 minute boundaries, rounding up their timeslot requirement. (The single odd timeslot appointment will always waste 1 timeslot). Now THAT is an interesting idea. I'll have to play with this in my head a bit (during really boring meetings) and get back to you. Thanks! Matthew Hartman Programmer

Re: [PERFORM] Most effective insert or replace

2009-07-03 Thread Matthew Wakeling
and http://www.intermine.org/ 3. HOT has been invented since then, and it won't play well with this method. Matthew -- Trying to write a program that can't be written is... well, it can be an enormous amount of fun! -- Computer Science Lecturer -- Sent via pgsql-performance

Re: [PERFORM] Utilizing multiple cores in a function call.

2009-06-30 Thread Hartman, Matthew
. To schedule an appointment, both the chair and nurse have to be available for the required times at the same time, while also respecting the pod/chair and pod/nurse assignments. It's more than incrementing/decrementing the total available time. Thanks, Matthew Hartman Programmer/Analyst Information

Re: [PERFORM] How would you store read/unread topic status?

2009-06-23 Thread Matthew Wakeling
. Perhaps you meant java.util.concurrent.ConcurrentHashMap? Be very careful. Matthew -- Now, you would have thought these coefficients would be integers, given that we're working out integer results. Using a fraction would seem really stupid. Well, I'm quite willing to be stupid here - in fact

Re: [PERFORM] Speeding up a query.

2009-06-17 Thread Hartman, Matthew
: interesting) problem. In my current template there are 17 chairs and 7 nurses. Chairs are grouped into pods of 2-4 chairs. Nurses cover one to many pods, allowing for a primary nurse per pod as well as floater nurses that cover multiple pods. Matthew Hartman Programmer/Analyst Information Management

Re: [PERFORM] Speeding up a query.

2009-06-17 Thread Hartman, Matthew
); There are indexes on matrix for timeslot,unit_id, timeslot,nurse_id, and unit_id,nurse_id. Matthew Hartman Programmer/Analyst Information Management, ICP Kingston General Hospital (613) 549- x4294

Re: [PERFORM] Speeding up a query.

2009-06-17 Thread Hartman, Matthew
nature of using a loop suggests that another tool might be more appropriate. Matthew Hartman Programmer/Analyst Information Management, ICP Kingston General Hospital (613) 549- x4294 -Original Message- From: pgsql-performance-ow...@postgresql.org [mailto:pgsql-performance-ow

Re: [PERFORM] performance with query

2009-06-16 Thread Matthew Wakeling
a nested loop. Then you're getting upset because it isn't using a nested loop. When you tell it to NEVER use anything (switching all join algorithms off), it ignores you and chooses the right plan anyway. Matthew -- You can configure Windows, but don't ask me how. -- Bill Gates -- Sent

[PERFORM] Speeding up a query.

2009-06-16 Thread Hartman, Matthew
.x ) nurse Where chair.id = nurse.id With matrix having 3,280 rows. Ugh. I have tried various indexes and clustering approachs with little success. Any ideas? Thanks, Matthew Hartman Programmer/Analyst Information Management, ICP Kingston General Hospital (613) 549- x4294

Re: [PERFORM] GiST index performance

2009-06-11 Thread Matthew Wakeling
on a profiling build? ...the best bet is probably to make a test build of Postgres in which your functions are linked directly into the main postgres executable. I'll give that a try. Oprofile scares me with the sheer number of options. Matthew -- Prolog doesn't have enough parentheses. -- Computer

Re: [PERFORM] GiST index performance

2009-06-11 Thread Matthew Wakeling
bioseg.so bioseg_gist_consistent I'm guessing my next step is to install a version of libc with debugging symbols? Matthew -- Some people, when confronted with a problem, think I know, I'll use regular expressions. Now they have two problems. -- Jamie Zawinski

Re: [PERFORM] GiST index performance

2009-06-11 Thread Matthew Wakeling
btree_gist.sobtree_gist.so gbt_num_consistent A quick grep in the postgres source for mcount reveals no hits. No idea what it does - there is no man page for it. Matthew -- I pause for breath to allow you to get over your shock that I really did cover all that in only five

Re: [PERFORM] GiST index performance

2009-06-11 Thread Matthew Wakeling
On Thu, 11 Jun 2009, Matthew Wakeling wrote: A quick grep in the postgres source for mcount reveals no hits. No idea what it does - there is no man page for it. Ah - that's part of gprof. I'll recompile without --enable-profiling and try again. Duh. Matthew -- What goes up must come down

Re: [PERFORM] GiST index performance

2009-06-11 Thread Matthew Wakeling
nvidia_drv.so(no symbols) 19321 0.7527 bioseg.sobioseg.so bioseg_gist_decompress 17365 0.6765 libmozjs.so.1d libmozjs.so.1d (no symbols) Matthew -- A good programmer is one who looks both ways before crossing a one-way street

Re: [PERFORM] GiST index performance

2009-06-10 Thread Matthew Wakeling
On Fri, 5 Jun 2009, Robert Haas wrote: On Thu, Jun 4, 2009 at 12:33 PM, Matthew Wakelingmatt...@flymine.org wrote: Do you have a recommendation for how to go about profiling Postgres, what profiler to use, etc? I'm running on Debian Linux x86_64. I mostly compile with --enable-profiling

[PERFORM] Censorship

2009-06-10 Thread Matthew Wakeling
. Is there someone here who can fix the problem? Matthew -- Psychotics are consistently inconsistent. The essence of sanity is to be inconsistently inconsistent. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http

Re: [PERFORM] Censorship

2009-06-10 Thread Matthew Wakeling
On Wed, 10 Jun 2009, Guillaume Cottenceau wrote: Matthew Wakeling matthew 'at' flymine.org writes: It appears that I am being censored. Do you seriously think that censorman would kill your previous mails, but would let a It appears that I am being censored mail go through? If it's

Re: [PERFORM] Censorship

2009-06-10 Thread Matthew Wakeling
On Wed, 10 Jun 2009, Gurjeet Singh wrote: There is a limit on the size of the mail that you can send to different mailing lists. Please try to remove/link your attachments if you are trying to send any. No, size is not an issue - it's only 3kB. Matthew -- Q: What's the difference between

Re: [PERFORM] Censorship

2009-06-10 Thread Matthew Wakeling
On Wed, 10 Jun 2009, Scott Mead wrote: Are you getting a bounce message?  They usually have the reason in there. No, I am not getting any bounce message. My email just goes into a black hole, and does not appear on the web site archives either. Matthew -- The only secure computer is one

[PERFORM] Explaining an EXPLAIN.

2009-06-10 Thread Matthew Wakeling
multiplying rows=361427 by rows=47. That would only give 16987069 rows. Any ideas/explanations? Matthew -- To be or not to be -- Shakespeare To do is to be -- Nietzsche To be is to do -- Sartre Do be do be do -- Sinatra -- Sent via pgsql

Re: [PERFORM] Explaining an EXPLAIN.

2009-06-10 Thread Matthew Wakeling
On Wed, 10 Jun 2009, Tom Lane wrote: ...the size of the join relation was estimated long before we even started to think about nestloop-with-inner-indexscan plans. That makes a lot of sense. Matthew -- Richards' Laws of Data Security: 1. Don't buy a computer. 2. If you must buy a computer

[PERFORM] GiST index performance

2009-06-04 Thread Matthew Wakeling
x86_64. Matthew -- Jadzia: Don't forget the 34th rule of acquisition: Peace is good for business. Quark: That's the 35th. Jadzia: Oh yes, that's right. What's the 34th again? Quark: War is good for business. It's easy to get them mixed up. -- Sent via pgsql-performance mailing list (pgsql

Re: [PERFORM] Very inefficient query plan with disjunction in WHERE clause

2009-06-02 Thread Matthew Wakeling
for this particular query+dataset. Yeah, being bound by the ORM can be annoying. What version of Postgres is this? Recent versions can sometimes do a bitmap index scan to satisfy an OR constraint. Matthew -- Anyone who goes to a psychiatrist ought to have his head examined. -- Sent via pgsql

Re: [PERFORM] Very inefficient query plan with disjunction in WHERE clause

2009-06-02 Thread Matthew Wakeling
for this particular query+dataset. Yeah, being bound by the ORM can be annoying. What version of Postgres is this? Recent versions can sometimes do a bitmap index scan to satisfy an OR constraint. Matthew -- I work for an investment bank. I have dealt with code written by stock exchanges. I have seen

Re: [PERFORM] Using index for bitwise operations?

2009-06-02 Thread Matthew Wakeling
what you mean. Matthew -- What goes up must come down. Ask any system administrator. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] raid10 hard disk choice

2009-05-21 Thread Matthew Wakeling
, but there may be a few situations where the reverse is true. Basically, the first option will only be faster if you are doing lots of seeking (small requests) in a single thread. As soon as you go multi-threaded or are looking at sequential scans, you're better off with more discs. Matthew

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Matthew Wakeling
each set of identical a and sort each by b. Matthew -- Riker: Our memory pathways have become accustomed to your sensory input. Data: I understand - I'm fond of you too, Commander. And you too Counsellor -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Matthew Wakeling
the index? Or how about this query: SELECT * FROM table1, table2 WHERE table1.fk = table2.id ORDER BY table1.id, table2.id where both id columns are UNIQUE with an index. Do we eliminate table2.id from the ORDER BY in this case? Matthew -- Programming today is a race between software

Re: [PERFORM] increase index performance

2009-05-14 Thread Matthew Wakeling
your search is always going to be better than munging together results from several indexes, even if the planner decides to turn it into a bitmap index scan (which will be more likely in PG8.4 with effective_concurrency set). Matthew -- I don't want the truth. I want something I can tell

Re: [PERFORM] increase index performance

2009-05-13 Thread Matthew Wakeling
of the tree under that, because you are not constraining for street_id. A much better index to answer your query is (city_id, house_id, floor_id) - then it can just look up straight away. Instead of the index returning 20 rows to check, it will return just the 2000. Matthew -- An ant

Re: [PERFORM] Any better plan for this query?..

2009-05-12 Thread Matthew Wakeling
simpler backend to start up each time. If you really want to get a decent performance out of Postgres, then use long-lived connections (which most real-world use cases will do) and prepare your queries in advance with parameters. Matthew -- import oz.wizards.Magic; if (Magic.guessRight

Re: [PERFORM] Any better plan for this query?..

2009-05-12 Thread Matthew Wakeling
a bottleneck given very short-lived connections. Matthew -- -. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. ||X|||\ /|||X|||\ /|||X|||\ /|||X|||\ /|||X|||\ /|||X|||\ /||| |/ \|||X|||/ \|||X|||/ \|||X|||/ \|||X|||/ \|||X|||/ \|||X

[PERFORM] Statistics use with functions

2009-05-08 Thread Matthew Wakeling
of 45502) causes major planning problems. Also, why does the BitmapAnd say zero actual rows? I understand this probably isn't Priority No. 1, and there are some interesting corner cases when n_distinct is higher than the histogram width, but would it be possible to fix this one up? Matthew -- I

Re: [PERFORM] Statistics use with functions

2009-05-08 Thread Matthew Wakeling
'::text) - Bitmap Index Scan on geneflankingregion__distance_equals (cost=0.00..1185.78 rows=91134 width=0) (actual time=16.565..16.565 rows=91004 loops=1) Index Cond: (lower(distance) = '10.0kb'::text) Total runtime: 199.282 ms (6 rows) Matthew -- It is better

Re: [PERFORM] GiST index performance

2009-05-07 Thread Matthew Wakeling
too. I can't see any reason for this is the GiST code - it all seems pretty tight to me. We probably need to do some profiling. Matthew -- I suppose some of you have done a Continuous Maths course. Yes? Continuous Maths? menacing stares from audience Whoah, it was like

Re: [PERFORM] Slow select performance despite seemingly reasonable query plan

2009-05-07 Thread Matthew Wakeling
have the directory_index option enabled on your ext3 filesystem. Matthew -- The third years are wandering about all worried at the moment because they have to hand in their final projects. Please be sympathetic to them, say things like ha-ha-ha, but in a sympathetic tone of voice

Re: [PERFORM] Slow select performance despite seemingly reasonable query plan

2009-05-07 Thread Matthew Wakeling
it's not getting too out of hand. I think quite a few systems do set it by default now. Matthew -- Me... a skeptic? I trust you have proof? -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [PERFORM] Any better plan for this query?..

2009-05-06 Thread Matthew Wakeling
On Wed, May 06, 2009 at 10:04:33PM +0800, Craig Ringer wrote: Can the genetic query optimizer come into play on small queries? On Wed, 6 May 2009, Kenneth Marshall wrote: No. Yes. But you would have had to have set some really weird configuration. Matthew -- And the lexer will say Oh look

Re: [PERFORM] [HACKERS] high shared buffer and swap

2009-05-05 Thread Matthew Wakeling
afterwards. Matthew -- Surely the value of C++ is zero, but C's value is now 1? -- map36, commenting on the No, C++ isn't equal to D. 'C' is undeclared [...] C++ should really be called 1 response to C++ -- shouldn't it be called D? -- Sent via pgsql-performance mailing list (pgsql-performance

Re: [PERFORM] GiST index performance

2009-04-22 Thread Matthew Wakeling
On Tue, 21 Apr 2009, Matthew Wakeling wrote: Unfortunately, it seems there is another bug in the picksplit function. My patch fixes a bug that reveals this new bug. The whole picksplit algorithm is fundamentally broken, and needs to be rewritten completely, which is what I am doing. I have

Re: [PERFORM] GiST index performance

2009-04-22 Thread Matthew Wakeling
On Wed, 22 Apr 2009, Matthew Wakeling wrote: I will post a patch when I have ported my bioseg code over to the seg data type. Here is my patch ported over to the seg contrib package, attached. Apply it to seg.c and all should be well. A similar thing needs to be done to cube, but I haven't

Re: [PERFORM] GiST index performance

2009-04-21 Thread Matthew Wakeling
, the different branches of the tree were unselective, so new entries would just get stuffed in anywhere, leading to a much more balanced tree. I shall have a proper fix to this problem later today. Matthew -- It's one of those irregular verbs - I have an independent mind, You are an eccentric, He

Re: [PERFORM] GiST index performance

2009-04-20 Thread Matthew Wakeling
On Fri, 17 Apr 2009, Matthew Wakeling wrote: I have done a bit of investigation, and I think I might have found the smoking gun I was looking for. I have found a bug in the contrib package seg, which has been copied into the bioseg data type as well. It causes the index to be created

Re: [PERFORM] GiST index performance

2009-04-17 Thread Matthew Wakeling
On Thu, 16 Apr 2009, Tom Lane wrote: Matthew, can you put together a self-contained test case with a similar slowdown? I have done a bit of investigation, and I think I might have found the smoking gun I was looking for. I just added a load of debug to the gist consistent function

[PERFORM] Really dumb planner decision

2009-04-16 Thread Matthew Wakeling
default_text_search_config = 'pg_catalog.english' Anything I can do to solve this? Matthew -- Surely the value of C++ is zero, but C's value is now 1? -- map36, commenting on the No, C++ isn't equal to D. 'C' is undeclared [...] C++ should really be called 1 response to C++ -- shouldn't it be called D

Re: [PERFORM] Really dumb planner decision

2009-04-16 Thread Matthew Wakeling
trivial for Postgres to do. Matthew -- I quite understand I'm doing algebra on the blackboard and the usual response is to throw objects... If you're going to freak out... wait until party time and invite me along -- Computer Science Lecturer -- Sent via pgsql-performance

Re: [PERFORM] Really dumb planner decision

2009-04-16 Thread Matthew Wakeling
On Thu, 16 Apr 2009, Grzegorz Jaśkiewicz wrote: On Thu, Apr 16, 2009 at 11:37 AM, Matthew Wakeling matt...@flymine.org wrote: I have a query that is executed really badly by Postgres. It is a nine table join, where two of the tables are represented in a view. If I remove one of the tables from

Re: [PERFORM] Really dumb planner decision

2009-04-16 Thread Matthew Wakeling
then? Matthew -- Contrary to popular belief, Unix is user friendly. It just happens to be very selective about who its friends are. -- Kyle Hearn -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http

Re: [PERFORM] Really dumb planner decision

2009-04-16 Thread Matthew Wakeling
incorporated into a query that will constrain it and cause it to be evaluated a lot quicker. This kind of scenario kind of guarantees a bad plan as soon as the number of tables reaches from_collapse_limit. Matthew -- Failure is not an option. It comes bundled with your Microsoft product

[PERFORM] GiST index performance

2009-04-16 Thread Matthew Wakeling
be improved in the future? Matthew -- We have always been quite clear that Win95 and Win98 are not the systems to use if you are in a hostile security environment. We absolutely do recognize that the Internet is a hostile environment. Paul Leach pau...@microsoft.com -- Sent via pgsql-performance

Re: [PERFORM] GiST index performance

2009-04-16 Thread Matthew Wakeling
On Thu, 16 Apr 2009, Kevin Grittner wrote: Matthew Wakeling matt...@flymine.org wrote: I have been doing some queries that are best answered with GiST indexes For what definition of best answered? Since an index is only a performance tuning feature (unless declared UNIQUE), and should never

Re: [PERFORM] GiST index performance

2009-04-16 Thread Matthew Wakeling
in these queries, which are 14 seconds and 38 minutes respectively. Matthew -- Doctor: Are you okay? You appear to be injured. Neelix: Aaah! Doctor: It's okay, it looks superficial. Neelix: Am I going to die? Doctor: Not unless you are allergic to tomatoes. This appears to be a sauce

Re: [PERFORM] GiST index performance

2009-04-16 Thread Matthew Wakeling
On Thu, 16 Apr 2009, Tom Lane wrote: Matthew, can you put together a self-contained test case with a similar slowdown? It isn't the smoking gun I thought it would be, but: CREATE TABLE a AS SELECT a FROM generate_series(1,100) AS a(a); CREATE TABLE b AS SELECT b FROM generate_series

Re: [PERFORM] GiST index performance

2009-04-16 Thread Matthew Wakeling
location__key_all index: 334 MB Matthew -- For those of you who are into writing programs that are as obscure and complicated as possible, there are opportunities for... real fun here -- Computer Science Lecturer -- Sent via pgsql-performance mailing list (pgsql

Re: [PERFORM] INSERT times - same storage space but more fields - much slower inserts

2009-04-15 Thread Matthew Wakeling
in the background in another thread, while fully honouring flush calls. When it is using the database connection is well-defined. I hope someone can find it useful. Matthew -- -. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. ||X|||\ /|||X|||\ /|||X|||\ /|||X|||\ /|||X|||\ /|||X

Re: [PERFORM] INSERT times - same storage space but more fields - much slower inserts

2009-04-15 Thread Matthew Wakeling
On Wed, 15 Apr 2009, Matthew Wakeling wrote: If anyone needs this code in Java, we have a version at http://www.intermine.org/ Download source code: http://www.intermine.org/wiki/SVNCheckout Javadoc: http://www.intermine.org/api/ Sorry, that should be http://www.flymine.org/api/ Matthew

Re: [PERFORM] Nested query performance issue

2009-04-14 Thread Matthew Wakeling
that this join is redundant. Because the join isn't redundant? You're making the assumption that for every score.game_id there is exactly one game.id that matches. Of course, you may have a unique constraint and foreign key/trigger that ensures this. Matthew -- The third years are wandering about all

Re: [PERFORM] Shouldn't the planner have a higher cost for reverse index scans?

2009-04-14 Thread Matthew Wakeling
issue that correlation isn't actually that good a measure of the cost of an index scan, but I'm not sure what is better, and feasible. Matthew -- Our riverbanks and seashores have a beauty all can share, provided there's at least one boot, three treadless tyres, a half-eaten pork pie, some oil

Re: [PERFORM] INSERT times - same storage space but more fields - much slower inserts

2009-04-14 Thread Matthew Wakeling
to help you implement a test setup for doing that, if you'd like. You can always do binary-format COPY. Matthew -- An ant doesn't have a lot of processing power available to it. I'm not trying to be speciesist - I wouldn't want to detract you from such a wonderful creature, but, well, there isn't

Re: [PERFORM] linux deadline i/o elevator tuning

2009-04-09 Thread Matthew Wakeling
single disc in the system that takes a long time to seek from one place to another. This assumption fails on both RAID arrays and SSDs, so I'd be interested to see some numbers to back that one up. Matthew -- import oz.wizards.Magic; if (Magic.guessRight())... -- Computer Science

<    1   2   3   4   5   6   >