Re: [PERFORM] vacuum locking

2003-10-24 Thread Stephen
I ran into the same problem with VACUUM on my Linux box. If you are running Linux, take a look at "elvtune" or read this post: http://groups.google.com/groups?q=stephen+vacuum+linux&hl=en&lr=&ie=UTF-8&se lm=gRdjb.7484%241o2.77%40nntp-post.primus.ca&rnum=3 Regards,

Re: [PERFORM] Query limitations (size, number of UNIONs ...)

2005-06-03 Thread Stephen Frost
f 'union'. Just a thought. Stephen signature.asc Description: Digital signature

Re: [PERFORM] Quad Opteron stuck in the mud

2005-07-13 Thread Stephen Frost
w/ a test case that others could reproduce where explain isn't returning? I think that would be very useful towards solving at least that issue... Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread Stephen Frost
een 32bit and 64bit as you'd see on a sparc platform. Enjoy, Stephen signature.asc Description: Digital signature

Re: [PERFORM] LEFT JOIN optimization

2005-09-11 Thread Stephen Frost
* Ksenia Marasanova ([EMAIL PROTECTED]) wrote: > Any tips are greatly appreciated. EXPLAIN ANALYZE of the same queries would be much more useful. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] LEFT JOIN optimization

2005-09-11 Thread Stephen Frost
for some reason) from a regular join to a left join with a filtering in the application which is probably a bad move... If you can use at least some filtering in the database I expect that'd help.. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Performance considerations for very heavy INSERT traffic

2005-09-13 Thread Stephen Frost
ing full commits but it might be something to consider. > When I finally get all of this sorted out and working reasonably optimally, > I'll be sure to come back and report what techniques/settings did and didn't > work for this workload. That'd be great, many thank

Re: [PERFORM] Database restore speed

2005-12-02 Thread Stephen Frost
client side? Or does binary mode not work with copy (that wouldn't suprise me, but perhaps copy could be made to support it)? The other thought, of course, is that you could use PITR for your backups instead of pgdump... Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Database restore speed

2005-12-02 Thread Stephen Frost
* Luke Lonergan ([EMAIL PROTECTED]) wrote: > On 12/2/05 12:18 PM, "Stephen Frost" <[EMAIL PROTECTED]> wrote: > > Just a thought, but couldn't psql be made to use the binary mode of > > libpq and do at least some of the conversion on the client side? Or >

Re: [PERFORM] MySQL is faster than PgSQL but a large margin in my program... any ideas why?

2005-12-21 Thread Stephen Frost
ansaction safe, and has problems with data reliability (aiui, equivilant to doing 'fsync = false' for Postgres). InnoDB, again iirc, is transaction safe and whatnot, and more akin to the default PostgreSQL setup. I expect some others will comment along these lines too, if my response isn&#

Re: [PERFORM] Advice on selecting good values for work_mem?

2006-12-07 Thread Stephen Frost
planner will, for example, do a hashjoin which is much faster than sorting and merge-joining, but takes alot of memory... They may say "hey, I like it being fast" but not consider what happens when alot of those queries run at once.. Thanks! Stephen signature.asc Description: Digital signature

Re: [PERFORM] PITR performance costs

2007-05-28 Thread Stephen Frost
he occational 16MB copy off the disk... Thanks, Stephen > On 28-May-07, at 12:31 PM, Heikki Linnakangas wrote: > > >Dave Cramer wrote: > >>Since PITR has to enable archiving does this not increase the > >>amount of disk I/O required ? > > > >

Re: [PERFORM] setting up raid10 with more than 4 drives

2007-05-29 Thread Stephen Frost
> AFAIK you can't RAID1 more than two drives, so the above doesn't make sense > to me. It's just more copies of the same data if it's really a RAID1, for the extra, extra paranoid. Basically, in the example above, I'd read it as "D1, D2, D5 have identical

Re: [PERFORM] setting up raid10 with more than 4 drives

2007-05-30 Thread Stephen Frost
r it will be slower to keep in sync on a heavy write system. I'm not sure, but I don't think most RAID1 systems do reads against all drives and compare the results before returning it to the caller... I'd be curious if I'm wrong. Thanks, Stephen si

Re: [PERFORM]

2007-06-25 Thread Stephen Frost
ster when doing a hash-join vs. a sort + merge-join. Could likely be because it doesn't think there's enough work memory available for the hash, which might change based on the values it gets from the statistics on how frequently something shows up, etc. Enjoy, Stephen signature.asc Description: Digital signature

Re: [PERFORM]

2007-06-25 Thread Stephen Frost
ly run on our data warehouse box, which is a nice dual-proc/dual-core DL385 w/ 16GB of ram. :) The annoying thing is that I can still run it out of memory sometimes, even w/ 16GB. :/ Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM]

2007-06-25 Thread Stephen Frost
s without doing 'explain analyze', but I think you have a good idea of the best plan for this query already... Thanks, Stephen signature.asc Description: Digital signature

[PERFORM] Optimising "in" queries

2007-08-21 Thread Stephen Davies
1138,1139}'::integer[])) -> Bitmap Index Scan on reading_sensor (cost=0.00..528.37 rows=28650 width=0) Index Cond: (sensor_id = ANY ('{1137,1138,1139}'::integer[])) (8 rows) TIA, Stephen Davies -- =

Re: [PERFORM] Optimising "in" queries

2007-08-23 Thread Stephen Davies
ted faster than =a or =b or =c. Am I wrong for PostgreSQL? Stephen On Wednesday 22 August 2007 22:55, Michael Glaesemann wrote: > On Aug 22, 2007, at 5:58 , Russell Smith wrote: > > Stephen Davies wrote: > >> select count(rdate),rdate from reading where sensor_id in > >>

Re: [PERFORM] Optimising "in" queries

2007-08-28 Thread Stephen Davies
l | numeric(7,3)| Indexes: "reading_pkey" PRIMARY KEY, btree (id) "unique_sensor_date" UNIQUE, btree (sensor_id, rdate) "date" btree (rdate) "reading_sensor" btree (sensor_id) Foreign-key constraints: "$1" FOREIGN KEY (sen

Re: [PERFORM] Optimising "in" queries

2007-08-28 Thread Stephen Davies
Herrera wrote: > I don't think you showed us the EXPLAIN ANALYZE results that Scott > requested. -- This email is for the person(s) identified above, and is confidential to the sender and the person(s). No one else is authorised to use or disseminate this email

Re: [PERFORM] query plan worse after analyze

2007-10-05 Thread Stephen Frost
ting the various plans up somewhere online (perhaps a pastebin like http://pgsql.privatepaste.com) instead of or in addition to sending it in the email. Thanks! Stephen signature.asc Description: Digital signature

Re: [PERFORM] mysql to postgresql, performance questions

2010-03-18 Thread Stephen Frost
are almost the > same). What's the best tool to time queries in postgresql? Can this be > done from pgadmin? As was mentioned elsewhere, certainly the best tool to test with is your actual application, if that's possible.. Or at least the language your application is in. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] too complex query plan for not exists query and multicolumn indexes

2010-03-19 Thread Stephen Frost
y more complicated). > As in my previous tests, this is only a testing environment: so all data > is in memory, no disk activity involved at all, no swap etc. Yea, system calls still aren't free. I would recommend, if you care about this query, bumping up your work_mem setting for it. Right now, PG is using an external sort (meaning- on-disk), but the data set appears to only be like 50M (49560kB). If you increased work_mem to, say, 128MB (for this query, maybe or maybe not for the entire system), it'd be able to do an in-memory sort (or maybe a hash or something else, if it makes sense), which would be faster. I'd probably rewrite this as a left-join too, to be honest, but based on what I'm saying, that'd probably get the same query plan as you had first anyway (the merge anti-join), so it's probably not necessary. I'd love to hear how PG performs with work_mem bumped up to something decent... Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Optimize date query for large child tables: GiST or GIN?

2010-05-20 Thread Stephen Frost
sure that those constraints are really the right ones and that they make sense? You're using a bunch of extract()'s there too, why not just specify a CHECK constraint on the date ranges which are allowed in the table..? Maybe I've misunderstood the whole point here, but I don't think so. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Optimize date query for large child tables: GiST or GIN?

2010-05-20 Thread Stephen Frost
day > user_stop_year || user_stop) THEN m.taken BETWEEN (user_start_year || user_start_day)::date AND ((user_stop_year || user_stop)::date + '1 year'::interval)::date -- I don't think you need/want this..? -- GROUP BY

Re: [PERFORM] Optimize date query for large child tables: GiST or GIN?

2010-05-20 Thread Stephen Frost
s in your partitioning setup. What is constraint_exclusion set to? What version of PG is this? Do the results og this query look at all correct to you? Have you considered an index on elevation, btw? How many records in that city table are there and how many are actually in that range? Thanks,

Re: [PERFORM] Optimize date query for large child tables: GiST or GIN?

2010-05-21 Thread Stephen Frost
ly efficient way (not doing seq. scans through everything because you're operating on every row for something). It needs to be a couple hundred-thousand rows, but it doesn't need to be the full data set, imv. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] PostgreSQL Function Language Performance: C vs PL/PGSQL

2010-05-26 Thread Stephen Frost
these stored procedures from within a C-based function which > does the logging, math, control logic, and builds the result sets and > cursors? Uhh, I'd guess 'no' to that one. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] PostgreSQL Function Language Performance: C vs PL/PGSQL

2010-05-26 Thread Stephen Frost
saction. That's not really relevant.. Is it called alot from the same backend/database connection? If so, and if you're using regular SELECT statements and the like (not EXECUTE), then they're getting prepared the first time they're used and that is kept across transactions.

Re: [PERFORM] PostgreSQL Function Language Performance: C vs PL/PGSQL

2010-05-26 Thread Stephen Frost
to be an overall improvment of, say, 10%, or a 10-fold improvment. :) Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] PostgreSQL Function Language Performance: C vs PL/PGSQL

2010-06-01 Thread Stephen Frost
built-ins now available in 8.4 (array_agg), that copying doesn't happen any more. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Overusing 1 CPU

2010-06-02 Thread Stephen Frost
tabase. If you're building alot of indexes then you probably want to split up the statements into multiple connections and run them in parallel. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] requested shared memory size overflows size_t

2010-06-02 Thread Stephen Frost
r any other PG operations (PG would use at most 4GB-shared_buffers, or so). Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] SeqScans on boolen values / How to speed this up?

2010-07-05 Thread Stephen Frost
ULL) OR > (persons.modified > indexing_persons.indexed)) >-> Seq Scan on persons (cost=0.00..4438.29 > rows=143629 width=16) (actual time=0.000..56.000 rows=143629 loops=1) >-> Hash (cost=2534.86..2534.86 rows=142286 width=16) > (actual tim

Re: [PERFORM] Question about partitioned query behavior

2010-07-06 Thread Stephen Frost
included (which, really, shouldn't be that big a deal). Did you look at what the other reply suggested? Do you have constraint_exclusion = 'on' in your postgresql.conf? Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Understanding tsearch2 performance

2010-07-14 Thread Stephen Frost
x27;s entirely possible that the index is *not* the fastest way to pull this data (it's nearly 10% of the table..), if the stats were better it might use a seq scan instead, not sure how bad the cost of the filter itself would be. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Slow SQL lookup due to every field being listed in SORT KEY

2010-09-10 Thread Stephen Frost
etting to use...)? Increasing work_mem is often a good idea if your system can afford it based on the number/kind of queries running concurrently. Note that you can also increase that setting for just a single role, single session, or even single query. Thanks, Stephen signature.as

Re: [PERFORM] Slow SQL lookup due to every field being listed in SORT KEY

2010-09-10 Thread Stephen Frost
which can often be due to missing pieces from the JOIN clause or misunderstanding of the database schema...). Stephen signature.asc Description: Digital signature

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-06 Thread Stephen Frost
ntly useful to test with.. Stephen signature.asc Description: Digital signature

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-06 Thread Stephen Frost
could try to replicate the lseek() contention? I can give it a shot, but the impression I had from the paper is that the lseek() contention wouldn't be seen without the changes to the lock manager...? Or did I misunderstand? Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-07 Thread Stephen Frost
es atomic changes instead of spinlocks for certain locking situations.. If that's all the MIT folks did, they certainly made it sound like alot more. :) Stephen signature.asc Description: Digital signature

Re: [PERFORM] large dataset with write vs read clients

2010-10-07 Thread Stephen Frost
running out of memory. > I think the first thing to do is look into using a connection pooler > like pgpool to reduce your connection memory overhead. Yeah.. Having the number of database connections be close to the number of processors is usually recommended. Stephen signature.asc Description: Digital signature

Re: [PERFORM] large dataset with write vs read clients

2010-10-07 Thread Stephen Frost
#x27;t great, but honestly is surprisingly good all things > considered. I'm kind of suprised at each connection taking 100MB, especially ones which are just doing simple inserts. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] large dataset with write vs read clients

2010-10-07 Thread Stephen Frost
all the same actual memory, not 100M per process. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] High load,

2011-01-27 Thread Stephen Frost
en checkpoints, so that the checkpoints aren't as big and painful. That can be done by making the background writer more aggressive. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] How to best use 32 15k.7 300GB drives?

2011-01-28 Thread Stephen Frost
When you run top, is your PG process constantly in 'D' state, or is it in 'R' state, or what? Might help figure some of that out. Note that parallelizing the query will help regardless of if it's disk bound or CPU bound, when you're running on the kind of hardware you're talking about (lots of spindles, multiple CPUs, etc). Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] How to best use 32 15k.7 300GB drives?

2011-01-28 Thread Stephen Frost
be even *faster* would be the run all 5 of those queries against the child tables in parallel (given that you have over 5 CPUs and enough memory that you don't start swapping). If it's still too big on the per-child basis, you might be able to use conditionals to do the first 100 strands, then the next hundred, etc. >I appreciate the comments thus far. Let's hope you'll always appreciate them. :) Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] How to best use 32 15k.7 300GB drives?

2011-01-28 Thread Stephen Frost
ex or > cluster. Just to share my experiences- I've found that creating a new table and inserting into it is actually faster than doing full-table updates, if that's an option for you. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Does exclusive locking improve performance?

2011-02-16 Thread Stephen Frost
Does it have less work to do? That's a bit harder to say but my guess is "not so much that you'd actually be able to notice it.".. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] application of KNN code to US zipcode searches?

2011-02-17 Thread Stephen Frost
* Mark Stosberg (m...@summersault.com) wrote: > Recommendations? PostGIS, geometry columns, and UTM.. I'm not sure where they are wrt adding KNN support, but it's something they've been anxious to have for a while, so I expect support will come quickly. Thanks,

Re: [PERFORM] maintenance_work_mem + create index

2011-03-24 Thread Stephen Frost
from the same problem that work_mem has, specifically that PG still won't allocate more than 1GB of memory for any single operation. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Postgres slower than MS ACCESS

2006-02-14 Thread Stephen Frost
e shortcuts but the times you're posting for Postgres seem quite far off based on the hardware and commands you've described... Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Postgres slower than MS ACCESS

2006-02-14 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > Stephen Frost <[EMAIL PROTECTED]> writes: > > While it's true that Access almost certainly takes some shortcuts, 24 > > minutes for an update across 1.2 millon rows seems an awefully long time > > for Postgres. > >

[PERFORM] why is bitmap index chosen for this query?

2006-05-18 Thread Stephen Byers
Could someone explain the results of the following?  This is with postgres 8.1.2 on a database that was just vacuum-verbose-analyzed.  I have packets_i4 index which I am expecting to be used with this query but as you can see, I have have to convince its usage by turning off other scans.  The total

Re: [PERFORM] why is bitmap index chosen for this query?

2006-05-18 Thread Stephen Byers
"Steinar H. Gunderson" <[EMAIL PROTECTED]> wrote: It sounds like PostgreSQL badly overestimates the cost of the index scan.Does the table perchance fit completely into memory, withouteffective_cache_size indicating that? Don't know the exact way to answer your question, but my initial instinct is

Re: [PERFORM] why is bitmap index chosen for this query?

2006-05-18 Thread Stephen Byers
I repeated explain analyze on the query 5 times and it came up with the same plan. You asked about index order and physical table order.  In general the index order is indeed close to the same order as the physical table order.  However, this query is likely an exception.  The data is actually fro

Re: [PERFORM] why is bitmap index chosen for this query?

2006-05-18 Thread Stephen Byers
06 at 12:38:18PM -0700, Stephen Byers wrote:> I repeated explain analyze on the query 5 times and it came up with the same plan.Yes, but did it end up with the same runtime? That's the interesting part --the plan will almost always be identical between explain analyze runs giventhat you haven&#x

Re: [PERFORM] why is bitmap index chosen for this query?

2006-05-18 Thread Stephen Byers
You may be comparing the values to Tom's suggestion to bump up work_mem.  Take a look at the original posting (Total runtime: 777208.041 ms for the bitmap scan)   -Steve "Steinar H. Gunderson" <[EMAIL PROTECTED]> wrote: On Thu, May 18, 2006 at 12:53:16PM -0700, Stephen By

Re: [PERFORM] 64-bit vs 32-bit performance ... backwards?

2006-06-12 Thread Stephen Frost
he cacheing, so as long as your OS can see all the memory you have in the box, that benefit of running 64bit isn't going to be seen on Postgres. On many other database systems (notably the 800-pound gorillas...) the database handle the cacheing and so wants to basically have control over all

Re: [PERFORM] Placement of 64-bit libraries (offtopic)

2006-06-13 Thread Stephen Frost
led 'multiarch' and is being discussed with LSB and other distros too, though I think it did mostly originated with Debian folks. Just my 2c. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] SAN performance mystery

2006-06-19 Thread Stephen Frost
lly dual-controller load-balanced and it works quite decently. Indeed, the EMC SANs are generally the high-priced ones too, so not really sure what to tell you about the poor performance you're seeing out of it. Your IT folks and/or your EMC rep. should be able to resolve that, really... En

Re: [PERFORM] SAN performance mystery

2006-06-19 Thread Stephen Frost
; >> that, really... > > > > > >The only exception I've heard to this is the Clarion AX150. We looked at > >one and we were warned off of it by some EMC gearheads. Yeah, the Clarion is the EMC "cheap" line, and I think the AX150 was the extra-cheap one which Dell rebranded and sold. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Opteron/FreeBSD/PostgreSQL performance poor

2006-07-05 Thread Stephen Frost
o the vacuum's run for? If it's 3 hours, then that might start to be an issue with disk I/O contention... > Hopefully I've supplied enough information to start diagnosing the > problem. Any ideas, thoughts, suggestions are greatly appreciated ... Just my 2c, hopefully you'll get some better answers too. :) Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Performance penalty for remote access of postgresql (8.1.3)? any experiance?

2006-07-19 Thread Stephen Frost
OLTP) per transaction. Enjoy, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Performance penalty for remote access of postgresql (8.1.3)? any experiance?

2006-07-19 Thread Stephen Frost
* Florian Weimer ([EMAIL PROTECTED]) wrote: > * Stephen Frost: > > Actually, can't you stick multiple inserts into a given 'statement'? > > ie: insert into abc (123); insert into abc (234); > > IIRC, this breaks with PQexecParams, which is the recommended m

Re: [PERFORM] Hardware upgraded but performance still ain't good enough

2006-08-08 Thread Stephen Frost
uSe (ES, Fedora, who knows) and pretty much any kernel you build using sources off of kernel.org or for any other distribution unless you know exactly what versions/patches they support. Feel free to contact me off-list if you'd like to continue this discussion since I don't really see

Re: [PERFORM] Hardware upgraded but performance still ain't good enough

2006-08-08 Thread Stephen Frost
ry useful tools too and can help with hardware decisions but you probably want to review your queries and make sure the database is performing as best it can with the setup you have today before throwing more hardware at it. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Hardware upgraded but performance still ain't good enough

2006-08-09 Thread Stephen Frost
e a distro-specific issue and what wouldn't), the support folks are willing to help debug it. Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] odd variances in count(*) times

2006-10-09 Thread Stephen Frost
ead and that might be inconsistant between the systems (mainly it may have to do with the gettimeofday() calls being implemented differently between Windows and Linux..). Thanks, Stephen signature.asc Description: Digital signature

Re: [PERFORM] Basic Postgresql Performance Question

2004-06-16 Thread Stephen Frost
;t make sense to split them across different nodes if they're accessing the same memory- every memory access would have to be checked) like the PostgreSQL server. > 2. It is advantageous to buy AMD 64 rather than the Pentium IV? Personally, I certainly think so. More registers, more me

Re: [PERFORM] Basic Postgresql Performance Question

2004-06-16 Thread Stephen Frost
register-starved i386 platforms which increases the speed for most applications where it usually wouldn't when recompiled for 64bit. Stephen signature.asc Description: Digital signature

Re: [PERFORM] Major differences between oracle and postgres performance - what can I do ?

2004-06-18 Thread Stephen Frost
mory if that were happening. Could this optimization be added to PostgreSQL? It sounds like a very reasonable thing to do. Hopefully there wouldn't be too much complexity needed to add it. Stephen signature.asc Description: Digital signature

Re: [PERFORM] Major differences between oracle and postgres performance - what can I do ?

2004-06-18 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > Stephen Frost <[EMAIL PROTECTED]> writes: > > * Tom Lane ([EMAIL PROTECTED]) wrote: > >> [... thinks for awhile ...] It seems possible that they may use sort > >> code that knows it is performing a DISTINCT operation and

Re: [PERFORM] Major differences between oracle and postgres performance - what can I do ?

2004-06-18 Thread Stephen Frost
* Stephen Frost ([EMAIL PROTECTED]) wrote: > systems does in 40 seconds. My only other concern is the Oracle system > having to do the write I/O while the postgres one doesn't... I don't > see an obvious way to get around that though, and I'm not sure if it'd

[PERFORM] Large # of rows in query extremely slow, not using index

2004-09-13 Thread Stephen Crowley
width=83) Filter: ((date = '2004-09-07'::date) AND ((stock)::text = 'QQQ'::text)) (2 rows) Any help would be appreciated --Stephen Table "public.island_history" Column | Type | Modifiers --+---

Re: [PERFORM] Large # of rows in query extremely slow, not using index

2004-09-13 Thread Stephen Crowley
On Mon, 13 Sep 2004 21:11:07 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: > Stephen Crowley <[EMAIL PROTECTED]> writes: > > Does postgres cache the entire result set before it begins returning > > data to the client? > > The backend doesn't, but libpq doe

Re: [PERFORM] Large # of rows in query extremely slow, not using index

2004-09-13 Thread Stephen Crowley
I get some extra time I'll look into libpq and see what is required to fix the API. Most thirdparty programs and existing JDBC apps won't work with the current paradigm when returning large result sets. Thanks, Stephen On Mon, 13 Sep 2004 21:49:14 -0400, Tom Lane <[EMAIL PROT

Re: [PERFORM] Data Warehouse Reevaluation - MySQL vs Postgres --

2004-09-14 Thread Stephen Frost
a.keyA, a.keyB, a.colA, a.colB from Y a left join X b using (keyA, keyB) where b.keyA is NULL and b.keyB is NULL; With the appropriate indexes, this is pretty fast but I think a merge would be much faster. Thanks, Ste

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-16 Thread Stephen Crowley
4-09-07'::date) AND ((stock)::text = 'MSFT'::text))" "Total runtime: 346759.000 ms" Nearly 8 minutes.. Why would it take this long? Is there anything else I can do to debug this? When I set enable_seqscan to OFF and force everything to use the index every stock I query ret

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-17 Thread Stephen Crowley
uot; "Total runtime: 201009.000 ms" So now this in all in proportion and works as expected.. the question is, why would the fact that it needs to be vaccumed cause such a huge hit in performance? When i vacuumed it did free up nearly 25% of the space. --Stephen On Fri, 17 Sep 2004 2

Re: [PERFORM] Large # of rows in query extremely slow, not using index

2004-09-23 Thread Stephen Crowley
Thanks for the explanation. So what sort of changes need to be made to the client/server protocol to fix this problem? On Thu, 23 Sep 2004 18:22:15 -0500 (EST), Kris Jurka <[EMAIL PROTECTED]> wrote: > > > On Tue, 14 Sep 2004, Stephen Crowley wrote: > > > Problem solv

Re: [PERFORM] Improve performance of query

2004-12-16 Thread Stephen Frost
ould having them be 'UNION ALL's work? Stephen signature.asc Description: Digital signature

Re: [PERFORM]

2005-01-20 Thread Stephen Frost
n, if you have time before you have to go into production with the new solution (sounds like you do- changing databases takes time anyway). > Thanks in advance for any help you may have, I'll do my best to keep > pgsql-performance up to date > on the results. Hope that helps. Others on here will correct me if I misspoke. :) Stephen signature.asc Description: Digital signature

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-20 Thread Stephen Frost
me way such that it can be spread across multiple machines, then if you need to combine the data have it be replicated using slony to a big box that has a view which joins all the tables and do your big queries against that. Just some thoughts. Stephen signature.asc Description: Digital signature

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-20 Thread Stephen Frost
* Herv? Piedvache ([EMAIL PROTECTED]) wrote: > Le Jeudi 20 Janvier 2005 15:30, Stephen Frost a écrit : > > * Herv? Piedvache ([EMAIL PROTECTED]) wrote: > > > Is there any solution with PostgreSQL matching these needs ... ? > > > > You might look into pg_pool. Anot

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-20 Thread Stephen Frost
/ the guy from Cox Communications and I thought he was using it :). > So, your only option is Oracle or another very expensive commercial > database. Or partition the data at the application layer. Stephen signature.asc Description: Digital signature

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-20 Thread Stephen Frost
which machine to contact for what data. Stephen signature.asc Description: Digital signature

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-20 Thread Stephen Frost
* [EMAIL PROTECTED] ([EMAIL PROTECTED]) wrote: > I think maybe a SAN in conjunction with tablespaces might be the answer. > Still need one honking server. That's interesting- can a PostgreSQL partition be acress multiple tablespaces? Stephen signature.asc Description: Digital signature

Re: [PERFORM] dell versus hp

2007-11-06 Thread Stephen Frost
#x27;ll help w/ whatever hardware you end up going with. Enjoy, Stephen signature.asc Description: Digital signature

Re: [PERFORM] TB-sized databases

2007-11-26 Thread Stephen Cook
I think either would work; both PostgreSQL and MS SQL Server have success stories out there running VLDBs. It really depends on what you know and what you have. If you have a lot of experience with Postgres running on Linux, and not much with SQL Server on Windows, of course the former would

Re: [PERFORM] TB-sized databases

2007-11-29 Thread Stephen Frost
_cost_query' variable where someone working in a data warehouse situation would get a notice if the query he's hand-crafting has a very high cost (in which case he could ctrl-c it if he thinks something is wrong, rather than waiting 5 hours before realizing he forgot a join clause), but the

[PERFORM] 8.3rc1 Out of memory when performing update

2008-01-24 Thread Stephen Denne
(id = $0) Stephen Denne Disclaimer: At the Datamail Group we value team commitment, respect, achievement, customer focus, and courage. This email with any attachments is confidential and may be subject to legal privilege. If it is not intended for you please advise by reply immediately, destroy

Re: [PERFORM] 8.3rc1 Out of memory when performing update

2008-01-24 Thread Stephen Denne
> "Stephen Denne" <[EMAIL PROTECTED]> writes: > > A simple update query, over roughly 17 million rows, > > populating a newly added column in a table, resulted in an > > out of memory error when the process memory usage reached > > 2GB. Could this be

Re: [PERFORM] 8.3rc1 Out of memory when performing update

2008-01-25 Thread Stephen Denne
I don't have a PostgreSQL build environment. It is now Friday night for me. I left the alternate query running, and will find out on Monday what happened. If I drop the fk constraint, and/or its index, would I still be affected by the leak you found? Regards, Stephen

Re: [PERFORM] 8.3rc1 Out of memory when performing update

2008-01-27 Thread Stephen Denne
>>"Stephen Denne" <[EMAIL PROTECTED]> writes: >>> I altered the update statement slightly, and reran the query. >>> The altered query has been running over 3 hours now, >>> without using lots of memory (38M private bytes). >>> 2046 temp

Re: [PERFORM] 8.3rc1 Out of memory when performing update

2008-01-27 Thread Stephen Denne
Tom Lane wrote: > "Stephen Denne" <[EMAIL PROTECTED]> writes: > > So dropping the fk constraint and index results in > successful query execution with constant memory usage. Does > this confirm that the memory leak you found is the one I was > suffering fro

Re: [PERFORM] Optimizing No matching record Queries

2008-02-12 Thread Stephen Denne
2113544.412 rows=0 loops=1) > Filter: (fklistingsourceid = 5525) Would it help to have a combined index on fklistingsourceid, entrydate? Regards, Stephen Denne. Disclaimer: At the Datamail Group we value team commitment, respect, achievement, customer focus, and courage

Re: [PERFORM] Weird issue with planner choosing seq scan

2008-02-24 Thread Stephen Denne
ght have a huge number of dead rows in your table, slowing down the sequential scan. (Likewise updating a third of the rows, changing an indexed field.) What do you get from: VACUUM VERBOSE u_counts; Regards, Stephen Denne. Disclaimer: At the Datamail Group we value team commitment, respect, achie

Re: [PERFORM] Weird issue with planner choosing seq scan

2008-02-25 Thread Stephen Denne
ansaction is still alive. Alternatively, it may be a different 2 million dead row versions now than earlier, and may simply be a side effect of your particular usage, and nothing to worry about. (Though it is exactly the same number of rows, which strongly hints at being exactly the same rows.) R

  1   2   3   >