[PERFORM] group commit

2014-06-04 Thread Evgeny Shishkin
Hi, i just wanted to know if group commit (as described in https://wiki.postgresql.org/wiki/Group_commit ) was committed. And if parallel replication is going to be introduced. Mysql 5.7 going to have intra-database parallel slave based on group commit on master.

Re: [PERFORM] slave wal is ahead of master

2014-03-19 Thread Evgeny Shishkin
On 19 Mar 2014, at 21:11, Sethu Prasad wrote: > fsync > ... taking more knowledge around this will shed some light to understand this > problem "slave ahead of master" > > "there was silence, because master hang." > ... replication halted here, master holds the latest copy which is missing at

Re: [PERFORM] Query taking long time

2014-03-11 Thread Evgeny Shishkin
Hello, new server with more ram will definitely help to keep your working set in memory. But if you want your queries be fast on cold (on disk) data, then you need more/faster disks. And work_mem = 1000MB is too much, better set to 32MB so you don’t get OOM Killer. And may be slightly lower sh

Re: [PERFORM] Slow SELECT by primary key? Postgres 9.1.2

2013-05-27 Thread Evgeny Shishkin
y on test_select (cost=0.00..8.36 rows=1 > width=21) (actual time=23.072..23.074 rows=1 loops=1) >Index Cond: (id = 50) > Total runtime: 23.192 ms > > > > > On Mon, May 27, 2013 at 10:21 AM, Evgeny Shishkin > wrote: > > On May 27, 2013, at 6:02

Re: [PERFORM] Slow SELECT by primary key? Postgres 9.1.2

2013-05-27 Thread Evgeny Shishkin
On May 27, 2013, at 6:02 PM, John Mudd wrote: > Postgres 9.1.2 on Ubuntu 12.04 > > Any reason why a select by primary key would be slower than a select that > includes an ORDER BY? I was really hoping using the primary key would give me > a boost. > You created my_key after data loading, a

Re: [PERFORM] performance database for backup/restore

2013-05-21 Thread Evgeny Shishkin
On May 21, 2013, at 5:18 PM, Jeison Bedoya wrote: > Hi people, i have a database with 400GB running in a server with 128Gb RAM, > and 32 cores, and storage over SAN with fiberchannel, the problem is when i > go to do a backup whit pg_dumpall take a lot of 5 hours, next i do a restore > and ta

Re: [PERFORM] Issues with OSX and SHMMAX?

2013-04-21 Thread Evgeny Shishkin
On Apr 22, 2013, at 1:29 AM, Josh Berkus wrote: > Folks, > > I've heard a rumor that the most recent update of OSX "mountain lion" > lowers the installed SHMMAX to 4MB, which prevents PostgreSQL from > installing. Can anyone verify this? > kern.sysv.shmmax: 4194304 mac os x 10.8.3 > -- > J

Re: [PERFORM] Performance on Bulk Insert to Partitioned Table

2012-12-24 Thread Evgeny Shishkin
On Dec 24, 2012, at 9:07 PM, Charles Gomes wrote: > By the way, I've just re-wrote the code to target the partitions individually > and I've got almost 4 times improvement. > Shouldn't it be faster to process the trigger, I would understand if there > was no CPU left, but there is lots of cpu

Re: [PERFORM] Why does the number of rows are different in actual and estimated.

2012-12-13 Thread Evgeny Shishkin
On Dec 14, 2012, at 3:36 AM, Tom Lane wrote: > Evgeny Shishkin writes: >> On Dec 14, 2012, at 3:09 AM, Andrew Dunstan wrote: >>> Well, it looks like it's choosing a join order that's quite a bit different >>> from the way the query is expressed, so the O

Re: [PERFORM] Why does the number of rows are different in actual and estimated.

2012-12-13 Thread Evgeny Shishkin
On Dec 14, 2012, at 3:09 AM, Andrew Dunstan wrote: > > On 12/13/2012 05:42 PM, Claudio Freire wrote: >> And it looks like it all may be starting to go south here: >>>-> Hash Join >>> (cost=9337.97..18115.71 rows=34489 width=244) (actual >>> time=418.0

Re: [PERFORM] Why does the number of rows are different in actual and estimated.

2012-12-13 Thread Evgeny Shishkin
On Dec 14, 2012, at 2:36 AM, Andrew Dunstan wrote: > > On 12/13/2012 05:12 PM, AI Rumman wrote: >> Why does the number of rows are different in actual and estimated? >> > > > Isn't that in the nature of estimates? An estimate is a heuristic guess at > the number of rows it will find for the

Re: [PERFORM] hash join vs nested loop join

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 8:57 AM, Evgeny Shishkin wrote: > > On Dec 12, 2012, at 8:44 AM, Huan Ruan wrote: > >> >> On 12 December 2012 15:33, Evgeny Shishkin wrote: >> Optimiser thinks that nested loop is more expensive, because of point PK >> lookups,

Re: [PERFORM] hash join vs nested loop join

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 8:25 AM, Huan Ruan wrote: > Hello All > > While investigating switching to Postgres, we come across a query plan that > uses hash join and is a lot slower than a nested loop join. > > I don't understand why the optimiser chooses the hash join in favor of the > nested loop

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 7:05 AM, David Boreham wrote: > On 12/11/2012 7:49 PM, Evgeny Shishkin wrote: >> Yeah, s3700 looks promising, but sata interface is limiting factor for this >> drive. >> I'm looking towards SMART ssd >&

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 6:41 AM, David Boreham wrote: > On 12/11/2012 7:38 PM, Evgeny Shishkin wrote: >> Which drives would you recommend? Besides intel 320 and 710. > Those are the only drive types we have deployed in servers at present (almost > all 710, but we have some 320 f

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 6:26 AM, David Boreham wrote: > On 12/11/2012 7:20 PM, Evgeny Shishkin wrote: >> Oh, there is no 100% safe system. > In this case we're discussing specifically "safety in the event of power loss > shortly after the drive indicates to the controlle

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 6:15 AM, David Boreham wrote: > On 12/11/2012 7:13 PM, Evgeny Shishkin wrote: >> Yes, i am aware of this issue. Never experienced this neither on intel 520, >> no ocz vertex 3. >> Have you heard of them on this list? > People have done plug-pul

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 6:02 AM, Craig Ringer wrote: > On 12/12/2012 09:44 AM, Evgeny Shishkin wrote: >> So far, more than a year already, i bought consumer ssds with 300-400$ hw >> raid. Cost effective and fast, may be not very safe, but so far so good. All >> data pro

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 5:29 AM, Craig Ringer wrote: > On 12/12/2012 09:17 AM, Evgeny Shishkin wrote: >> >> Actually most of low-end SSDs don't do write caching, they do not have >> enough ram for that. Sandforce for example. >> > Or, worse, some of them

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 5:03 AM, Craig Ringer wrote: > On 12/12/2012 06:44 AM, Evgeny Shishkin wrote: >> >> On Dec 12, 2012, at 2:41 AM, Niels Kristian Schjødt >> wrote: >> >>> Are you using a hardware based raid controller with them? >>> >>

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
n and other cloud shit. > Den 11/12/2012 20.11 skrev "Evgeny Shishkin" : > > On Dec 11, 2012, at 10:54 PM, Niels Kristian Schjødt > wrote: > >> And what is your experience so far? >> > Increased tps by a factor of 10, database no longer a limiting facto

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 11, 2012, at 10:54 PM, Niels Kristian Schjødt wrote: > And what is your experience so far? > Increased tps by a factor of 10, database no longer a limiting factor of application. And it is cheaper than brand rotating drives. > Den 11/12/2012 18.16 skrev "Evgeny Shish

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 11, 2012, at 5:35 PM, Niels Kristian Schjødt wrote: > > Den 11/12/2012 kl. 14.29 skrev Craig Ringer : > >> On 12/11/2012 06:04 PM, Niels Kristian Schjødt wrote: >>> >>> Maybe I should mention, that I never see more than max 5Gb out of my total >>> 32Gb being in use on the server… Can

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-10 Thread Evgeny Shishkin
On Dec 11, 2012, at 2:51 AM, Niels Kristian Schjødt wrote: > Pitch > ## > I previously posted this question > http://archives.postgresql.org/pgsql-performance/2012-11/msg00289.php about a > performance i

Re: [PERFORM] Have: Seq Scan - Want: Index Scan - what am I doing wrong?

2012-10-16 Thread Evgeny Shishkin
On Oct 17, 2012, at 3:52 AM, Chris Ruprecht wrote: > Hi guys, > > PG = 9.1.5 > OS = winDOS 2008R8 > > I have a table that currently has 207 million rows. > there is a timestamp field that contains data. > more data gets copied from another database into this database. > How do I make this do a

Re: [PERFORM] Two identical systems, radically different performance

2012-10-08 Thread Evgeny Shishkin
On Oct 9, 2012, at 3:24 AM, Tomas Vondra wrote: > On 9.10.2012 00:33, Evgeny Shishkin wrote: >>> >>> pgbench: Old server >>> >>>pgbench -i -s 100 -U test >>>pgbench -U test -c ... -t ... >>> >>>-c -t TPS &g

Re: [PERFORM] Two identical systems, radically different performance

2012-10-08 Thread Evgeny Shishkin
On Oct 9, 2012, at 2:44 AM, Claudio Freire wrote: > On Mon, Oct 8, 2012 at 7:25 PM, Craig James wrote: But why? What have I overlooked? >>> >>> Do you have readahead properly set up on the new one? >> >> >> # blockdev --getra /dev/sdb1 >> 256 > > > It's probably this. 256 is way too l

Re: [PERFORM] Two identical systems, radically different performance

2012-10-08 Thread Evgeny Shishkin
On Oct 9, 2012, at 1:45 AM, Craig James wrote: > This is driving me crazy. A new server, virtually identical to an old one, > has 50% of the performance with pgbench. I've checked everything I can think > of. > > The setups (call the servers "old" and "new"): > > old: 2 x 4-core Intel Xeon

Re: [PERFORM] Two identical systems, radically different performance

2012-10-08 Thread Evgeny Shishkin
On Oct 9, 2012, at 2:06 AM, Craig James wrote: > > > On Mon, Oct 8, 2012 at 2:57 PM, Evgeny Shishkin wrote: > > On Oct 9, 2012, at 1:45 AM, Craig James wrote: > >> I tested both the RAID10 data disk and the RAID1 xlog disk with bonnie++. >> The xlog

Re: [PERFORM] Two identical systems, radically different performance

2012-10-08 Thread Evgeny Shishkin
On Oct 9, 2012, at 1:45 AM, Craig James wrote: > This is driving me crazy. A new server, virtually identical to an old one, > has 50% of the performance with pgbench. I've checked everything I can think > of. > > The setups (call the servers "old" and "new"): > > old: 2 x 4-core Intel Xeon

[PERFORM] wrong join result set estimate

2012-09-28 Thread Evgeny Shishkin
Hello, i have a problem with relatively easy query. EXPLAIN ANALYZE SELECT content.* FROM content JOIN blog ON blog.id = content.blog_id JOIN community_prop ON blog.id = community_prop.blog_id JOIN community ON community.id = community_prop.id WHERE community.id IN (33, 55, 61, 1741, 75, 90, 1

Re: [PERFORM] hardware advice

2012-09-27 Thread Evgeny Shishkin
On Sep 28, 2012, at 1:36 AM, Scott Marlowe wrote: > On Thu, Sep 27, 2012 at 3:16 PM, Claudio Freire > wrote: >> On Thu, Sep 27, 2012 at 6:08 PM, David Boreham >> wrote: We went from Dunnington to Nehalem, and it was stunning how much better the X5675 was compared to the E7450

Re: [PERFORM] hardware advice

2012-09-27 Thread Evgeny Shishkin
On Sep 28, 2012, at 1:20 AM, Shaun Thomas wrote: > On 09/27/2012 04:08 PM, Evgeny Shishkin wrote: > >> from benchmarking on my r/o in memory database, i can tell that 9.1 >> on x5650 is faster than 9.2 on e2440. > > How did you run those benchmarks? I find that incredi

Re: [PERFORM] hardware advice

2012-09-27 Thread Evgeny Shishkin
Hello, from benchmarking on my r/o in memory database, i can tell that 9.1 on x5650 is faster than 9.2 on e2440. I do not have x5690, but i have not so loaded e2660. If you can give me a dump and some queries, i can bench them. Nevertheless x5690 seems more efficient on single threaded workloa