On 2011-05-09 22:32, Chris Hoover wrote:
The issue we are running into is how do we benchmark this server,
specifically, how do we get valid benchmarks for the Fusion IO card?
Normally to eliminate the cache effect, you run iozone and other
benchmark suites at 2x the ram. However, we can't
On 05/09/2011 11:13 PM, Shaun Thomas wrote:
Take a look at /proc/sys/vm/dirty_ratio and
/proc/sys/vm/dirty_background_ratio if you have an older Linux system,
or /proc/sys/vm/dirty_bytes, and /proc/sys/vm/dirty_background_bytes
with a newer one.
On older systems for instance, those are set to
I have 8-core server, I wanted to ask whether a query can be divided for
multiple processors or cores, if it could be what to do in postgresql
Thanks
On 10/05/11 10:40, Aren Cambre wrote:
> how are you reading through the table? if you are using OFFSET, you
> owe me a steak dinner.
>
>
> Nope. :-)
>
> Below is my exact code for the main thread. The C# PLINQ statement is
> highlighted. Let me know if I can help to explain this.
Lookin
> How many times was the kernel tested with this much memory, for example
> ? (never??)
This is actually *extremely* relevant.
Take a look at /proc/sys/vm/dirty_ratio and /proc/sys/vm/dirty_background_ratio
if you have an older Linux system, or /proc/sys/vm/dirty_bytes, and
/proc/sys/vm/dirty_b
>
> how are you reading through the table? if you are using OFFSET, you
> owe me a steak dinner.
>
>
Nope. :-)
Below is my exact code for the main thread. The C# PLINQ statement is
highlighted. Let me know if I can help to explain this.
NpgsqlConnection arrestsConnection = new NpgsqlC
On Mon, May 9, 2011 at 10:15 PM, Aren Cambre wrote:
>> Your OS won't *see* eight processors if you turn of HT. :-)
>> I'm going to pursue this digression just a little further, because
>> it probably will be biting you sooner or later. We make sure to
>> configure the BIOS on our database server
On Mon, May 9, 2011 at 8:15 PM, Aren Cambre wrote:
>> Your OS won't *see* eight processors if you turn of HT. :-)
>> I'm going to pursue this digression just a little further, because
>> it probably will be biting you sooner or later. We make sure to
>> configure the BIOS on our database servers
>
> Your OS won't *see* eight processors if you turn of HT. :-)
>
> I'm going to pursue this digression just a little further, because
> it probably will be biting you sooner or later. We make sure to
> configure the BIOS on our database servers to turn off
> hyperthreading. It really can make a
>
> so follow the advice above. we need to see pg_stat_activity, and/or
> pg_locks while your test is running (especially take note of pg_lock
> records with granted=f)
Attached.
The database is named de. The process with procpid 3728 has the SQL query
for my "main" thread--the one that reads th
On Mon, 9 May 2011, David Boreham wrote:
On 5/9/2011 6:32 PM, Craig James wrote:
Maybe this is a dumb question, but why do you care? If you have 1TB RAM
and just a little more actual disk space, it seems like your database will
always be cached in memory anyway. If you "eliminate the cach ef
Craig James wrote:
Maybe this is a dumb question, but why do you care? If you have 1TB
RAM and just a little more actual disk space, it seems like your
database will always be cached in memory anyway. If you "eliminate
the cach effect," won't the benchmark actually give you the wrong
real-li
On 5/9/2011 6:32 PM, Craig James wrote:
Maybe this is a dumb question, but why do you care? If you have 1TB
RAM and just a little more actual disk space, it seems like your
database will always be cached in memory anyway. If you "eliminate
the cach effect," won't the benchmark actually give y
2011/5/9 Chris Hoover:
I've got a fun problem.
My employer just purchased some new db servers that are very large. The
specs on them are:
4 Intel X7550 CPU's (32 physical cores, HT turned off)
1 TB Ram
1.3 TB Fusion IO (2 1.3 TB Fusion IO Duo cards in a raid 10)
3TB Sas Array (48 15K 146GB spi
2011/5/9 Chris Hoover :
> I've got a fun problem.
> My employer just purchased some new db servers that are very large. The
> specs on them are:
> 4 Intel X7550 CPU's (32 physical cores, HT turned off)
> 1 TB Ram
> 1.3 TB Fusion IO (2 1.3 TB Fusion IO Duo cards in a raid 10)
> 3TB Sas Array (48 15
On 5/5/11 12:47 AM, Sethu Prasad wrote:
> I did the hot standby configured earlier and at that time I started
> using(querying) the standby database.
>
> May be something missed on the archive command.
Most likely, yes. PostgreSQL shouldn't start up under such
circumstances, but apparently you f
On 5/9/11 1:41 PM, Merlin Moncure wrote:
> I'm looking for a good ready-to-run broad spectrum (tests cpu bound,
> i/o bound, various cases, various sizes) benchmark. I tried dbt5 and
> got it compiled after some effort but it immediately fails upon
> running so I punted. Anybody have any ideas wh
On 05/09/2011 05:59 PM, Kevin Grittner wrote:
I'm going to pursue this digression just a little further, because
it probably will be biting you sooner or later. We make sure to
configure the BIOS on our database servers to turn off
hyperthreading. It really can make a big difference in performa
On 05/09/2011 04:32 PM, Chris Hoover wrote:
So, does anyone have any suggestions/experiences in benchmarking
storage when the storage is smaller then 2x memory?
If you do the Linux trick to drop its caches already mentioned, you can
start a database test with zero information in memory. In t
Aren Cambre wrote:
>>>- Core i7 processor--4 physical cores, but OS sees 8 cores
>>>via hyper-threading
>>
>> Most benchmarks I've seen comparing hyper-threading show that
>> PostgreSQL performs better if you don't try to convince it that
>> one core is actually two different cores. Wit
On Mon, May 9, 2011 at 4:50 PM, Aren Cambre wrote:
>> Are you sure you are really using > 1 connection? While your test is
>> running, log onto postgres with psql and grab the output of
>> pg_stat_activity a few times. What do you see?
>
> Thanks. If a connection corresponds to a process, then t
>
> > Postgres won't use more than 1 CPU core.
>
> One *connection* to PostgreSQL won't directly use more than one
> core. As Merlin suggests, perhaps you're really only running one
> query at a time? The other possibility is that you're somehow
> acquiring locks which cause one process to block
>
> Are you sure you are really using > 1 connection? While your test is
> running, log onto postgres with psql and grab the output of
> pg_stat_activity a few times. What do you see?
>
Thanks. If a connection corresponds to a process, then this suggests I am
using 1 connection for my main threa
Aren Cambre wrote:
> Postgres won't use more than 1 CPU core.
One *connection* to PostgreSQL won't directly use more than one
core. As Merlin suggests, perhaps you're really only running one
query at a time? The other possibility is that you're somehow
acquiring locks which cause one process
On Mon, May 9, 2011 at 4:23 PM, Aren Cambre wrote:
> I have a multi-threaded app. It uses ~22 threads to query Postgres.
> Postgres won't use more than 1 CPU core. The 22-threaded app only has 3% CPU
> utilization because it's mostly waiting on Postgres.
> Here's the details:
> The app has a "main
On 5/9/11 9:59 , Kevin Grittner wrote:
You don't need to do that; you can specify an opclass for the index
to tell it that you don't want to order by the normal collation, but
rather in a way which will allow the index to be useful for pattern
matching:
http://www.postgresql.org/docs/9.0/intera
I have a multi-threaded app. It uses ~22 threads to query Postgres.
Postgres won't use more than 1 CPU core. The 22-threaded app only has 3% CPU
utilization because it's mostly waiting on Postgres.
Here's the details:
The app has a "main" thread that reads table A's 11,000,000 rows, one at a
tim
On 5/9/2011 3:11 PM, Merlin Moncure wrote:
The problem with bonnie++ is that the results aren't valid, especially
the read tests. I think it refuses to even run unless you set special
switches.
I only care about writes ;)
But definitely, be careful with the tools. I tend to prefer small
prog
On Mon, May 9, 2011 at 3:59 PM, David Boreham wrote:
>
>> hm, if it was me, I'd write a small C program that just jumped
>> directly on the device around and did random writes assuming it wasn't
>> formatted. For sequential read, just flush caches and dd the device
>> to /dev/null. Probably some
On 05/09/2011 03:32 PM, Chris Hoover wrote:
So, does anyone have any suggestions/experiences in benchmarking storage
when the storage is smaller then 2x memory?
We had a similar problem when benching our FusionIO setup. What I did
was write a script that cleared out the Linux system cache bef
On May 9, 2011, at 1:32 PM, Chris Hoover wrote:
> 1.3 TB Fusion IO (2 1.3 TB Fusion IO Duo cards in a raid 10)
Be careful here. What if the entire card hiccups, instead of just a device on
it? (We've had that happen to us before.) Depending on how you've done your
raid 10, either all your parit
hm, if it was me, I'd write a small C program that just jumped
directly on the device around and did random writes assuming it wasn't
formatted. For sequential read, just flush caches and dd the device
to /dev/null. Probably someone will suggest better tools though.
I have a program I wrote ye
On Mon, May 9, 2011 at 3:32 PM, Chris Hoover wrote:
> I've got a fun problem.
> My employer just purchased some new db servers that are very large. The
> specs on them are:
> 4 Intel X7550 CPU's (32 physical cores, HT turned off)
> 1 TB Ram
> 1.3 TB Fusion IO (2 1.3 TB Fusion IO Duo cards in a ra
I'm looking for a good ready-to-run broad spectrum (tests cpu bound,
i/o bound, various cases, various sizes) benchmark. I tried dbt5 and
got it compiled after some effort but it immediately fails upon
running so I punted. Anybody have any ideas where I could look?
merlin
--
Sent via pgsql-per
I've got a fun problem.
My employer just purchased some new db servers that are very large. The
specs on them are:
4 Intel X7550 CPU's (32 physical cores, HT turned off)
1 TB Ram
1.3 TB Fusion IO (2 1.3 TB Fusion IO Duo cards in a raid 10)
3TB Sas Array (48 15K 146GB spindles)
The issue we are
Marcus Engene wrote:
> On 5/9/11 8:57 , Kevin Grittner wrote:
>>
>> That could be a difference is collations. What do you get from
>> the query on this page for each database?:
>>
>> http://wiki.postgresql.org/wiki/Server_Configuration
> There's indeed a different collation. Why is this affecti
On 5/9/11 8:57 , Kevin Grittner wrote:
That could be a difference is collations. What do you get from the
query on this page for each database?:
http://wiki.postgresql.org/wiki/Server_Configuration
-Kevin
There's indeed a different collation. Why is this affecting? Can i force
a column
Marcus Engene wrote:
> I have a table with a few million rows and this index:
> CREATE INDEX bond_item_common_x7 ON bond_item_common
> ((lower(original_filename)));
> Dropping the wildcard for the like, both databases uses the index.
>
> Is there a way to convince Postgres to try not to do f
Marcus Engene writes:
> There are about 2M rows on bonddump and 4M rows on bond90.
> bonddump is on a 8MB RAM machine, bond90 is on a 72MB RAM machine.
> The table is analyzed properly both places.
I'll bet one database was initialized in C locale and the other not.
regar
Dear list,
I have a table with a few million rows and this index:
CREATE INDEX bond_item_common_x7 ON bond_item_common
((lower(original_filename)));
There are about 2M rows on bonddump and 4M rows on bond90.
bonddump is on a 8MB RAM machine, bond90 is on a 72MB RAM machine.
The table is anal
On 05/06/2011 05:13 PM, Thomas Hägi wrote:
the query "SELECT * FROM data.logs ORDER BY re_timestamp DESC LIMIT
100" does use seq scans on all tables instead of using the existing
indexes which takes ages. when issuing the the same query to one of
the child tables directly ("SELECT * FROM data.l
* Thomas Hägi:
> how can i get postgres to use the indexes when querying the master
> table?
I believe that this is a new feature in PostgreSQL 9.1 ("Allow
inheritance table queries to return meaningfully-sorted results").
--
Florian Weimer
BFK edv-consulting GmbH http://w
42 matches
Mail list logo
